Countdown to Zero - Counter Use Cases in Aerospike

Countdown to Zero
Counter Use Cases in Aerospike

2 Proprietary & Confidential | All rights reserved. © 2018 Aerospike Inc.
 I’m Ronen Botzer, and I’m a solutions architect at Aerospike.
 I’ve worked at Aerospike since June 2014, first on the Python and PHP clients.
 I’m active on StackOverflow and the Aerospike community forum.
 This is the third in a series of tech talks about data modeling.
 Check out the slides from the previous ASUG Israel meetups.
Welcome to Aerospike User Group Israel #3

 A Quick Overview of Aerospike Data Model and Storage
 Counter Use Cases
 Data Capping – Telecom
 Frequency Capping – Ad Tech
 Consolidating Counters
 Mitigating Hot Counters
 Strong Consistency
 Counters
 Ticket Inventory – Countdown to Zero
 Summary
Agenda

Aerospike is a Primary Key Database
Objects stored in Aerospike are called records
A bin holds the value of a supported data type: integer, double, string, bytes, list, map,
geospatial
Every record is uniquely identified by the 3-tuple (namespace, set, user-key)
A record contains one or more bins
(namespace, set, user-key)
EXP – Expiration Timestamp
LUT – Last Update Time
GEN – Generation
RECORD
EXP LUT GEN BIN1 BIN2

 Aerospike is a row-oriented distributed database
 Rows (records) contain one or more columns (bins)
 Similar to an RDBMS with primary-key table lookups
 Single record transactions
 Namespaces can be configured for strong consistency
Aerospike Concepts
Aerospike RDBMS
Namespace Tablespace or Database
Set Table
Record Row
Bin Column
Bin type
Integer
Double
String
Bytes
List (Unordered, Ordered)
Map (Unordered,
K-Ordered, KV-Ordered)
GeoJSON

 Integer
 8B of storage (64-bit)
 In places where signed values are needed (secondary indexes) the integer is
signed in the range of –(2^63) to (2^63) – 1.
 Double
 64-bit IEEE-754
 Both Integer and Double values can be used for counters.
 Support the atomic increment() operation.
 Can be stored in special data-in-index namespaces.
Numeric Data Types in Aerospike

Namespace Storage

Data Capping
namespace users {
memory-size 80G
replication-factor 2
prefer-uniform-balance true
partition-tree-sprigs 8192
storage-engine device {
write-block-size 128K
data-in-memory false
read-page-cache true
post-write-queue 1024
device /dev/nvme1n1p1
}
}

('users', 'mobile', 'cf296d9a-0b77-4dd0-8d2b-91e59a6f02d2')
--
name: 'Jasper Madison Jr.'
age: 27
payment-method: [
{ 'type': 'visa', 'last4’: 6164,
'expires': '2019-09'},
{
'type': 'mastercard', 'last4': 7147,
'expires': '2023-03'}
]
number: [ 1, 408, 5551212 ]
data: 10711339520
 Atomically increment() the integer value of data to implement a counter
 What's the problem with this approach?
Data Capping

 Think about the access patterns.
 The naive implementation has the user's data stored in a single record.
 Minimizes the amount of memory used by this namespace.
 Reduces the number of reads.
 Problem: the data counter updates frequently.
 Solution: split the data counter into a separate in-memory namespace.
 Consider using the data-in-index optimization.
Optimizing Counters – Namespace Considerations

Optimizing Counters – Namespace Considerations
namespace counters {
memory-size 20G
default-ttl 1d
replication-factor 2
partition-tree-sprigs 8192
prefer-uniform-balance true
single-bin true
data-in-index true
storage-engine device {
data-in-memory true
filesize 40G
file /opt/aerospike/data/counters.dat
}
}

('counters', 'mobdata', 'cf296d9a-0b77-4dd0-8d2b-91e59a6f02d2|2019-02')
--
: 10711339520
• Initialize this record with an explicit 0 value each month
• On all writes (each increment) set the TTL to -1, i.e. NEVER_EXPIRE
• Combine the increment with a read. After every update the app knows the latest value
• Archive the counter after rolling to the new month
Data Capping – Counter Using Data in Index

from aerospike_helpers.operations import operations as oh
key = ('counters', mobdata', 'cf296d9a-0b77-4dd0-8d2b-91e59a6f02d2|2019-02')
ops = [
oh.increment('', 1500),
oh.read('')
]
ttl = aerospike.TTL_NEVER_EXPIRE # AKA ttl -1
(key, meta, bins) = client.operate(key, ops, {'ttl': ttl},
{'timeout': 50})
Data Capping – Increment and Read in a single Transaction

('counters', 'adcap', 'cf296d9a-0b77-4dd0-8d2b-91e59a6f02d2|adx123|2019-03-05')
--
: 3
 Keep track of ads served to a user in a given day.
 Compound key of userID | adID | day
 Increment should work as an upsert.
 Initial TTL of 24 hours (use the default TTL of 1 day).
 Do not update the TTL if the record exists, by setting it to -2.
Frequency Capping

Frequency Capping
from aerospike import exception as e
key = ('counters', 'adcap', 'cf296d9a-0b77-4dd0-8d2b-91e59a6f02d2|adx123|2019-03-05')
try:
(key, meta, bins) = client.get(key, {'timeout': 20})
if bins[''] < AD_LIMIT:
client.increment(key, '', 1, {'ttl': aerospike.TTL_DONT_UPDATE}) # aka ttl -2
# continue and attempt to serve the ad
except e.RecordNotFound:
ttl = aerospike.TTL_NAMESPACE_DEFAULT # AKA ttl 0, inherit the default-ttl
client.put(key, '', 1, {'ttl': ttl},
policy={'exists': aerospike.POLICY_EXISTS_CREATE})

('counters', 'adcap', 'cf296d9a-0b77-4dd0-8d2b-91e59a6f02d2|cmp123|2019-03-05')
--
: {
'adx123': 2,
'adx456': 3
}
 Keep track of all campaign ads served to a user in a given day
 Compound key of userID | campaignID | day
 Map increment should work as an upsert, returning the current count
 Initial TTL of 24 hours (default TTL is 1 day)
 Do not update the TTL if the record exists, by setting it to -2.
 Only numeric data works with data-in-index. Use data-in-memory/single-bin.
Frequency Capping – Consolidating Counters

Frequency Capping – Consolidating Counters
key = ('counters', 'adcap', 'cf296d9a-0b77-4dd0-8d2b-91e59a6f02d2|cmp123|2019-03-05')
try:
val = client.map_get_by_key(key, 'ads', 'adx123', aerospike.MAP_RETURN_VALUE)
if val < AD_LIMIT:
# continue and attempt to serve the ad
client.map_increment(key, '', 'adx123', 1, {}, {'ttl': aerospike.TTL_DONT_UPDATE})
except e.RecordNotFound as err:
try:
ops = [
mh.map_put('', 'adx123', 1, {}),
mh.map_get_by_key('', 'adx123', aerospike.MAP_RETURN_VALUE)
]
ttl = aerospike.TTL_NAMESPACE_DEFAULT # AKA ttl 0, inherit the default-ttl
(key, meta, bins) = client.operate(key, ops, {'ttl': ttl})

Solution:
 Identify hot counters with a key busy (error code 14) exception.
 Shard into several records <key||1> .. <key||N>. These will distribute to different nodes.
Reading the counter:
 How do you know if this counter is sharded? EAFP or Check ahead?
 EAFP approach:
 Overload the counter with a string. Assume all records aren't sharded.
 Catch any bin incompatible (error code 12) exception.
 Check-ahead approach:
 Check for the existence of a shard key with exists().
 If this counter is sharded, use a batch-read to fetch all the shards, and combine in the app.
Hot Counters

('counters', 'adcap', 'cf296d9a-0b77-4dd0-8d2b-91e59a6f02d2|adx123|2019-03-05') -- : 🚀 U+1F680
And
('counters', 'adcap', 'cf296d9a-0b77-4dd0-8d2b-91e59a6f02d2|adx123|2019-03-05||1') -- : 14
Either try to increment the original record and fail with an incompatible bin error (code 12), or check
for the existence of a shard key. One of those being true should lead to batch read the shard keys.
Hot Counters

 In an AP system you have no guarantee that a write succeeded or failed on both the
master and replica, even when the cluster is stable (timeouts). It is best effort.
 Stale reads can happen from the slave before the transaction completes.
 Subsequent dirty reads if the write happens to one or both and a timeout occurs.
 An AP system will lose writes during network partitions.
 Aerospike 4.0 passed Jepsen testing of Strong Consistency (as did MongoDB,
CockroachDB).
 In a stable Aerospike 4 cluster with RF=2 the performance is the same as AP mode.
 If you want to be sure that your counters are accurate, you should use a namespace that is
defined as strong-consistency true.
Strong Consistency Mode

(Integer counter , Unordered Unique transactionIDs [ 1, 4, 7, 3, 9])
• Generate a transaction ID for each call. For example, use 4B for the client (process ID)
masked with a 4 byte transaction counter in the process.
1. In one transaction
a. Increment the counter value atomically.
b. list-prepend the transaction ID while declaring the list should be unique.
c. Trim the list to N recent transactions (by index).
d. Return the value of the counter.
2. If it succeeded then we’re done.
3. If it fails on a uniqueness violation for the transaction ID, then we’re done. This
transaction already happened (error code 24 ElementExistsError).
4. If the transaction is ‘inDoubt’ go to step 1.
Counters in Strong Consistency Mode

Shows (Integer seats , Unordered Unique carts { c1, c2, c3, c4, … })
Cart: cartID => [ttl, status, quantity]
• This application handles ticket reservations for shows.
• A key identifies a specific show's inventory.
• Assume there is no assigned seating.
• The application should never overbook a show by selling more tickets than are available.
• The counter is the pair of seats remaining and shopping carts.
• When a show becomes available for booking, it is initialized with a number of seats.
• This example relies on Strong Consistency.
Ticket Inventory – Countdown to Zero

Shows (Integer seats , Unordered Unique carts { c1, c2, c3, c4, … )}
• Assuming that the most recently known seats quantity for the show was greater than
zero, you’d reserve seats with the following transaction:
1. Decrement the seats value.
2. map-set carts with the cart with aerospike.MAP_WRITE_FLAGS_CREATE_ONLY
3. Return the value of seats.
• If the reservation was 'inDoubt' repeat, then go to the next step on success or
ElementExistsError (error code 24).

• If the returned value is sub-zero, roll back with the transaction:
1. map-remove-by-key the cart ID from carts.
2. Increment seats by the cart quantity.
• If the rollback was 'inDoubt' check if the cart exists. If it does, repeat.

• If an item is removed from a cart, or the entire cart explicitly dumped, you’d read the value
of cartID in one map-get from the carts bin, then create a transaction to
1. map-remove-by-key the cartID from carts.
2. map-increment seats with the cart’s quantity.
• Periodically check for abandoned carts using a map-get-by-value( [ttl, WILDCARD] ) and
then treat each matched cart as if it was being explicitly removed.
• When a cart has been fully checked out, remove the map entry from carts , because the
inventory reservation is now permanent.

 We talked about the numeric data types in Aerospike.
 We modeled counters in Aerospike using different approaches.
 We talked about Strong Consistency mode.
 We discussed using Strong Consistency for implementing accurate counters.
What next?
 Take a look at the slides from the previous two Israeli ASUG meetups.
 Go to GitHub; clone the code samples repo; run it; read the code.
 Read the Aerospike blog. Get familiar with all the database features.
 Participate in the community forum (https://discuss.aerospike.com), StackOverflow’s
aerospike tag.
Summary

Reference
 https://www.aerospike.com/docs/guide/data-types.html
 https://www.aerospike.com/docs/architecture/primary-index.html#single-bin-optimization
 https://www.aerospike.com/blog/aerospike-4-strong-consistency-and-jepsen/
 https://jepsen.io/consistency
 https://discuss.aerospike.com/t/handling-timeout-in-case-of-counter-bin/5196/4
 https://www.aerospike.com/docs/architecture/consistency.html#strong-consistency-mode
 https://www.aerospike.com/docs/guide/consistency.html#indoubt-errors
Code Samples
 https://github.com/rbotzer/aerospike-cdt-examples
Aerospike Training
 https://www.aerospike.com/training/
 https://academy.aerospike.com/
More material you can explore:

Thank You!
Any questions?
ronen@aerospike.com

Countdown to Zero - Counter Use Cases in Aerospike

More Related Content

What's hot (20)

Similar to Countdown to Zero - Counter Use Cases in Aerospike (20)

Recently uploaded (20)

Countdown to Zero - Counter Use Cases in Aerospike

Editor's Notes