Introduction to data modeling with apache cassandra

©2013 DataStax Conﬁdential. Do not distribute without consent.
@PatrickMcFadin
Patrick McFadin 
Chief Evangelist for Apache Cassandra
Introduction to Data Modeling
with Apache Cassandra
1

My Background
…ran into this problem

Gave it my best shot
shard 1 shard 2 shard 3 shard 4
router
client
Patrick,
All your wildest
dreams will come
true.

ACID vs CAP
ACID
CAP - Pick two
Atomic - All or none
Consistency - Only valid data is written
Isolation - One operation at a time
Durability - Once committed, it stays that way
Consistency - All data on cluster
Availability - Cluster always accepts writes
Partition tolerance - Nodes in cluster can’t talk to each other
Cassandra let’s you tune this

Relational Data Models
• 5 normal forms
• Foreign Keys
• Joins
deptId First Last
1 Edgar Codd
2 Raymond Boyce
id Dept
1 Engineering
2 Math
Employees
Department

Relational Modeling
Data
Models
Application

Cassandra Modeling
Data
Models
Application

CQL vs SQL
• No joins
• No aggregations
deptId First Last
1 Edgar Codd
2 Raymond Boyce
id Dept
1 Engineering
2 Math
Employees
Department
SELECT e.First, e.Last, d.Dept
FROM Department d, Employees e
WHERE ‘Codd’ = e.Last
AND e.deptId = d.id

Denormalization
• Combine table columns into a single view
• No joins
SELECT First, Last, Dept
FROM employees
WHERE id = ‘1’
id First Last Dept
1 Edgar Codd Engineering
2 Raymond Boyce Math
Employees

No more sequences
• Great for auto-creation of Ids
• Guaranteed unique
• Needs ACID to work. (Sorry. No sharding)
INSERT INTO user (id, firstName, LastName)
VALUES (seq.nextVal(), ‘Ted’, ‘Codd’)

No sequences???
• Almost impossible in a distributed system
• Couple of great choices
• Natural Key - Unique values like email
• Surrogate Key - UUID
• Universal Unique ID
• 128 bit number represented in character form
• Easily generated on the client
• Same as GUID for the MS folks
99051fe9-6a9c-46c2-b949-38ef78858dd0

KillrVideo.com
• Hosted on Azure
• Code on GitHub
• Also on your USB
• Data Model for examples

Entity Table
• Simple view of a single
user
• UUID used for ID
• Simple primary key // Users keyed by id
CREATE TABLE users (
userid uuid,
firstname text,
lastname text,
email text,
created_date timestamp,
PRIMARY KEY (userid)
);
SELECT firstname, lastname
FROM user
WHERE userId = 99051fe9-6a9c-46c2-b949-38ef78858dd0

CQL Collections
• Meant to be dynamic part of table
• Update syntax is very different from insert
• Reads require all of collection to be read

CQL Set
• Set is sorted by CQL type comparator
INSERT INTO collections_example (id, set_example)
VALUES(1, {'1-one', '2-two'});
set_example set<text>
Collection name Collection type CQLType

CQL Set Operations
• Adding an element to the set
• After adding this element, it will sort to the beginning.
• Removing an element from the set
UPDATE collections_example
SET set_example = set_example + {'3-three'} WHERE id = 1;
SET set_example = set_example + {'0-zero'} WHERE id = 1;
SET set_example = set_example - {'3-three'} WHERE id = 1;

CQL List
• Ordered by insertion
• Use with caution
list_example list<text>
Collection name Collection type
INSERT INTO collections_example (id, list_example)
VALUES(1, ['1-one', '2-two']);
CQLType

CQL List Operations
• Adding an element to the end of a list
• Adding an element to the beginning of a list
• Deleting an element from a list
SET list_example = list_example + ['3-three']
WHERE id = 1;
SET list_example = ['0-zero'] + list_example
WHERE id = 1;
SET list_example = list_example - ['3-three'] WHERE id = 1;

CQL Map
• Key and value
• Key is sorted by CQL type comparator
INSERT INTO collections_example (id, map_example)
VALUES(1, { 1 : 'one', 2 : 'two' });
map_example map<int,text>
Collection name Collection type Value CQLTypeKey CQLType

CQL Map Operations
• Add an element to the map
• Update an existing element in the map
• Delete an element in the map
SET map_example[3] = 'three'
WHERE id = 1;
SET map_example[3] = 'tres'
WHERE id = 1;
DELETE map_example[3]
FROM collections_example
WHERE id = 1;

Entity with collections
• Same type of entity
• SET type for dynamic data
• tags for each video
// Videos by id
CREATE TABLE videos (
videoid uuid,
userid uuid,
name text,
description text,
location text,
location_type int,
preview_image_location text,
tags set<text>,
added_date timestamp,
PRIMARY KEY (videoid)
);

Index (or lookup) tables
• Table arranged to find data
• Denormalized for speed
• Find videos for a user
// One-to-many from user point of view (lookup table)
CREATE TABLE user_videos (
userid uuid,
videoid uuid,
name text,
PRIMARY KEY (userid, added_date, videoid)
) WITH CLUSTERING ORDER BY (added_date DESC, videoid ASC);

Primary Key
• First column name is the Partition Key
• Subsequent are the Clustering Columns
• Videos will be ordered by added_date and
videoId per user
userid uuid,
videoid uuid,
name text,

Primary key relationship
PRIMARY KEY (userId,added_date,videoId)

Partition Key

Partition Key Clustering Columns

A12378E55F5A32

2005:12:1:102005:12:1:92005:12:1:82005:12:1:7
5F22A0BC
F2B3652CFFB3652D7AB3652C
A12378E55F5A32
SELECT videoId FROM user_videos
WHERE userId = A12378E55F5A32
AND added_date = ‘2005-12-1’
AND videoId = 5F22A0BC

Clustering Order
• Clustering Columns have default order
• Use to specify order
• Bonus: Sorts on disk for speed
userid uuid,
videoid uuid,
name text,

Multiple Lookups
• Same data
• Different lookup pattern // Index for tag keywords
CREATE TABLE videos_by_tag (
tag text,
videoid uuid,
name text,
tagged_date timestamp,
PRIMARY KEY (tag, videoid)
);
// Index for tags by first letter in the tag
CREATE TABLE tags_by_letter (
first_letter text,
tag text,
PRIMARY KEY (first_letter, tag)
);

Many to Many Relationships
• Two views
• Different directions
• Insert data in a batch
// Comments for a given video
CREATE TABLE comments_by_video (
videoid uuid,
commentid timeuuid,
userid uuid,
comment text,
PRIMARY KEY (videoid, commentid)
) WITH CLUSTERING ORDER BY (commentid DESC);
// Comments for a given user
CREATE TABLE comments_by_user (
userid uuid,
commentid timeuuid,
videoid uuid,
comment text,
PRIMARY KEY (userid, commentid)
) WITH CLUSTERING ORDER BY (commentid DESC);

Example 1: Weather Station
• Weather station collects data
• Cassandra stores in sequence
• Application reads in sequence

Use case
• Store data per weather station
• Store time series in order: first to last
• Get all data for one weather station
• Get data for a single date and time
• Get data for a range of dates and times
Needed Queries
Data Model to support queries

Data Model
• Weather Station Id and Time
are unique
• Store as many as needed
CREATE TABLE temperature (
weather_station text,
year int,
month int,
day int,
hour int,
temperature double,
PRIMARY KEY (weather_station,year,month,day,hour)
);
INSERT INTO temperature(weather_station,year,month,day,hour,temperature)
VALUES (‘10010:99999’,2005,12,1,7,-5.6);
VALUES (‘10010:99999’,2005,12,1,8,-5.1);
VALUES (‘10010:99999’,2005,12,1,9,-4.9);
VALUES (‘10010:99999’,2005,12,1,10,-5.3);

Storage Model - Logical View
2005:12:1:7
-5.6
2005:12:1:8
-5.1
2005:12:1:9
-4.9
SELECT weather_station,hour,temperature
FROM temperature
WHERE weatherstation_id='10010:99999';
10010:99999
10010:99999
10010:99999
weather_station hour temperature
2005:12:1:10
-5.3
10010:99999

2005:12:1:12
-5.4
2005:12:1:11
-4.9-5.3-4.9-5.1
2005:12:1:7
-5.6
Storage Model - Disk Layout
2005:12:1:8 2005:12:1:9
10010:99999
2005:12:1:10
Merged, Sorted and Stored Sequentially
SELECT weather_station,hour,temperature
FROM temperature
WHERE weatherstation_id='10010:99999';

Query patterns
• Range queries
• “Slice” operation on disk
SELECT weatherstation,hour,temperature
FROM temperature
WHERE weatherstation=‘10010:99999'
AND year = 2005 AND month = 12 AND day = 1
AND hour >= 7 AND hour <= 10;
Single seek on disk
2005:12:1:12
-5.4
2005:12:1:11
-4.9-5.3-4.9-5.1
2005:12:1:7
-5.6
2005:12:1:8 2005:12:1:9
10010:99999
2005:12:1:10
Partition key for locality

Query patterns
• Range queries
• “Slice” operation on disk
Programmers like this
Sorted by event_time
2005:12:1:7
-5.6
2005:12:1:8
-5.1
2005:12:1:9
-4.9
10010:99999
10010:99999
10010:99999
weather_station hour temperature
2005:12:1:10
-5.3
10010:99999
SELECT weatherstation,hour,temperature
FROM temperature
WHERE weatherstation=‘10010:99999'
AND year = 2005 AND month = 12 AND day = 1
AND hour >= 7 AND hour <= 10;

Thank you!
Bring the questions
Follow me on twitter
@PatrickMcFadin

Introduction to data modeling with apache cassandra

More Related Content

What's hot (20)

Similar to Introduction to data modeling with apache cassandra (20)

More from Patrick McFadin (15)

Recently uploaded (20)

Introduction to data modeling with apache cassandra