KeyValue Stores
22 likes3,812 views
The document presents an overview of key-value stores and NoSQL databases, highlighting CouchDB, Redis, and Cassandra among others. It discusses their features, scalability, data modeling, and performance characteristics, presenting specific Ruby libraries for each. The conclusion emphasizes that each database addresses different challenges, suggesting best use cases for each technology.
![Redis
require 'redis'
redis = Redis.new
# Strings
redis['foo'] = 'bar' # => 'bar'
redis['foo'] # => 'bar'
# Expirations
redis.expire('foo', 5) # will expire existing key 'foo' in 5 sec
redis.set('foo', 'bar', 5) # set 'foo' with 5 sec expiration
# Counters
redis.incr('counter') # => 1
redis.incr('counter', 10) # => 11
redis.decr('counter') # => 10](https://image.slidesharecdn.com/keyvalue-091203180617-phpapp01/85/KeyValue-Stores-15-320.jpg)
![Redis
# Lists
%w(1st 2nd 3rd).each { |item| redis.push_tail('logs', item) }
redis.list_range('logs', 0, -1) # => ["1st", "2nd", "3rd"]
redis.pop_head('logs') # => "1st"
redis.pop_tail('logs') # => "3rd"
# Sets
%w(one two).each { |item| redis.set_add('foo-tags', item) }
%w(two three).each { |item| redis.set_add('bar-tags', item) }
redis.set_intersect('foo-tags', 'bar-tags') # => ["two"]
redis.set_union('foo-tags', 'bar-tags') # => ["three", "two",
"one"]](https://image.slidesharecdn.com/keyvalue-091203180617-phpapp01/85/KeyValue-Stores-16-320.jpg)
![CouchDB
View: relation_blog_posts
function(doc){
if(doc.type=="post"){
emit([doc.blog_id,
doc.created_at],
doc);
}
}](https://image.slidesharecdn.com/keyvalue-091203180617-phpapp01/85/KeyValue-Stores-30-320.jpg)
![CouchDB
View: relation_blog_posts
GET
/db/design_doc/relation_blog_posts?
startkey=[blog_1]](https://image.slidesharecdn.com/keyvalue-091203180617-phpapp01/85/KeyValue-Stores-31-320.jpg)
KeyValue Stores
- 1. KeyValue Stores Jedi Master Edition
- 2. Who? Antonio Garrote @antoniogarrote Mauro Pompilio @malditogeek Pablo Delgado @pablete
- 3. Agenda •Why? •Definitions •CouchDB •Redis •Cassandra •Ruby Libraries •Demo application •Data modeling •Benchmark
- 4. Why? •Scalability •Availability •Fault Tolerance •Schema-free •Ease of use •Performance •Elasticity •blah blah blah
- 6. NoSQL != NoSQL No SQL Not Only SQL
- 7. Taxonomy •Key-value stores: Redis, Voldemort, Cassandra •Column-oriented datastores: Cassandra, HBase •Document collection databases: CouchDB, MongoDB •Graph database: Neo4J, AllegroGraph •Data structure store: Redis
- 8. CouchDB relax! •Damien Katz •Erlang - OTP compliant •schema-less documents •high availability •completely distributed •made for the web
- 9. CouchDB B-Trees . MapReduce . MVCC
- 10. Ruby Libraries •CouchDB •Pure: net/http + JSON implementation •Thin wrapper: Couchrest http://github.com/jchris/couchrest •ORM/ActiveRecord: ActiveCouch, CouchObject, RelaxDB ..etc http://github.com/arunthampi/activecouch http://github.com/paulcarey/relaxdb
- 11. CouchDB •Rocks •Simplicity and elegance •Much more than a DB •New possibilities for web apps •Sucks •Speed •Speed •Speed
- 12. Redis il meglio d'Italia classy as a tasty as Giulietta a pizza
- 13. Redis •Salvatore 'antirez' Sanfilippo •ANSI C - POSIX compliant •MemCache-like (on steroids) •Data structures store: •strings •counters •lists •sets + sorted sets (>= 1.1)
- 14. Ruby Libraries •Redis •Client: redis-rb http://github.com/ezmobius/redis-rb •Hash/Object mapper: Ohm http://github.com/soveran/ohm •ORM: RedisRecord http://github.com/malditogeek/redisrecord
- 15. Redis require 'redis' redis = Redis.new # Strings redis['foo'] = 'bar' # => 'bar' redis['foo'] # => 'bar' # Expirations redis.expire('foo', 5) # will expire existing key 'foo' in 5 sec redis.set('foo', 'bar', 5) # set 'foo' with 5 sec expiration # Counters redis.incr('counter') # => 1 redis.incr('counter', 10) # => 11 redis.decr('counter') # => 10
- 16. Redis # Lists %w(1st 2nd 3rd).each { |item| redis.push_tail('logs', item) } redis.list_range('logs', 0, -1) # => ["1st", "2nd", "3rd"] redis.pop_head('logs') # => "1st" redis.pop_tail('logs') # => "3rd" # Sets %w(one two).each { |item| redis.set_add('foo-tags', item) } %w(two three).each { |item| redis.set_add('bar-tags', item) } redis.set_intersect('foo-tags', 'bar-tags') # => ["two"] redis.set_union('foo-tags', 'bar-tags') # => ["three", "two", "one"]
- 17. Redis •Rocks •Speed, in memory dataset •Asynch non-blocking persistence •Non-blocking replication •Data structures with atomic operations •Ease of use and deployment •Sucks •Sharding (client-side only at the moment) •Datasets > RAM •Very frequent code updates (?)
- 18. Redis Upcoming coolness... •1.1 •Sorted sets (ZSET), append-only journaling •1.2 •HASH type, JSON dump tool •1.3 •Virtual memory (datasets > RAM) •1.4 •Redis-cluster proxy: consistent hashing and fault tollerant nodes •1.5 •Optimizations, UDP GET/SET
- 19. Cassandra BigTable Dynamo by + by
- 20. Cassandra Structure Storage System over P2P network •Developed at Facebook •Java •Dynamo: partition and replication •Bigtable: Log-structured ColumnFamily data model
- 21. Ruby Libraries •Cassandra •Client: cassandra http://github.com/fauna/cassandra •ORM: cassandra_object http://github.com/NZKoz/cassandra_object •ORM: BigRecord http://github.com/openplaces/bigrecord
- 22. Cassandra •Rocks •High Availability •Incremental Scalability •Minimal Administration •No Single Point of Failure •Sucks •Thrift API (...not so bad) •Change Schema, restart server •The Logo
- 24. Data Modeling •Class mapping •ID generation •Relationships •one-to-one •one-to-many •many-to-many •Index sorting •Pagination •Data filtering
- 25. Cassandra •Class mapping • ColumnFamily :Blog, :Post •ID generation •UUID.new(Time.now) •Relationships •Use ColumnFamily :PostsforUser to hold all posts that belong to a user
- 26. Cassandra •Index sorting •Columns within a ColumnFamily are stored in sorted order. Keys are also sorted (if OrderPreservingPartitioner) •Pagination •for keys get_range (start, finish, count) •for columns get_slice (start, finish, count) •Data filtering •Use get_range/get_slice and play around with start/finish
- 27. Redis •Class mapping • Namespaced keys: 'Post:5:title' •ID generation •Redis counters: incr('Post:ids') •Relationships •Redis lists: push_tail('Post:5:_rating_ids', 4)
- 28. Redis •Index sorting •Redis sort: •sort 'Post:list', by 'Post:*:score', get 'Post:*:id' •Pagination •Redis lists: list_range('Post:list', 0, -9) •Data filtering •Lookups: 'Post:permalink:fifth_post' => 5
- 29. CouchDB •Type attribute in each document •CouchDB automatic ID generation •Related document IDs in the attributes •Views with complex keys •Special attributes for view functions
- 30. CouchDB View: relation_blog_posts function(doc){ if(doc.type=="post"){ emit([doc.blog_id, doc.created_at], doc); } }
- 31. CouchDB View: relation_blog_posts GET /db/design_doc/relation_blog_posts? startkey=[blog_1]
- 32. VPork •Utility for load-testing a distributed hash table. •Allows you to test raw throughput via concurrent read/writes operations. •Hardware: •2 x comodity servers: CoreDuo 2.5Ghz, 4Gb RAM, 7200RPM disks •CouchDB: 2 instances, round-robin balanced •Cassandra: 2 instances •Redis: 1 instance http://github.com/antoniogarrote/vpork
- 33. VPork Throughput with read probability 0.2
- 34. VPork Throughput with read probability 0.5
- 35. VPork Throughput with read probability 0.8
- 36. Conclusions •Complementary to relational solutions •Each K/V address a different problem •Best use case: •CouchDB: distributed/scalable Javascript-only app (no backend) •Cassandra: big amount of writes, no SPOF •Redis: datasets < RAM, lookups, cache, buffers
- 37. Credits •All sponsored products, company names, brand names, trademarks and logos are the property of their respective owners. •Alfa Romeo Giulietta: http://www.flickr.com/photos/ mauboi/3296469097/ •Pizza: http://reportingfrombelgium.wordpress.com/2009/ 05/20/belgian-summer-holidays/ •Sammy: http://www.yuddy.com/celebrity/Sammy-Davis- Jr/bio •Everything else is from teh internets and is free.