SlideShare a Scribd company logo
Redis — The AK-47 Of Post-relational Databases
Karel Minařík
Karel Minařík
→ Independent web designer and developer

→ Ruby, Rails, Git, CouchDB propagandista in .cz

→ Previously: Flash Developer; Art Director; Information Architect;… (see LinkedIn)

→ @karmiq at Twitter




→   karmi.cz



                                                                                      Redis
Redis — The AK-47 of Post-relational Databases
WEAPON DESIGN

AK-47



Designed at the end of WWII by Mikhail Kalashnikov

Assault Rifle, not a “submachine gun”

Simple design

Designed for mass production & low-quality manufacturing

Extreme reliability, at the cost of accuracy


                                                     Redis
http://www.youtube.com/watch?v=J6c3DLlM9KA#t=8m32s



                                                     Redis
Reliability



              Redis
Redis — The AK-47 of Post-relational Databases
SIMPLICITY

The Redis Manifesto




We're against complexity.
We believe designing systems is a fight against complexity.
Most of the time the best way to fight complexity is by not
creating it at all.




http://antirez.com/post/redis-manifesto.html            Redis
SIMPLICITY

Redis and NoSQL




What's wrong with RDBMS when used for (many) tasks
that don't need all this complexity? The data model: non
scalable, time complexity hard to predict, and can't model
many common problems well enough.




http://antirez.com/post/MongoDB-and-Redis.html               Redis
Assault rifles as the crucial invention for
                                      new rules of warfare: fighting in shorter
                                      distances and shooting while on the run.


http://www.youtube.com/watch?v=a1KBsqvKpXk#t=7m50s

                                                                               Redis
Memory is the new disk.
Disk is the new tape.
— Jim Gray

http://www.infoq.com/news/2008/06/ram-is-disk

http://research.microsoft.com/en-us/um/people/gray/talks/Flash_is_Good.ppt   Redis
Redis: Main Features




                       Redis
Simplicity       Speed
Predictability   Versatility
Reliability      Low Footprint
SIMPLICITY

Installation



git  clone  http://github.com/antirez/redis
cd  redis
make

./src/redis-­‐server

./src/redis-­‐cli




                                              Redis
SIMPLICITY

The Zen of Redis




(...) what Redis provides are data structures (...)

http://antirez.com/post/MongoDB-and-Redis.html#c1537   Redis
REDIS

Data Structures


Strings

Lists

Sets

Sorted Sets

Hashes



                  Redis
REDIS DATA STRUCTURES

Strings




SET  key  "value"  ~  4  billion  of  keys
GET  key
=>  "value"
DEL  key



                                             Redis
REDIS DATA STRUCTURES

Fetch multiple keys at once



SET  key1  "value1"
SET  key2  "value2"

MGET  key1  key2
=>  "value1"
=>  "value2"


                              Redis
REDIS DATA STRUCTURES

Expiration



EXPIRE  key  5
GET  key
=>  "value"
TTL  key
=>  1
GET  key
=>  (nil)


                        Redis
Usage

Cache
http://antirez.com/post/redis-as-LRU-cache.html

Sessions
https://github.com/mattmatt/redis-session-store




                                                  Redis
REDIS DATA STRUCTURES

Atomic Increments


GET  key
=>  nil
INCR  key
=>  1
INCR  key
=>  2
GET  key
=>  2

                        Redis
Usage

Counters (downloads, hits, votes, …)
$  curl  http://example.com/downloads/file1.mpg
INCR  downloads:total
INCR  downloads:total:today
INCR  downloads:total:2011-­‐05-­‐10
INCR  downloads:/downloads/file1.mpg:total
INCR  downloads:/downloads/file1.mpg:today
INCR  downloads:/downloads/file1.mpg:2011-­‐05-­‐10




                                                Redis
Usage

Counters (downloads, hits, votes, …)
#  Total  downloads  for  server,  all  time
GET  downloads:total
#  Total  downloads  for  server,  today
GET  downloads:total:today
#  Total  downloads  for  file
GET  downloads:/downloads/file1.mpg:total

#  Total  downloads  for  file  today
INCR  downloads:/downloads/file1.mpg:today
...

                                               Redis
Usage

Counters (downloads, hits, votes, …)
#  Expire  at  2011-­‐05-­‐10  23:59:59
EXPIREAT  downloads:total:today  1305064799




All this runs at super-sonic speed, with minimal overhead and resource consumption.
See implementation for Rubygems.org: https://gist.github.com/296921
However, you'll hit denormalization bloat once you start adding metrics (eg. downloads per
country, per category, …)




                                                                                       Redis
Usage

Variations: Rate limiting
$  curl  http://api.example.com/list.json
INCR  api:<TOKEN>:hits
=>  1

 if INCR('api:abc123:hits') > LIMIT
   return 420 Enhance Your Calm
 end




#  Every  hour...
DEL  api:<TOKEN>:hits


                                            Redis
Usage

Generating unique IDs

INCR  global:users_ids
=>  1
SET  users:1:username  "john"

INCR  global:users_ids
=>  2
SET  users:2:username  "mary"




                                Redis
REDIS DATA STRUCTURES

Lists

LPUSH  key  1           RPOP  key
=>  1                   =>  "1"
LPUSH  key  2           LRANGE  key  0  -­‐1
=>  2                   =>  "3"
LPUSH  key  3           =>  "2"
=>  3                   LLEN  key
LRANGE  key  0  -­‐1    =>  2
=>  "3"                 LTRIM  key  0  1
=>  "2"                 =>  OK
=>  "1"
                                               Redis
Usage

Indexes (list of comments, ...)
LPUSH  article:comments  <ID>

Timelines (of all sorts: messages, logs, …)
LPUSH  user:<ID>:inbox  "message  from  Alice"
LPUSH  user:<ID>:inbox  "message  from  Bob"
#  Limit  the  messages  to  100
LTRIM  user:<ID>:inbox  0  99
#  Get  last  10  messages
LRANGE  user:<ID>:inbox  0  9
#  Get  next  10  messages
LRANGE  user:<ID>:inbox  10  19
                                                 Redis
Usage

Queues
#  Publisher
RPUSH  queue  "task-­‐1"
RPUSH  queue  "task-­‐2"

#  Worker  (blocks  and  waits  for  tasks)
BLPOP  queue  0




                                              Redis
Usage

Queues
# publisher.sh
for i in {1..10}; do
  redis-cli RPUSH "queue" "task-$i"
done

# worker.sh
while true; do
  redis-cli BLPOP "queue" 0
done




                       Demo
                                      Redis
REDIS DATA STRUCTURES

Resque: Background Processing from Github



                                                                                                RPUSH




                                          }
                             LPOP




   O(1)                                      … tens or millions of items …




                        https://github.com/defunkt/resque/blob/v1.13.0/lib/resque.rb#L133-138




                                                                                                        Redis
MESSAGE QUEUES

RestMQ




                 Redis
TALK ON ASYNCHRONOUS PROCESSING

The Code of the Forking Paths




Slides
http://www.slideshare.net/karmi/the-code-of-the-forking-paths-asynchronous-processing-with-resque-and-amqp

Video (in Czech)
http://multimedia.vse.cz/media/Viewer/?peid=51c06c512f4645289c4e9c749dc85acc1d


                                                                                                     Redis
REDIS DATA STRUCTURES

Sets

SADD  key  1            SISMEMBER  key  1
=>  1                   =>  "1"
SADD  key  2            SISMEMBER  key  5
=>  2                   =>  "0"
SADD  key  3            SRANDMEMBER  key
=>  3                   =>  "<RAND>"
SMEMBERS  key           SREM  key  3
=>  "3"                 =>  1
=>  "1"
=>  "2"
                                            Redis
REDIS DATA STRUCTURES

Set Operations

SADD  A  1              SADD  B  1
SADD  A  2              SADD  B  3

SMEMBERS  A             SMEMBERS  B
=>  "1"                 =>  "1"
=>  "2"                 =>  "3"




                                      Redis
REDIS DATA STRUCTURES

Set Operations

[1,2]               [1,3]


                               Union                              SUNION  A  B
                                                                  =>  1
                                                                  =>  2
[1,2]               [1,3]                                         =>  3
                               Intersection                       SINTER  A  B
                                                                  =>  1


[1,2]               [1,3]


                               Difference                         SDIFF  A  B
                                                                  =>  2

http://en.wikipedia.org/wiki/Set_(mathematics)#Basic_operations                  Redis
Usage

Ad serving
SADD  ads:cars      "Check  out  Toyota!"
SADD  ads:cars      "Check  out  Ford!"
...
SADD  ads:movies  "Check  out  Winter's  Bone!"
...

SRANDMEMBER  ads:cars
SRANDMEMBER  ads:movies

Note: Time complexity is O(1). “Check out ODER BY RAND()!”

                                                             Redis
Usage

Relations (Friends/followers)
SADD  users:A:follows  B

SADD  users:B:follows  C
SADD  users:B:follows  D

SADD  users:C:follows  A
SADD  users:C:follows  D




                                Redis
Usage

Relations (Friends/followers)
# Joint network of A and B
SUNION  users:A:follows  users:B:follows
1)  "C"
2)  "D"
3)  "B"




                                           Redis
Usage

Relations (Friends/followers)
#  Common  for  A  and  B
SINTER  users:A:follows  users:B:follows
[]
#  Common  for  B  and  C
SINTER  users:B:follows  users:C:follows
1)  "D"
#  Unique  to  B  compared  to  C
SDIFF  users:B:follows  users:C:follows
1)  "C"


                                           Redis
Usage

Relations (Friends/followers)

#  Whom  I  follow...
SADD  friends  A                SMEMBERS  friends
SADD  friends  B                1)  "A"
                                2)  "B"
#  Who  follows  me...
SADD  followers  B              SMEMBERS  followers
SADD  followers  C              1)  "C"
                                2)  "B"


                                                      Redis
Usage

Relations (Friends/followers)

#  Mutual  relationships
SINTER  friends  followers
1)  "B"
#  Who  does  not  follow  me  back?
SDIFF  friends  followers
1)  "A"
#  Who  am  I  not  following  back?
SDIFF  followers  friends
1)  "C"

                                       Redis
Mining the Social Web
                                           Analyzing Data from Facebook, Twitter, LinkedIn, and Other Social Media Sites
                                           Matthew A. Russell
                                           http://books.google.com/books?id=SYM1lrQdrdsC&lpg=PP1&pg=PA94




http://oreilly.com/catalog/0636920010203                                                                                   Redis
Usage

Relations (Article tags/categories)

SADD  tags:ruby  article-­‐1

SADD  tags:java  article-­‐2

SADD  tags:web    article-­‐1
SADD  tags:web    article-­‐2




                                      Redis
Usage

Relations (Article tags/categories)
#  ruby  OR  java
SUNION  tags:ruby  tags:java
1)  "article-­‐2"
2)  "article-­‐1"
#  ruby  AND  java
SINTER  tags:ruby  tags:java
[]
#  web  AND  NOT  ruby
SDIFF  tags:web  tags:ruby
1)  "article-­‐2"
                                      Redis
Usage

Friends Online
#  My  friends
SADD  friends  A
SADD  friends  B
SADD  friends  C
#  Friend  A  connects
SADD  online:fresh  A
SUNIONSTORE  online  online:fresh  online:stale
#  Who's  online  now?
SINTER  friends  online
[A]
                                                  Redis
Usage

Friends Online
#  Every  minute,  rename  the  "fresh"  to  "stale"  ...
RENAME  online:fresh  online:stale
#  ...  and  update  the  "online"  set
SUNIONSTORE  online  online:fresh  online:stale
#  Friend  B  connects
SADD  online:fresh  B
SUNIONSTORE  online  online:fresh  online:stale
#  Who's  online  now?
SINTER  friends  online
[A,B]
                                                            Redis
Usage

Friends Online
#  Time  passes  ...
#  Rename  the  "fresh"  to  "stale",  every  minute  ...
RENAME  online:fresh  online:stale
#  ...  and  update  the  "online"  set
SUNIONSTORE  online  online:fresh  online:stale
#  Who's  online  now?
SINTER  friends  online
[B]



                                                            Redis
REDIS DATA STRUCTURES

Sorted Sets

ZADD  key  100  A       ZREVRANGE  key  0  -­‐1
ZADD  key  10    C      1)  "A"
ZADD  key  80    B      2)  "B"
                        3)  "C"

ZRANGE  key  0  -­‐1    ZINCRBY  key  10  C
1)  "C"                 "20"
2)  "B"
3)  "A"




                                              Redis
REDIS DATA STRUCTURES

Sorted Sets

ZREVRANGE  key  0  -­‐1   ZREVRANGEBYSCORE  
WITHSCORES                key  100  50
1)  "A"                 1)  "A"
2)  "100"               2)  "B"
3)  "B"
4)  "80"
5)  "C"
6)  "20"




                                           Redis
Usage

Leaderboards
#  User  A  got  10  points
ZINCRBY  scores  10  A
#  User  B  got  15  points
ZINCRBY  scores  15  B
#  User  A  got  another  10  points
ZINCRBY  scores  10  A
#  Display  scores
ZREVRANGE  scores  0  -­‐1  WITHSCORES
1)  "A"
2)  "20"
3)  "B"
4)  "15"
                                         Redis
Usage

Inverted Index
#  Index  document  A
ZINCRBY  index:foo  1  document-­‐A
ZINCRBY  index:foo  1  document-­‐A
ZINCRBY  index:bar  1  document-­‐A
#  Index  document  B
ZINCRBY  index:foo  1  document-­‐B
ZINCRBY  index:baz  1  document-­‐B
#  Search  for  token  foo,  sort  by  occurences
ZREVRANGE  index:foo  0  -­‐1  WITHSCORES
1)  "document-­‐A"
2)  "2"
3)  "document-­‐B"
4)  "1"

                                                    Redis
REDIS' SORTED SETS AS AN INDEX

Inverted Index




https://gist.github.com/928605   Redis
Usage

Counters (downloads, hits, votes, …)
#  Total  downloads  for  server,  all  time
GET  downloads:total
#  Total  downloads  for  server,  today
        Mm'kay. But I need the
GET  downloads:total:today
     “leaderboard” of downloads!
#  Total  downloads  for  file
GET  downloads:/downloads/file1.mpg:total

#  Total  downloads  for  file  today
INCR  downloads:/downloads/file1.mpg:today
...

                                               Redis
Usage

Counters (downloads, hits, votes, …)
INCR  downloads:total
INCR  downloads:total:today
...
                                            Easy!
#  >  Individual  files
#      GET  /downloads/file1.mpg
ZINCRBY  downloads:files:total  1  /downloads/file1.mpg
ZINCRBY  downloads:files:2011-­‐05-­‐10  1  /downloads/file1.mpg
#      GET  /downloads/file2.mpg
ZINCRBY  downloads:files:total  1  /downloads/file2.mpg
ZINCRBY  downloads:files:2011-­‐05-­‐10  1  /downloads/file2.mpg
...

                                                             Redis
Usage

Counters (downloads, hits, votes, …)
#  10  most  downloaded,  all  the  time
ZREVRANGE  downloads:files:total  0  10  WITHSCORES

#  10  most  downloaded  on  2011-­‐05-­‐10
ZREVRANGE  downloads:files:2011-­‐05-­‐10  0  10  WITHSCORES

#  10  most  downloaded  between  2011-­‐05-­‐10  and  <OTHER  DATE>
ZUNIONSTORE  downloads:timeline  2  
                        downloads:files:2011-­‐05-­‐10  
                        <OTHER  DATE>

ZREVRANGE  downloads:timeline  0  10  WITHSCORES




https://gist.github.com/974306                                    Redis
REDIS DATA STRUCTURES

Hashes

HMSET  users:1  username  j  name  John
HMSET  users:2  username  m  name  Mary
HGETALL  users:1
1)  "username"
2)  "j"
...
HKEYS  users:1
1)  "username"
2)  "name"
HSET  users:1  score  100
HGET  users:1  score
1)  "100"
                                          Redis
Usage

Structured data (Articles, users, …)
HMSET  articles:1  title          "Redis  is  cool!"  
                                  content      "I  recently  ..."  
                                  published  "2011-­‐05-­‐10"
HGETALL  articles:1
1)  "title"
2)  "Redis  is  cool!"
3)  "content"
4)  "I  recently  ..."
5)  "published"
6)  "2011-­‐05-­‐10"


HSET  articles:1  title  "Redis  is  very  cool!"
HGET  articles:1  title
"Redis  is  very  cool!"



                                                                       Redis
Usage

User Preferences (No login)
#  Save  preferences  from  <FORM>
HMSET  prefs:<COOKIE  HASH>  background  #ccc  color  #333

#  Keep  it  for  one  year
EXPIRE  prefs:<COOKIE  HASH>  31556926

#  Retrieve  preferences
HGETALL  prefs:<COOKIE  HASH>




                                                       Redis
REDIS FEATURES

Publish/Subscribe

SUBSCRIBE    log.error
PUBLISH        log.error  "ERROR"

PSUBSCRIBE  log.*                   “AMQP”

PUBLISH        log.error  "ERROR"
=>  "ERROR"
=>  "ERROR"

PUBLISH        log.info  "INFO"
=>  "INFO"

                    Demo
                                        Redis
REDIS FEATURES

Durability


BGSAVE
/usr/local/var/db/redis/dump.rdb

BGREWRITEAOF
#  redis.conf
appendonly  yes



http://redis.io/topics/persistence   Redis
REDIS FEATURES

Virtual Memory


#  redis.conf
vm-­‐enabled  yes
Allows to work with data sets bigger then available RAM.
Swaps less often used values to disk.
Keys must still fit into RAM.

#  Make  Redis  disk-­‐bound  database
vm-­‐max-­‐memory  0




The future? Redis Diskstore (disk-bound by default, cache in between server and disk)

http://redis.io/topics/virtual-memory                                              Redis
REDIS FEATURES

Replication


#  slave.conf
slaveof  127.0.0.1  6379
$  redis-­‐server  slave.conf




The future? Redis Cluster (Dynamo-like, Distributed hash table)

http://redis.io/topics/replication                                Redis
REDIS FEATURES

Scripting (experimental)
$  git  clone  https://github.com/antirez/redis  -­‐b  scripting  redis-­‐scripting
$  cd  redis-­‐scripting
$  make
$  echo  port  7397  |  ./src/redis-­‐server  -­‐
$  ./src/redis-­‐cli



SET  myscript  "return  'HELLO'"


EVAL  "return  loadstring(redis('get',  KEYS[1]))()"    
          1  myscript
=>  "HELLO"




http://antirez.com/post/scripting-branch-released.html                                Redis
REDIS RESOURCES

The Interactive Documentation




                           PROMPT




http://redis.io/commands            Redis
REDIS RESOURCES

Simon Willison Tutorial




http://simonwillison.net/static/2010/redis-tutorial/   Redis
REDIS RESOURCES

Redis Implementation Details




http://pauladamsmith.com/articles/redis-under-the-hood.html   Redis
REDIS RESOURCES

Redis Use Cases




http://highscalability.com/blog/2010/12/6/what-the-heck-are-you-actually-using-nosql-for.html   Redis
REDIS RESOURCES

The Original Metaphore…




http://flazz.me/redis-the-ak-47-of-databases   Redis
Thanks!
  d

More Related Content

PDF
Paris Redis Meetup Introduction
PDF
Redis SoCraTes 2014
PDF
Kicking ass with redis
PPTX
Redis Use Patterns (DevconTLV June 2014)
PDF
Everything you always wanted to know about Redis but were afraid to ask
PDF
Redis basics
KEY
Redis in Practice
PDF
Introduction to redis - version 2
Paris Redis Meetup Introduction
Redis SoCraTes 2014
Kicking ass with redis
Redis Use Patterns (DevconTLV June 2014)
Everything you always wanted to know about Redis but were afraid to ask
Redis basics
Redis in Practice
Introduction to redis - version 2

What's hot (20)

PPT
Redis And python at pycon_2011
PDF
Redis - Usability and Use Cases
PDF
Introduction to Redis
PDF
Redis and its many use cases
ODP
An Introduction to REDIS NoSQL database
PPTX
Caching solutions with Redis
PPTX
PPTX
Redis and it's data types
KEY
Scaling php applications with redis
PPTX
Redis Indices (#RedisTLV)
PDF
Extend Redis with Modules
ODP
Introduction to Redis
PPTX
Introduction to Redis
PDF
Redis 101
PPTX
Redis/Lessons learned
PDF
Advanced Redis data structures
PDF
Redis for the Everyday Developer
PDF
Node.js in production
PPTX
Redis Functions, Data Structures for Web Scale Apps
PDF
Nodejs - A quick tour (v5)
Redis And python at pycon_2011
Redis - Usability and Use Cases
Introduction to Redis
Redis and its many use cases
An Introduction to REDIS NoSQL database
Caching solutions with Redis
Redis and it's data types
Scaling php applications with redis
Redis Indices (#RedisTLV)
Extend Redis with Modules
Introduction to Redis
Introduction to Redis
Redis 101
Redis/Lessons learned
Advanced Redis data structures
Redis for the Everyday Developer
Node.js in production
Redis Functions, Data Structures for Web Scale Apps
Nodejs - A quick tour (v5)
Ad

Viewers also liked (20)

PPTX
High-Volume Data Collection and Real Time Analytics Using Redis
PDF
RestMQ - HTTP/Redis based Message Queue
PDF
Interaktivita, originalita a návrhové vzory
PDF
Vizualizace dat a D3.js [EUROPEN 2014]
PDF
Redis installation
PPTX
Using puppet, foreman and git to develop and operate a large scale internet s...
PDF
Mapping, Interlinking and Exposing MusicBrainz as Linked Data
PDF
Continuously-Integrated Puppet in a Dynamic Environment
PDF
JSON and the APInauts
PDF
Better encryption & security with MariaDB 10.1 & MySQL 5.7
PDF
PPT
Ruby application based on http
PDF
vSphere APIs for performance monitoring
PDF
Dlsecyx pgroammr (Dyslexic Programmer - cool stuff for scaling)
PPT
Symfonytn
PDF
PostgreSQL Materialized Views with Active Record
PDF
The Complete MariaDB Server Tutorial - Percona Live 2015
PDF
Taking Control of Chaos with Docker and Puppet
PDF
Detecting headless browsers
ODP
Monitoring in an Infrastructure as Code Age
High-Volume Data Collection and Real Time Analytics Using Redis
RestMQ - HTTP/Redis based Message Queue
Interaktivita, originalita a návrhové vzory
Vizualizace dat a D3.js [EUROPEN 2014]
Redis installation
Using puppet, foreman and git to develop and operate a large scale internet s...
Mapping, Interlinking and Exposing MusicBrainz as Linked Data
Continuously-Integrated Puppet in a Dynamic Environment
JSON and the APInauts
Better encryption & security with MariaDB 10.1 & MySQL 5.7
Ruby application based on http
vSphere APIs for performance monitoring
Dlsecyx pgroammr (Dyslexic Programmer - cool stuff for scaling)
Symfonytn
PostgreSQL Materialized Views with Active Record
The Complete MariaDB Server Tutorial - Percona Live 2015
Taking Control of Chaos with Docker and Puppet
Detecting headless browsers
Monitoring in an Infrastructure as Code Age
Ad

Similar to Redis — The AK-47 of Post-relational Databases (20)

PDF
Redispresentation apac2012
PDF
Mini-Training: Redis
PDF
Redis — memcached on steroids
PDF
Redis+Spark Structured Streaming: Roshan Kumar
PDF
r2con 2017 r2cLEMENCy
PPTX
Tailoring Redis Modules For Your Users’ Needs
PDF
Introduction to Redis
KEY
WebClusters, Redis
PDF
RedisGears
PPTX
REDIS327
PDF
RedisGears: Meir Shpilraien
PDF
Intro to Map Reduce
PDF
Discover System Facilities inside Your Android Phone
PDF
Getting Ready to Use Redis with Apache Spark with Dvir Volk
PDF
Reusing your existing software on Android
PDF
Building a Large Scale Recommendation Engine with Spark and Redis-ML with Sha...
PDF
REDIS intro and how to use redis
PDF
MongoDB.local Austin 2018: MongoDB Ops Manager + Kubernetes
PDF
2012 09 MariaDB Boston Meetup - MariaDB 是 Mysql 的替代者吗
PPTX
Developing a Redis Module - Hackathon Kickoff
Redispresentation apac2012
Mini-Training: Redis
Redis — memcached on steroids
Redis+Spark Structured Streaming: Roshan Kumar
r2con 2017 r2cLEMENCy
Tailoring Redis Modules For Your Users’ Needs
Introduction to Redis
WebClusters, Redis
RedisGears
REDIS327
RedisGears: Meir Shpilraien
Intro to Map Reduce
Discover System Facilities inside Your Android Phone
Getting Ready to Use Redis with Apache Spark with Dvir Volk
Reusing your existing software on Android
Building a Large Scale Recommendation Engine with Spark and Redis-ML with Sha...
REDIS intro and how to use redis
MongoDB.local Austin 2018: MongoDB Ops Manager + Kubernetes
2012 09 MariaDB Boston Meetup - MariaDB 是 Mysql 的替代者吗
Developing a Redis Module - Hackathon Kickoff

More from Karel Minarik (20)

PDF
Elasticsearch (Rubyshift 2013)
PDF
Elasticsearch in 15 Minutes
PDF
Realtime Analytics With Elasticsearch [New Media Inspiration 2013]
PDF
Elasticsearch And Ruby [RuPy2012]
PDF
Shell's Kitchen: Infrastructure As Code (Webexpo 2012)
PDF
Elastic Search: Beyond Ordinary Fulltext Search (Webexpo 2011 Prague)
PDF
Your Data, Your Search, ElasticSearch (EURUKO 2011)
PDF
CouchDB – A Database for the Web
PDF
Spoiling The Youth With Ruby (Euruko 2010)
PDF
Verzovani kodu s Gitem (Karel Minarik)
PDF
Představení Ruby on Rails [Junior Internet]
PDF
Efektivni vyvoj webovych aplikaci v Ruby on Rails (Webexpo)
PDF
Úvod do Ruby on Rails
PDF
Úvod do programování 7
PDF
Úvod do programování 6
PDF
Úvod do programování 5
PDF
Úvod do programování 4
PDF
Úvod do programování 3 (to be continued)
PDF
Historie programovacích jazyků
PDF
Úvod do programování aneb Do nitra stroje
Elasticsearch (Rubyshift 2013)
Elasticsearch in 15 Minutes
Realtime Analytics With Elasticsearch [New Media Inspiration 2013]
Elasticsearch And Ruby [RuPy2012]
Shell's Kitchen: Infrastructure As Code (Webexpo 2012)
Elastic Search: Beyond Ordinary Fulltext Search (Webexpo 2011 Prague)
Your Data, Your Search, ElasticSearch (EURUKO 2011)
CouchDB – A Database for the Web
Spoiling The Youth With Ruby (Euruko 2010)
Verzovani kodu s Gitem (Karel Minarik)
Představení Ruby on Rails [Junior Internet]
Efektivni vyvoj webovych aplikaci v Ruby on Rails (Webexpo)
Úvod do Ruby on Rails
Úvod do programování 7
Úvod do programování 6
Úvod do programování 5
Úvod do programování 4
Úvod do programování 3 (to be continued)
Historie programovacích jazyků
Úvod do programování aneb Do nitra stroje

Recently uploaded (20)

PDF
Bridging biosciences and deep learning for revolutionary discoveries: a compr...
PDF
Unlocking AI with Model Context Protocol (MCP)
PPTX
PA Analog/Digital System: The Backbone of Modern Surveillance and Communication
PDF
Agricultural_Statistics_at_a_Glance_2022_0.pdf
PDF
Spectral efficient network and resource selection model in 5G networks
PPTX
Detection-First SIEM: Rule Types, Dashboards, and Threat-Informed Strategy
PPTX
Cloud computing and distributed systems.
PPTX
Understanding_Digital_Forensics_Presentation.pptx
PDF
Advanced methodologies resolving dimensionality complications for autism neur...
PDF
Dropbox Q2 2025 Financial Results & Investor Presentation
PDF
Building Integrated photovoltaic BIPV_UPV.pdf
PDF
NewMind AI Monthly Chronicles - July 2025
DOCX
The AUB Centre for AI in Media Proposal.docx
PPTX
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
PDF
KodekX | Application Modernization Development
PDF
Encapsulation_ Review paper, used for researhc scholars
PDF
Shreyas Phanse Resume: Experienced Backend Engineer | Java • Spring Boot • Ka...
PDF
Mobile App Security Testing_ A Comprehensive Guide.pdf
PDF
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
PDF
Review of recent advances in non-invasive hemoglobin estimation
Bridging biosciences and deep learning for revolutionary discoveries: a compr...
Unlocking AI with Model Context Protocol (MCP)
PA Analog/Digital System: The Backbone of Modern Surveillance and Communication
Agricultural_Statistics_at_a_Glance_2022_0.pdf
Spectral efficient network and resource selection model in 5G networks
Detection-First SIEM: Rule Types, Dashboards, and Threat-Informed Strategy
Cloud computing and distributed systems.
Understanding_Digital_Forensics_Presentation.pptx
Advanced methodologies resolving dimensionality complications for autism neur...
Dropbox Q2 2025 Financial Results & Investor Presentation
Building Integrated photovoltaic BIPV_UPV.pdf
NewMind AI Monthly Chronicles - July 2025
The AUB Centre for AI in Media Proposal.docx
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
KodekX | Application Modernization Development
Encapsulation_ Review paper, used for researhc scholars
Shreyas Phanse Resume: Experienced Backend Engineer | Java • Spring Boot • Ka...
Mobile App Security Testing_ A Comprehensive Guide.pdf
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
Review of recent advances in non-invasive hemoglobin estimation

Redis — The AK-47 of Post-relational Databases

  • 1. Redis — The AK-47 Of Post-relational Databases Karel Minařík
  • 2. Karel Minařík → Independent web designer and developer → Ruby, Rails, Git, CouchDB propagandista in .cz → Previously: Flash Developer; Art Director; Information Architect;… (see LinkedIn) → @karmiq at Twitter → karmi.cz Redis
  • 4. WEAPON DESIGN AK-47 Designed at the end of WWII by Mikhail Kalashnikov Assault Rifle, not a “submachine gun” Simple design Designed for mass production & low-quality manufacturing Extreme reliability, at the cost of accuracy Redis
  • 6. Reliability Redis
  • 8. SIMPLICITY The Redis Manifesto We're against complexity. We believe designing systems is a fight against complexity. Most of the time the best way to fight complexity is by not creating it at all. http://antirez.com/post/redis-manifesto.html Redis
  • 9. SIMPLICITY Redis and NoSQL What's wrong with RDBMS when used for (many) tasks that don't need all this complexity? The data model: non scalable, time complexity hard to predict, and can't model many common problems well enough. http://antirez.com/post/MongoDB-and-Redis.html Redis
  • 10. Assault rifles as the crucial invention for new rules of warfare: fighting in shorter distances and shooting while on the run. http://www.youtube.com/watch?v=a1KBsqvKpXk#t=7m50s Redis
  • 11. Memory is the new disk. Disk is the new tape. — Jim Gray http://www.infoq.com/news/2008/06/ram-is-disk http://research.microsoft.com/en-us/um/people/gray/talks/Flash_is_Good.ppt Redis
  • 13. Simplicity Speed Predictability Versatility Reliability Low Footprint
  • 14. SIMPLICITY Installation git  clone  http://github.com/antirez/redis cd  redis make ./src/redis-­‐server ./src/redis-­‐cli Redis
  • 15. SIMPLICITY The Zen of Redis (...) what Redis provides are data structures (...) http://antirez.com/post/MongoDB-and-Redis.html#c1537 Redis
  • 17. REDIS DATA STRUCTURES Strings SET  key  "value"  ~  4  billion  of  keys GET  key =>  "value" DEL  key Redis
  • 18. REDIS DATA STRUCTURES Fetch multiple keys at once SET  key1  "value1" SET  key2  "value2" MGET  key1  key2 =>  "value1" =>  "value2" Redis
  • 19. REDIS DATA STRUCTURES Expiration EXPIRE  key  5 GET  key =>  "value" TTL  key =>  1 GET  key =>  (nil) Redis
  • 21. REDIS DATA STRUCTURES Atomic Increments GET  key =>  nil INCR  key =>  1 INCR  key =>  2 GET  key =>  2 Redis
  • 22. Usage Counters (downloads, hits, votes, …) $  curl  http://example.com/downloads/file1.mpg INCR  downloads:total INCR  downloads:total:today INCR  downloads:total:2011-­‐05-­‐10 INCR  downloads:/downloads/file1.mpg:total INCR  downloads:/downloads/file1.mpg:today INCR  downloads:/downloads/file1.mpg:2011-­‐05-­‐10 Redis
  • 23. Usage Counters (downloads, hits, votes, …) #  Total  downloads  for  server,  all  time GET  downloads:total #  Total  downloads  for  server,  today GET  downloads:total:today #  Total  downloads  for  file GET  downloads:/downloads/file1.mpg:total #  Total  downloads  for  file  today INCR  downloads:/downloads/file1.mpg:today ... Redis
  • 24. Usage Counters (downloads, hits, votes, …) #  Expire  at  2011-­‐05-­‐10  23:59:59 EXPIREAT  downloads:total:today  1305064799 All this runs at super-sonic speed, with minimal overhead and resource consumption. See implementation for Rubygems.org: https://gist.github.com/296921 However, you'll hit denormalization bloat once you start adding metrics (eg. downloads per country, per category, …) Redis
  • 25. Usage Variations: Rate limiting $  curl  http://api.example.com/list.json INCR  api:<TOKEN>:hits =>  1 if INCR('api:abc123:hits') > LIMIT return 420 Enhance Your Calm end #  Every  hour... DEL  api:<TOKEN>:hits Redis
  • 26. Usage Generating unique IDs INCR  global:users_ids =>  1 SET  users:1:username  "john" INCR  global:users_ids =>  2 SET  users:2:username  "mary" Redis
  • 27. REDIS DATA STRUCTURES Lists LPUSH  key  1 RPOP  key =>  1 =>  "1" LPUSH  key  2 LRANGE  key  0  -­‐1 =>  2 =>  "3" LPUSH  key  3 =>  "2" =>  3 LLEN  key LRANGE  key  0  -­‐1 =>  2 =>  "3" LTRIM  key  0  1 =>  "2" =>  OK =>  "1" Redis
  • 28. Usage Indexes (list of comments, ...) LPUSH  article:comments  <ID> Timelines (of all sorts: messages, logs, …) LPUSH  user:<ID>:inbox  "message  from  Alice" LPUSH  user:<ID>:inbox  "message  from  Bob" #  Limit  the  messages  to  100 LTRIM  user:<ID>:inbox  0  99 #  Get  last  10  messages LRANGE  user:<ID>:inbox  0  9 #  Get  next  10  messages LRANGE  user:<ID>:inbox  10  19 Redis
  • 29. Usage Queues #  Publisher RPUSH  queue  "task-­‐1" RPUSH  queue  "task-­‐2" #  Worker  (blocks  and  waits  for  tasks) BLPOP  queue  0 Redis
  • 30. Usage Queues # publisher.sh for i in {1..10}; do redis-cli RPUSH "queue" "task-$i" done # worker.sh while true; do redis-cli BLPOP "queue" 0 done Demo Redis
  • 31. REDIS DATA STRUCTURES Resque: Background Processing from Github RPUSH } LPOP O(1) … tens or millions of items … https://github.com/defunkt/resque/blob/v1.13.0/lib/resque.rb#L133-138 Redis
  • 33. TALK ON ASYNCHRONOUS PROCESSING The Code of the Forking Paths Slides http://www.slideshare.net/karmi/the-code-of-the-forking-paths-asynchronous-processing-with-resque-and-amqp Video (in Czech) http://multimedia.vse.cz/media/Viewer/?peid=51c06c512f4645289c4e9c749dc85acc1d Redis
  • 34. REDIS DATA STRUCTURES Sets SADD  key  1 SISMEMBER  key  1 =>  1 =>  "1" SADD  key  2 SISMEMBER  key  5 =>  2 =>  "0" SADD  key  3 SRANDMEMBER  key =>  3 =>  "<RAND>" SMEMBERS  key SREM  key  3 =>  "3" =>  1 =>  "1" =>  "2" Redis
  • 35. REDIS DATA STRUCTURES Set Operations SADD  A  1 SADD  B  1 SADD  A  2 SADD  B  3 SMEMBERS  A SMEMBERS  B =>  "1" =>  "1" =>  "2" =>  "3" Redis
  • 36. REDIS DATA STRUCTURES Set Operations [1,2] [1,3] Union SUNION  A  B =>  1 =>  2 [1,2] [1,3] =>  3 Intersection SINTER  A  B =>  1 [1,2] [1,3] Difference SDIFF  A  B =>  2 http://en.wikipedia.org/wiki/Set_(mathematics)#Basic_operations Redis
  • 37. Usage Ad serving SADD  ads:cars      "Check  out  Toyota!" SADD  ads:cars      "Check  out  Ford!" ... SADD  ads:movies  "Check  out  Winter's  Bone!" ... SRANDMEMBER  ads:cars SRANDMEMBER  ads:movies Note: Time complexity is O(1). “Check out ODER BY RAND()!” Redis
  • 38. Usage Relations (Friends/followers) SADD  users:A:follows  B SADD  users:B:follows  C SADD  users:B:follows  D SADD  users:C:follows  A SADD  users:C:follows  D Redis
  • 39. Usage Relations (Friends/followers) # Joint network of A and B SUNION  users:A:follows  users:B:follows 1)  "C" 2)  "D" 3)  "B" Redis
  • 40. Usage Relations (Friends/followers) #  Common  for  A  and  B SINTER  users:A:follows  users:B:follows [] #  Common  for  B  and  C SINTER  users:B:follows  users:C:follows 1)  "D" #  Unique  to  B  compared  to  C SDIFF  users:B:follows  users:C:follows 1)  "C" Redis
  • 41. Usage Relations (Friends/followers) #  Whom  I  follow... SADD  friends  A SMEMBERS  friends SADD  friends  B 1)  "A" 2)  "B" #  Who  follows  me... SADD  followers  B SMEMBERS  followers SADD  followers  C 1)  "C" 2)  "B" Redis
  • 42. Usage Relations (Friends/followers) #  Mutual  relationships SINTER  friends  followers 1)  "B" #  Who  does  not  follow  me  back? SDIFF  friends  followers 1)  "A" #  Who  am  I  not  following  back? SDIFF  followers  friends 1)  "C" Redis
  • 43. Mining the Social Web Analyzing Data from Facebook, Twitter, LinkedIn, and Other Social Media Sites Matthew A. Russell http://books.google.com/books?id=SYM1lrQdrdsC&lpg=PP1&pg=PA94 http://oreilly.com/catalog/0636920010203 Redis
  • 44. Usage Relations (Article tags/categories) SADD  tags:ruby  article-­‐1 SADD  tags:java  article-­‐2 SADD  tags:web    article-­‐1 SADD  tags:web    article-­‐2 Redis
  • 45. Usage Relations (Article tags/categories) #  ruby  OR  java SUNION  tags:ruby  tags:java 1)  "article-­‐2" 2)  "article-­‐1" #  ruby  AND  java SINTER  tags:ruby  tags:java [] #  web  AND  NOT  ruby SDIFF  tags:web  tags:ruby 1)  "article-­‐2" Redis
  • 46. Usage Friends Online #  My  friends SADD  friends  A SADD  friends  B SADD  friends  C #  Friend  A  connects SADD  online:fresh  A SUNIONSTORE  online  online:fresh  online:stale #  Who's  online  now? SINTER  friends  online [A] Redis
  • 47. Usage Friends Online #  Every  minute,  rename  the  "fresh"  to  "stale"  ... RENAME  online:fresh  online:stale #  ...  and  update  the  "online"  set SUNIONSTORE  online  online:fresh  online:stale #  Friend  B  connects SADD  online:fresh  B SUNIONSTORE  online  online:fresh  online:stale #  Who's  online  now? SINTER  friends  online [A,B] Redis
  • 48. Usage Friends Online #  Time  passes  ... #  Rename  the  "fresh"  to  "stale",  every  minute  ... RENAME  online:fresh  online:stale #  ...  and  update  the  "online"  set SUNIONSTORE  online  online:fresh  online:stale #  Who's  online  now? SINTER  friends  online [B] Redis
  • 49. REDIS DATA STRUCTURES Sorted Sets ZADD  key  100  A ZREVRANGE  key  0  -­‐1 ZADD  key  10    C 1)  "A" ZADD  key  80    B 2)  "B" 3)  "C" ZRANGE  key  0  -­‐1 ZINCRBY  key  10  C 1)  "C" "20" 2)  "B" 3)  "A" Redis
  • 50. REDIS DATA STRUCTURES Sorted Sets ZREVRANGE  key  0  -­‐1   ZREVRANGEBYSCORE   WITHSCORES key  100  50 1)  "A" 1)  "A" 2)  "100" 2)  "B" 3)  "B" 4)  "80" 5)  "C" 6)  "20" Redis
  • 51. Usage Leaderboards #  User  A  got  10  points ZINCRBY  scores  10  A #  User  B  got  15  points ZINCRBY  scores  15  B #  User  A  got  another  10  points ZINCRBY  scores  10  A #  Display  scores ZREVRANGE  scores  0  -­‐1  WITHSCORES 1)  "A" 2)  "20" 3)  "B" 4)  "15" Redis
  • 52. Usage Inverted Index #  Index  document  A ZINCRBY  index:foo  1  document-­‐A ZINCRBY  index:foo  1  document-­‐A ZINCRBY  index:bar  1  document-­‐A #  Index  document  B ZINCRBY  index:foo  1  document-­‐B ZINCRBY  index:baz  1  document-­‐B #  Search  for  token  foo,  sort  by  occurences ZREVRANGE  index:foo  0  -­‐1  WITHSCORES 1)  "document-­‐A" 2)  "2" 3)  "document-­‐B" 4)  "1" Redis
  • 53. REDIS' SORTED SETS AS AN INDEX Inverted Index https://gist.github.com/928605 Redis
  • 54. Usage Counters (downloads, hits, votes, …) #  Total  downloads  for  server,  all  time GET  downloads:total #  Total  downloads  for  server,  today Mm'kay. But I need the GET  downloads:total:today “leaderboard” of downloads! #  Total  downloads  for  file GET  downloads:/downloads/file1.mpg:total #  Total  downloads  for  file  today INCR  downloads:/downloads/file1.mpg:today ... Redis
  • 55. Usage Counters (downloads, hits, votes, …) INCR  downloads:total INCR  downloads:total:today ... Easy! #  >  Individual  files #      GET  /downloads/file1.mpg ZINCRBY  downloads:files:total  1  /downloads/file1.mpg ZINCRBY  downloads:files:2011-­‐05-­‐10  1  /downloads/file1.mpg #      GET  /downloads/file2.mpg ZINCRBY  downloads:files:total  1  /downloads/file2.mpg ZINCRBY  downloads:files:2011-­‐05-­‐10  1  /downloads/file2.mpg ... Redis
  • 56. Usage Counters (downloads, hits, votes, …) #  10  most  downloaded,  all  the  time ZREVRANGE  downloads:files:total  0  10  WITHSCORES #  10  most  downloaded  on  2011-­‐05-­‐10 ZREVRANGE  downloads:files:2011-­‐05-­‐10  0  10  WITHSCORES #  10  most  downloaded  between  2011-­‐05-­‐10  and  <OTHER  DATE> ZUNIONSTORE  downloads:timeline  2                          downloads:files:2011-­‐05-­‐10                          <OTHER  DATE> ZREVRANGE  downloads:timeline  0  10  WITHSCORES https://gist.github.com/974306 Redis
  • 57. REDIS DATA STRUCTURES Hashes HMSET  users:1  username  j  name  John HMSET  users:2  username  m  name  Mary HGETALL  users:1 1)  "username" 2)  "j" ... HKEYS  users:1 1)  "username" 2)  "name" HSET  users:1  score  100 HGET  users:1  score 1)  "100" Redis
  • 58. Usage Structured data (Articles, users, …) HMSET  articles:1  title          "Redis  is  cool!"                                    content      "I  recently  ..."                                    published  "2011-­‐05-­‐10" HGETALL  articles:1 1)  "title" 2)  "Redis  is  cool!" 3)  "content" 4)  "I  recently  ..." 5)  "published" 6)  "2011-­‐05-­‐10" HSET  articles:1  title  "Redis  is  very  cool!" HGET  articles:1  title "Redis  is  very  cool!" Redis
  • 59. Usage User Preferences (No login) #  Save  preferences  from  <FORM> HMSET  prefs:<COOKIE  HASH>  background  #ccc  color  #333 #  Keep  it  for  one  year EXPIRE  prefs:<COOKIE  HASH>  31556926 #  Retrieve  preferences HGETALL  prefs:<COOKIE  HASH> Redis
  • 60. REDIS FEATURES Publish/Subscribe SUBSCRIBE    log.error PUBLISH        log.error  "ERROR" PSUBSCRIBE  log.* “AMQP” PUBLISH        log.error  "ERROR" =>  "ERROR" =>  "ERROR" PUBLISH        log.info  "INFO" =>  "INFO" Demo Redis
  • 62. REDIS FEATURES Virtual Memory #  redis.conf vm-­‐enabled  yes Allows to work with data sets bigger then available RAM. Swaps less often used values to disk. Keys must still fit into RAM. #  Make  Redis  disk-­‐bound  database vm-­‐max-­‐memory  0 The future? Redis Diskstore (disk-bound by default, cache in between server and disk) http://redis.io/topics/virtual-memory Redis
  • 63. REDIS FEATURES Replication #  slave.conf slaveof  127.0.0.1  6379 $  redis-­‐server  slave.conf The future? Redis Cluster (Dynamo-like, Distributed hash table) http://redis.io/topics/replication Redis
  • 64. REDIS FEATURES Scripting (experimental) $  git  clone  https://github.com/antirez/redis  -­‐b  scripting  redis-­‐scripting $  cd  redis-­‐scripting $  make $  echo  port  7397  |  ./src/redis-­‐server  -­‐ $  ./src/redis-­‐cli SET  myscript  "return  'HELLO'" EVAL  "return  loadstring(redis('get',  KEYS[1]))()"              1  myscript =>  "HELLO" http://antirez.com/post/scripting-branch-released.html Redis
  • 65. REDIS RESOURCES The Interactive Documentation PROMPT http://redis.io/commands Redis
  • 66. REDIS RESOURCES Simon Willison Tutorial http://simonwillison.net/static/2010/redis-tutorial/ Redis
  • 67. REDIS RESOURCES Redis Implementation Details http://pauladamsmith.com/articles/redis-under-the-hood.html Redis
  • 68. REDIS RESOURCES Redis Use Cases http://highscalability.com/blog/2010/12/6/what-the-heck-are-you-actually-using-nosql-for.html Redis
  • 69. REDIS RESOURCES The Original Metaphore… http://flazz.me/redis-the-ak-47-of-databases Redis