Riak Intro

Jon Meredith Basho Technologies

Tuesday, December 7, 2010

WHAT IS IT?


Riak is
a scalable, highly-available, networked
key/value store.

basho

Inspired by Amazon’s Dynamo
...with some enhancements.

Open Source

basho

Riak is
a scalable, highly-available, networked
key/value store.


• Stores and retrieves data against a key.

• Encode your data how you like it.

• Keys are organized into buckets.

basho

BASIC OPERATIONS

get
put
delete

basho

• Riak also stores metadata against keys

• Content Type, Charset, Encoding, ...

• Custom application metadata

basho

bucket: pivotal
key: greeting
value: “<h1>Hi Pivotal<h1>”
metadata:
content-type=text/html
charset=utf8

basho

> PUT /riak/pivotal/greeting HTTP/1.1
> User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3
> Host: 127.0.0.1:8098
> Accept: */*
> content-type: text/html
> charset: utf8
> Content-Length: 19
>
> <h1>Hi Pivotal<h1>

< HTTP/1.1 200 OK
< X-Riak-Vclock: a85hYGBgzWDKBVIsTNPiOTOYEhnzWBm+5uUf5YMIszUnMdcLv4dKsOTDJVhYPx1JwiLMeH/
bU6iwWj6yMUwtt6yQJbIA
< Vary: Accept-Encoding
< Server: MochiWeb/1.1 WebMachine/1.6 (eat around the stinger)
< Link: </riak/pivotal>; rel="up"
< Date: Mon, 24 May 2010 21:44:06 GMT
< Content-Type: text/html
< Content-Length: 19
<

basho

> GET /riak/pivotal/greeting HTTP/1.1
> User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3
> Host: localhost:8098
> Accept: */*
>
< HTTP/1.1 200 OK
< X-Riak-Vclock: a85hYGBgymDKBVIsrDk3LTOYEhnzWBmUmtOP8kGE2ZqTmNf8Xg6VCAJJZAEA
< Vary: Accept-Encoding
< Server: MochiWeb/1.1 WebMachine/1.6 (eat around the stinger)
< Link: </riak/pivotal>; rel="up"
< Last-Modified: Tue, 18 May 2010 21:32:02 GMT
< ETag: 1VW7QmCuyofVCuQbqBNVYh
< Date: Tue, 18 May 2010 21:32:06 GMT
< Content-Type: text/html; charset=utf8
< Content-Length: 19
<
* Connection #0 to host localhost left intact
* Closing connection #0
<h1>Hi Pivotal</h1>

basho

OTHER INTERFACES TOO!

• Protocol Buffers Client

• Messages encoded using Protocol Buffers

• 3-10x speedups depending on data

• Native Erlang client

basho

Riak was designed to scale horizontally.

basho

#Replicas - N R+W > N
#Get response - R
#Put responses - W for quorum


WILL THE REAL OBJECT PLEASE STEP
FORWARD?

• Riak always accepts writes - even during network partitions and other badness

• Objects are versioned to automatically detect ancestors/siblings

• Conﬂicts are detected on reads

• Eventually consistent

basho

That sounds great...
but can I do anything without a key?


Query datasets with
Map / Reduce

basho

basho

function(v) {
var m = v.values[0].data.
match('w*','g');
var r = [];
for(var i in m)
if (m[i] != '') {
var o = {};
o[m[i]] = 1;
r.push(o);
}
return r;
}

basho

map()
function(v) {
var m = v.values[0].data.
match('w*','g');
var r = [];
for(var i in m)
if (m[i] != '') {
var o = {};
o[m[i]] = 1;
r.push(o);
}
return r;
}

basho

map()

map() map()

map() map()

map() map()

map()
basho

R1

R8 R2

R7 R3

R6 R4

R5
basho

R2
reduce()
6
5
8
7
4
3
1
function(v) {
var r = {};
for (var i in v) {
for(var w in v[i]) {
if (w in r)
r[w] += v[i][w];
else
r[w] = v[i][w];
}
}
return [r];
}

basho

Store relationships
with links

basho

ch ild
people/dean
child
chi
ld
people/bob people/alice

people/claire
basho

he r
mot
ch ild
father people/dean
child mo
the
r
chi
ld
people/bob people/alice

people/claire
basho

he r
hild m ot
grandc
ch ild
father so n people/dean
child mo
the
daughter r
chi
dau ld
people/bob people/alice ght
grandc er
hild

people/claire
basho

he r
m ot
ch ild
child mo
the
daughter r
chi
dau ld
er
GET /riak/person/bob/people,child,_
key link link
bucket tag
people/claire
basho

he r
mot
ch ild
child mo
the
daughter r
chi
dau ld
er
GET /riak/person/bob/people,child,_/_,child,_
key link1 link1 link2 link2
bucket tag bucket tag
people/claire
basho

he r
mot
ch ild
child mo
the
daughter r
chi
dau ld
er
GET /riak/person/bob/people,child,1/people,child,_

people/claire
basho

COMPARED TO RDBMS

• No schema • No schema migrations - version your
objects
• No transactions
• Noreplication topologies - just add
• No joins (but we do have map/reduce) nodes to scale
• No secondary indices (but we do have
links!)


CLIENT LIBRARIES

Ruby

Javascript
Java

.NET linq Twisted Python

basho

RIAK IN PRODUCTION

basho

MOCHI MEDIA
www.mochimedia.com

• Multiple clusters in production

• Sessions (web and Flash) - 5 nodes, ~20MM keys - highest trafﬁc
Developer
• User data storage - 3 nodes, bucket-per-game

• URL Shortener - 2 nodes, bucket-per-game

• Social graph - 3 nodes, OAuth credentials, relationships

Text and link here but
basho

MOZILLA TEST PILOT
http://bit.ly/aC0i4f

• Data collector for anonymized Firefox usage statistics

• High peak throughput - pure writes

• Large MapReduce queries for analytics “Riak meets or exceeds all the write
performance requirements...it was nice to
see predictable performance and
stability under load.”

basho

WEGEO

• Mobile Productivity / Collaboration App
- Task lists, Shopping lists, Calendar

• Event and Check-in logs

• Monthly reports archive “Scalability I could understand, on
a platform I could understand.”
• Third-party coupon and deal aggregation

basho

GETTING RIAK
Prebuild Packages The Bleeding Edge

https://github.com/basho

http://bitbucket.org/basho
http://downloads.basho.com


http://wiki.basho.com
follow twitter.com/basho/team
riak-users@lists.basho.com
#riak on Freenode

basho

Riak Intro

More Related Content

Similar to Riak Intro (20)

Recently uploaded (20)

Riak Intro