OSCON 2011 Learning CouchDB

Co-Founder and
Technical Director

from Vermont

Battery Park - Burlington, Vermont by Marty Desilets, on Flickr

Author

http://oreilly.com/catalog/9781449303129/ http://oreilly.com/catalog/9781449303433/

OS X
$ brew install couchdb

OR

$ sudo port install couchdb

OR

Couchbase Server:
http://info.couchbase.com/couchbaseEEdownload.html

Ubuntu
$ sudo aptitude install couchdb

OR

Couchbase Server:

Red Hat Enterprise Linux 5
$ sudo yum install couchdb

OR

Couchbase Server:

Windows
Binary:
http://wiki.apache.org/couchdb/Windows_binary_installer

OR

Couchbase Server:

Futon:
http://localhost:5984/_utils/

cURL
$ curl -iX GET 'http://localhost:5984/'

HTTP/1.1 200 OK
Server: CouchDB/1.0.2 (Erlang OTP/R14B)
{
"couchdb": "Welcome",
"version": "1.0.2"
}

Document-Oriented
Schema-less

Stores key/value pairs

Key is a string representing a document’s ID

Value is a document represented as a JSON Object:
{
"_id": "fe8a4a1ca998daafaaa04d7d9000187f",
"title": "Learning CouchDB"
}

Bene ts of JSON
Human-readable and simple data interchange format

Data structures from many programming languages can be easily converted
to and from JSON

Lightweight—doesn’t consume much bandwidth

Easily integrated into JavaScript applications (e.g. CouchApps)

JSON Objects
A JSON Object is a collection of name/value pairs.

The name is a String.

The value can be a:
• Boolean (false or true)
• JSON Array (e.g. ["a", "b", "c"])
• JSON Object
• Number
• String
• JSON NULL (null)

Example JSON Object
{
"_id":"978-0-596-15589-6",
"title":"CouchDB: The De nitive Guide",
"subtitle":"Time to Relax",
"authors":[
"J. Chris Anderson",
"Jan Lehnardt",
"Noah Slater"
],
"publisher":"O'Reilly Media",
"released":"2010-01-19",
"pages":272
}

Indexed Queries
MapReduce “views” generate B+ tree indexes

Indexes are incrementally updated by Map functions

Views are queried for result sets:
{ // …
"rows": [
{
"id": "fe8a4a1ca998daafaaa04d7d9000187f",
"key":"Learning CouchDB",
"value": null
}
]
}

Multi-Version
Concurrency Control (MVCC)
Each document stores its current revision number and hash:
{
"_id": "fe8a4a1ca998daafaaa04d7d9000187f",
"_rev": "1-bd6ed00168af92895217953f69504bb4",
"title":"Learning CouchDB"
}

Optimistic concurrency ensures that a document has not been modi ed since
it was last retrieved

Old revisions are saved for replication purposes, but are eventually pruned

Replication
Peer-based, bi-directional replication of databases

In multi-master scenarios, replicate database A → B
and database B → A

Replication can happen continuously or after having being disconnected for long
periods of time (or never connected)

Document update con icts are agged and can then be processed by your
application or by the end-user

Shared Nothing
Horizontally scalable

Eventually consistent (once replication completes)

Always available (just add nodes)

Partition-tolerant (replication can happen later)

HTTP API
No custom binary protocol (well, there is Hovercraft)

Everyone speaks HTTP—trivially easy to interface with CouchDB

HTTP is:
• distributed
• scalable
• cacheable

Under The Hood
Atomic, Consistent, Isolated, and Durable (ACID)

“Crash-only” design—no shutdown process

Append-only B-tree les

Runs on the Erlang Open Telecom Platform (OTP)

Erlang is a highly-concurrent functional programming language

Erlang OTP is well suited for distributed systems

When You Might
Consider CouchDB
You’ve found yourself denormalizing your SQL database for better performance

Your domain model is a “ t” for documents (e.g. a CMS)

Your application is read-heavy

You need a high level of concurrency and can give up consistency in exchange

You need horizontal scalability

You want your database to be able to run anywhere, even on mobile devices,
and even when disconnected from the cluster

Trade-Oﬀs
No ad hoc queries. You need to know what you’re going to want to query
ahead of time

No concept of “joins”. You can relate data, but watch out for consistency issues

Transactions are limited to document boundaries

CouchDB trades storage space for performance

Reads scale well, writes scale well with third-party software (open source)

Other Alternatives to SQL
MongoDB
http://www.mongodb.org/

Redis
http://redis.io/

Cassandra
http://cassandra.apache.org/

Riak
http://www.basho.com/

HBase (a database for Hadoop)
http://hbase.apache.org/

Don’t be so quick to get rid of SQL! There are
many problems for which an SQL database is a
good t. SQL is a very powerful and exible
query language.

JavaScript
couch.js (synchronous)
http://localhost:5984/_utils/script/couch.js

jquery.couch.js (asynchronous)
http://localhost:5984/_utils/script/jquery.couch.js

jqCouch
http://plugins.jquery.com/project/jqcouch

Perl
CouchDB-Client
http://search.cpan.org/dist/CouchDB-Client/

POE-Component-Client-CouchDB
http://search.cpan.org/dist/POE-Component-Client-CouchDB/

CouchDB-View
http://search.cpan.org/dist/CouchDB-View/

CouchDB-Deploy
http://search.cpan.org/dist/CouchDB-Deploy/

PHP
Sag
http://www.saggingcouch.com/

PHPCouch
http://www.phpcouch.org/

PHPillow
http://arbitracker.org/phpillow.html

PHP CouchDB Extension (PECL)
http://www.topdog.za.net/php_couchdb_extension

Doctrine2 ODM

Python
CouchDB Python Library
http://code.google.com/p/couchdb-python/

CouchDBKit
http://couchdbkit.org/

Paisley: CouchDB client for Twisted
https://launchpad.net/paisley/

Ruby
CouchRest (“close to the metal”)
https://github.com/couchrest/couchrest

CouchRest Model (“close to shiny metal with rounded edges”)
https://github.com/couchrest/couchrest_model

Couch Potato
https://github.com/langalex/couch_potato

CouchFoo (ActiveRecord API)
https://github.com/georgepalmer/couch_foo

CouchObject
http://couchobject.rubyforge.org/

Java
Ektorp
http://code.google.com/p/ektorp/

CouchDB4J
https://github.com/mbreese/couchdb4j

.NET
EasyCouchDB
https://github.com/hhariri/EasyCouchDB
Relax
https://github.com/arobson/Relax/
Divan (C#)
https://github.com/foretagsplatsen/Divan
FunctionalDivan (F#)
https://github.com/kolosy/FunctionalDivan
Ottoman
https://github.com/sinesignal/ottoman
SharpCouch
http://code.google.com/p/couchbrowse/

Erlang
Couchbeam—Erlang CouchDB Kit
https://github.com/benoitc/couchbeam

erlang_couchdb
https://github.com/ngerakines/erlang_couchdb/

eCouch
http://code.google.com/p/ecouch/

Scala
SCouchDB
https://github.com/debasishg/scouchdb

Lisp
Chillax
https://github.com/sykopomp/chillax

ClouchDB (a Common Lisp library)
http://common-lisp.net/project/clouchdb/

RESTful HTTP API
curl -iX PUT 'http://localhost:5984/mydb'

REST and Relaxation
Messages are self-described via HTTP headers and HTTP status codes

URIs identify resources

HTTP methods de ne operations on resources

Hypermedia controls are possible through show and list functions
(if you don’t consider JSON to be hypermedia)

Request Header Examples
User-Agent: curl/7.21.6…

Host: localhost:5984

Accept: application/json

Content-Type: application/json

Content-Length: 15

If-None-Match: "2-bbd27429fd1a0daa2b946cbacb22dc3e"

If-Match: "2-bbd27429fd1a0daa2b946cbacb22dc3e"

Response Header Examples

Date: Tue, 07 Jun 2011 19:30:33 GMT

Content-Type: application/json

Content-Length: 211

Cache-Control: must-revalidate

Location: http://localhost:5984/mydb/mydoc

Etag: "1-967a00dﬀ5e02add41819138abb3284d"

Successful 2xx
200 OK

201 Created

202 Accepted

Redirection 3xx
301 Moved Permanently

304 Not Modi ed

Client Error 4xx
400 Bad Request

401 Unauthorized

403 Forbidden

405 Method Not Allowed

404 Not Found

409 Con ict

412 Precondition Failed

Server Error 5xx
500 Internal Server

GET a Resource

HTTP/1.1 200 OK
{
"version": "1.0.2"
}

Get Just HEADers
$ curl -iX HEAD 'http://localhost:5984/'

HTTP/1.1 200 OK

PUT a Database
$ curl -iX PUT 'http://localhost:5984/mydb'

HTTP/1.1 201 Created
Location: http://localhost:5984/mydb
{"ok":true}

POST a Document
$ curl -iX POST 'http://localhost:5984/mydb'
-H "Content-Type: application/json"
-d '{}'

Location: http://localhost:5984/mydb/0a72c9c36bd169818dc97ed18b000aa4

{
"ok":true,
"id":"0a72c9c36bd169818dc97ed18b000aa4",
"rev":"1-967a00dﬀ5e02add41819138abb3284d"
}

PUT a Document
$ curl -iX PUT 'http://localhost:5984/mydb/0a72c9c36bd169818dc97ed18b000aa4'
-d '{
"_rev":"1-967a00dﬀ5e02add41819138abb3284d",
"title":"Learning CouchDB"
}'


{
"ok":true,
"id":"0a72c9c36bd169818dc97ed18b000aa4",
"rev":"2-516027e3179a22a22e06874c374e8ef0"
}

DELETE a Document
$ curl -iX DELETE 'http://localhost:5984/mydb/0a72c9c36bd169818dc97ed18b000aa4
?rev=2-516027e3179a22a22e06874c374e8ef0'

HTTP/1.1 200 OK

{
"ok":true,
"id":"0a72c9c36bd169818dc97ed18b000aa4",
"rev":"3-e9a5aa1c486eee23c84fa028bc904991"
}

Create a Document
$ curl -iX POST 'http://localhost:5984/mydb'
-d '{"_id":"mydoc"}'

Location: http://localhost:5984/mydb/mydoc

{
"ok":true,
"id":"mydoc",
"rev":"1-967a00dﬀ5e02add41819138abb3284d"
}

Read a Document
$ curl -iX GET 'http://localhost:5984/mydb/mydoc'

HTTP/1.1 200 OK
Etag: "1-967a00dﬀ5e02add41819138abb3284d"

{
"_id":"mydoc",
"_rev":"1-967a00dﬀ5e02add41819138abb3284d"
}

Update a Document
$ curl -iX PUT 'http://localhost:5984/mydb/mydoc'
-d '{
"_id":"mydoc",
"_rev":"1-967a00dﬀ5e02add41819138abb3284d",
"title":"CouchDB: The De nitive Guide"
}'


{
"ok":true,
"id":"mydoc",
"rev":"2-bbd27429fd1a0daa2b946cbacb22dc3e"
}

CouchDB allows for conditional requests,
saving bandwidth and processing.

Conditional GET
-H 'If-None-Match: "2-bbd27429fd1a0daa2b946cbacb22dc3e"'

HTTP/1.1 304 Not Modi ed
Etag: "2-bbd27429fd1a0daa2b946cbacb22dc3e"
Content-Length: 0

Delete a Document
$ curl -iX DELETE 'http://localhost:5984/mydb/mydoc'
-H 'If-Match: "2-bbd27429fd1a0daa2b946cbacb22dc3e"'

HTTP/1.1 200 OK

{
"ok":true,
"id":"mydoc",
"rev":"3-29d2ef6e0d3558a3547a92dac51f3231"
}

Read a Deleted Document

HTTP/1.1 404 Object Not Found
{
"error":"not_found",
"reason":"deleted"
}

Read a Deleted Document
$ curl -iX GET 'http://localhost:5984/mydb/mydoc
?rev=3-29d2ef6e0d3558a3547a92dac51f3231'

HTTP/1.1 200 OK
Etag: "3-29d2ef6e0d3558a3547a92dac51f3231"

{
"_id":"mydoc",
"_rev":"3-29d2ef6e0d3558a3547a92dac51f3231",
"_deleted":true
}

Fetch Revisions
$ curl -iX GET 'http://localhost:5984/mydb/mydoc
?rev=3-29d2ef6e0d3558a3547a92dac51f3231&revs=true'

HTTP/1.1 200 OK
{
// …
"_revisions":{
"start":3,
"ids":[
"29d2ef6e0d3558a3547a92dac51f3231",
"bbd27429fd1a0daa2b946cbacb22dc3e",
"967a00dﬀ5e02add41819138abb3284d"
]
}
}

Do not rely on older revisions! Revisions
are used for concurrency control and for
replication. Old revisions are removed
during database compaction.

Replication Party!

Cloned by Asha ten Broeke, on Flickr

My Business Card
(business_card.json)
{
   "fn":"Bradley Holt",
   "url":"http://bradley-holt.com/",
   "role":"Technical Director",
   "org":"Found Line, Inc.",
   "email":"bradley.holt@foundline.com",
   "adr":{
       "street-address":"7 Kilburn Street",
       "locality":"Burlington",
       "region":"Vermont",
       "postal-code":"05401",
       "country-name":"USA"
   },
   "tel":"802-383-4737 ext. 11"
}

When POSTing a document, CouchDB will
assign a UUID as the document’s ID if one
is not speci ed.

POST My Business Card
$ curl -iX POST 'http://localhost:5984/cards'
-d @business_card.json

Location: http://localhost:5984/cards/fe8a4a1ca998daafaaa04d7d90000574

{
"ok":true,
"id":"fe8a4a1ca998daafaaa04d7d90000574",
"rev":"1-2526abc0e10aa39167ﬀ7b16a77b4dad"
}

You Replicate Me
$ curl -iX POST 'http://localhost:5984/_replicate'
-d '{
"source": "https://oscon-tutorial.iriscouch.com/cards",
"target": "cards",
"create_target": true
}'

HTTP/1.1 200 OK

{
"ok":true,
//…
}

GET All Documents
$ curl -iX GET 'http://localhost:5984/cards/_all_docs'

HTTP/1.1 200 OK
{
// …
"rows": [
{
"id": "fe8a4a1ca998daafaaa04d7d90000574",
// …
}
]
}

GET One Document
$ curl -iX GET 'http://localhost:5984/cards/fe8a4a1ca998daafaaa04d7d90000574'

HTTP/1.1 200 OK
Etag: "2-3a584544d14db477911bd420a24b7055"
{
"_id": "fe8a4a1ca998daafaaa04d7d90000574",
"_rev": "2-3a584544d14db477911bd420a24b7055",
"fn": "Bradley Holt",
"url": "http://bradley-holt.com/",
// …
}

Create Your Business Card
(business_card.json)
{
   "fn":"Your Name Here",
   "url":"http://yoururl.com/",
   "role":"Your Role",
   "org":"Your Organization",
   "email":"youremail@yoururl.com",
   "adr":{
       "street-address":"Your Street",
       "locality":"Your City",
       "region":"Your State",
       "postal-code":"Your Postal Code",
       "country-name":"Your Country"
   },
   "tel":"Your Phone Number"
}

POST Your Business Card
$ curl -iX POST 'http://localhost:5984/cards'
-d @business_card.json


Continuous
Push Replication
-d '{
"source": "cards",
"target": "https://oscon-tutorial.iriscouch.com/cards",
"continuous": true
}'

HTTP/1.1 200 OK

{
"ok":true,
//…
}

Continuous Pull Replication
-d '{
"source": "https://oscon-tutorial.iriscouch.com/cards",
"target": "cards",
"continuous": true
}'

HTTP/1.1 200 OK

{
"ok":true,
//…
}

GET All Documents Again
$ curl -iX GET 'http://localhost:5984/tutorial/_all_docs'

HTTP/1.1 200 OK
{
// …
"rows": [
{
"id": "fe8a4a1ca998daafaaa04d7d90000574",
// …
},
// …
]
}

Cancel Push Replication
-d '{
"source": "cards",
"target": "https://oscon-tutorial.iriscouch.com/cards/",
"cancel": true
}'

HTTP/1.1 200 OK

{
"ok":true,
//…
}

Cancel Pull Replication
-d '{
"source": "https://oscon-tutorial.iriscouch.com/cards/",
"target": "cards",
"cancel": true
}'

HTTP/1.1 200 OK

{
"ok":true,
//…
}

MapReduce Views
function(doc) { if (doc.title) { emit(doc.title); } }

MapReduce consists of Map and Reduce steps
which can be distributed in a way that takes
advantage of the multiple processor cores
found in modern hardware.

Create a Database with cURL
$ curl -iX PUT 'http://localhost:5984/books'

Location: http://localhost:5984/books
{"ok":true}

The First Document
(978-0-596-15589-6.json)
{
"_id":"978-0-596-15589-6",
"title":"CouchDB: The De nitive Guide",
"subtitle":"Time to Relax",
"authors":[
"J. Chris Anderson",
"Jan Lehnardt",
"Noah Slater"
],
"formats":[
"Print",
"Ebook",
"Safari Books Online"
],
"released":"2010-01-19",
"pages":272
}

POST the First Document
$ curl -iX POST 'http://localhost:5984/books'
-d @978-0-596-15589-6.json

Location: http://localhost:5984/books/978-0-596-15589-6

{
"ok":true,
"id":"978-0-596-15589-6",
"rev":"1-8c1d735475a0401d984c4aﬀ2fe796a5"
}

The Second Document
(978-0-596-52926-0.json)
{
"_id":"978-0-596-52926-0",
"title":"RESTful Web Services",
"subtitle":"Web services for the real world",
"authors":[
"Leonard Richardson",
"Sam Ruby"
],
"formats":[
"Print",
"Ebook",
],
"released":"2007-05-08",
"pages":448
}

POST the Second Document
-d @978-0-596-52926-0.json


{
"ok":true,
"id":"978-0-596-52926-0",
"rev":"1-12538b07773519133043157220a63d8b"
}

The Third Document
(978-1-565-92580-9.json)
{
"_id":"978-1-565-92580-9",
"title":"DocBook: The De nitive Guide",
"authors":[
"Norman Walsh",
"Leonard Muellner"
],
"formats":[
"Print"
],
"released":"1999-10-28",
"pages":648
}

POST the Third Document
-d @978-1-565-92580-9.json


{
"ok":true,
"id":"978-1-565-92580-9",
"rev":"1-b945cb4799a1ccdd1689eae0e44124f1"
}

The Fourth Document
(978-0-596-80579-1.json)
{
"_id":"978-0-596-80579-1",
"title":"Building iPhone Apps with HTML, CSS, and JavaScript",
"subtitle":"Making App Store Apps Without Objective-C or Cocoa",
"authors":[
"Jonathan Stark"
],
"formats":[
"Print",
"Ebook",
],
"released":"2010-01-08",
"pages":192
}

POST the Fourth Document
-d @978-0-596-80579-1.json


{
"ok":true,
"id":"978-0-596-80579-1",
"rev":"1-09ce09fef75068834da99957c7b14cf2"
}

…or Just Replicate
-d '{
"source": "https://oscon-tutorial.iriscouch.com/books",
"target": "books"
}'

HTTP/1.1 200 OK

{
"ok":true,
//…
}

GET All Documents
$ curl -iX GET 'http://localhost:5984/books/_all_docs'

HTTP/1.1 200 OK
{
// …
"rows": [
{ "id": "978-0-596-15589-6", // … },
{ "id": "978-0-596-52926-0", // … },
{ "id": "978-0-596-80579-1", // … },
{ "id": "978-1-565-92580-9", // … }
]
}

Map and Reduce are written as JavaScript
functions that are de ned within views.

emit(key, value)
Accepts two arguments—both are optional and default to null

First argument is the key to be indexed

Second argument is the value to be associated with the key

Arbitrary JSON values are allowed for both the key and value

The ID of the document being mapped is implicitly included with each
call to the emit function

One-To-One Mapping
(On Emit Per Document)

Map Book Titles
function(doc) { // JSON object representing a doc to be mapped
if (doc.title) { // make sure this doc has a title
emit(doc.title); // emit the doc’s title as the key
}
}

Temporary views can be used during
development but should be saved
permanently to design documents
for production.

Titles View
(titles_view.json)
{
"map": "function(doc) { if (doc.title) { emit(doc.title); } }"
}

POST Titles Temporary View
$ curl -iX POST 'http://localhost:5984/books/_temp_view'
-d @titles_view.json

HTTP/1.1 200 OK

{
"total_rows": 4,
"oﬀset": 0,
"rows": [
// …
]
}

“Titles” View Rows
key id value

"Building iPhone Apps with
"978-0-596-80579-1" null
HTML, CSS, and JavaScript"

"CouchDB: The Deﬁnitive
"978-0-596-15589-6" null
Guide"

"DocBook: The Deﬁnitive
"978-1-565-92580-9" null
Guide"

"RESTful Web Services" "978-0-596-52926-0" null

One-To-Many Mapping
(Multiple Emits Per Document)

Map Book Formats
if (doc.formats) { // make sure this doc has a formats eld
for (var i in doc.formats) {
emit(doc.formats[i]); // emit each format as the key
}
}
}

Formats View
(formats_view.json)
{
"map": "function(doc) { if (doc.formats) { for (var i in doc.formats)
{ emit(doc.formats[i]); } } }"
}

POST Formats
Temporary View
-d @formats_view.json

HTTP/1.1 200 OK

{
"total_rows": 10,
"oﬀset": 0,
"rows": [
// …
]
}

“Formats” View Rows
key id value
"Ebook" "978-0-596-15589-6" null
"Ebook" "978-0-596-52926-0" null
"Ebook" "978-0-596-80579-1" null
"Print" "978-0-596-15589-6" null
"Print" "978-0-596-52926-0" null
"Print" "978-0-596-80579-1" null
"Print" "978-1-565-92580-9" null
"Safari Books Online" "978-0-596-15589-6" null

Map Book Authors
if (doc.authors) { // make sure this doc has an authors eld
for (var i in doc.authors) {
emit(doc.authors[i]); // emit each author as the key
}
}
}

Authors View
(authors_view.json)
{
"map": "function(doc) { if (doc.authors) { for (var i in doc.authors)
{ emit(doc.authors[i]); } } }"
}

POST Authors
Temporary View
-d @authors_view.json

HTTP/1.1 200 OK

{
"total_rows": 8,
"oﬀset": 0,
"rows": [
// …
]
}

“Authors” View Rows
key id value
"J. Chris Anderson" "978-0-596-15589-6" null
"Jan Lehnardt" "978-0-596-15589-6" null
"Jonathan Stark" "978-0-596-80579-1" null
"Leonard Muellner" "978-1-565-92580-9" null
"Leonard Richardson" "978-0-596-52926-0" null
"Noah Slater" "978-0-596-15589-6" null
"Norman Walsh" "978-1-565-92580-9" null
"Sam Ruby" "978-0-596-52926-0" null

When querying a view, the key and id
elds can be used to select a row or range
of rows, and to group. Rows can optionally
be grouped by key elds.

Built-in Reduce Functions
Function Output

_count Returns the number of mapped values in the set

_sum Returns the sum of the set of mapped values

Returns numerical statistics of the mapped values in the set
_stats
including the sum, count, min, and max

Updated Formats View
(formats_view.json)
{
{ emit(doc.formats[i]); } } }",
"reduce": "_count"
}

POST Updated Formats
Temporary View

HTTP/1.1 200 OK

{
"rows": [
{
"key": null,
"value": 10
}
]
}

Format Count, Not Grouped

key value
null 10

POST Updated Formats
Temporary View, Grouped
$ curl -iX POST 'http://localhost:5984/books/_temp_view?group=true'

HTTP/1.1 200 OK

{
"rows": [
// …
]
}

Format Count, Grouped

key value
"Ebook" 3
"Print" 4
"Safari Books Online" 3

The _count function can count arbitrary values,
including NULLs. However, the _sum and _stats
functions require numbers as values.

Updated Book Formats Map
function(doc) {
if (doc.formats) {
for (var i in doc.formats) {
emit(doc.formats[i], doc.pages); // now emit pages as value
}
}
}

(formats_view.json)
{
{ emit(doc.formats[i], doc.pages); } } }",
"reduce": "_sum"
}

Sum of Pages by Format

key value
"Ebook" 912
"Print" 1560
"Safari Books Online" 912

Stats

sum, count, minimum, maximum,
sum over all square roots

(formats_view.json)
{
{ emit(doc.formats[i], doc.pages); } } }",
"reduce": "_stats"
}

Stats of Pages by Format
key value
{"sum":912,"count":3,"min":192,
"Ebook"
"max":448,"sumsqr":311552}

{"sum":1560,"count":4,"min":192,
"Print"
"max":648,"sumsqr":731456}

{"sum":912,"count":3,"min":192,
"max":448,"sumsqr":311552}

The built-in Reduce functions should serve your
needs most of the time.

Parameters
Keys: An array of mapped key and document IDs in the form of [key,id] where id is
the document ID.

Values: An array of mapped values.

Rereduce: Whether or not the Reduce function is being called recursively on its
own output.

Reduce Function Skeleton
function(keys, values, rereduce) { }

Count Equivalent
function(keys, values, rereduce) {
if (rereduce) {
return sum(values);
} else {
return values.length;
}
}

Sum Equivalent
function(keys, values, rereduce) {
return sum(values);
}

MapReduce Limitations
Full-text indexing and ad hoc searching
• couchdb-lucene
https://github.com/rnewson/couchdb-lucene
• ElasticSearch and CouchDB
https://github.com/elasticsearch/elasticsearch/wiki/Couchdb-integration

Geospatial indexing and search (two dimensional)
• GeoCouch
https://github.com/couchbase/geocouch
• Geohash (e.g. c20g0b2t2882)
http://geohash.org/

You can query for all rows, a single contiguous
range of rows, or even rows matching a
speci ed key.

Map Book Releases
function(doc) {
if (doc.released) {
emit(doc.released.split("-"), doc.pages);
}
}

Save the “Releases”
View in Futon

Default Design Document
(default_design.json)
{
"_id": "_design/default",
"language": "javascript",
"views": {
"releases": {
"map": "function(doc) { if (doc.released) { emit(doc.released.split('-'),
doc.pages); } }",
"reduce": "_stats"
}
}
}

Save the “Default”
Design Document with cURL
-d @default_design.json

Location: http://localhost:5984/books/_design/default

{
"ok": true,
"id": "_design/default",
"rev": "1-1776c504d0927e80e515acb952327797"
}

“Releases”
$ curl -iX GET 'http://localhost:5984/books/_design/default/_view/releases
?group=true'

HTTP/1.1 200 OK
{
"rows": [
{"key": ["1999","10","28"], /* … */ },
{"key": ["2007","05","08"], /* … */ },
{"key": ["2010","01","08"], /* … */ },
{"key": ["2010","01","19"], /* … */ }
]
}

“Releases”
key value
{"sum":648,"count":1,"min":648,
["1999","10","28"]
"max":648,"sumsqr":419904}

{"sum":448,"count":1,"min":448,
["2007","05","08"]
"max":448,"sumsqr":200704}

{"sum":192,"count":1,"min":192,
["2010","01","08"]
"max":192,"sumsqr":36864}

{"sum":272,"count":1,"min":272,
["2010","01","19"]
"max":272,"sumsqr":73984}

“Releases”, Level 1 Grouping

?group=true&group_level=1'

HTTP/1.1 200 OK
{
"rows": [
{"key": ["1999"], /* … */ },
{"key": ["2007"], /* … */ },
{"key": ["2010"], /* … */ }
]
}

key value
{"sum":648,"count":1,"min":648,
["1999"]
"max":648,"sumsqr":419904}

{"sum":448,"count":1,"min":448,
["2007"]
"max":448,"sumsqr":200704}

{"sum":464,"count":2,"min":192,
["2010"]
"max":272,"sumsqr":110848}

?group=true&group_level=2'

HTTP/1.1 200 OK
{
"rows": [
{"key": ["1999","10"], /* … */ },
{"key": ["2007","05"], /* … */ },
{"key": ["2010","01"], /* … */ }
]
}

key value
{"sum":648,"count":1,"min":648,
["1999","10"]
"max":648,"sumsqr":419904}

{"sum":448,"count":1,"min":448,
["2007","05"]
"max":448,"sumsqr":200704}

{"sum":464,"count":2,"min":192,
["2010","01"]
"max":272,"sumsqr":110848}

By Key
$ curl -iGX GET 'http://localhost:5984/books/_design/default/_view/releases'
-d reduce=false
--data-urlencode key='["1999", "10", "28"]'

HTTP/1.1 200 OK

{"total_rows":4,"oﬀset":0,"rows":[
{"id":"978-1-565-92580-9","key":["1999","10","28"],"value":648}
]}

By Start and End Keys
$ curl -iGX GET 'http://localhost:5984/books/_design/default/_view/formats'
-d reduce=false
--data-urlencode startkey='"Ebook"'
--data-urlencode endkey='"Print"'

HTTP/1.1 200 OK

{"id":"978-0-596-15589-6","key":"Ebook","value":272},
{"id":"978-0-596-15589-6","key":"Print","value":272},
{"id":"978-1-565-92580-9","key":"Print","value":648}
]}

By Start and End Keys
and Document IDs
-d reduce=false
--data-urlencode startkey='"Ebook"'
--data-urlencode startkey_docid='978-0-596-52926-0'
--data-urlencode endkey='"Print"'
--data-urlencode endkey_docid='978-0-596-52926-0'

HTTP/1.1 200 OK
]}

Limiting, Skipping & Reversing Results

Limit
-d reduce=false
-d limit=5

HTTP/1.1 200 OK

]}

Skip
-d reduce=false
-d limit=5
-d skip=5

HTTP/1.1 200 OK

{"id":"978-0-596-15589-6","key":"Safari Books Online","value":272},
{"id":"978-0-596-52926-0","key":"Safari Books Online","value":448},
{"id":"978-0-596-80579-1","key":"Safari Books Online","value":192}
]}

The skip and limit parameters can be used
to implement pagination. When skipping a
larger number of rows, it is more eﬃcient to
set the skip parameter’s value to 1 and use
the previous page’s last key and document
ID as the startkey and startkey_docid.

Reversing Output
$ curl -iGX GET 'http://localhost:5984/books/_design/default/_view/titles'
-d reduce=false
-d descending=true

HTTP/1.1 200 OK

{"id":"978-0-596-52926-0","key":"RESTful Web Services","value":448},
{"id":"978-1-565-92580-9","key":"DocBook: The De nitive Guide","value":648},
{"id":"978-0-596-15589-6","key":"CouchDB: The De nitive Guide","value":272},
{"id":"978-0-596-80579-1","key":"Building iPhone Apps with HTML, CSS, and
JavaScript","value":192}
]}

When reversing output, the values for startkey
and endkey must be swapped as well as the
values for startkey_docid and endkey_docid, if
speci ed. This is because output is reversed
before rows are ltered.

include_docs Parameter
-d reduce=false
-d include_docs=true

HTTP/1.1 200 OK

{ // …
"rows": [ {
// …
"doc": {
"_id": "978-0-596-80579-1",
"_rev": "1-09ce09fef75068834da99957c7b14cf2",
"title": "Building iPhone Apps with HTML, CSS, and JavaScript",
// …

Emit Document as Value
function(doc) {
if (doc.title) {
emit(doc.title, doc);
}
}

Get the View
-d reduce=false
-d limit=1

HTTP/1.1 200 OK

{"id":"978-0-596-80579-1","key":"Building iPhone Apps with HTML, CSS, and
JavaScript","value":192}
]}

Get a Document
From the View
$ curl -iX GET 'http://localhost:5984/books/978-0-596-80579-1'

HTTP/1.1 200 OK
{
"_id": "978-0-596-80579-1",
"_rev": "1-09ce09fef75068834da99957c7b14cf2",
"title": "Building iPhone Apps with HTML, CSS, and JavaScript",
"subtitle": "Making App Store Apps Without Objective-C or Cocoa",
"authors": ["Jonathan Stark"],
"publisher": "O'Reilly Media",
"formats": ["Print", "Ebook", "Safari Books Online"],
"released":" 2010-01-08",
"pages":192
}

Get a Cached Document From the
View
$ curl -iX GET 'http://localhost:5984/books/978-0-596-80579-1'
-H 'If-None-Match: "1-09ce09fef75068834da99957c7b14cf2"'

HTTP/1.1 304 Not Modi ed
Etag: "1-09ce09fef75068834da99957c7b14cf2"
Content-Length: 0

Load Balancing
Send POST, PUT, and DELETE requests to a write-only master node

Setup continuous replication from the master node to multiple read-only nodes

Load balance GET, HEAD, and OPTIONS requests amongst
read-only nodes
• Apache HTTP Server (mod_proxy)
• nginx
• HAProxy
• Varnish
• Squid

Clustering
(Partitioning/Sharding)
BigCouch
https://github.com/cloudant/bigcouch
• Clusters modeled after Amazon’s Dynamo approach
Lounge
http://tilgovi.github.com/couchdb-lounge/
• Proxy, partitioning, and sharding
Pillow
https://github.com/khellan/Pillow
• “…a combined router and rereducer for CouchDB.”

local.ini
[couchdb]
;max_document_size = 4294967296 ; bytes
[httpd]
;port = 5984
;bind_address = 127.0.0.1
; Uncomment next line to trigger basic-auth popup on unauthorized requests.
;WWW-Authenticate = Basic realm="administrator"
;…

GET Con g
$ curl -iX GET 'http://localhost:5984/_con g'

HTTP/1.1 200 OK
{
"httpd_design_handlers": { /* … */ },
"uuids": { /* … */ },
"stats": { /* … */ },
"httpd_global_handlers": { /* … */ },
"attachments": { /* … */ },
"query_server_con g": { /* … */ },
// …
}

GET Con g Section
$ curl -iX GET 'http://localhost:5984/_con g/uuids'

HTTP/1.1 200 OK
{
"algorithm": "sequential"
}

GET Con g Section Key
$ curl -iX GET 'http://localhost:5984/_con g/uuids/algorithm'

HTTP/1.1 200 OK
"sequential"

PUT Con g Section Key
$ curl -iX PUT 'http://localhost:5984/_con g/uuids/algorithm'
-d '"utc_random"'

HTTP/1.1 200 OK

"sequential"

Con guration settings can also be edited from
within Futon

Admin Party
Default settings after install are that everyone is admin

By default, CouchDB will only listen on the loopback address

You’ll want to change the default settings if you let CouchDB listen on a public IP
address!

SSL Con guration
[daemons]
;…
httpsd = {couch_httpd, start_link, [https]}
;…

[ssl]
cert_ le = /full/path/to/server_cert.pem
key_ le = /full/path/to/server_key.pem
;…

Connect to Port 6984 for SSL
$ curl -iX GET 'https://localhost:6984/'

HTTP/1.1 200 OK
{
"version": "1.1.0"
}

GET All Users
$ curl -iX GET 'http://localhost:5984/users/_all_docs'

HTTP/1.1 200 OK
{
// …
"rows": [
// …
]
}

GET a UUID for
Password Salting
$ curl -iX GET 'http://localhost:5984/_uuids'

HTTP/1.1 200 OK
{
"uuids": [
"f0f1353b46013ec80533346f7f000c8b"
]
}

Concatenate and Hash Password
and Salt
$ echo -n "supersecurepasswordf0f1353b46013ec80533346f7f000c8b" | openssl
sha1

(stdin)= 3cfa095ef255121dcc2fe223b907246a6d1591e5

Bob (bob.json)
{
"_id": "org.couchdb.user:bob",
"type": "user",
"name": "bob",
"roles": [],
"password_sha": "3cfa095ef255121dcc2fe223b907246a6d1591e5",
"salt": "f0f1353b46013ec80533346f7f000c8b"
}

POST a User
$ curl -iX POST 'http://localhost:5984/_users'
-d @bob.json

Location: http://localhost:5984/_users/org.couchdb.user:bob

{
"ok": true,
"id": "org.couchdb.user:bob",
"rev": "1-fbca8d4c136c2c322e88249b78eeafe6"
}

Basic Authentication
$ curl -iX GET 'http://bob:supersecurepassword@localhost:5984/'

HTTP/1.1 200 OK
{
"version": "1.0.2"
}

Cookie Authentication
$ curl -iX POST 'http://localhost:5984/_session'
-d "name=bob"
-d "password=supersecurepassword"

HTTP/1.1 200 OK
Set-Cookie: AuthSession=Ym9iOjRERjExRjk3On4WFg1s…; Version=1; Path=/;
HttpOnly

{
"ok": true,
"name": "bob",
"roles": []
}

Cookie Authentication
-b "AuthSession=Ym9iOjRERjExRjk3On4WFg1s…"

HTTP/1.1 200 OK

{
"version": "1.0.2"
}

Server Admin
Can:
• Create databases (PUT /db)
• Delete databases (DELETE /db)
• Create design documents (PUT /db/_design/foo)
• Update design documents (PUT /db/_design/foo?rev=1-3BA)
• Delete design documents (DELETE /db/_design/foo?rev=2-C8F)
• Trigger compaction (POST /db/_compact)
• Trigger view cleanup (POST /db/_view_cleanup)
• View active tasks (GET /_active_tasks)
• Restart the server (POST /_restart)
• Read con g settings (GET /_con g)
• Update con g settings (PUT /_con g)

Create a Server Admin
$ curl -iX PUT 'http://localhost:5984/_con g/admins/bob'
-d '"supersecurepassword"'

HTTP/1.1 200 OK
""

Delete Server Admin
$ curl -iX DELETE 'http://bob:supersecurepassword@
localhost:5984/_con g/admins/bob'

HTTP/1.1 200 OK
"-hashed-54520b20f1b3a5c24d36c051aa1268de7543144c,
20edd87862631adab51442b9b10e0a65"

Create Server Admins
from Futon

Database Admin
Can:
• Create design documents (PUT /db/_design/foo)
• Update design documents (PUT /db/_design/foo?rev=1-3BA)
• Delete design documents (DELETE /db/_design/foo?rev=2-C8F)
• Add database admins and database readers
• Remove database admins and database readers
• Set database revision limit (PUT /db/_revs_limit)
• Execute temporary views (POST /db/_temp_view)
• Trigger compaction (POST /db/_compact)
• Trigger view cleanup (POST /db/_view_cleanup)

Database Reader
Can:
• Read all types of documents
• Except for design documents:
• Create documents (POST /mydb/mydoc)
• Update documents (PUT /mydb/mydoc?rev=1-3BA)
• Delete documents (DELETE /mydb/mydoc?rev=2-C8F)

Database Security Object
(security.json)
{
"admins" : {
"names": ["bob"],
"roles": []
},
"readers" : {
"names": ["joe"],
"roles": []
}
}

PUT Security Object
$ curl -iX PUT 'http://localhost:5984/mydb/_security'
-d @security.json

HTTP/1.1 200 OK

{
"ok":true
}

Validation Functions
De ned within design documents

Function passed:
• New document
• Old document
• User context:
• Database name (db)
• User name (name)
• Roles (roles)

Read-Only for Non-Admins
function(newDoc, oldDoc, userCtx) {
if (-1 === userCtx.roles.indexOf('_admin')) {
throw({forbidden: 'Only admins may update documents.'});
}
}

Auth Design Document
{
"_id": "_design/auth",
"language": "javascript",
"validate_doc_update": "function(newDoc, oldDoc, userCtx) { if (-1 ===
userCtx.roles.indexOf('_admin')) { throw({forbidden: 'Only admins may update
documents.'}); } }"
}

Con ict Resolution
Replication will eventually lead to con icting revisions

Both con icted revisions will be preserved, but one “wins”

Con icted documents will get a special "_con icts" ag

Create a view to nd con icted documents:
if (doc._con icts) { for (var i in doc._con icts) { emit(doc._con icts[i]); } }

Have end-user or application logic handle a merge

Clear the "_con icts" ag with a document update

Changes Feed
$ curl -iX GET 'http://localhost:5984/books/_changes'

HTTP/1.1 200 OK
{"results":[
{"seq":1,"id":"978-0-596-52926-0","changes":[{"rev":"1-12538b0…"}]},
{"seq":2,"id":"978-1-565-92580-9","changes":[{"rev":"1-b945cb4…"}]},
{"seq":3,"id":"978-0-596-80579-1","changes":[{"rev":"1-09ce09fe…"}]},
{"seq":4,"id":"978-0-596-15589-6","changes":[{"rev":"1-8c1d7354…"}]}
],
"last_seq":4}

Partial Replicas:
Filter Functions
Filter functions are de ned within design documents under the " lters" key.
Example of an "authors" lter function:
function(doc, req) { return 'author' == doc.collection }

The lter function (and its containing design document) can then be referenced
during replication:
" lter":"default/authors"

Filter functions can be parameterized:
function(doc, req) { return req.query.collection == doc.collection }

Parameters values can be speci ed during replication:
"query_params": { "collection":"publisher" }

Partial Replicas:
Specifying IDs
Provide an array of IDs when replicating:
"doc_ids": [
"be231efa93502b3286aae0ed7b000aed",
"be231efa93502b3286aae0ed7b001a6a",
"be231efa93502b3286aae0ed7b002916",
"be231efa93502b3286aae0ed7b003458",
"be231efa93502b3286aae0ed7b00375e"
]

Show Functions
function(doc, req) {
return {
"body": "<h1>" + doc.title + "</h1>",
"headers": {
"Content-Type": "text/html"
}
};
}

GET /db/_design/<design_name>/_show/<show_name>/<doc_id>

List Functions
function(head, req) {
var row;
start({
"headers": {
"Content-Type": "text/html"
}
});
while (row = getRow()) {
send("<h1>" + row.key + "</h1>");
}
}

GET /db/_design/<design_name>/_list/<list_name>/<view_name>

Rewriting to an Attachment
{
"from": "",
"to": "index.html",
"method": "GET",
"query": {}
}

From:
GET /db/_design/app/_rewrite

To:
GET /db/_design/app/index.html

Rewriting to a List
{
"from": "books",
"to": "_list/books/all",
"query": {
"limit": 10
}
}

From:
GET /db/_design/app/_rewrite/books

To:
GET /db/_design/app/_list/books/all?limit=10

Rewriting to a Show
{
"from": "books/:id",
"to": "_show/books/:id",
"query": {}
}

From:
GET /db/_design/app/_rewrite/books/978-0-596-15589-6

To:
GET /db/_design/app/_show/books/978-0-596-15589-6

“vhosts” Con g Section
{
"example.com": "/db/_design/app/_rewrite",
"www.example.com": "/db/_design/app/_rewrite"
}

All requests with a Host HTTP header of example.com or www.example.com will
now be handled by the rewrite rules de ned in the app design document.

Database Compaction
$ curl -iX POST 'http://localhost:5984/db/_compact'

HTTP/1.1 202 Accepted
{
"ok": true
}

View Compaction
$ curl -iX POST 'http://localhost:5984/db/_compact/default'

{
"ok": true
}

View Cleanup
$ curl -iX POST 'http://localhost:5984/db/_view_cleanup'

{
"ok": true
}

CouchApp
Applications built using CouchDB, JavaScript and HTML5

CouchDB is a database, web server and application server

No more middle-tier between your data and presentation layers:
• CouchDB becomes both the data and application tiers
• User agent becomes the presentation tier
Data and application can be replicated together

CouchApp Tool:
http://couchapp.org/

Mobile Couchbase for Android & iOS
http://www.couchbase.com/products-and-services/mobile-couchbase

Hosting is available through Iris Couch or
Cloudant.

CouchDB Resources
CouchDB: The De nitive Guide CouchDB Wiki
by J. Chris Anderson, Jan Lehnardt, and http://wiki.apache.org/couchdb/
Noah Slater (O’Reilly)
978-0-596-15589-6 Beginning CouchDB
by Joe Lennon (Apress)
Writing and Querying MapReduce Views in 978-1-430-27237-3
CouchDB
by Bradley Holt (O’Reilly)
978-1-449-30312-9

Scaling CouchDB
by Bradley Holt (O’Reilly)
063-6-920-01840-7

OSCON 2011 Learning CouchDB

More Related Content

What's hot (20)

Similar to OSCON 2011 Learning CouchDB (20)

More from Bradley Holt (15)

Recently uploaded (20)

OSCON 2011 Learning CouchDB

Editor's Notes