Building a CMS on top of NoSQL (for ParisJUG)

... In which I tell a
story of building
a CMS on top of
‘NoSQL’
(*)

(*) HBase and SOLR

IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

... and hopefully
warn you on
what YOU will
encounter in the
near future.
IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

/usr/bin/whoami

» co-founder of Outerthought

» scalable content applications
» content management & publishing
» Java, REST and now NoSQL
» open source product portfolio

IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.orgTHIS NOTEBOOK BELONGS TO: 3

» Daisy: content- and knowledge management
www.daisycms.org

» Lily: scalable store and search
THIS N OTE B OOK B ELO N GS TO :
www.lilycms.org

» Kauri: RESTcentric internet app development
www.kauriproject.org

IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org 4

Petite annonce semi-commercial
» Devoxx 2010 ! (ex-Javapolis)

» 15-19 Novembre, Anvers, Belgique

» Track NoSQL/Cloud
» Speakers: Tom White (Cloudera/Hadoop O’Reilly
author), Jonathan Ellis (Cassandra), Michael Stack
(HBase)
» Produits: MongoDB, Voldemort, Elastic Search
» Cases: Twitter, Facebook, Adobe


Devoxx NoSQL/Cloud track


This story is about


The typical CMS ‘architecture’

database (+opt. ﬁlesystem) (+ opt. full-text indexes)



application cache




more cache

application cache




client (+cache?)

more cache

application cache



Hitting the scale spot
» Sweet spot of # documents: (100)Ks, not Ms

» Not everything could be solved with increasing
heap size
» cold cache at startup
» OOME’s
» we didn’t want to step in the PHP/RDBMS trap
(of dynamic database schemes)
» The cost of ﬂexibility


What we found hard to scale
» access control (dynamically evaluated against rule set)

» facet browsing (compute facet counts in RAM)

» all the nifty stuff people were using our
software for

» ... anything that required random access
to in-memory-cache data for computations


Beyond the ‘scaling’ problem
» three-prong data layer

fs

» result set merging (between MySQL & Lucene)
» happened in appcode/memory

» ‘transactions’, set operations = hard


Beyond the three-prong problem

» errrr..... “Failover” ..... ?

» = symptom of enterprise success


If we would be able to add more nodes ...

scalability

» True Distribution availability

performance

... in the line of ﬁre


Solution 1

» do MORE inside the database


Infrastructural (master/slave)

e !
as
ta b
d a
o r e
m

18

e !
a s
ta b
da
o r e
n m
e ve


s !
u s se
e b
sa g
mes
d d
’s a
l et


ff!
! s tu
B C
JD
r !
o ve t
S w00
I! JM
RM


http://bigdatamatters.com/bigdatamatters/2010/04/high-availability-with-oracle.html


Business Development 101
user interest

budget


Solution II: Enter The Cambrian Explosion

Cassandra

NoSQL
neo4j


NoSQL

» the era of Polyglot Persistence

» the Tower of Bable

» the (B)Le(e|a)ding Edge


NoSQL typology

» Key/Value stores

» Document Databases

» Column (Family) Databases
C

» Graph Databases


NoSQL tool selection

» the luxury of choice
(but remember polyglot persistence)

» survival of the fittest

» inflated expectations + nifty marketing

NOTE If your data fits in single node RAM
memory, DON’T go NoSQL (just yet)


C

Requirements, phase I
» automatic scaling to large data sets

» fault-tolerance: replication, automatic handling of failing nodes

» a ﬂexible data model supporting sparse data

» runs on commodity hardware

» efﬁcient random access to data

» open source, ability to participate in the development thus
drive the direction of the project
» some preference for a Java-based solution


C

Requirements, phase II

» After careful consideration, we realized the
important choices were also:
» consistency: no chance of having two conﬂicting
versions of a row
» atomic updates of a single row, single-row
transactions
» bonus points for MapReduce integration
» e.g. full-text index rebuilding


That brought us to HBase, which bought us:
» a datamodel where you can have column
families which keep all versions and others
which do not, which ﬁts very well on our
CMS document model
» ordered tables with the ability to do range
scans on them, which allows to build
scalable indexes on top of it
» HDFS, a convenient place to store large blobs

» Apache license and community, a familiar
environment for us


HBase

» hbase.apache.org + Cloudera CDH distro

» Open Source (Google) BigTable
implementation
» HDFS as underlying DFS (≈GFS)

» ZooKeeper as lock service (≈Chubby)

» Integration with Hadoop MapReduce


BigTable
column family

{
"contents:" "anchor:cnnsi.com" "anchor:my.look.ca"

"com.cnn.www"
"<html>..."
"<html>..."
"<html>..." t6
t5
t3
"CNN" t9 "CNN.com" t8
} row

ure 1: A slice of an example table that stores Web pages. The row name is a reversed URL. The contents column family con-
the page contents, and the anchor column family contains the text of any anchors that reference the page. CNN’s home page
key cell
ferenced by both the Sports Illustrated and the MY-look home pages, so the row contains columns named anchor:cnnsi.com
anchor:my.look.ca. Each anchor cell has one version; the contents column has three versions, at timestamps t 3 , t5 , and t6 .

We settled on this data model after examining a variety Column Families
otential uses of a Bigtable-like system. As one con- 3

e example that drove some ofTECHNOLOGIEPARKdecisions,ZWIJNAARDE (GENT) » are grouped into sets called column fami-
IIC »
our design 3 » B-9052 Column keys www.outerthought.org

Data Model
HBase Datamodel
•
» Sparse, multi-dimensional map map
Sparse, multi dimensional
(row, column, timestamp) → cell cell
(row, column, timestamp)

•
» Column = Column Family:Column Qualiﬁer
Column = Column Family:Column Qualiﬁer
Columns
Fam1:Qual1

Rows
t1
AK v1 t2
v2
Timestamps

t2>t1
7
Tuesday, August 17, 2010

Regions
» Lexicographically sorted set of rows
» default size : 256MB

» Hosted by region servers
row 1

row 200
split
row 201

row 350

writes


Storage architecture

© lars george


Storage organisation
Region

Memstore

HLog
(Append only
WAL on HDFS)
HFile HFile
(on HDFS) (on HDFS)
(Sequence File)
(one per RS)

Region

HFile: Immutable sorted map (byte[] byte[])
(row, column, timestamp) cell value

© Amandeep Khurana

14

Writing
Region
Write
Memstore

HLog
(Append only
WAL on HDFS)
HFile HFile
(on HDFS) (on HDFS)
(Sequence File)
(one per RS)

Region


© Amandeep Khurana

14

Flush
Region

Memstore Flush

HLog
(Append only Small
WAL on HDFS)
HFile HFile
(Sequence File)
(on HDFS) (on HDFS) HFile
(one per RS)

Region


© Amandeep Khurana

14

Compaction
Region

Memstore

HLog
(Append only Small
WAL on HDFS)
HFile HFile
(Sequence File)
(on HDFS) (on HDFS) HFile
(one per RS) Compaction
Region


© Amandeep Khurana

14

Stable
Region

Memstore

HLog
(Append only
WAL on HDFS)
HFile HFile HFile
(on HDFS) (on HDFS) (on HDFS)
(Sequence File)
(one per RS)

Region

© Amandeep Khurana

15

Reading
Region

Read
Memstore

HLog
(Append only
WAL on HDFS)
HFile HFile HFile
(on HDFS) (on HDFS) (on HDFS)
(Sequence File)
(one per RS)

Region

© Amandeep Khurana

15

HBase APIs

» Java

» REST

» Thrift

» Ruby shell

» Java M/R


HBase Java API

» Get
(byte arrays, mostly)
» Put

» Scan

» Delete

» MapReduce Source / Sink


Interesting HBase-related
projects

» AvroHBase Avro: Hadoop RPC + ser/deser

» HBasene

» HBase Explorer

» asyncHbase


» OK, so now we have a data store !


» However, content repository =
store + search !
u ch
o


a s
w
t !
h a
T asy ...)
e er
w ev
(h o


Search ponderings

» CMS = two types of search
» structured, ‘logic’ search
» numbers, strings
» based on logic (SQL, anyone?)

» information retrieval (or: full-text search)
» text
» based on statistics


Search ponderings

» All of that, at scale


Structured Search
» HBase Indexing Library
» idea from Google App Engine datastore indexes
» http://code.google.com/appengine/articles/
index_building.html

rowkey col col rowkey col

order
A val3 foo6 val2-B

B val2 foo7 val3-A

content table index table A


Full-text / IR search

» Lucene?
» no sharding (for scale)
» no replication (for availability)
» batched index updates (not real-time)


Beyond Lucene
» Katta
» scalable architecture, however only search, no indexing

» Elastic Search
» very young (sorry)

» hbasene et al.
» stores inverted index in HBase, might not scale all features

» SOLR
» widely used, schema, facets, query syntax, cloud branch


?
+
=
r ?
! O
as y
E

Remember distribution ?
Remember secondary indexes ?

➙ Need for reliable queuing


Connecting things
» we needed a reliable bridge between our
main storage (HBase) and our index/search
server(s) (SOLR)
» indexing, reindexing, mass reindexing (M/R)

» we need a reliable method of updating
HBase secondary indexes
» all of that eventually to run distributed

» distribution means coping with failure


Solution

» ... a QUEUE ! Meh.

» ACMEMessageQueue ? Bzzzzzt.
We wanted fault-safe HBase persistence for
the queues.
Also for ease of administration.
» ➙ WAL & Queue implemented on top of
HBase tables


WAL / Queue
» WAL » Queue
» guaranteed execution » triggering of async
of synchronous actions actions
» call doesn’t return before » e.g. (re)index (updated)
secondary action ﬁnishes record with SOLR back-end
» e.g. update secondary actions » size depends on speed of
» if all goes well, back-end process
size = #concurrent ops
» useful outside of Lily context
as well!


The Sum
» Lily model (records & ﬁelds)

» mapped onto HBase (=storage)

» indexed and searchable through
SOLR
» using a WAL/Queue mechanism
implemented in HBase
» runtime based on Kauri

» with client/server comms via Avro
(and a REST interface with JSON)


Lily Content Model

» Records > Fields

» Field types: the usual base types + blobs + link
ﬁelds
» ... so we can model relationships again
(and have free versioning while at it)


Architecture

Roadmap

» Available now = learning material
(architecture, model, API, Javadoc)
+ developer playground ‘proof of architecture’
➥ www.lilycms.org

» End of October = fully distributed release re!
the
early
» from there on, ca. 3-monthly releases N

leading up to Lily 1.0


License

» Apache


Documentation


Questions?

http://www.ﬂickr.com/photos/leehaywood/4237636853/


Thanks for your
hospitality and
attention !

THIS NOTEBOOK BELONGS TO:
» stevenn@outerthought.org

Noteblock_03.indd 1 23/05/10 14:42
» @stevenn


Building a CMS on top of NoSQL (for ParisJUG)

More Related Content

Similar to Building a CMS on top of NoSQL (for ParisJUG) (20)

More from NGDATA (8)

Recently uploaded (20)

Building a CMS on top of NoSQL (for ParisJUG)

Editor's Notes