C* Summit EU 2013: Cassandra Internals

CASSANDRA EU 2013

CASSANDRA
INTERNALS

Aaron Morton

@aaronmorton

!

Co-Founder & Principal Consultant

www.thelastpickle.com
Licensed under a Creative Commons Attribution-NonCommercial 3.0 New Zealand License

#CassandraEU

About The Last Pickle.

Work with clients to deliver and improve
Apache Cassandra based solutions.

Apache Cassandra Committer, DataStax MVP,
Hector Maintainer, Apache Usergrid Committer.
Based in New Zealand & Austin, TX.


#CassandraEU

Architecture
Code

#CassandraEU

Cassandra Architecture.
Clients

API's

Cluster Aware

Cluster Unaware

Disk


#CassandraEU

Cassandra Cluster Architecture.
Clients

API's

Cluster Aware

Cluster Aware

Cluster Unaware

Cluster Unaware

Disk

Disk

Node 1


API's

Node 2

#CassandraEU

Dynamo Cluster Architecture.
Clients

API's

Dynamo

Dynamo

Database

Database

Disk

Disk

Node 1


API's

Node 2

#CassandraEU

Architecture
API
Dynamo
Database

#CassandraEU

API Transports.
!

Thrift
Native Binary
!


#CassandraEU

Thrift Transport.
!

//Custom TServer implementations
o.a.c.thrift.CustomTThreadPoolServer
o.a.c.thrift.CustomTHsHaServer


#CassandraEU

API Transports.


Thrift
Native Binary

#CassandraEU

Native Binary Transport.
!

Beta in Cassandra 1.2, now GA.
Uses Netty.
CQL 3 only.

#CassandraEU

o.a.c.transport.Server.run()
!

//Setup the Netty server
new ExecutionHandler()
new NioServerSocketChannelFactory()
ServerBootstrap.setPipelineFactory()


#CassandraEU

o.a.c.transport.Message.Dispatcher.messageReceived()
!

//Process message from client
ServerConnection.validateNewMessage()
Request.execute()
ServerConnection.applyStateTransition()
Channel.write()


#CassandraEU

Messages.
!

Deﬁned in the Native Binary
Protocol
$SRC/doc/native_protocol.spec


#CassandraEU

API Services.
!

JMX
Thrift
CQL 3
!


#CassandraEU

JMX Management Beans.
!

Spread around the code
base.
Interfaces named *MBean

#CassandraEU

JMX Management Beans.
!

Registered with names such
as org.apache.cassandra.db:
type=StorageProxy


#CassandraEU

o.a.c.thrift.CassandraServer
!

// Implements Thrift Interface
// Access control
// Input validation
// Mapping to/from Thrift and internal types


#CassandraEU

Thrift Interface.
!

Thrift IDL

$SRC/interface/cassandra.thrift


#CassandraEU

o.a.c.thrift.CassandraServer.get_slice()
!

// get columns for one row
Tracing.begin()
ClientState cState = state()
cState.hasColumnFamilyAccess()

multigetSliceInternal()
!


#CassandraEU

CassandraServer.multigetSliceInternal()
!

// get columns for may rows
ThriftValidation.validate*()
// Create ReadCommands
getSlice()
!


#CassandraEU

CassandraServer.getSlice()
!

// Process ReadCommands
// return Thrift types
!

readColumnFamily()
thriftifyColumnFamily()
!


#CassandraEU

CassandraServer.readColumnFamily()
!

// Process ReadCommands
// Return ColumnFamilies
!

StorageProxy.read()
!


#CassandraEU

o.a.c.cql3.QueryProcessor
!

// Prepares and executes CQL3 statements
// Used by Thrift & Native transports
// Access control
// Input validation
// Returns transport.ResultMessage
!
!


#CassandraEU

CQL3 Grammar.
!

ANTLR Grammar
$SRC/o.a.c.cql3/Cql.g


#CassandraEU

o.a.c.cql3.statements.ParsedStatement
!

// Subclasses generated by ANTLR
// Tracks bound term count
// Prepare CQLStatement
prepare()


#CassandraEU

o.a.c.cql3.statements.CQLStatement
!

checkAccess(ClientState state)
validate(ClientState state)
execute(ConsistencyLevel cl,

QueryState state,

List<ByteBuffer> variables)


#CassandraEU

statements.SelectStatement.RawStatement
!

// Implements ParsedStatement
// Input validation
prepare()


#CassandraEU

statements.SelectStatement.execute()
!

// Create ReadCommands
StorageProxy.read()


#CassandraEU

Dynamo Layer.

o.a.c.service
o.a.c.net
!

o.a.c.dht
o.a.c.gms
o.a.c.locator
o.a.c.stream

#CassandraEU

o.a.c.service.StorageProxy
!

// Cluster wide storage operations
// Select endpoints & check CL available
// Send messages to Stages
// Wait for response
// Store Hints

#CassandraEU

o.a.c.service.StorageService
!

// Ring operations
// Track ring state
// Start & stop ring membership
// Node & token queries


#CassandraEU

o.a.c.service.IResponseResolver
!

preprocess(MessageIn<T> message)
resolve() throws
DigestMismatchException
!

RowDigestResolver
RowDataResolver
RangeSliceResponseResolver


#CassandraEU

Response Handlers / Callback.

implements IAsyncCallback<T>
!

response(MessageIn<T> msg)
!


#CassandraEU

o.a.c.service.ReadCallback.get()
//Wait for blockfor & data response
condition.await(timeout,

TimeUnit.MILLISECONDS)
!

throw ReadTimeoutException()
!

resolver.resolve()

#CassandraEU

o.a.c.service.StorageProxy.fetchRows()
!

getLiveSortedEndpoints()
new RowDigestResolver()
new ReadCallback()
MessagingService.sendRR()
--------------------------------------ReadCallback.get() # blocking
catch (DigestMismatchException ex)
catch (ReadTimeoutException ex)

#CassandraEU

Dynamo Layer
!

o.a.c.service
o.a.c.net
!

o.a.c.dht
o.a.c.gms
o.a.c.locator
o.a.c.stream

#CassandraEU

o.a.c.net.MessagingService.verb<<enum>>
!

MUTATION
READ
REQUEST_RESPONSE
TREE_REQUEST
TREE_RESPONSE
(And more...)

#CassandraEU

o.a.c.net.MessagingService.verbHandlers
!

new EnumMap<Verb,

IVerbHandler>(Verb.class)


#CassandraEU

o.a.c.net.IVerbHandler<T>
!

doVerb(MessageIn<T> message,

String id);
!


#CassandraEU

o.a.c.net.MessagingService.verbStages
!

new EnumMap<MessagingService.Verb,

Stage>(MessagingService.Verb.class)


#CassandraEU

o.a.c.net.MessagingService.receive()
!

runnable = new MessageDeliveryTask(

message, id, timestamp);
!

StageManager.getStage(

message.getMessageType());
!

stage.execute(runnable);

#CassandraEU

o.a.c.net.MessageDeliveryTask.run()
!

// If dropable and rpc_timeout
MessagingService.incrementDroppedMessages(v
erb);
return;
!

MessagingService.getVerbHandler(verb)
verbHandler.doVerb(message, id)

#CassandraEU

Architecture
API Layer
Dynamo Layer
Database Layer

#CassandraEU

Database Layer
!

o.a.c.concurrent
o.a.c.db
!

o.a.c.cache
o.a.c.io
o.a.c.trace

#CassandraEU

o.a.c.concurrent.StageManager
!

stages = new EnumMap<Stage,
ThreadPoolExecutor>(Stage.class);
!

getStage(Stage stage)


#CassandraEU

o.a.c.concurrent.Stage
!

READ
MUTATION
GOSSIP
REQUEST_RESPONSE
ANTI_ENTROPY
(And more...)

#CassandraEU

Database Layer.

o.a.c.concurrent
o.a.c.db
!

o.a.c.cache
o.a.c.io
o.a.c.trace

#CassandraEU

o.a.c.db.Table
!

// Keyspace
open(String table)
getColumnFamilyStore(String cfName)
!

getRow(QueryFilter ﬁlter)
apply(RowMutation mutation,

boolean writeCommitLog)

#CassandraEU

o.a.c.db.ColumnFamilyStore
!

// Column Family
getColumnFamily(QueryFilter ﬁlter)
getTopLevelColumns(...)
!

apply(DecoratedKey key,

ColumnFamily columnFamily,

SecondaryIndexManager.Updater

indexer)

#CassandraEU

o.a.c.db.IColumnContainer
!

addColumn(IColumn column)
remove(ByteBuffer columnName)
!

ColumnFamily
SuperColumn
!

(Removed in 2.0)

#CassandraEU

o.a.c.db.ISortedColumns
!

addColumn(IColumn column,

Allocator allocator)
removeColumn(ByteBuffer name)
!

ArrayBackedSortedColumns
AtomicSortedColumns
TreeMapBackedSortedColumns

#CassandraEU

o.a.c.db.Memtable
!

put(DecoratedKey key,

ColumnFamily columnFamily,
SecondaryIndexManager.Updater
indexer)
!

ﬂushAndSignal(CountDownLatch latch,

Future<ReplayPosition>

context)

#CassandraEU

o.a.c.db.ReadCommand
!

getRow(Table table)
!

SliceByNamesReadCommand
SliceFromReadCommand
RangeSliceCommand
(Additional classes for paging in 2.0)


#CassandraEU

o.a.c.db.IDiskAtomFilter
!

getMemtableColumnIterator(...)
getSSTableColumnIterator(...)
!

IdentityQueryFilter
NamesQueryFilter
SliceQueryFilter


#CassandraEU

Summary
CustomTThreadPoolServer

Message.Dispatcher

CassandraServer

API

QueryProcessor

Dynamo

ReadCommand
StorageProxy
IResponseResolver
IAsyncCallback
MessagingService
IVerbHandler

Table


ColumnFamilyStore

IDiskAtomFilter

Database

#CassandraEU

Thanks.

!


#CassandraEU

Aaron Morton

@aaronmorton


!

Licensed under a Creative Commons Attribution-NonCommercial 3.0 New Zealand License

C* Summit EU 2013: Cassandra Internals

More Related Content

What's hot (18)

Viewers also liked (6)

Similar to C* Summit EU 2013: Cassandra Internals (19)

More from DataStax Academy (20)

Recently uploaded (20)

C* Summit EU 2013: Cassandra Internals