Database Shrading and cassandra architecture

DATABASE SHARDING AND
CASSANDRA ARCHITECTURE
SOUPIK CHOWDHURY

THE PROBLEM
● Consider a very popular shopping cart application (ex. Amazon)
● Each transaction from the user needs to be maintained in a database.
● Millions of people all around the world are buying items daily.
● A point arises when size of the database exceeds the maximum memory limit of
the server.

SOLUTIONS
● One solution can be introducing more memory or buying bigger machines.
○ There’s an upper limit to the above approach due to hardware constraints.
○ This is called vertical scaling.
● The second solution can be, vertical partitioning i.e moving few columns to a
different table.
○ This is not same as normalization. We can use vertical partitioning even on a normalized
database.
○ This reduces the size of each entry. However again there will be an upper bound
● The third solution can be, horizontally partition the table and store each partition in
a different server.
○ This is also called database sharding.

DATABASE SHARDING
● Process of breaking up a large table into smaller chunks called shards
● This is partitioning the table horizontally,
● Each table will have same schema but entire different rows.
● The partitioning can be done on basis on some key or non key attribute or set of
attributes.
● For example, we can partitioning the data based on geographical location of the
users.

ADVANTAGES OF SHARDING
● Facilitates horizontal scaling on databases. Now there is no upper bound on data
that can get into a particular table.
● Increases request efficiency. Now the same request is served by a smaller version
of the database so is faster.
● For read-write-heavy databases, the efficiency is further increased if sharding is
done based on some factor like geographical location and storing corresponding
shards in servers nearby those locations.

DISADVANTAGES OF SHARDING
● Manual implementation of sharding is not easy. Might not work at all.
● Identifying the property or properties to shard on is the key factor and affects the
performance completely. So choosing wrong set of properties might become costly
than giving advantage. (Example, after sharding, we need to join across shards to
serve the request)
● Might be difficult to return to the unsharded version.
● Not every database supports sharding by default. We need to manually implement
for them. This might become unnecessary complexity when our application is
small.

INTRODUCTION TO CASSANDRA
● Cassandra is a distributed, nosql database management system designed by
Apache.
● It is a peer-to-peer, distributed database that runs on a cluster of homogenous
nodes.
● Handles large volume of data, while providing high availability.
● Provides high read and write throughput.

SOME PROPERTIES OF CASSANDRA
● Distributed - Every node in Cassandra has same role. There is no single point of
failure. Data is distributed across the cluster but there is no master and each node
can service any request.
● Replication - Replication strategies are configurable. This avoids single point of
failure, as if a node goes out of service, the requests gets served by another node.
● Eventual Consistency - Since data is replicated across nodes, we need to ensure
data is synchronized across replicas. Cassandra follows eventual consistency
model which states that provided there are no new updates, all replicas will
eventually return the last updated value.
● Consistent Hashing

CONSISTENT HASHING
● A hash function is a function that maps one piece of data - typically
describing some kind of object, often of arbitrary size to another piece of
data, typically an integer, known as hash code or simply hash.
● Hash functions are widely used. Some examples can be cryptographic
hashing of sensitive information in websites, hashing data to compress
them, hashing strings for substring matching etc.
● Distributed Hashing - The hash table might grow a lot in size. So we can
partition it into several parts and each part is hosted in different servers.

CONSISTENT HASHING (CONTD…)
● Rehashing problem - If one of the server crashes, the keys needs to be
redistributed across the remaining servers. Even if a single server is added or
removed, all keys are likely to get rehashed into different buckets (at least most of
them). Thus, rehashing is expensive in normal distributed hashing.
● Consistent hashing - Solves rehashing problem i.e minimizes the rehashing when
servers are added/removed. Consistent hashing is a distributed hashing scheme
that operates independently of servers or objects in a distributed hash table by
assigning them a position on an abstract circle, or hash ring. This allows servers
and objects to scale without affecting the overall system.

CONSISTENT HASHING (IMPLEMENTATION)
● Map all the hash values on the circle using a proper hash function (that maps to an
angle in [0-2𝝅] radians).
● Map all the servers as well to the circle using the same hash function. (or different
if needed).
● Assign the hash values to the servers that is nearest to it in clockwise direction.

CONSISTENT HASHING IN CASSANDRA
● Consistent hashing is used in Cassandra for sharding and also load balancing.
● Let the database be sharded using key. We use a hash function h , where h(key)
maps to the circle.
● Suppose we have k servers. Map these servers in the circle as well. All the key
values are mapped to the nearest server in clockwise direction.
● Now suppose a request comes with key = key1. We compute h(key1) and then
serve the request using that server which is nearest in clockwise sense.

REPLICATION IN CASSANDRA
● The database admin has to fix a number K (called Replication factor).
● Data corresponding to a given key is stored not only in nearest server in clockwise
direction but nearest K servers.
● If a server goes down, all the requests meant for that server goes to the next server
in clockwise sense.

REFERENCES
● Gaurav Sen’s lecture videos : Cassandra , Consistent Hashing
● Wikipedia : Cassandra , Sharding
● Stackoverflow : Sharding

Database Shrading and cassandra architecture

More Related Content

What's hot (20)

Similar to Database Shrading and cassandra architecture (20)

Recently uploaded (20)

Database Shrading and cassandra architecture