SlideShare a Scribd company logo
In-memory database
Problems
- Multiple Redis instances on 1 server
- Each instance contains some types of data
- 200000 users active /s
- About 5 milions records in memory
- Master-Slave structures
* Redis 1 instances → 1 core → Bottleneck read/write
* No cluster
* Scale out ???
In-memory database
Requirements
● Horizontal Scale
● Fast read for real time access
● Use full core of CPU
● High availability
Replacements
● Aerospike
● Redis cluster
● Apache ignite
Aerospike
● Written in C
● Opensource
● Community Edition (free) and Enterprise
Edititon
● 1177 stars on Github
Aerospike
● Sharding supported
● ACID (Atomicity, Consistency, Isolation, Durability)
●
Data can be saved to disk or RAM or hybrid storage
(indexes in RAM, data in disk)
● Self managing
● Optimized for SSD
● Reduce money (Less server, SSD is more cheaper than
RAM)
● Proved by AppNexus, Inmobi,...
In-memory database
Aerospike
● Multicast
● No master/slave node. Each node contains
some different primary shards and some
different replica shards
Storage
● Pure DRAM without persistence
● DRAM with persistence
● Flash storage (SSDs)
Namespace
– 1 database = many namespaces
– How data is stored: RAM or DISK
– How many replicas
– When record expire
Sets
● Group of records
● 1 namespace = many sets
Record
● Key
● Meta-data (generation, time to live)
● Bins (1 or more bins) → as rows and their value
in DBMS
Apache Ignite
Many features
● Written in java
● In-memory database
● Data grid
● Accelator for Hadoop and Spark
….
In-memory database
In-memory database
Shared RDDs among Spark jobs
Indexed SQL Query
● SparkSQL does not index RDD
● IgniteSQL can index its cached data by field or
using lucene index for text.
Examples:
val cacheRdd = igniteContext.fromCache("partitioned")
cacheRdd.savePairs(sparkContext.parallelize(1 to 10000,
10).map(i => (i, i)))
val cacheRdd = igniteContext.fromCache("partitioned")
val result = cacheRdd.sql(
"select _val from Integer where val > ? and val < ?",10,100)
public class Person implements Serializable {
/** Person ID (indexed). */
@QuerySqlField(index = true)
private long id;
/** First name (not-indexed). */
@QuerySqlField
private String name;
/** Resume text (create LUCENE-based TEXT index for this
field). */
@QueryTextField
private String resume;
/** Salary (indexed). */
@QuerySqlField(index = true)
private double salary;
…..
}

More Related Content

PPTX
in-memory database system and low latency
PPTX
In-memory Databases
PPTX
In memory databases presentation
PPTX
WiredTiger Overview
ODP
Efficient data maintaince in GlusterFS using Databases
PPTX
Some key value stores using log-structure
PPTX
What'sNnew in 3.0 Webinar
PDF
A Technical Introduction to WiredTiger
in-memory database system and low latency
In-memory Databases
In memory databases presentation
WiredTiger Overview
Efficient data maintaince in GlusterFS using Databases
Some key value stores using log-structure
What'sNnew in 3.0 Webinar
A Technical Introduction to WiredTiger

What's hot (20)

PPTX
Get More Out of MongoDB with TokuMX
ODP
Gluster Data Tiering
PPTX
Get More Out of MySQL with TokuDB
POTX
WiredTiger MongoDB Integration
PPTX
WiredTiger Overview
PPT
Redis e Memcached - Daniel Naves - Omnilogic
PDF
Ten Reasons Why You Should Prefer PostgreSQL to MySQL
PPTX
Introduction to TokuDB v7.5 and Read Free Replication
PPTX
Percona FT / TokuDB
PDF
MongoDB 3.0 and WiredTiger (Event: An Evening with MongoDB Dallas 3/10/15)
PPTX
A Technical Introduction to WiredTiger
PPTX
An introduction to SQL Server in-memory OLTP Engine
PDF
Optimizing RocksDB for Open-Channel SSDs
PDF
State of Gluster Performance
PPTX
Presentation db2 best practices for optimal performance
PDF
Storage in hadoop
PDF
Optimizing columnar stores
PDF
PGDay.Seoul 2016 lightingtalk
PPTX
IMC Summit 2016 Breakout - Brian Bulkowski - NVMe, Storage Class Memory and O...
DOC
IBM DB2 LUW/UDB DBA Training by www.etraining.guru
Get More Out of MongoDB with TokuMX
Gluster Data Tiering
Get More Out of MySQL with TokuDB
WiredTiger MongoDB Integration
WiredTiger Overview
Redis e Memcached - Daniel Naves - Omnilogic
Ten Reasons Why You Should Prefer PostgreSQL to MySQL
Introduction to TokuDB v7.5 and Read Free Replication
Percona FT / TokuDB
MongoDB 3.0 and WiredTiger (Event: An Evening with MongoDB Dallas 3/10/15)
A Technical Introduction to WiredTiger
An introduction to SQL Server in-memory OLTP Engine
Optimizing RocksDB for Open-Channel SSDs
State of Gluster Performance
Presentation db2 best practices for optimal performance
Storage in hadoop
Optimizing columnar stores
PGDay.Seoul 2016 lightingtalk
IMC Summit 2016 Breakout - Brian Bulkowski - NVMe, Storage Class Memory and O...
IBM DB2 LUW/UDB DBA Training by www.etraining.guru
Ad

Viewers also liked (14)

PDF
Plan d`action stratégique en matière de violence conjugale et intrafamiliale ...
PDF
Expense checklist
PPTX
Ley de Acuerdos y Convenios Intergubernativos
PPTX
Arquitectura
DOCX
Cover jadwal sail ok
PDF
Kevin Coughlin Brand Presentation TCNJ 9/23/13
PPTX
Rendición Pública de Cuentas. Final 2015 - Inicial 2016
PPTX
Lucha contra la violencia hacia las mujeres, en números
PPTX
Santa Cruz cómo vamos - Panel 1
PPTX
Lineamientos metodológicos para la formulación de los PTDI
PDF
โครงงานคอมพิวเตอร์
PDF
Marketing in the Trenches
PPT
Plan de Desarrollo Económico y Social 2016 - 2020
PPT
Plan d`action stratégique en matière de violence conjugale et intrafamiliale ...
Expense checklist
Ley de Acuerdos y Convenios Intergubernativos
Arquitectura
Cover jadwal sail ok
Kevin Coughlin Brand Presentation TCNJ 9/23/13
Rendición Pública de Cuentas. Final 2015 - Inicial 2016
Lucha contra la violencia hacia las mujeres, en números
Santa Cruz cómo vamos - Panel 1
Lineamientos metodológicos para la formulación de los PTDI
โครงงานคอมพิวเตอร์
Marketing in the Trenches
Plan de Desarrollo Económico y Social 2016 - 2020
Ad

Similar to In-memory database (20)

PDF
Effectively deploying hadoop to the cloud
PDF
Edge performance with in memory nosql
PPTX
Intro to Apache Spark
PPTX
Intro to Apache Spark
PPTX
Introduction to AWS Big Data
PPT
Apache Spark™ is a multi-language engine for executing data-S5.ppt
PDF
Boston Apache Spark User Group (the Spahk group) - Introduction to Spark - 15...
PPTX
Introduction to Spark
PDF
Ingesting Over Four Million Rows Per Second With QuestDB Timeseries Database ...
PPTX
Apache Spark II (SparkSQL)
PPTX
Hadoop Architecture_Cluster_Cap_Plan
ODP
MySQL HA
PPTX
Apache Spark
PDF
Advanced Spark Programming - Part 1 | Big Data Hadoop Spark Tutorial | CloudxLab
PDF
Introduction to Apache Spark Ecosystem
PDF
Crimson: Ceph for the Age of NVMe and Persistent Memory
PDF
JDD 2016 - Michal Matloka - Small Intro To Big Data
PPTX
Spark Study Notes
PDF
Spark SQL
PDF
Elasticsearch for Logs & Metrics - a deep dive
Effectively deploying hadoop to the cloud
Edge performance with in memory nosql
Intro to Apache Spark
Intro to Apache Spark
Introduction to AWS Big Data
Apache Spark™ is a multi-language engine for executing data-S5.ppt
Boston Apache Spark User Group (the Spahk group) - Introduction to Spark - 15...
Introduction to Spark
Ingesting Over Four Million Rows Per Second With QuestDB Timeseries Database ...
Apache Spark II (SparkSQL)
Hadoop Architecture_Cluster_Cap_Plan
MySQL HA
Apache Spark
Advanced Spark Programming - Part 1 | Big Data Hadoop Spark Tutorial | CloudxLab
Introduction to Apache Spark Ecosystem
Crimson: Ceph for the Age of NVMe and Persistent Memory
JDD 2016 - Michal Matloka - Small Intro To Big Data
Spark Study Notes
Spark SQL
Elasticsearch for Logs & Metrics - a deep dive

Recently uploaded (20)

PDF
How UI/UX Design Impacts User Retention in Mobile Apps.pdf
PDF
NewMind AI Weekly Chronicles - August'25 Week I
PPTX
Understanding_Digital_Forensics_Presentation.pptx
PDF
Network Security Unit 5.pdf for BCA BBA.
PDF
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
PDF
cuic standard and advanced reporting.pdf
PDF
NewMind AI Monthly Chronicles - July 2025
PPTX
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
PDF
GDG Cloud Iasi [PUBLIC] Florian Blaga - Unveiling the Evolution of Cybersecur...
PDF
Unlocking AI with Model Context Protocol (MCP)
PDF
Diabetes mellitus diagnosis method based random forest with bat algorithm
PDF
GamePlan Trading System Review: Professional Trader's Honest Take
PPTX
Detection-First SIEM: Rule Types, Dashboards, and Threat-Informed Strategy
PDF
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
PDF
Review of recent advances in non-invasive hemoglobin estimation
PPT
Teaching material agriculture food technology
PDF
Bridging biosciences and deep learning for revolutionary discoveries: a compr...
PDF
Approach and Philosophy of On baking technology
PDF
Shreyas Phanse Resume: Experienced Backend Engineer | Java • Spring Boot • Ka...
DOCX
The AUB Centre for AI in Media Proposal.docx
How UI/UX Design Impacts User Retention in Mobile Apps.pdf
NewMind AI Weekly Chronicles - August'25 Week I
Understanding_Digital_Forensics_Presentation.pptx
Network Security Unit 5.pdf for BCA BBA.
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
cuic standard and advanced reporting.pdf
NewMind AI Monthly Chronicles - July 2025
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
GDG Cloud Iasi [PUBLIC] Florian Blaga - Unveiling the Evolution of Cybersecur...
Unlocking AI with Model Context Protocol (MCP)
Diabetes mellitus diagnosis method based random forest with bat algorithm
GamePlan Trading System Review: Professional Trader's Honest Take
Detection-First SIEM: Rule Types, Dashboards, and Threat-Informed Strategy
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
Review of recent advances in non-invasive hemoglobin estimation
Teaching material agriculture food technology
Bridging biosciences and deep learning for revolutionary discoveries: a compr...
Approach and Philosophy of On baking technology
Shreyas Phanse Resume: Experienced Backend Engineer | Java • Spring Boot • Ka...
The AUB Centre for AI in Media Proposal.docx

In-memory database

  • 2. Problems - Multiple Redis instances on 1 server - Each instance contains some types of data - 200000 users active /s - About 5 milions records in memory - Master-Slave structures * Redis 1 instances → 1 core → Bottleneck read/write * No cluster * Scale out ???
  • 4. Requirements ● Horizontal Scale ● Fast read for real time access ● Use full core of CPU ● High availability
  • 5. Replacements ● Aerospike ● Redis cluster ● Apache ignite
  • 6. Aerospike ● Written in C ● Opensource ● Community Edition (free) and Enterprise Edititon ● 1177 stars on Github
  • 7. Aerospike ● Sharding supported ● ACID (Atomicity, Consistency, Isolation, Durability) ● Data can be saved to disk or RAM or hybrid storage (indexes in RAM, data in disk) ● Self managing ● Optimized for SSD ● Reduce money (Less server, SSD is more cheaper than RAM) ● Proved by AppNexus, Inmobi,...
  • 9. Aerospike ● Multicast ● No master/slave node. Each node contains some different primary shards and some different replica shards
  • 10. Storage ● Pure DRAM without persistence ● DRAM with persistence ● Flash storage (SSDs)
  • 11. Namespace – 1 database = many namespaces – How data is stored: RAM or DISK – How many replicas – When record expire
  • 12. Sets ● Group of records ● 1 namespace = many sets
  • 13. Record ● Key ● Meta-data (generation, time to live) ● Bins (1 or more bins) → as rows and their value in DBMS
  • 14. Apache Ignite Many features ● Written in java ● In-memory database ● Data grid ● Accelator for Hadoop and Spark ….
  • 17. Shared RDDs among Spark jobs
  • 18. Indexed SQL Query ● SparkSQL does not index RDD ● IgniteSQL can index its cached data by field or using lucene index for text. Examples: val cacheRdd = igniteContext.fromCache("partitioned") cacheRdd.savePairs(sparkContext.parallelize(1 to 10000, 10).map(i => (i, i))) val cacheRdd = igniteContext.fromCache("partitioned") val result = cacheRdd.sql( "select _val from Integer where val > ? and val < ?",10,100)
  • 19. public class Person implements Serializable { /** Person ID (indexed). */ @QuerySqlField(index = true) private long id; /** First name (not-indexed). */ @QuerySqlField private String name; /** Resume text (create LUCENE-based TEXT index for this field). */ @QueryTextField private String resume; /** Salary (indexed). */ @QuerySqlField(index = true) private double salary; ….. }