SlideShare a Scribd company logo
Omid: A Transactional Framework for HBase
Francisco Perez-Sorrosal
Ohad Shacham
Hadoop Summit SJ
June 29th, 2016
Outline
 Background
 Basic Concepts
 Use cases
 Architecture
 Transaction Management
 High Availability
 Performance
 Summary
Hadoop Summit SJ (June 29th 2016)2
 New Big data apps → new requirements:
● Low-latency
● Incremental data processing
● e.g. Percolator
 Multiple clients updating same data concurrently
● Problem: Conflicts/Inconsistencies may arise
● Solution: Transactional Access to Data
Background
Hadoop Summit SJ (June 29th 2016)3
 Transaction → Abstract UoW to manage data with certain
guarantees
● ACID
● Relational databases
 Big data → NoSQL datastores → Transactions in NoSQL
● Relaxed Guarantees:
○ e.g. Atomicity, Consistency
Background
● Hard to Scale
○ Data partition
○ Data replication
Hadoop Summit SJ (June 29th 2016)4
 Flexible
 Reliable
 High Performant
 Scalable
…OLTP framework that allows BigData apps to
execute ACID transactions on top of HBase
+ =
Consistency in
BigData Apps
Omid is a…
Hadoop Summit SJ (June 29th 2016)5
Why use Omid?
 Simplifies development of apps requiring consistency
● Multi-row/multi-table transactions on HBase
● Simple & well-known interface
 Good performance & reliability
 Lock-free
 Snapshot Isolation
 HBase is a blackbox
● No HBase code modification
● No changes on table schemas
 Used successfully at Yahoo
Hadoop Summit SJ (June 29th 2016)6
Snapshot Isolation
▪ Transaction T2 overlaps in time with T1 & T3, but spatially:
● T1 ∩ T2 = ∅
● T2 ∩ T3 = { R4 } Transactions T2 and T3 conflict
▪ Transaction T4 does not have conflicts
TxId
T1
T2
T3
T4
Time Overlap Spatial Overlap (WriteSet)
R1 R2 R3 R4
R3 R4
R2 R4
R1 R3
Hadoop Summit SJ (June 29th 2016)7
Sieve
Use Cases: Sieve @ Yahoo
HBase
Internet
Crawler Doc Proc Aggregation
Omid
Feeder
Real-Time
Index
Notifications
Transactional Data Flow
Hadoop Summit SJ (June 29th 2016)8
Hive Metastore Thrift Server
Use Cases:
HBase
HBaseStore
Omid
Hadoop Summit SJ (June 29th 2016)
ObjectStore
Relational
Database
9
Transactional App
Architectural Components
HBase
Omid Client
Transaction Status Oracle
(TSO)
Timestamp
Oracle
Get Start/Commit
Timestamps
Start/Commit TXs
Keep track &
Validate TXs
Commit Table
Compactor
Commit data
R/W data
Guarantee
SI
App Table
Shadow
CellsApp TableApp Table
Shadow
Cells
Hadoop Summit SJ (June 29th 2016)10
Client APIs
▪ Transaction Manager → Create Transactional contexts
Transaction begin();
void commit(Transaction tx);
void rollback(Transaction tx);
▪ Transactional Tables (TTable) → Data access
Result get(Transaction tx, Get g);
void put(Transaction tx, Put p);
ResultScanner getScanner(Transaction tx, Scan s);
Hadoop Summit SJ (June 29th 2016)11
TX Management (Begin TX phase)
Omid Client TSO TO Table/SC CommitTable
Begin TX Get ST
ST=1
TX(ST=1)
R/W Ops for TX (ST=1)
App
Begin TX
R/W Ops (within TX context)
TX Context
R/W Results for TX with ST=1
Read Ops:
Get right results
for TX’s SnapshotWrite Ops:
Build Writeset
for TX
Hadoop Summit SJ (June 29th 2016)12
TX Management (Commit TX Phase)
Omid Client TSO TO Table/SC CommitTable
Commit TX (Writeset)
Get CT
CT=2
TX(CT=2)
App
Commit TX
Check Conflicts
of TX Writeset
in Conflict Map
Persist commit details (ST/CT) for TX
Hadoop Summit SJ (June 29th 2016)13
TX Management (Complete TX Phase)
Omid Client TSO TO Table/SC CommitTable
Update SC for TX (ST=1/CT=2)
App
Complete commit (Cleanup entry for TX with ST=1)
Result
Hadoop Summit SJ (June 29th 2016)14
Transactional App
High Availability
HBase
Omid Client
Transaction Status Oracle
Timestamp
Oracle
Get Start/Commit
Timestamps
Start/Commit TXs
Commit Table
Compactor
Commit data
R/W data
Guarantee
SI
App Table
Shadow
CellsApp TableApp Table
Shadow
Cells
Single
point of
failure
Hadoop Summit SJ (June 29th 2016)15
Timestamp
Oracle
Transaction Status Oracle
Transactional App
High Availability
HBase
Omid Client
Transaction Status Oracle
Timestamp
Oracle
Get Start/Commit
Timestamps
Start/Commit TXs
Commit Table
Compactor
Commit data
R/W data
Guarantee
SI
App Table
Shadow
CellsApp TableApp Table
Shadow
CellsRecovery
State
Primary
/
Backup
Hadoop Summit SJ (June 29th 2016)16
High Availability – Failing Scenario
Omid Client TSO P TSO B Table/SC CommitTableApp
Begin TX
Begin TX Get ST
ST=1
TX(ST=1)
TX 1
TO
Data Store Commit Table
Write(k1, v1) (ST=1)
TX 1 Write(k1, v1)
(k1, v1, 1)
Hadoop Summit SJ (June 29th 2016)17
High Availability – Failing Scenario
Omid Client TSO P TSO B Table/SC CommitTableApp TO
Data Store Commit Table
Write(k2, v2) (ST=1)
Write(k2, v2)
(k1, v1, 1)
(k2, v2, 1)
Commit TX 1{k1, k2}
Commit TX 1
Get CT
CT=2
Persist commit details for TX 1
Hadoop Summit SJ (June 29th 2016)18
High Availability – Failing Scenario
Omid Client TSO B Table/SC CommitTableApp
Begin TX
Begin TX Get ST
ST=3
TX(ST=3)
TX 3
TO
Data Store Commit Table
Read(k1) (ST=3)
TX 3 Read(k1)
(k1, v1, 1)
(k1, v1, 1)
(k2, v2, 1)Hadoop Summit SJ (June 29th 2016)19
High Availability – Failing Scenario
Omid Client TSO B Table/SC CommitTableApp TO
Data Store Commit Table
Return TX 1 CT
(k1, v1, 1)
! exist
! exist
Read(k2) (ST=3)
(k2, v2, 1)
TX 3 Read(k2)
(k2, v2, 1)
CT = 2
Return TX 1 CT
v2
(1, 2)Hadoop Summit SJ (June 29th 2016)20
Timestamp
Oracle
Transaction Status Oracle
Transactional App
High Availability
HBase
Omid Client
Transaction Status Oracle
Timestamp
Oracle
Get Start/Commit
Timestamps
Start/Commit TXs
Commit Table
Compactor
R/W data
Guarantee
SI
App Table
Shadow
CellsApp TableApp Table
Shadow
CellsRecovery
State
Hadoop Summit SJ (June 29th 2016)21
High Availability – Solution
Omid Client TSO P TSO B Table/SC CommitTableApp
Begin TX
Begin TX Get ST
ST=1
TX(ST=1,E=1)
TX 1, 1
TO
Data Store Commit Table
Write(k1, v1) (ST=1)
TX 1 Write(k1, v1)
(k1, v1, 1)
Hadoop Summit SJ (June 29th 2016)22
High Availability – Solution
Omid Client TSO P TSO B Table/SC CommitTableApp TO
Data Store Commit Table
Write(k2, v2) (ST=1)
Write(k2, v2)
(k1, v1, 1)
(k2, v2, 1)
Commit TX 1{k1, k2}
Commit TX 1
Get CT
CT=2
Persist commit details for TX 1
Hadoop Summit SJ (June 29th 2016)23
High Availability – Solution
Omid Client TSO B Table/SC CommitTableApp
Begin TX
Begin TX Get ST
ST=3
TX(ST=3,E=3)
TX 3,3
TO
Data Store Commit Table
Read(k1) (ST=3)
TX 3 Read(k1)
(k1, v1, 1)
(k1, v1, 1)
(k2, v2, 1)Hadoop Summit SJ (June 29th 2016)24
High Availability – Solution
Omid Client TSO B Table/SC CommitTableApp TO
Data Store Commit Table
Return TX1 CT
(k1, v1, 1)
! exist
(k2, v2, 1)
Invalid
Try invalidate
(1, -, invalid)
! exist
Read(k2) (ST=3)
(k2, v2, 1)
TX 3 Read(k2)
Hadoop Summit SJ (June 29th 2016)25
High Availability – Solution
Omid Client TSO B Table/SC CommitTableApp TO
Data Store Commit Table
Return TX 1 CT
(k1, v1, 1)
! exist
! exist
(k2, v2, 1)
(1, 2, invalid)Hadoop Summit SJ (June 29th 2016)26
High Availability
 No runtime overhead in mainstream execution
• Minor overhead after failover
 TSO uses regular writes
 Leases for leader election
• Lease status check before/after writing to Commit Table
Hadoop Summit SJ (June 29th 2016)27
Perf. Improvements: Read-Only Txs
Omid Client TSO/TO Table/SC
Begin TX
TX(ST=1)
Read Ops for TX (ST=1)
App
Begin TX
Read Ops (in TX context)
TX Context
Read Results in Snapshot
Commit TX
Writeset is ∅, so no need to contact TSO!!!Success
Hadoop Summit SJ (June 29th 2016)28
TSO
HBase
Perf. Improvements: Commit Table Writes
Omid
Client
HBase
TSO
Commit
Table
Commit
Data
Omid
Client
Commit
Data
Hadoop Summit SJ (June 29th 2016)29
HBase
TSO
Perf. Improvements: Commit Table Writes
Omid
Client
HBase
TSO
Commit
Table
Commit
Data
Omid
Client
Commit
Data
Hadoop Summit SJ (June 29th 2016)30
0
50
100
150
200
250
300
350
400
1 2 4 6
Tps*103
Commit Table: # Region servers
Omid Throughput with Improvements
Hadoop Summit SJ (June 29th 2016)31
Summary
 Transactions in NoSQL
• Use cases in incremental big data processing
• Snapshot Isolation: Scalable consistency model
 Omid
• Web-scale TPS for HBase
• Reliable and performant
• Battle-tested
• http://omid.incubator.apache.org/
Hadoop Summit SJ (June 29th 2016)32
Questions?
Hadoop Summit SJ (June 29th 2016)33

More Related Content

PPTX
Foundations of streaming SQL: stream & table theory
PPTX
Google cloud Dataflow & Apache Flink
PDF
Parallelization of Structured Streaming Jobs Using Delta Lake
PPTX
Apache Beam: A unified model for batch and stream processing data
PPTX
Re-envisioning the Lambda Architecture : Web Services & Real-time Analytics ...
PDF
Google Cloud Dataflow and lightweight Lambda Architecture for Big Data App
PPTX
Yahoo - Moving beyond running 100% of Apache Pig jobs on Apache Tez
Foundations of streaming SQL: stream & table theory
Google cloud Dataflow & Apache Flink
Parallelization of Structured Streaming Jobs Using Delta Lake
Apache Beam: A unified model for batch and stream processing data
Re-envisioning the Lambda Architecture : Web Services & Real-time Analytics ...
Google Cloud Dataflow and lightweight Lambda Architecture for Big Data App
Yahoo - Moving beyond running 100% of Apache Pig jobs on Apache Tez

What's hot (20)

PDF
Big Data Tools in AWS
PPTX
Next Gen Big Data Analytics with Apache Apex
PDF
IEEE International Conference on Data Engineering 2015
PDF
Unified, Efficient, and Portable Data Processing with Apache Beam
PDF
Large-Scale Stream Processing in the Hadoop Ecosystem
PPTX
Discover How IBM Uses InfluxDB and Grafana to Help Clients Monitor Large Prod...
PPTX
Lambda architecture with Spark
PPTX
Data Pipeline at Tapad
PPTX
Real-Time Data Pipelines with Kafka, Spark, and Operational Databases
PDF
Why is My Stream Processing Job Slow? with Xavier Leaute
PDF
Change Data Feed in Delta
PDF
Large-Scale Stream Processing in the Hadoop Ecosystem
PDF
Arbitrary Stateful Aggregations using Structured Streaming in Apache Spark
PDF
Large-Scale Stream Processing in the Hadoop Ecosystem - Hadoop Summit 2016
PPTX
[Pulsar summit na 21] Change Data Capture To Data Lakes Using Apache Pulsar/Hudi
PPTX
Unified Batch & Stream Processing with Apache Samza
PDF
Bellevue Big Data meetup: Dive Deep into Spark Streaming
PDF
Realtime Reporting using Spark Streaming
PDF
Delta from a Data Engineer's Perspective
PPTX
Scaling Cloud-Scale Translytics Workloads with Omid and Phoenix
Big Data Tools in AWS
Next Gen Big Data Analytics with Apache Apex
IEEE International Conference on Data Engineering 2015
Unified, Efficient, and Portable Data Processing with Apache Beam
Large-Scale Stream Processing in the Hadoop Ecosystem
Discover How IBM Uses InfluxDB and Grafana to Help Clients Monitor Large Prod...
Lambda architecture with Spark
Data Pipeline at Tapad
Real-Time Data Pipelines with Kafka, Spark, and Operational Databases
Why is My Stream Processing Job Slow? with Xavier Leaute
Change Data Feed in Delta
Large-Scale Stream Processing in the Hadoop Ecosystem
Arbitrary Stateful Aggregations using Structured Streaming in Apache Spark
Large-Scale Stream Processing in the Hadoop Ecosystem - Hadoop Summit 2016
[Pulsar summit na 21] Change Data Capture To Data Lakes Using Apache Pulsar/Hudi
Unified Batch & Stream Processing with Apache Samza
Bellevue Big Data meetup: Dive Deep into Spark Streaming
Realtime Reporting using Spark Streaming
Delta from a Data Engineer's Perspective
Scaling Cloud-Scale Translytics Workloads with Omid and Phoenix
Ad

Viewers also liked (20)

PDF
Omid Efficient Transaction Mgmt and Processing for HBase
KEY
Real Time BI with Hadoop
PDF
IoT Crash Course Hadoop Summit SJ
PPTX
Using Hadoop for Cognitive Analytics
PDF
Making the leap to BI on Hadoop by Mariani, dave @ atscale
PPTX
Curb your insecurity with HDP
PPTX
The Path to Wellness through Big Data
PPTX
Combining Machine Learning frameworks with Apache Spark
PPTX
What the #$* is a Business Catalog and why you need it
PDF
The Future of Apache Storm
PPTX
HIPAA Compliance in the Cloud
PPTX
Real Time Machine Learning Visualization with Spark
PPTX
Open Source Ingredients for Interactive Data Analysis in Spark
PDF
Machine Learning for Any Size of Data, Any Type of Data
PPTX
A New "Sparkitecture" for modernizing your data warehouse
PPTX
Extreme Analytics @ eBay
PPTX
Faster, Faster, Faster: The True Story of a Mobile Analytics Data Mart on Hive
PPTX
The Future of Apache Hadoop an Enterprise Architecture View
PDF
Intro to Spark with Zeppelin Crash Course Hadoop Summit SJ
PPTX
Accelerating Data Warehouse Modernization
Omid Efficient Transaction Mgmt and Processing for HBase
Real Time BI with Hadoop
IoT Crash Course Hadoop Summit SJ
Using Hadoop for Cognitive Analytics
Making the leap to BI on Hadoop by Mariani, dave @ atscale
Curb your insecurity with HDP
The Path to Wellness through Big Data
Combining Machine Learning frameworks with Apache Spark
What the #$* is a Business Catalog and why you need it
The Future of Apache Storm
HIPAA Compliance in the Cloud
Real Time Machine Learning Visualization with Spark
Open Source Ingredients for Interactive Data Analysis in Spark
Machine Learning for Any Size of Data, Any Type of Data
A New "Sparkitecture" for modernizing your data warehouse
Extreme Analytics @ eBay
Faster, Faster, Faster: The True Story of a Mobile Analytics Data Mart on Hive
The Future of Apache Hadoop an Enterprise Architecture View
Intro to Spark with Zeppelin Crash Course Hadoop Summit SJ
Accelerating Data Warehouse Modernization
Ad

Similar to Omid: A Transactional Framework for HBase (20)

PPTX
Omid: A transactional Framework for HBase
PDF
Apache Kafka, and the Rise of Stream Processing
PDF
Scio - Moving to Google Cloud, A Spotify Story
PDF
Unlocking the Power of Lakehouse Architectures with Apache Pulsar and Apache ...
PDF
Typesafe & William Hill: Cassandra, Spark, and Kafka - The New Streaming Data...
PPTX
Big Data with SQL Server
PPTX
Change Data Capture to Data Lakes Using Apache Pulsar and Apache Hudi - Pulsa...
PDF
NoLambda: Combining Streaming, Ad-Hoc, Machine Learning and Batch Analysis
PDF
Strata Stinger Talk October 2013
PDF
Distributed Database Design Decisions to Support High Performance Event Strea...
PPT
Moving Towards a Streaming Architecture
PPTX
Traitement temps réel chez Streamroot - Golang Paris Juin 2016
PPT
MongoDB at the Silicon Valley iPhone and iPad Developers' Meetup
PPTX
Map Reduce Online
PDF
Event Driven Microservices
PDF
Streaming SQL Foundations: Why I ❤ Streams+Tables
PDF
So you think you can stream.pptx
PDF
Distributed Real-Time Stream Processing: Why and How 2.0
PDF
Streaming Data Lakes using Kafka Connect + Apache Hudi | Vinoth Chandar, Apac...
PPT
HDFS-HC: A Data Placement Module for Heterogeneous Hadoop Clusters
Omid: A transactional Framework for HBase
Apache Kafka, and the Rise of Stream Processing
Scio - Moving to Google Cloud, A Spotify Story
Unlocking the Power of Lakehouse Architectures with Apache Pulsar and Apache ...
Typesafe & William Hill: Cassandra, Spark, and Kafka - The New Streaming Data...
Big Data with SQL Server
Change Data Capture to Data Lakes Using Apache Pulsar and Apache Hudi - Pulsa...
NoLambda: Combining Streaming, Ad-Hoc, Machine Learning and Batch Analysis
Strata Stinger Talk October 2013
Distributed Database Design Decisions to Support High Performance Event Strea...
Moving Towards a Streaming Architecture
Traitement temps réel chez Streamroot - Golang Paris Juin 2016
MongoDB at the Silicon Valley iPhone and iPad Developers' Meetup
Map Reduce Online
Event Driven Microservices
Streaming SQL Foundations: Why I ❤ Streams+Tables
So you think you can stream.pptx
Distributed Real-Time Stream Processing: Why and How 2.0
Streaming Data Lakes using Kafka Connect + Apache Hudi | Vinoth Chandar, Apac...
HDFS-HC: A Data Placement Module for Heterogeneous Hadoop Clusters

More from DataWorks Summit/Hadoop Summit (20)

PPT
Running Apache Spark & Apache Zeppelin in Production
PPT
State of Security: Apache Spark & Apache Zeppelin
PDF
Unleashing the Power of Apache Atlas with Apache Ranger
PDF
Enabling Digital Diagnostics with a Data Science Platform
PDF
Revolutionize Text Mining with Spark and Zeppelin
PDF
Double Your Hadoop Performance with Hortonworks SmartSense
PDF
Hadoop Crash Course
PDF
Data Science Crash Course
PDF
Apache Spark Crash Course
PDF
Dataflow with Apache NiFi
PPTX
Schema Registry - Set you Data Free
PPTX
Building a Large-Scale, Adaptive Recommendation Engine with Apache Flink and ...
PDF
Real-Time Anomaly Detection using LSTM Auto-Encoders with Deep Learning4J on ...
PPTX
Mool - Automated Log Analysis using Data Science and ML
PPTX
How Hadoop Makes the Natixis Pack More Efficient
PPTX
HBase in Practice
PPTX
The Challenge of Driving Business Value from the Analytics of Things (AOT)
PDF
Breaking the 1 Million OPS/SEC Barrier in HOPS Hadoop
PPTX
From Regulatory Process Verification to Predictive Maintenance and Beyond wit...
PPTX
Backup and Disaster Recovery in Hadoop
Running Apache Spark & Apache Zeppelin in Production
State of Security: Apache Spark & Apache Zeppelin
Unleashing the Power of Apache Atlas with Apache Ranger
Enabling Digital Diagnostics with a Data Science Platform
Revolutionize Text Mining with Spark and Zeppelin
Double Your Hadoop Performance with Hortonworks SmartSense
Hadoop Crash Course
Data Science Crash Course
Apache Spark Crash Course
Dataflow with Apache NiFi
Schema Registry - Set you Data Free
Building a Large-Scale, Adaptive Recommendation Engine with Apache Flink and ...
Real-Time Anomaly Detection using LSTM Auto-Encoders with Deep Learning4J on ...
Mool - Automated Log Analysis using Data Science and ML
How Hadoop Makes the Natixis Pack More Efficient
HBase in Practice
The Challenge of Driving Business Value from the Analytics of Things (AOT)
Breaking the 1 Million OPS/SEC Barrier in HOPS Hadoop
From Regulatory Process Verification to Predictive Maintenance and Beyond wit...
Backup and Disaster Recovery in Hadoop

Recently uploaded (20)

PDF
Mobile App Security Testing_ A Comprehensive Guide.pdf
PDF
[발표본] 너의 과제는 클라우드에 있어_KTDS_김동현_20250524.pdf
PDF
Unlocking AI with Model Context Protocol (MCP)
PDF
Chapter 3 Spatial Domain Image Processing.pdf
PPTX
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
PDF
CIFDAQ's Market Insight: SEC Turns Pro Crypto
PDF
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
PDF
Review of recent advances in non-invasive hemoglobin estimation
PPT
Teaching material agriculture food technology
PPTX
MYSQL Presentation for SQL database connectivity
PDF
Advanced Soft Computing BINUS July 2025.pdf
PDF
Modernizing your data center with Dell and AMD
PDF
TokAI - TikTok AI Agent : The First AI Application That Analyzes 10,000+ Vira...
PPTX
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
PDF
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
PDF
Optimiser vos workloads AI/ML sur Amazon EC2 et AWS Graviton
DOCX
The AUB Centre for AI in Media Proposal.docx
PDF
Advanced methodologies resolving dimensionality complications for autism neur...
PPTX
Big Data Technologies - Introduction.pptx
PDF
NewMind AI Weekly Chronicles - August'25 Week I
Mobile App Security Testing_ A Comprehensive Guide.pdf
[발표본] 너의 과제는 클라우드에 있어_KTDS_김동현_20250524.pdf
Unlocking AI with Model Context Protocol (MCP)
Chapter 3 Spatial Domain Image Processing.pdf
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
CIFDAQ's Market Insight: SEC Turns Pro Crypto
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
Review of recent advances in non-invasive hemoglobin estimation
Teaching material agriculture food technology
MYSQL Presentation for SQL database connectivity
Advanced Soft Computing BINUS July 2025.pdf
Modernizing your data center with Dell and AMD
TokAI - TikTok AI Agent : The First AI Application That Analyzes 10,000+ Vira...
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
Optimiser vos workloads AI/ML sur Amazon EC2 et AWS Graviton
The AUB Centre for AI in Media Proposal.docx
Advanced methodologies resolving dimensionality complications for autism neur...
Big Data Technologies - Introduction.pptx
NewMind AI Weekly Chronicles - August'25 Week I

Omid: A Transactional Framework for HBase

  • 1. Omid: A Transactional Framework for HBase Francisco Perez-Sorrosal Ohad Shacham Hadoop Summit SJ June 29th, 2016
  • 2. Outline  Background  Basic Concepts  Use cases  Architecture  Transaction Management  High Availability  Performance  Summary Hadoop Summit SJ (June 29th 2016)2
  • 3.  New Big data apps → new requirements: ● Low-latency ● Incremental data processing ● e.g. Percolator  Multiple clients updating same data concurrently ● Problem: Conflicts/Inconsistencies may arise ● Solution: Transactional Access to Data Background Hadoop Summit SJ (June 29th 2016)3
  • 4.  Transaction → Abstract UoW to manage data with certain guarantees ● ACID ● Relational databases  Big data → NoSQL datastores → Transactions in NoSQL ● Relaxed Guarantees: ○ e.g. Atomicity, Consistency Background ● Hard to Scale ○ Data partition ○ Data replication Hadoop Summit SJ (June 29th 2016)4
  • 5.  Flexible  Reliable  High Performant  Scalable …OLTP framework that allows BigData apps to execute ACID transactions on top of HBase + = Consistency in BigData Apps Omid is a… Hadoop Summit SJ (June 29th 2016)5
  • 6. Why use Omid?  Simplifies development of apps requiring consistency ● Multi-row/multi-table transactions on HBase ● Simple & well-known interface  Good performance & reliability  Lock-free  Snapshot Isolation  HBase is a blackbox ● No HBase code modification ● No changes on table schemas  Used successfully at Yahoo Hadoop Summit SJ (June 29th 2016)6
  • 7. Snapshot Isolation ▪ Transaction T2 overlaps in time with T1 & T3, but spatially: ● T1 ∩ T2 = ∅ ● T2 ∩ T3 = { R4 } Transactions T2 and T3 conflict ▪ Transaction T4 does not have conflicts TxId T1 T2 T3 T4 Time Overlap Spatial Overlap (WriteSet) R1 R2 R3 R4 R3 R4 R2 R4 R1 R3 Hadoop Summit SJ (June 29th 2016)7
  • 8. Sieve Use Cases: Sieve @ Yahoo HBase Internet Crawler Doc Proc Aggregation Omid Feeder Real-Time Index Notifications Transactional Data Flow Hadoop Summit SJ (June 29th 2016)8
  • 9. Hive Metastore Thrift Server Use Cases: HBase HBaseStore Omid Hadoop Summit SJ (June 29th 2016) ObjectStore Relational Database 9
  • 10. Transactional App Architectural Components HBase Omid Client Transaction Status Oracle (TSO) Timestamp Oracle Get Start/Commit Timestamps Start/Commit TXs Keep track & Validate TXs Commit Table Compactor Commit data R/W data Guarantee SI App Table Shadow CellsApp TableApp Table Shadow Cells Hadoop Summit SJ (June 29th 2016)10
  • 11. Client APIs ▪ Transaction Manager → Create Transactional contexts Transaction begin(); void commit(Transaction tx); void rollback(Transaction tx); ▪ Transactional Tables (TTable) → Data access Result get(Transaction tx, Get g); void put(Transaction tx, Put p); ResultScanner getScanner(Transaction tx, Scan s); Hadoop Summit SJ (June 29th 2016)11
  • 12. TX Management (Begin TX phase) Omid Client TSO TO Table/SC CommitTable Begin TX Get ST ST=1 TX(ST=1) R/W Ops for TX (ST=1) App Begin TX R/W Ops (within TX context) TX Context R/W Results for TX with ST=1 Read Ops: Get right results for TX’s SnapshotWrite Ops: Build Writeset for TX Hadoop Summit SJ (June 29th 2016)12
  • 13. TX Management (Commit TX Phase) Omid Client TSO TO Table/SC CommitTable Commit TX (Writeset) Get CT CT=2 TX(CT=2) App Commit TX Check Conflicts of TX Writeset in Conflict Map Persist commit details (ST/CT) for TX Hadoop Summit SJ (June 29th 2016)13
  • 14. TX Management (Complete TX Phase) Omid Client TSO TO Table/SC CommitTable Update SC for TX (ST=1/CT=2) App Complete commit (Cleanup entry for TX with ST=1) Result Hadoop Summit SJ (June 29th 2016)14
  • 15. Transactional App High Availability HBase Omid Client Transaction Status Oracle Timestamp Oracle Get Start/Commit Timestamps Start/Commit TXs Commit Table Compactor Commit data R/W data Guarantee SI App Table Shadow CellsApp TableApp Table Shadow Cells Single point of failure Hadoop Summit SJ (June 29th 2016)15
  • 16. Timestamp Oracle Transaction Status Oracle Transactional App High Availability HBase Omid Client Transaction Status Oracle Timestamp Oracle Get Start/Commit Timestamps Start/Commit TXs Commit Table Compactor Commit data R/W data Guarantee SI App Table Shadow CellsApp TableApp Table Shadow CellsRecovery State Primary / Backup Hadoop Summit SJ (June 29th 2016)16
  • 17. High Availability – Failing Scenario Omid Client TSO P TSO B Table/SC CommitTableApp Begin TX Begin TX Get ST ST=1 TX(ST=1) TX 1 TO Data Store Commit Table Write(k1, v1) (ST=1) TX 1 Write(k1, v1) (k1, v1, 1) Hadoop Summit SJ (June 29th 2016)17
  • 18. High Availability – Failing Scenario Omid Client TSO P TSO B Table/SC CommitTableApp TO Data Store Commit Table Write(k2, v2) (ST=1) Write(k2, v2) (k1, v1, 1) (k2, v2, 1) Commit TX 1{k1, k2} Commit TX 1 Get CT CT=2 Persist commit details for TX 1 Hadoop Summit SJ (June 29th 2016)18
  • 19. High Availability – Failing Scenario Omid Client TSO B Table/SC CommitTableApp Begin TX Begin TX Get ST ST=3 TX(ST=3) TX 3 TO Data Store Commit Table Read(k1) (ST=3) TX 3 Read(k1) (k1, v1, 1) (k1, v1, 1) (k2, v2, 1)Hadoop Summit SJ (June 29th 2016)19
  • 20. High Availability – Failing Scenario Omid Client TSO B Table/SC CommitTableApp TO Data Store Commit Table Return TX 1 CT (k1, v1, 1) ! exist ! exist Read(k2) (ST=3) (k2, v2, 1) TX 3 Read(k2) (k2, v2, 1) CT = 2 Return TX 1 CT v2 (1, 2)Hadoop Summit SJ (June 29th 2016)20
  • 21. Timestamp Oracle Transaction Status Oracle Transactional App High Availability HBase Omid Client Transaction Status Oracle Timestamp Oracle Get Start/Commit Timestamps Start/Commit TXs Commit Table Compactor R/W data Guarantee SI App Table Shadow CellsApp TableApp Table Shadow CellsRecovery State Hadoop Summit SJ (June 29th 2016)21
  • 22. High Availability – Solution Omid Client TSO P TSO B Table/SC CommitTableApp Begin TX Begin TX Get ST ST=1 TX(ST=1,E=1) TX 1, 1 TO Data Store Commit Table Write(k1, v1) (ST=1) TX 1 Write(k1, v1) (k1, v1, 1) Hadoop Summit SJ (June 29th 2016)22
  • 23. High Availability – Solution Omid Client TSO P TSO B Table/SC CommitTableApp TO Data Store Commit Table Write(k2, v2) (ST=1) Write(k2, v2) (k1, v1, 1) (k2, v2, 1) Commit TX 1{k1, k2} Commit TX 1 Get CT CT=2 Persist commit details for TX 1 Hadoop Summit SJ (June 29th 2016)23
  • 24. High Availability – Solution Omid Client TSO B Table/SC CommitTableApp Begin TX Begin TX Get ST ST=3 TX(ST=3,E=3) TX 3,3 TO Data Store Commit Table Read(k1) (ST=3) TX 3 Read(k1) (k1, v1, 1) (k1, v1, 1) (k2, v2, 1)Hadoop Summit SJ (June 29th 2016)24
  • 25. High Availability – Solution Omid Client TSO B Table/SC CommitTableApp TO Data Store Commit Table Return TX1 CT (k1, v1, 1) ! exist (k2, v2, 1) Invalid Try invalidate (1, -, invalid) ! exist Read(k2) (ST=3) (k2, v2, 1) TX 3 Read(k2) Hadoop Summit SJ (June 29th 2016)25
  • 26. High Availability – Solution Omid Client TSO B Table/SC CommitTableApp TO Data Store Commit Table Return TX 1 CT (k1, v1, 1) ! exist ! exist (k2, v2, 1) (1, 2, invalid)Hadoop Summit SJ (June 29th 2016)26
  • 27. High Availability  No runtime overhead in mainstream execution • Minor overhead after failover  TSO uses regular writes  Leases for leader election • Lease status check before/after writing to Commit Table Hadoop Summit SJ (June 29th 2016)27
  • 28. Perf. Improvements: Read-Only Txs Omid Client TSO/TO Table/SC Begin TX TX(ST=1) Read Ops for TX (ST=1) App Begin TX Read Ops (in TX context) TX Context Read Results in Snapshot Commit TX Writeset is ∅, so no need to contact TSO!!!Success Hadoop Summit SJ (June 29th 2016)28
  • 29. TSO HBase Perf. Improvements: Commit Table Writes Omid Client HBase TSO Commit Table Commit Data Omid Client Commit Data Hadoop Summit SJ (June 29th 2016)29
  • 30. HBase TSO Perf. Improvements: Commit Table Writes Omid Client HBase TSO Commit Table Commit Data Omid Client Commit Data Hadoop Summit SJ (June 29th 2016)30
  • 31. 0 50 100 150 200 250 300 350 400 1 2 4 6 Tps*103 Commit Table: # Region servers Omid Throughput with Improvements Hadoop Summit SJ (June 29th 2016)31
  • 32. Summary  Transactions in NoSQL • Use cases in incremental big data processing • Snapshot Isolation: Scalable consistency model  Omid • Web-scale TPS for HBase • Reliable and performant • Battle-tested • http://omid.incubator.apache.org/ Hadoop Summit SJ (June 29th 2016)32
  • 33. Questions? Hadoop Summit SJ (June 29th 2016)33