SlideShare a Scribd company logo
Real Time
Analytics with
HBase
Ivo Mitov
datafusion.eu
2016
Introduction
● Integration gateway
● Authentication, authorization, throttling,
routing, transformation and
orchestration
● Multiple partners
● Multiple backends
● High-volume data - 10 000 TPS
Monitoring and analytics
● Nagios
● Oracle
● Hadoop cluster
● MapReduce jobs
● Lack of real time information:
partner invocations
error rate
latency
HBase
BigTable - sparse, distributed, persistent
multidimensional, sorted map
Column-oriented DBMS on top of HDFS
Row key, column family, column qualifier,
version, value
HBase
HBase Coprocessors
Endpoints
Observers
RegionObserver
RegionServerObserver
WALObserver
MasterObserver
HBase client
Attribute descriptor
Metric definition
filter expression
Channel adapter
Nomenclature manager
HBase gateway
HBase schema
● NТAttribute (name,’a’:value,id)
● NТMetric (name,’m’:[type][attributes],id)
● TEvent ([TID][CID],’e’:[metrics],timestamp)
● TMetric
([MID][rounded TS],’m’:type,agg value)
([MID][rounded TS],
’m’:[type][TID][CIS],value)
TEvent coprocessor
TMetric coprocessor
Thank you!
ivo.mitov@datafusion.eu

More Related Content

PPTX
The future of Big Data tooling
PPT
Big Data: Improving capacity utilization of transport companies
PDF
Data Science Toolchain 101
PPTX
Big Data - Part I
PPTX
Big Data - Part IV
PPTX
Big Data - Part III
PPTX
Big Data - Part II
PPTX
Intro to bigdata on gcp (1)
The future of Big Data tooling
Big Data: Improving capacity utilization of transport companies
Data Science Toolchain 101
Big Data - Part I
Big Data - Part IV
Big Data - Part III
Big Data - Part II
Intro to bigdata on gcp (1)

What's hot (20)

PDF
WSO2Con Asia 2014 - Simultaneous Analysis of Massive Data Streams in real-tim...
PPTX
Introduction of big data unit 1
PPTX
Big data technology unit 3
PPTX
Big Data and Hadoop
PDF
NoSQL Introduction
PPTX
Big Data Unit 4 - Hadoop
PPTX
SQLSat 245 - Por Onde Começar no BigData
PPT
BigData Analytics with Hadoop and BIRT
PDF
Elephant in the room: A DBA's Guide to Hadoop
PPTX
HDFS and Hadoop
PDF
Future of Data - Big Data
PPTX
Big Data Analytics for Non-Programmers
PDF
Graph-Powered Digital Asset Management with Neo4j
ODP
Graphing Your Data
PPTX
Bigdata
PDF
Simultaneous analysis of massive data streams in real time and batch
PPTX
How Linked Data Can Speed Information Discovery
PPTX
PDF
Hdfs Dhruba
PPT
The World of Structured Storage System
WSO2Con Asia 2014 - Simultaneous Analysis of Massive Data Streams in real-tim...
Introduction of big data unit 1
Big data technology unit 3
Big Data and Hadoop
NoSQL Introduction
Big Data Unit 4 - Hadoop
SQLSat 245 - Por Onde Começar no BigData
BigData Analytics with Hadoop and BIRT
Elephant in the room: A DBA's Guide to Hadoop
HDFS and Hadoop
Future of Data - Big Data
Big Data Analytics for Non-Programmers
Graph-Powered Digital Asset Management with Neo4j
Graphing Your Data
Bigdata
Simultaneous analysis of massive data streams in real time and batch
How Linked Data Can Speed Information Discovery
Hdfs Dhruba
The World of Structured Storage System
Ad

Viewers also liked (7)

PPT
Tweeting beyond Facts – The Need for a Linguistic Perspective
PDF
Data science challenges in flight search
PDF
Computer vision and image processing for dental products
PDF
Real-time information analysis: social networks and open data
PDF
DBPedia-past-present-future
PPT
Crowdsourced hedge funds
PPTX
Wavelet analysis of financial datasets
Tweeting beyond Facts – The Need for a Linguistic Perspective
Data science challenges in flight search
Computer vision and image processing for dental products
Real-time information analysis: social networks and open data
DBPedia-past-present-future
Crowdsourced hedge funds
Wavelet analysis of financial datasets
Ad

Similar to Real-time analytics with HBase (20)

PDF
Realtime analytics with_hadoop
PPT
Real-Time Video Analytics Using Hadoop and HBase (HBaseCon 2013)
PPT
HBaseCon 2013: Apache Hadoop and Apache HBase for Real-Time Video Analytics
PDF
Hadoop World 2011: Building Realtime Big Data Services at Facebook with Hadoo...
PDF
HBase ArcheTypes
ODP
HBase introduction talk
PPTX
TriHUG January 2012 Talk by Chris Shain
PDF
Thug feb 23 2015 Chen Zhang
PPT
Chicago Data Summit: Apache HBase: An Introduction
PPTX
Real-time Analytics for Data-Driven Applications
PDF
Tugdual Grall - Real World Use Cases: Hadoop and NoSQL in Production
PDF
Hbase: an introduction
PPTX
Hbasepreso 111116185419-phpapp02
PPTX
Apache phoenix: Past, Present and Future of SQL over HBAse
PDF
Hadoop at datasift
PPTX
Apache Phoenix and HBase: Past, Present and Future of SQL over HBase
PPTX
Apache Phoenix and HBase: Past, Present and Future of SQL over HBase
PPTX
HBase in Practice
PDF
Hbase 20141003
PPTX
HBase in Practice
Realtime analytics with_hadoop
Real-Time Video Analytics Using Hadoop and HBase (HBaseCon 2013)
HBaseCon 2013: Apache Hadoop and Apache HBase for Real-Time Video Analytics
Hadoop World 2011: Building Realtime Big Data Services at Facebook with Hadoo...
HBase ArcheTypes
HBase introduction talk
TriHUG January 2012 Talk by Chris Shain
Thug feb 23 2015 Chen Zhang
Chicago Data Summit: Apache HBase: An Introduction
Real-time Analytics for Data-Driven Applications
Tugdual Grall - Real World Use Cases: Hadoop and NoSQL in Production
Hbase: an introduction
Hbasepreso 111116185419-phpapp02
Apache phoenix: Past, Present and Future of SQL over HBAse
Hadoop at datasift
Apache Phoenix and HBase: Past, Present and Future of SQL over HBase
Apache Phoenix and HBase: Past, Present and Future of SQL over HBase
HBase in Practice
Hbase 20141003
HBase in Practice

More from Data Science Society (20)

PDF
[Data Meetup] Data Science in Finance - Factor Models in Finance
PDF
[Data Meetup] Data Science in Finance - Building a Quant ML pipeline
PPTX
[Data Meetup] Data Science in Journalism - Tanbih, QCRI and MIT
PPTX
Computer Vision in Real Estate
PPTX
ML in Proptech - Concept to Production
PPTX
Lessons Learned: Linked Open Data implemented in 2 Use Cases
PPT
AI methods for localization in noisy environment
PPTX
Object Identification and Detection Hackathon Solution
PPTX
Data Science for Open Innovation in SMEs and Large Corporations
PDF
Air Pollution in Sofia - Solution through Data Science by Kiwi team
PPTX
Machine Learning in Astrophysics
PPTX
#AcademiaDatathon Finlists' Solution of Crypto Datathon Case
PPTX
Coreference Extraction from Identric’s Documents - Solution of Datathon 2018
PDF
DNA Analytics - What does really goes into Sausages - Datathon2018 Solution
PDF
Relationships between research tasks and data structure (basic methods and a...
PDF
Data science tools - A.Marchev and K.Haralampiev
PDF
Problems of Application of Machine Learning in the CRM - panel
PDF
Disruptive as Usual: New Technologies and Data Value Professor Severino Mereg...
PDF
Intelligent Question Answering Using the Wisdom of the Crowd, Preslav Nakov
PDF
Master class Hristo Hadjitchonev - Aubg
[Data Meetup] Data Science in Finance - Factor Models in Finance
[Data Meetup] Data Science in Finance - Building a Quant ML pipeline
[Data Meetup] Data Science in Journalism - Tanbih, QCRI and MIT
Computer Vision in Real Estate
ML in Proptech - Concept to Production
Lessons Learned: Linked Open Data implemented in 2 Use Cases
AI methods for localization in noisy environment
Object Identification and Detection Hackathon Solution
Data Science for Open Innovation in SMEs and Large Corporations
Air Pollution in Sofia - Solution through Data Science by Kiwi team
Machine Learning in Astrophysics
#AcademiaDatathon Finlists' Solution of Crypto Datathon Case
Coreference Extraction from Identric’s Documents - Solution of Datathon 2018
DNA Analytics - What does really goes into Sausages - Datathon2018 Solution
Relationships between research tasks and data structure (basic methods and a...
Data science tools - A.Marchev and K.Haralampiev
Problems of Application of Machine Learning in the CRM - panel
Disruptive as Usual: New Technologies and Data Value Professor Severino Mereg...
Intelligent Question Answering Using the Wisdom of the Crowd, Preslav Nakov
Master class Hristo Hadjitchonev - Aubg

Recently uploaded (20)

PPTX
Business Ppt On Nestle.pptx huunnnhhgfvu
PDF
168300704-gasification-ppt.pdfhghhhsjsjhsuxush
PDF
Galatica Smart Energy Infrastructure Startup Pitch Deck
PPTX
Computer network topology notes for revision
PPTX
Business Acumen Training GuidePresentation.pptx
PPTX
Microsoft-Fabric-Unifying-Analytics-for-the-Modern-Enterprise Solution.pptx
PPTX
mbdjdhjjodule 5-1 rhfhhfjtjjhafbrhfnfbbfnb
PPTX
Introduction to Knowledge Engineering Part 1
PDF
Foundation of Data Science unit number two notes
PDF
Clinical guidelines as a resource for EBP(1).pdf
PDF
BF and FI - Blockchain, fintech and Financial Innovation Lesson 2.pdf
PPTX
DISORDERS OF THE LIVER, GALLBLADDER AND PANCREASE (1).pptx
PPT
Quality review (1)_presentation of this 21
PPT
Reliability_Chapter_ presentation 1221.5784
PPTX
Acceptance and paychological effects of mandatory extra coach I classes.pptx
PPTX
IBA_Chapter_11_Slides_Final_Accessible.pptx
PPTX
The THESIS FINAL-DEFENSE-PRESENTATION.pptx
PDF
.pdf is not working space design for the following data for the following dat...
PPTX
IB Computer Science - Internal Assessment.pptx
PDF
“Getting Started with Data Analytics Using R – Concepts, Tools & Case Studies”
Business Ppt On Nestle.pptx huunnnhhgfvu
168300704-gasification-ppt.pdfhghhhsjsjhsuxush
Galatica Smart Energy Infrastructure Startup Pitch Deck
Computer network topology notes for revision
Business Acumen Training GuidePresentation.pptx
Microsoft-Fabric-Unifying-Analytics-for-the-Modern-Enterprise Solution.pptx
mbdjdhjjodule 5-1 rhfhhfjtjjhafbrhfnfbbfnb
Introduction to Knowledge Engineering Part 1
Foundation of Data Science unit number two notes
Clinical guidelines as a resource for EBP(1).pdf
BF and FI - Blockchain, fintech and Financial Innovation Lesson 2.pdf
DISORDERS OF THE LIVER, GALLBLADDER AND PANCREASE (1).pptx
Quality review (1)_presentation of this 21
Reliability_Chapter_ presentation 1221.5784
Acceptance and paychological effects of mandatory extra coach I classes.pptx
IBA_Chapter_11_Slides_Final_Accessible.pptx
The THESIS FINAL-DEFENSE-PRESENTATION.pptx
.pdf is not working space design for the following data for the following dat...
IB Computer Science - Internal Assessment.pptx
“Getting Started with Data Analytics Using R – Concepts, Tools & Case Studies”

Real-time analytics with HBase