How to Scale BI and Analytics with Hadoop-based Platforms

How to Scale BI & Analytics
with Hadoop-based Platforms

Arcadia Data. Proprietary and Confidential2
Moderator:
Steve Wooledge
VP, Marketing

Presented by:
Boris Evelson
Vice President, Principal Analyst
Priyank Patel
Co-Founder, CPO

Agenda:
1. Systems Of Insight (SOI) – Next Gen BI:
How to scale SOI with native Hadoop BI platforms;
Boris Evelson, Forrester
2. Scale BI & Visual Analytics with Big Data;
Priyank Patel, Arcadia Data
3. Q&A

Poll:
a) Gathering knowledge - thinking about Hadoop or other scale-out data platforms.
b) Developing strategy - defining architecture, selecting tools.
c) Piloting - have big data analytics platform in place and beginning to experiment
d) Deployed - have defined use case and end-users are accessing and analyzing data
Where are you with your big data deployment?
Submit your answers in the “vote” tab of BrightTALK!

© 2017 Forrester. REPRODUCTION PROHIBITED.
Systems Of Insight (SOI) – Next Gen BI
How to scale SOI with native Hadoop BI platforms
Boris Evelson, VP and Principal Analyst
March 2, 2017

© 2017 Forrester Research, Inc. Reproduction Prohibited 8© 2017 Forrester Research, Inc. Reproduction Prohibited 8
“An insights-driven
business harnesses
and applies data
and analytics at
every opportunity
to differentiate its
products and
customer
experiences.”
Source: Forrester Research August 1, 2016 The Insights-Driven Business report

What are the most important goals/drivers your organization
considered when planning BI strategy?
16%
24%
28%
30%
30%
31%
31%
36%
37%
0% 5% 10% 15% 20% 25% 30% 35% 40%
Achieve better business transparency
Ensure compliance and reduce risks
Develop better products and services
Improve data quality and consistency
Improve business planning
Monitor, improve, and optimize process performance
Gain competitive advantage
Make better informed business decisions
Improve customer interaction and satisfaction
Base: 1249 global data and analytics technology decisions-makers
Source: Business Technographics® Global Data & Analytics Survey, 2016

Forrester clients report solid BI ROI
› 59% say BI is their top priority *
› More than 45% report solid double-digit ROI on BI **
investments within two years
› Industry leaders invest 38% more in BI as a percentage of
their IT budget ***
*Source: Forrester's Global Business Technographics Data And Analytics Survey, 2016, Base: 1859 Global data and analytics decision-makers
**Source: Forrester’s Q4 2016 Global Business Intelligence Business Case Online Survey
***Source: Forrester’s Business Technographics® Global Data And Analytics Survey, 2014

Challenges

TECHNOLOGY
› Single BI platform
› Streamlined data
architecture
› Centralized support
› Single version of the truth
BUSINESS
› I just want to get my job done
› Single version of the truth is
not my top priority
› Good enough but timely
data/info is good enough for
me

› An average organization only
leverages 50% of its
structured and …
› … 25% of its unstructured
data for decision making.
› But deeper investigations
often show that less than
10% of unstructured and less
then 20% of structured data
are only being turned into
information.
Source: Forrester’s Business Technographics® Global Data And Analytics Survey, 2016

Majority of analytical apps
are still being built using
spreadsheets
› 66% report >50% of BI content
in spreadsheets
› 15% report >80%
Source: Forrester’s Business Technographics® Global Data And Analytics Survey, 2016

Solutions

Business
agility
Big data
Artificial
intelligence

We have entered the Age of the Customer

Awareness
Dangerous
Formidable
Execution
Clueless
Paralyzed
CI Channel integration
MR Market responsiveness
KD Knowledge dissemination
DP Digital psychology
CM Change management
BI Business intelligence
IE Infrastructure elasticity
PA Process architecture
SI Software innovation
SC Sourcing & supply chain
10 dimensions of business agility
Source: Forrester’s “The 10 Dimensions Of Business Agility” report

Awareness
Dangerous
Formidable
Execution
Clueless
Paralyzed
Lower performers
CI
MR
KD
DP
CM
BI
IE
PA
SI
SC
Awareness
Dangerous
Formidable
Execution
Clueless
Paralyzed
Higher performers
CIMRKD
DP
CM
BI
IE
PASI
SC
Source: Forrester’s “The 10 Dimensions Of Business Agility” report

Four
components of
Forrester Agile
BI
Source: Forrester’s “It’s Time For A User-Driven Enterprise BI Strategy” report

Big data is not just about multiple Vs
Velocity
Variety
Volume Variability

Four questions to ask yourself: Are you ready (or not) for
big data?
› What are the typical analytics issues
where big data Is clearly not the answer?
› What are the typical analytics issues and
requirements where a different technology,
not necessarily big data, may be the
answer
› What are the typical business and
technical requirements where big data
may indeed be the answer?
› What are the implications of upgrading
analytics to big data?

Four reasons to consider native Hadoop/Spark BI platform
› Do your business requirements call for
linear scalability?
› Do your business requirements call for
both SQL (schema on write) and NoSQL
(schema on read) based applications?
› Is network traffic causing a bottleneck in
your BI applications?
› Do you need to keep data and applications
together?

Source: Forrester’s “It's Time To Upgrade Business Intelligence To Systems Of Insight” report
Agile BI
Systems of
Insight
Earlier
Generation BI
& Analytics
Big Data
More
Less
BusinessAgility
MoreLess
Data Availability
Systems of
insight extend
your business
technology
agenda

Systems of
insight extend
your business
technology
agenda
Source: Forrester’s “Digital Insights Are The New Currency Of Business” report

› Contextual insights
(embedded, pervasive BI)
› Insights to execution / action
(actionable BI)
› Continuous feedback loop for
learning and improvement
More Systems Of Insight vs BI best practices and
technologies

Recommendations
Source: Forrester’s “Digital Insights Are The New Currency Of Business” report.
Goal Single version of the truth
Winning, serving and retaining
customers
Sponsorship CIO CMO, CFO, VP Sales, CRO, CSO
People
IT (Information Technology) and
business silos
Business drives, IT supports (BT –
Business Technology)
Technology Silo’ed
Pervasive, contextual, actionable,
self-learning and improving
BI Systems Of Insight

FORRESTER.COM
Thank you
FORRESTER.COM
Thank you
Boris Evelson
bevelson@forrester.com
http://www.forrester.com/Boris-Evelson
http://blogs.forrester.com/boris_evelson
https://twitter.com/bevelson
https://www.linkedin.com/in/bevelson
https://www.facebook.com/ForresterBI

Poll:
a) Development tools (e.g. Spark, MapReduce)
b) SQL engines (e.g. Hive, Impala, SparkSQL, Drill)
c) Traditional BI tools (e.g. Tableau, Qlik, MicroStrategy)
d) Hadoop-native, distributed BI platforms
e) Other (please specify in the comments section)
How do you plan to give users access to analyze their data?
Submit your answers in the “vote” tab of BrightTALK!

Arcadia Data. Proprietary and Confidential
Scale BI & Visual Analytics with Big Data
Priyank Patel
Co-founder and VP Products

Challenges with BI tools on Big Data
 Data summarization
 Big data fidelity loss
 No access to real-time data
 Higher security risk
 Management and
operational complexity
 High TCO with multiple
systems
35
BI/VIZ
TOOLS
BI/SERVER
(CUBES)
DATA MART
(EXTRACTS)
DATA WAREHOUSE
DATA USERS
/ANALYSTS
Order Book Market Data
Electronic
Communications
Trader Data OATS
Operational Data Sources

Challenges with BI tools on Big Data
 Data summarization
 Big data fidelity loss
 No access to real-time data
 Higher security risk
 Management and
operational complexity
 High TCO with multiple
systems
36
Order Book Market Data
Electronic
Communications
Trader Data OATS
100s of silos

Start with a Data Lake strategy …
ALL
DATA
Sensor data Clickstreams Security Logs CRM data Transactions
Data Lake

But a Data Lake alone is not enough…
38
BI/VIZ
TOOLS
BI/SERVER
(CUBES)
DATA MART
(EXTRACTS)
DATA WAREHOUSE
(EXTRACTS)
ALL
DATA
<EXTRACTS>
DATA USERS
/ANALYSTS
Data
Lake
Data Lake becomes a
data dump
Data Consumption
Problems Remain

Arcadia Makes it Simple
Data Lake

Data-native BI & Visual Analytics
Arcadia Data 2016. Proprietary and Confidential
Arcadia Data
is a
Hadoop-native
platform
that connects
business
users to big
data
Di st r i but ed
BI & Anal yt i cs
Engi ne r uns
on each Hadoop
node
User s connect
vi a a web
br owser
BROWSER BASEDDATA-DRIVEN APPS
BROWSER BASED
BIG DATA OS
Distributed execution,
data storage (HDFS, S3, object stores) , metadata,
security
DATA-NATIVE COMPUTE ENGINE
On-Premise : Scales inside Hadoop Clusters
In-Cloud : Elastically scales with compute resources
WEB BASED INTERFACE
Drag & drop interface focused on BI and exploratory
analytics, edit and publish from the same place

AGILITY
Explore quickly & directly -
don’t start with data marts,
cubes, or extracts
APPLICATIONS
Actionable applications with
embeddability
Why does a data-native architecture matter for scaling BI ?
 Simple visual interface to exploration and
semantic modeling on ALL of your data
 Active data store continuously models data
based on usage for fast concurrent access
 Production-quality dashboards
and customer applications.
 Support for real time as well as
free text based analysis.
 Point-and-click micro-
segmentation and time-series
event analytics

Runs directly on your Hadoop or cloud cluster. No cubes. No extracts.
Hundreds
of concurrent
business users
Sub-second
performance
for production reports
Thousands of
shared data driven
applications
100s of
billions
of rows
Agility in a big data environment

Hadoop Cluster
Results
(100x Faster)
Eliminate dependence on cubes
Consumption Layer
Processing Layer
Smart Acceleration™
1. Start with exploration of raw data, no
need to determine design of
acceleration structures such as cubes
ahead of time
2. Recommendation engine generates
AVs (derived forms of raw data)
based on dynamic data usage within
Hadoop cluster
3. Re-routes data queries to AVs
transparently providing automated
acceleration when needed for
production/high concurrency uses
 Automatically modeled and
maintained within Hadoop cluster
 Keep logical data models simple
without needing to target specific data
cube structures
1
2
3Queries
Queries
automatically
redirected
In-memory
Analytical Views
Recommendation
Engine
Stores Derived Forms of
Raw Data in Hadoop
Raw Data in Hadoop

Access all data : Relational, Real Time, NoSQL and Search

AGILITY
Explore quickly & directly -
don’t start with data marts,
cubes, or extracts
APPLICATIONS
Actionable applications with
embeddability
Why does a data-native architecture matter for scaling BI ?
 Production-quality dashboards
and customer applications.
 Support for real time as well as
free text based analysis.
 Point-and-click micro-
segmentation, event analytics,
dimension/measure correlations

Juxtaposing Real-time and Historical in One View
Visuals are
coherent and
permit interaction
across data
sources
Real-time
feed from
Apache Solr
or Spark
Streaming
Drill to
detail in
Kudu or HDFS

Cross-connection Data Blending
Visuals are
coherent and
permit interaction
across data
sources
Visual
from
Oracle
Visual
from
Teradata
Visuals
from
Apache Hadoop

Build Modern Web Applications Driven by Your Big Data
HR
5
5"

Ad tech
Trade surveillance for high
velocity trade volume across
exchanges to identify and
prevent abusive trade behavior
Cybersecurity app to capture
investigative workflows, real-
time incident response, and
guided data exploration
Developed a new SaaS self-
service analytics platform
to give their customers better
marketing attribution
Gives global brand
managers digital campaign
intelligence across 100+
brands
INNOVATION
REDUCE RISK
Government
Improve patient outcomes
on 10+ million members by
predicting and controlling re-
admission risk.
Turn IoT data from enterprise
data servers into meaningful
lifecycle analytics data
service

For three years, we've been evaluating the
market for a BI product...
Arcadia Enterprise is the first product we found
that provides truly on-cluster Hadoop BI
…Its execution model and user self-service
approach deliver performance at Hadoop scale,
and lets us develop our analytics quickly.
— Terry McFadden
Associate Director, Global Business Services,
Procter & Gamble
“
”

Scaling BI and Visual Analytics enables value from Big Data
51
AGILITY
Explore quickly & directly - don’t
start with data marts, cubes, or
extracts
APPLICATIONS
Actionable applications,
no coding required
ARCHITECTURE
A powerful,
simplified architecture

More Resources:
• Forrester Research: How to Scale Business Intelligence
with Hadoop-Based Platforms
• https://www.arcadiadata.com/lp/forrester-research-scale-hadoop-BI
• Forrester Wave: Native Hadoop BI, Q3 2016
• https://www.arcadiadata.com/lp/forrester-wave-hadoop-bi-research-report/
Arcadia Data will send out links to these reports after the webinar

Q&A

Four Approaches for Big Data Analytics
55
Data-Native Visual
Analytics
Data-Native
Application
Fast SQL + BI Tools
(ODBC/JDBC, Hive, Spark,
Impala, Drill …)
BI Server
Scale
Agilit
y
Static cubes only.
No granular data access.Won’t scale. Summaries only.
Simple SQL. 1-5 users.
Real-time & dynamic.
100s to 1000s of users.
Cubes
Edge Node
Move Data to BI Server
Separate BI Server
BI Server
No access to real-
time, streaming,
unstructured data

Big Data Analytics: Alternatives
56
Capability
Separate
BI Server
Hadoop SQL Engines
+ BI Tool
Big Data
“Cubes”
Data-Native
Visual Analytics
Dashboards and reporting ✓ ✓ ✓ ✓
Real-time visualizations ✘ ✘ ✘ ✓
Data Applications ✘ ✘ ✘ ✓
High user concurrency ✓ ✘ ✓ ✓
Ad-hoc drill to detail ✘ -- ✘ ✓
In-Hadoop advanced analytics
(e.g., customer engagement flows, micro-segmentation) ✘ ✘ ✘ ✓
Multi-structured data access
(e.g. NoSQL, S3, files, search) -- ✓ ✘ ✓
Unified Security ✘ ✘ ✘ ✓
Unified Administration ✘ ✘ ✘ ✓
Lower TCO ✘ ✘ ✘ ✓

Data-Native Visual Analytics Architecture
NO DATA MOVEMENT
 No data extracts
 Analytics at the highest granularity
SELF-SERVICE BI ON BIG DATA
 Web-based UI
 Collaboration on “live” data apps
NATIVE MANAGEMENT & SECURITY
 Single system to manage and
secure
 Integrated with Apache Sentry,
Apache Ranger
57
Real-Time Streams and Processing
Cloud On-Prem
Batch and Interactive

How to Scale BI and Analytics with Hadoop-based Platforms

More Related Content

What's hot (20)

Similar to How to Scale BI and Analytics with Hadoop-based Platforms (20)

More from Arcadia Data (14)

Recently uploaded (20)

How to Scale BI and Analytics with Hadoop-based Platforms