Impact of big data on analytics

Impact of Big Data on Analytics
Mamatha Upadhyaya

2
Big Data & Analytics
Copyright © 2014 Capgemini. All rights reserved.
Impact of Big Data on Analytics | Mamatha Upadhyaya
Big Data and Analytics Summit 2014
The terms Big Data
and Analytics are
used simultaneously

3
However, analytics, predictive modeling, advanced analytics,
data science is not new!

4
So what does this mean for Analytics?
Yes, the amount of data that is available to us is exploding
And Big Data Platforms and Commodity Hardware and bringing in additional capabilities
So what does this
mean for Analytics?
Media is rife with Big Data and Analytics
AND The Data Scientist makes it from Nerd to the most cool person!!
…makes it to on top of CIO agenda
Big Data and analytics is touted as the panacea for all problems

5
Data Science perspective
– A Data Science perspective
Big Data and AnalyticsImpact of

6
A brief history of Data Science
Pre 1800s 1800-1900 1900-1940 1940-1960 1960 1970 1980 1990 2000 2010
 Text/ string search
 1974 Peter Naur “Concise Survey of Computer
Methods”, Data Science, Datalogy
 Knuth – Art of Computer Programming.
 1976 – SAS Institute
 1977 The International Association for
Statistical Computing (IASC).
Computer Science
Data Technology
Visualization
Mathematics/ OR
Statistics
 Probability
 Correlation
 Bayes Theorem.
 Regression, Least
Squares
 Time Series.
 Theoretical Foundations of Modern Stats
 Hypothesis, DOE
 Mathematical Statistics.
 Bayesian Methods
 Time Series Methods (Box Cox,
Survival, etc.)
 Stochastic Methods.
 Simulation, Markov
 Computational Statistics.
 Decision Science
 Pattern recognition
 Machine learning.
 Liebniz – Binary Logic.
 Babbage, Lovelace
 Boolean Algebra
 Punch cards.
 Turing machines
 Information Theory
 Weiner & Cybernetics
 Von Neumann Architecture.
 Calculus
 Logarithms
 Newton-Raphson.
 1989 First KDD Workshop
 Gregory Piatetsky-Shapiro.
 Sort & Search Algorithms –
Dijkstra, Kruskal, Shell Sort, …
 Heuristics – Simulated Annealing, …
 Graph Algorithms
 Multigrid methods
 Tree based methods.
 Database Marketing
 Data Mining, Knowledge Discovery
 “Data science, classification, and related methods.”
 William Cleveland: Data Science
 Leo Breimann: Statistical Modeling: 2 Cultures.
 Optimization Methods
 Fourier and other transforms
 Matrix & Generalizations
 Non-euclidean geometries.
 Applications to Military,
manufacturing,
Communications.
 1962 John W. Tukey, Future of
Data Analysis
 Networks
 Assignment Problems
 Automation
 Scheduling.
 First IBM
Computers
 DBMS.
 Removable Disk drives
 Relational DBMS.
 Desktop, floppy
 SQL, OOP
 High level languages.
 William Playfair
 Charles Minard
 Florence Nightingale.
 Catrography
 Astronomical Charts.
 John Tukey
 Jacques Bertin.  Edward Tufte.
 Grammar of Graphics
 Word Cloud, Tag Cloud.

7
Drivers of change
Data
Availability
Technology
Ability to
Handle
Structured and
unstructured
data
Platform Cost Agility
Business
Expectation
Digital
Experience
Strategic
Initiatives
New Business Models

8
TABLE OF CONTENTS
 Data drivers
 Technology drivers
 So what does all of this increased activity
mean to Talent!

9
8 TB data everyday
10 TB data everyday
152 million blogs on the internet
Data availability
Possible
Technically but
very expensive Not efficient enough to
handle the amount &
type of data generated
by newer internet-scale
technologies
Big Data
Internal data
M&A
New Tech
adoption
Need to access
Unstructured data
External data
2 billion internet users
* Hortonworks CEO Rob Bearden
Digital Customer
IOT
Legacy data management system are not designed to handle heavy
demand of data consumption
“85% of that data is coming from net-new data
sources.” – mobile, social media, and web- and
machine-generated data*… and this will increase.
RFID

10
Improving traditional analytics
Understand Act Measure
Change in Market Share
Campaign Effectiveness
Revenue/ profit metrics/
reports
Customer Churn/
retention reports
Customer Next
Best Offer
Cross Sell/ Up
Sell Opportunities
CRM (Customer
Info, billing, etc.)
Subscription and
Usage Summary
External data
(Demography, etc.)
Customer Profiling
 Demographic Segments
 Behavior (usage/ profit/
satisfaction/ etc.) based
profiles.
Product Association and Product Mix
Customer Profitability/ Life Time Value
Product Purchase
Propensity Score
Targeted
Retentions
Strategies
Existing Customer Analytics Insights
CDR, IPDR data
Customer Service
Data
Network Data
Usage Based
Profiling Customer Links/
Network Analysis
Drivers of
Satisfaction Network
Performance and
Service Levels
New Analytics from Big Data
Social Media Data and Analysis
Social Media and
Web Data
Sentiment
Analysis Social Media
Influence Analysis
Drivers of
Sentiment
Churn Scores

11
…but it is all about business value
KS Statistic
Accuracy Ratio
ROC Curve
Gini Coefficient
Implemented Cut-Off
Cut-off neighborhood
Shift
Baseline Population
Current Population
NewApplicants
PSI:
Distributional Shift
Scorecard Score
Strategic KPIs
Reduce Costs
Regulations
Compliance
Increase
Revenue

12
Engineering analytics:
Makes it a reality with Big Data + Data Science on top of traditional mathematical models
Various Data
 Usage Pattern, Health Monitoring
 Alarms & Control, Benchmarking
 Predictive Rules for optimal recommendation from connected
Assets
 Failure Prediction & Root Cause Analysis
 Resource Scheduling.
 Design for Reliability
 Benchmark data/content Mining
 Reduced Order Modeling for high volume Simulation
and Testing
 Supplier Risk Modeling
 Weight & Cost analytics.
Design, Testing & Production Operation Service
Manufacturer User Service Provider
Analytics
Engines
Mfg & other Guideline
Specification &
Performance
Benchmark Reports
System Topology
Financial
Sensors/Telemetry –
usage, operations
setting, events
/alarms logs, etc.
Failure/ Warranty
Claims
Field/Technical
Inspection Notes
Contract/Service
History
Social Media and
Third Party
Reliability
Testing/Simulation
Supplier/OEM
transactions
Value Chain
First Time Right Product Design
Connected Assets, Operations Control &
Predictive Maintenance
Supplier
Medium Volume, Low Speed,
Domain Specific
High Speed, High Volume and
Domain Neutral
Data
Behaviors

13
As more data becomes available to the Data Scientist,
so does the complexity…
Product Affinities
Private Label Analysis
Customer Purchase Patterns
Campaign Response Effectiveness
Shrinkage & Productivity Analysis
Category Scorecard & Contribution
to GM Promotion Decomposition
Promotion Mix Optimization
Product Price Optimization
Pricing with Consumer
Perception Analysis
Assortments Optimization for
Market Basket Analysis
Shopper Trip Mission Analysis
Shopper Market Basket
Shopper Brand Sentiment Analysis
Product Behavior Scan
Adjacencies Analysis
Out of Shelf Analytics
Scoring of Stores/ Retail
Chains
Cross-Channel Order
Management
Inventory Optimization
at DCs (SCM)
Demand/ Volume
Forecasting
Social Impact on
Category/ Brand
Consumption
Promotion Halo/
Cannibalization
Pricing Elasticity Analysis
Shopper Segmentation
Shopper Demographics
Shopper Loyalty Base
RFM Analysis
Product/ Brand
Switching
Trial & Repeat
Category
Uniqueness,
Popularity Indices
Category Leakage
Tree
Store Clustering
Category/ Brand
Offer Conversion
Cross-Sell
Up-Sell
Shopper
Assortment
Price
Promotion
ProductCompetition Category
Tactics
DataNeeds
+
Other
Consumer +
Survey+
Social Data
+
Household
Panel +
Loyalty/ CRM
Data
+
Syndicated +
Promotions
Data
(IRi/ Nielsen)
+
POS Data
+ Campaign +
Shipments +
Public* Data
Public* Data includes
Weather, Census,
Topography,
Ordinance etc
Maturity Stages

14
Availability of data is changing the way we address some
traditional business problem
Pharmaceutical
Companies have used
physical surveys to
identify KOL. Big data
and analytics is
pioneering the way to
use a data driven
objective approach to
identifying and
monitoring KOL
 Selection of right KOLs can help in better utilization of these marketing funds
 A key success factor for these marketing spends is the correct methodology to
identify KOLs
 Managing brand perception for the key Opinion Leaders is crucial for Brand
Management.

15
TABLE OF CONTENTS
 Data drivers
 So what does all of this increased activity
mean to Talent!

16
Technology
New Technologies are allowing us to
manage this at a fraction of the cost &
faster than ever before.
Traditional
DataWarehouse
BIG DATA
1/30 of the cost
Data does not have to be isolated in
repressive silos

17
Big Data technologies are enabling a new approach
Response time
Volume
Hadoop
Data warehouses
PB
TB
GB
Hour Min Sec SubSec
In-memory
databases
Event
processing
tools
Real-time
Applications

18
Data and technology implications…
Model development
 Reduced timelines
• Access all data from a „data lake‟
• Data discovery and visualization tools to
reduce EDA timelines
• In-memory/ in-database/ high performance
analytics and parallelized algorithms
 Increased analytical capability
• Implement techniques like Graph/ Network
analysis, ensemble methods, matrix
algorithms at scale
• Analyze structured and unstructured data on
one platform
 Improved accuracy
• Analyze much larger data sets
• Ability to personalize for a segment of one, for
e.g. targeting).
Model deployment
 Seamless deployment (In-database, PMML)
• Decreases error in deployment
 Big data deployment
• Analytics on exabytes, scoring in MB/ sec
 Real time deployment
• Response (alert/ recommendations) in
milliseconds or less
 Adaptive, machine learning algorithms
• “learn” and respond to recent events
 Availability and velocity of data leads to
change in analytical approach
• for e.g. Can move from „complex algorithms
for precision prediction of failure modes‟ to
„real time monitoring, alerts and control
processes‟.

19
Decreased
response time
Customer experience
Information is becoming the
new battleground
Business expectation

20
Analytics is playing an ever important role
Increased Focus on identifying the
customer across all channels
Segmentation to Micro segmentation
to the individual
Personalized Messaging and offers –
Increased Individual Customer Centricity
Gradual evolution of Customer Analytics
Past
 Customer segments who are
most likely to respond to
targeted campaigns for new
products offers
 Can tailor offers to specific to
each customer segment
 Mostly delivered through mass
mail campaigns and in store
promotions.
Now
 Micro segmentation
 Analyze customer behavior
and buying patterns across
channels
 Delivery through email, web,
mass mail campaigns.
Moving toward
 Historical individual customer behavior
and buying patterns across channels
 Individual customer consumption
pattern
 In-store basket analytics
 Additional dimensions Location & time
 Targeted Strategies to pre-empt
customers from visiting competition
 Instantaneous Delivery in store or a
proactive delivery via mobile to bring
the customer to store.
Segment to Individual to Individual @ time, place and behavior
You have
purchased Cheese,
here are the
offers on Bagels
You are within
2 KMs of a store
offering 50% off
garden furniture
Do you
need coffee?

21
Analytics is only as good as the implementation…
Analytics has long excelled in silos …as
the amount of data and business
expectation increases, this will no longer
be feasible
IT will move from a facilitator role to an
enabler role
Decreased response time will mean end to end integrations – enterprise architecture
teams will need to be involved…
The Data Science team will have to work along with technology teams to effectively
serve the end customer

22
much of which is outside the
organization
Increased availability of data
Analytics as a Service and Data
Monetization
New service models
Decreasing Time value of data!

23
TABLE OF CONTENTS
 Data drivers
 So what does all of this
increased activity mean to
Talent !

24
Scalability and industrialization to address skill shortage
Technical skills (Coding, Statistics,
Math)
+ Perseverance
+Creativity
+ Intuition
+Presentation Skills
+Business Savvy
= Great Data Scientist!
Key to a Great Data Scientist
 Identified four Data Scientist clusters based
on how data scientists think about
themselves and their work, not
• Years of experience,
• Academic degrees, favorite tools
• Titles, pay scales, org charts.
 Most successful data scientists are
those with substantial, deep expertise
in at least one aspect of data science,
be it statistics, big data, or business
communication
 T-Shaped Skills.

25
…so can analytics solve all our problems
Help us acquire customers
Product Recommendation
engine
Solve World Hunger Crop Sciences
Keep us fit!
Catch the bad guys Numbers
Win FIFA
“German national football team uses real
time analytics for a competitive edge”
Get you married! Dating sites, Matchmaking
Analytics in Healthcare

26
So what is it Big Data and Analytics cannot do!!!

The information contained in this presentation is proprietary.
Rightshore® is a trademark belonging to Capgemini.
www.capgemini.com/bim
About Capgemini
With more than 130,000 people in over 40 countries, Capgemini
is one of the world's foremost providers of consulting, technology
and outsourcing services. The Group reported 2013 global
revenues of EUR 10.1 billion.
Together with its clients, Capgemini creates and delivers
business and technology solutions that fit their needs and drive
the results they want. A deeply multicultural organization,
Capgemini has developed its own way of working, the
Collaborative Business Experience™, and draws on Rightshore®,
its worldwide delivery model.

Impact of big data on analytics

More Related Content

What's hot (20)

Viewers also liked (6)

Similar to Impact of big data on analytics (20)

More from Capgemini (20)

Recently uploaded (20)

Impact of big data on analytics