SlideShare a Scribd company logo
Overview of Big Data and
Analytics in practice
Vivek Murugesan
Contents
● What is Big Data?
● What is Analytics?
● Why do companies bother?
● Why now?
● Industries & Use cases
● Why should I bother?
Big data is about
these 4 Vs
Overview of analytics and big data in practice
Analytics
Is the process of iterative,
methodical exploration of
an organization’s data
with emphasis on
statistical Analysis. To
enable data-driven
decision making.
Why do companies bother…?
Why do companies care…?
● Digital innovation and disruptions
○ Netflix vs Blockbuster
○ Amazon’s disruptive innovation
○ Google vs GPS
○ Traditional advertising vs social media
advertising
○ Competitive advantage
Why now?
● Storage has become cheaper
● Availability of infrastructure at
cloud
● Open source
● Data Science and Machine
learning moving beyond
research
Data everywhere in every domain
❖ Web - content, link structure, clicks
❖ Retail - customer details, point of sale, inventory
❖ Medical - literature, patient history, drug details …
❖ Financial - stocks, currencies, financial news, commodities
❖ Insurance - customer history, claim details …
❖ Telecom - call detail records, customer history & profile …
❖ Banking - customer transactions, profile …
❖ Travel & Hospitality - travel itinerary, schedule …
Industries
● Medical, Healthcare and Life
Sciences
● Automobile and Manufacturing
● Travel and Hospitality
● Retail and Ecommerce
● Web, Social Media and Digital
Media
● Telecommunication
● Banking, Finance and Insurance
● Energy
● Sports, Media and Entertainment
● Niche areas like autonomous
driving, image video processing,
etc,.
Medical, Healthcare and Life Sciences
● Cancer research with pattern recognition on
cells
● Clinical trials with millions of compositions for
drugs
● Prediction of diseases with tests and
probabilistic studies ex: Diabetes and Down
syndrome prediction
● Collection and storage of test results like scan
reports, blood test reports, etc,.
● Image processing, text processing and
complex pattern recognition analysis etc,.
● Analyzing literatures and patents to find out
cure for diseases
Automobile and manufacturing
● One of the frontrunners of adopting big data and
analytics even before the cloud computing (during the
cluster computing days)
● Analyzing vast amount,
○ Customer feedbacks
○ Inventory data
○ Repair and life of parts report
○ Competitive information
○ Market research data
● To come out with best design that will sustain long time
in the market
● Some of these analysis could run for months together
● Design arrived at will be tested under simulation
environment
Travel and Hospitality
● Revenue management was one of the
technique that resurrected the airline
industry that was close to its death during
early 90’s
● Similar techniques are used with
hospitality industry as well with increasing
number of hotels and the kind of
competitive market it has became
● Growing number of Online portals shows
the amount of competition in this industry
● Data generated and consumed in this
industry really huge
Retail and Ecommerce
● Inventory tracking across franchises
● Relationship between inventory overrun and
discounts
● Recommending right products in subseconds to
close the purchase lifecycle of the customers
appropriately
● Imagine the scaling problems faced by online
retailers like Amazon, Flipkart, etc,. With
millions of products and millions customers to
handle
● The capability to handle the price elasticity in
the market
● Example use case of Best buy vs Amazon
Web, Social media and Digital media
● With the amount of tweets and posts that twitter and
facebook handle it is daunting task for them to be notifying
the right set of people
● The kind of job recommendation and PYMK does by
Linkedin is a really hard problem to handle at that scale
● Advertisement industry in the digital media has a really
complicated ecosystem,
○ With so many publishers, agencies and advertisements
○ To satisfy so many parameters like number of impressions, CTR,
conversion, etc,.
● Such a complicated ecosystem is handling online bidding
at micro seconds to choose the advertisement to show for
each page
Telecommunication
● More than 16 players in India running under
a very tight margin in call rate
● For them to get revenue they have to
squeeze out interest through every single
customer,
○ By targeting them with right offer and promotion at
right time
○ They operate at micro segments of size 1000s out of
their 160 Million customers
● Huge number mobile subscribers moving all
over and making lot of calls
● All these generate a lot of data in the form
CDRs, etc,.
● And all of these needs to be processed,
stored, analyzed and archived appropriately
Banking, Finance and Insurance
● Banks run lot of promotions in the form of sending emails,
sms, etc,. To its customers
● They get profit for every single conversion out of these
campaigns
● Imaging how hard it is to choose the right set of customers
to target with right set offers to maximize the revenues out
of these campaigns
● People who work in finance industry like stock market etc,.
Has a large volume of data in wide variety of forms to
consume to mine for any meaningful insights to come out
with right strategy for investment
● Processing claims and detection of frauds is really hard
problem to solve at scale
● Insurance firms have started utilizing sophisticated
techniques like text processing on the claim statements to
detect frauds
Energy
● Amount of image processing in
analyzing satellite images to locate
the point of energy source is
humongous
● Any small amount of precision of
error can also introduce a huge loss
● Hence the results need to be
optimized with huge number of
iterations to minimize the error
Sports, Media and Entertainment
● Football clubs and IPL franchises have started
modeling the players to arrive at optimal
strategy to play with
● For example NZ cricket team at some point of
time started utilizing the systems to an extent
to automate the team selection
● Media and Entertainment needs to be up to
date with social media to compete with them
and against their peers
Online
advertising
industry
Showing the
interactions in
all directions
and the
companies
playing in the
space
List of Techniques
● Statistical testing, models (regression, forecasting etc,.)
● Machine learning (pattern recognition, classification, clustering, segmentation,
etc,.)
● Application Simulation and Optimization (Revenue management, Supply
chain management, set covering, network problems etc,.)
● Recommendation (personalized, non-personalized, association rule mining,
etc,.)
● Text analytics, Image processing
Few more domains
● Education, Academics, E-learning
● Networking - Security, Adaptive routing
● Niche areas - Autonomous driving, Reinforcement learning, etc,.
● Multimedia - Audio and Video analytics
Why Should I bother…?
Why should I bother…?
● Industry growing rapidly
● More organizations adopting
● Technology trends
● Skill gap and projection
● Skills getting obsolete
Overview of analytics and big data in practice
Overview of analytics and big data in practice
Industry 4.0
Cyber-Physical
Systems (CPS) are
integrations of
computation,
networking, and
physical processes.
Embedded computers
and networks monitor
and control the
physical processes,
with feedback loops
where physical
processes affect
computations and
vice versa.
Questions...
Contact: vivek.murugesan@gmail.com
Linkedin: https://in.linkedin.com/in/vivek-murugesan-aa183416

More Related Content

PDF
Overview - IBM Big Data Platform
PDF
Big data ibm keynote d advani presentation
PDF
IBM-Why Big Data?
PDF
Ibm big data
PDF
What is big data - Architectures and Practical Use Cases
PDF
Big Data Scotland 2017
PDF
Strategyzing big data in telco industry
PDF
Big Data & Analytics Architecture
Overview - IBM Big Data Platform
Big data ibm keynote d advani presentation
IBM-Why Big Data?
Ibm big data
What is big data - Architectures and Practical Use Cases
Big Data Scotland 2017
Strategyzing big data in telco industry
Big Data & Analytics Architecture

What's hot (20)

PDF
Business case for Big Data Analytics
PDF
Introduction to Data Mining, Business Intelligence and Data Science
PPTX
Requirements document for big data use cases
PDF
Big Data Overview
PDF
"Industrializing Machine Learning – How to Integrate ML in Existing Businesse...
PDF
Telco Big Data 2012 Highlights
PDF
National Conference - Big Data - 31 Jan 2015
PPTX
Monitizing Big Data at Telecom Service Providers
PDF
Big data analytic market opportunity
PDF
Key Considerations for Putting Hadoop in Production SlideShare
PPTX
Big Data use cases in telcos
PDF
Telco Big Data Workshop Sample
PDF
"Empower Developers with HPE Machine Learning and Augmented Intelligence", Dr...
PDF
Fight Fraud with Big Data Analytics
PDF
Use cases for Hadoop and Big Data Analytics - InfoSphere BigInsights
PDF
Big Data Use Cases
PPTX
Gov & Private Sector Regulatory Compliance: Using Hadoop to Address Requirements
PDF
Big data case study collection
PDF
Bigdata based fraud detection
PPTX
Keynote talk at Financial Times Forum - BigData and Advanced Analytics at SIB...
Business case for Big Data Analytics
Introduction to Data Mining, Business Intelligence and Data Science
Requirements document for big data use cases
Big Data Overview
"Industrializing Machine Learning – How to Integrate ML in Existing Businesse...
Telco Big Data 2012 Highlights
National Conference - Big Data - 31 Jan 2015
Monitizing Big Data at Telecom Service Providers
Big data analytic market opportunity
Key Considerations for Putting Hadoop in Production SlideShare
Big Data use cases in telcos
Telco Big Data Workshop Sample
"Empower Developers with HPE Machine Learning and Augmented Intelligence", Dr...
Fight Fraud with Big Data Analytics
Use cases for Hadoop and Big Data Analytics - InfoSphere BigInsights
Big Data Use Cases
Gov & Private Sector Regulatory Compliance: Using Hadoop to Address Requirements
Big data case study collection
Bigdata based fraud detection
Keynote talk at Financial Times Forum - BigData and Advanced Analytics at SIB...
Ad

Similar to Overview of analytics and big data in practice (20)

PDF
Data science Applications in the Enterprise
PDF
Presentation on developments in hiring and fintech for HKU Executive certific...
PDF
Nicholas Gorski: Real-time revenue science at Twitter
PDF
Big data: Bringing competition policy to the digital era – VARIAN – November ...
PPTX
Startup InsurTech Award - Anorak Technologies Limited
PDF
Customer trust and viral growth presentation for partners
PDF
Marketing Interrupted - The perfect match data science and marketing webinar
PPTX
Changing landscape of Real Estate Industry with Digital Marketing and Technology
PDF
Panel: Powering Business Decision Making
 
PPTX
Mobile cdn market ppt
PDF
Data Drive Better Sales Conversions - Dawn of the Data Age Lecture Series
PDF
Rohit Talwar- Broadening the Ancillary Revenue Stream for the Airline Retail ...
PPTX
Cognitive computing market vendors by size, share & growth strategies 2...
PDF
Financial analysis for product managers
PPTX
Big Data: Banking Industry Use Case
PDF
[AI Webinar Series P1] - How Advanced Text Analytics Can Increase the Operati...
PDF
Digitisation in-insurance-presentation-samuel t-1
PDF
Introduction to Big Data Analytics Unit 1 .pdf
Data science Applications in the Enterprise
Presentation on developments in hiring and fintech for HKU Executive certific...
Nicholas Gorski: Real-time revenue science at Twitter
Big data: Bringing competition policy to the digital era – VARIAN – November ...
Startup InsurTech Award - Anorak Technologies Limited
Customer trust and viral growth presentation for partners
Marketing Interrupted - The perfect match data science and marketing webinar
Changing landscape of Real Estate Industry with Digital Marketing and Technology
Panel: Powering Business Decision Making
 
Mobile cdn market ppt
Data Drive Better Sales Conversions - Dawn of the Data Age Lecture Series
Rohit Talwar- Broadening the Ancillary Revenue Stream for the Airline Retail ...
Cognitive computing market vendors by size, share & growth strategies 2...
Financial analysis for product managers
Big Data: Banking Industry Use Case
[AI Webinar Series P1] - How Advanced Text Analytics Can Increase the Operati...
Digitisation in-insurance-presentation-samuel t-1
Introduction to Big Data Analytics Unit 1 .pdf
Ad

Recently uploaded (20)

PDF
Shreyas Phanse Resume: Experienced Backend Engineer | Java • Spring Boot • Ka...
PDF
Per capita expenditure prediction using model stacking based on satellite ima...
PDF
Dropbox Q2 2025 Financial Results & Investor Presentation
PDF
Bridging biosciences and deep learning for revolutionary discoveries: a compr...
PDF
Peak of Data & AI Encore- AI for Metadata and Smarter Workflows
PDF
Machine learning based COVID-19 study performance prediction
PDF
solutions_manual_-_materials___processing_in_manufacturing__demargo_.pdf
PDF
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
PDF
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
PDF
[발표본] 너의 과제는 클라우드에 있어_KTDS_김동현_20250524.pdf
PDF
Network Security Unit 5.pdf for BCA BBA.
PDF
cuic standard and advanced reporting.pdf
PDF
How UI/UX Design Impacts User Retention in Mobile Apps.pdf
PDF
Advanced IT Governance
PPT
“AI and Expert System Decision Support & Business Intelligence Systems”
PDF
Advanced methodologies resolving dimensionality complications for autism neur...
PDF
NewMind AI Weekly Chronicles - August'25 Week I
PPTX
20250228 LYD VKU AI Blended-Learning.pptx
PPTX
Big Data Technologies - Introduction.pptx
PPTX
Understanding_Digital_Forensics_Presentation.pptx
Shreyas Phanse Resume: Experienced Backend Engineer | Java • Spring Boot • Ka...
Per capita expenditure prediction using model stacking based on satellite ima...
Dropbox Q2 2025 Financial Results & Investor Presentation
Bridging biosciences and deep learning for revolutionary discoveries: a compr...
Peak of Data & AI Encore- AI for Metadata and Smarter Workflows
Machine learning based COVID-19 study performance prediction
solutions_manual_-_materials___processing_in_manufacturing__demargo_.pdf
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
[발표본] 너의 과제는 클라우드에 있어_KTDS_김동현_20250524.pdf
Network Security Unit 5.pdf for BCA BBA.
cuic standard and advanced reporting.pdf
How UI/UX Design Impacts User Retention in Mobile Apps.pdf
Advanced IT Governance
“AI and Expert System Decision Support & Business Intelligence Systems”
Advanced methodologies resolving dimensionality complications for autism neur...
NewMind AI Weekly Chronicles - August'25 Week I
20250228 LYD VKU AI Blended-Learning.pptx
Big Data Technologies - Introduction.pptx
Understanding_Digital_Forensics_Presentation.pptx

Overview of analytics and big data in practice

  • 1. Overview of Big Data and Analytics in practice Vivek Murugesan
  • 2. Contents ● What is Big Data? ● What is Analytics? ● Why do companies bother? ● Why now? ● Industries & Use cases ● Why should I bother?
  • 3. Big data is about these 4 Vs
  • 5. Analytics Is the process of iterative, methodical exploration of an organization’s data with emphasis on statistical Analysis. To enable data-driven decision making.
  • 6. Why do companies bother…?
  • 7. Why do companies care…? ● Digital innovation and disruptions ○ Netflix vs Blockbuster ○ Amazon’s disruptive innovation ○ Google vs GPS ○ Traditional advertising vs social media advertising ○ Competitive advantage
  • 8. Why now? ● Storage has become cheaper ● Availability of infrastructure at cloud ● Open source ● Data Science and Machine learning moving beyond research
  • 9. Data everywhere in every domain ❖ Web - content, link structure, clicks ❖ Retail - customer details, point of sale, inventory ❖ Medical - literature, patient history, drug details … ❖ Financial - stocks, currencies, financial news, commodities ❖ Insurance - customer history, claim details … ❖ Telecom - call detail records, customer history & profile … ❖ Banking - customer transactions, profile … ❖ Travel & Hospitality - travel itinerary, schedule …
  • 10. Industries ● Medical, Healthcare and Life Sciences ● Automobile and Manufacturing ● Travel and Hospitality ● Retail and Ecommerce ● Web, Social Media and Digital Media ● Telecommunication ● Banking, Finance and Insurance ● Energy ● Sports, Media and Entertainment ● Niche areas like autonomous driving, image video processing, etc,.
  • 11. Medical, Healthcare and Life Sciences ● Cancer research with pattern recognition on cells ● Clinical trials with millions of compositions for drugs ● Prediction of diseases with tests and probabilistic studies ex: Diabetes and Down syndrome prediction ● Collection and storage of test results like scan reports, blood test reports, etc,. ● Image processing, text processing and complex pattern recognition analysis etc,. ● Analyzing literatures and patents to find out cure for diseases
  • 12. Automobile and manufacturing ● One of the frontrunners of adopting big data and analytics even before the cloud computing (during the cluster computing days) ● Analyzing vast amount, ○ Customer feedbacks ○ Inventory data ○ Repair and life of parts report ○ Competitive information ○ Market research data ● To come out with best design that will sustain long time in the market ● Some of these analysis could run for months together ● Design arrived at will be tested under simulation environment
  • 13. Travel and Hospitality ● Revenue management was one of the technique that resurrected the airline industry that was close to its death during early 90’s ● Similar techniques are used with hospitality industry as well with increasing number of hotels and the kind of competitive market it has became ● Growing number of Online portals shows the amount of competition in this industry ● Data generated and consumed in this industry really huge
  • 14. Retail and Ecommerce ● Inventory tracking across franchises ● Relationship between inventory overrun and discounts ● Recommending right products in subseconds to close the purchase lifecycle of the customers appropriately ● Imagine the scaling problems faced by online retailers like Amazon, Flipkart, etc,. With millions of products and millions customers to handle ● The capability to handle the price elasticity in the market ● Example use case of Best buy vs Amazon
  • 15. Web, Social media and Digital media ● With the amount of tweets and posts that twitter and facebook handle it is daunting task for them to be notifying the right set of people ● The kind of job recommendation and PYMK does by Linkedin is a really hard problem to handle at that scale ● Advertisement industry in the digital media has a really complicated ecosystem, ○ With so many publishers, agencies and advertisements ○ To satisfy so many parameters like number of impressions, CTR, conversion, etc,. ● Such a complicated ecosystem is handling online bidding at micro seconds to choose the advertisement to show for each page
  • 16. Telecommunication ● More than 16 players in India running under a very tight margin in call rate ● For them to get revenue they have to squeeze out interest through every single customer, ○ By targeting them with right offer and promotion at right time ○ They operate at micro segments of size 1000s out of their 160 Million customers ● Huge number mobile subscribers moving all over and making lot of calls ● All these generate a lot of data in the form CDRs, etc,. ● And all of these needs to be processed, stored, analyzed and archived appropriately
  • 17. Banking, Finance and Insurance ● Banks run lot of promotions in the form of sending emails, sms, etc,. To its customers ● They get profit for every single conversion out of these campaigns ● Imaging how hard it is to choose the right set of customers to target with right set offers to maximize the revenues out of these campaigns ● People who work in finance industry like stock market etc,. Has a large volume of data in wide variety of forms to consume to mine for any meaningful insights to come out with right strategy for investment ● Processing claims and detection of frauds is really hard problem to solve at scale ● Insurance firms have started utilizing sophisticated techniques like text processing on the claim statements to detect frauds
  • 18. Energy ● Amount of image processing in analyzing satellite images to locate the point of energy source is humongous ● Any small amount of precision of error can also introduce a huge loss ● Hence the results need to be optimized with huge number of iterations to minimize the error
  • 19. Sports, Media and Entertainment ● Football clubs and IPL franchises have started modeling the players to arrive at optimal strategy to play with ● For example NZ cricket team at some point of time started utilizing the systems to an extent to automate the team selection ● Media and Entertainment needs to be up to date with social media to compete with them and against their peers
  • 20. Online advertising industry Showing the interactions in all directions and the companies playing in the space
  • 21. List of Techniques ● Statistical testing, models (regression, forecasting etc,.) ● Machine learning (pattern recognition, classification, clustering, segmentation, etc,.) ● Application Simulation and Optimization (Revenue management, Supply chain management, set covering, network problems etc,.) ● Recommendation (personalized, non-personalized, association rule mining, etc,.) ● Text analytics, Image processing
  • 22. Few more domains ● Education, Academics, E-learning ● Networking - Security, Adaptive routing ● Niche areas - Autonomous driving, Reinforcement learning, etc,. ● Multimedia - Audio and Video analytics
  • 23. Why Should I bother…?
  • 24. Why should I bother…? ● Industry growing rapidly ● More organizations adopting ● Technology trends ● Skill gap and projection ● Skills getting obsolete
  • 27. Industry 4.0 Cyber-Physical Systems (CPS) are integrations of computation, networking, and physical processes. Embedded computers and networks monitor and control the physical processes, with feedback loops where physical processes affect computations and vice versa.