Big data analytics

25-07-2019Big Data Analytics by Vikram Neerugatti1

Big Data Analytics
Vikram Neerugatti
Sri Venkateswara University, Tirupati.
vikramneerugatti@gmail.com
vikram@smartnutsandbolts.com
www.vikramneerugatti.com
www.smartnutsandbolts.com
Vikram Neerugatti
Vikram Nandini

Content
 What is Big Data
 Varieties of Data
 Unstructured Data
 Trends in Data Storage
 Industry Examples of Big Data

What is Big Data
Two men operating a mainframe computer, circa 1960. It’s amazing how
today’s smartphone holds so much more data than this huge 1960’s
relic. (Photo by Pictorial Parade/Archive Photos)

What is Big Data
 Big Data is the next generation of data
warehousing.
 twenty-first century, when the Age of Big Data
Analytics was in its infancy.
 It ’s not an overnight phenomenon.
 Reasons for now
 Computing perfect storm
 Data perfect storm
 Convergence perfect storm

Computing perfect storm
 Big Data analytics are the natural result of four
major global trends:
1. Moore ’s Law (which basically says that
technology always gets cheaper),
2. mobile computing (that smart phone or mobile
tablet in your hand),
3. social networking (Facebook, Foursquare,
Pinterest, etc.),
4. and cloud computing (you don ’t even have to own
hardware or software anymore; you can rent or
lease someone else ’s).

Data Perfect Storm
 Volumes of transactional data have been around
for decades for most big firms, but the flood gates
have now opened
 with more volume , and the velocity and variety—
the three Vs—of data that has arrived in
unprecedented ways.
 This perfect storm of the three Vs makes it
extremely complex and cumbersome
 with the current data management and
 analytics technology and practices.

Convergence perfect storm
 Traditional data management and analytics
software and hardware
 technologies, open-source technology, and
commodity hardware
 are merging to create new alternatives for IT and
business executives
 to address Big Data analytics.

What is Big Data
 People are able to store that much data now and
more than they ever before.
 We have reached this tipping point where they
don ’t have to make decisions about which half to
keep or how much history to keep.
 It ’s now economically feasible to keep all of your
history and all of your variables and go back later
when you have a new question and start looking
for an answer.
 That hadn ’t been practical up until just recently.

What is Big Data
 Certainly the advances in blade technology and
the idea
 that Google brought to market of you take lots
and lots of small Intel
 servers and you gang them together and use
their potential in aggregate.
 That is the super computer of the future.

Evolution of data systems
 Dependent (Early Days).
 Data systems were fairly new and users didn't
know quite know what they wanted. IT assumed
that “Build it and they shall come.”
 Independent (Recent Years).
 Users understood what an analytical platform was
and worked together with IT to define the business
needs and approach for deriving insights for their fi
rm.
 Interdependent (Big Data Era).
 Interactional stage between various companies,
creating more social collaboration beyond your
firm’s walls.

Big data
 Here is how the McKinsey study defi nes Big
Data:
 Big data refers to datasets whose size is beyond
the ability of typical
 database software tools to capture, store,
manage, and analyze.
 big data in many sectors today will range from a
few
 dozen terabytes to multiple petabytes (thousands
of terabytes). 2

Big Data Analytics
 The real challenge is identifying or developing
most cost-effective and reliable methods for
extracting value from all the terabytes and
petabytes of data now available.
 That ’s where Big Data analytics become
necessary.
 Comparing traditional analytics to Big Data
analytics is like comparing a cart to a tractor
 The differences in speed, scale, and complexity
are tremendous

Why now?
Timeline of Recent Technology Developments

Big Data
 The industry has an evolving definition around
Big Data that is currently defined by three
dimensions:
 1. Volume
 2. Variety
 3. Velocity

Big Data
 Volume
 Data variety is the assortment of data.
 Traditionally data, especially operational data, is
“structured” as it is put into a database based on the type
of data (i.e., character, numeric, floating point, etc.).
 Over the past couple of decades, data has increasingly
become “unstructured” as the sources of data have
proliferated beyond operational applications.
 Oftentimes, text, audio, video, image, geospatial, and
Internet data (including click streams and log files) are
considered unstructured data .
 Semi-structured data
 customer name + date of call + complaint
 Velocity

Varieties of Data
 The variety of data sources continues to increase.
 Internet data (i.e., clickstream, social media, social
networking links)
 Primary research (i.e., surveys, experiments,
observations)
 Secondary research (i.e., competitive and
marketplace data, industry reports, consumer data,
business data)
 Location data (i.e., mobile device data, geospatial
data)
 Image data (i.e., video, satellite image, surveillance)
 Supply chain data (i.e., EDI, vendor catalogs and
pricing, quality information)
 Device data (i.e., sensors, PLCs, RF devices, LIMs,

Varieties of Data
 The wide variety of data leads to complexities in
ingesting the data into data storage.
 The variety of data also complicates the
transformation (or the changing of data into a
form
 that can be used in analytics processing) and
analytic computation of the processing of the
data.

Unstructured Data
 structured data (the kind that is easy to define,
store, and analyze)
 Unstructured data (the kind that tends to defy
easy definition, takes up lots of storage capacity,
and is typically more difficult to analyze).
 Unstructured data is basically information that
either does not have a
 predefined data model and/or does not fi t well
into a relational database.
 Unstructured information is typically text heavy,
but may contain data such as dates, numbers,
and facts as well.

Unstructured Data
 The term semi-structured data is used to describe
structured data that doesn't ’t fit into a formal
structure of data models.
 However, semi-structured data does contain tags
that separate semantic elements, which includes
the capability to enforce hierarchies within the
data.

Unstructured Data
 but here are the main takeaways that we would like to
share with you:
 The amount of data (all data, everywhere) is doubling every
two years.
 Our world is becoming more transparent. We, in turn, are
beginning to accept this as we become more comfortable with
parting with data that we used to consider sacred and private.
 Most new data is unstructured. Specifically, unstructured data
represents almost 95 percent of new data, while structured
data represents only 5 percent.
 Unstructured data tends to grow exponentially, unlike
structured data, which tends to grow in a more linear fashion.
 Unstructured data is vastly underutilized. Imagine huge
deposits of oil or other natural resources that are just sitting
there, waiting to be used. That ’s the current state of
unstructured data as of today. Tomorrow will be a different
story because there ’s a lot of money to be made for smart
individuals and companies that can mine unstructured data

Is Big Data analytics worth the effort?
 Yes, without a doubt Big Data analytics is worth
the effort.
 It will be a competitive advantage, and it ’s likely
to play a key role in sorting winners from losers in
our ultracompetitive global economy.

From a business perspective, you
’ll need to learn how to:
 Use Big Data analytics to drive value for your
enterprise that aligns with your core competencies
and creates a competitive advantage for your
enterprise
 Capitalize on new technology capabilities and
leverage your existing technology assets
 Enable the appropriate organizational change to
move towards fact based decisions, adoption of new
technologies, and uniting people from multiple
disciplines into a single multidisciplinary team
 Deliver faster and superior results by embracing and
capitalizing on the ever-increasing rate of change that
is occurring in the global market place

Big Data analytics uses a wide
variety of advanced analytics

Advanced Analytics to provide:

Big Data Business Models

Enabling Big Data Analytic
Applications

The key to success for organizations seeking to
take advantage of this opportunity is:
 Leverage all your current data and enrich it with
new data sources
 Enforce data quality policies and leverage today’
s best technology and people to support the
policies
 Relentlessly seek opportunities to imbue your
enterprise with fact based decision making
 Embed your analytic insights throughout your
organization

Trends in Data Storage
 Following are differing types of storage systems:
 Distributed File Systems
 NoSQL Databases
 NewSQL Databases
 Big Data Querying Platforms

 Big Data Querying Platforms:
 Technologies that provide query facades infront of
big data stores such as distributed file systems or
NoSQL databases.
 The main concern is providing a high-level
interface, e.g. via SQL3 like query languages and
achieving low query latencies.

Industry Examples of Big Data
 collected from thought leaders in subjects and
industries such as
 Digital Marketing
 Financial Services
 Advertising
 and Healthcare.

Digital Marketing and the Non-line
World
 Don ’t Abdicate Relationships
 Is IT Losing Control of Web Analytics?

Database Marketers, Pioneers of
Big Data
 Database marketing is really concerned with
building databases containing information about
individuals, using that information to better
understand those individuals, and communicating
effectively with some of those individuals to drive
business value.

Big Data and the New School of
Marketing
 New School marketers deliver what today ’s
consumers want: relevant interactive
communication across the digital power channels:
email, mobile, social, display and the web.
 Consumers Have Changed. So Must
Marketers
 The Right Approach: Cross-Channel Lifecycle
Marketing
 It really starts with the capture of customer
permission, contact information, and preferences for
multiple channels.

Lifecycle Marketing approach: conversion, repurchase,
stickiness, win-back, and re-permission

Marketing
 Social and Affiliate Marketing
 Empowering Marketing with Social
Intelligence

Fraud and Big Data
 One of the most common forms of fraudulent
activity is credit card fraud.
 The credit card fraud rate in United States and
other countries is increasing.
 In order to prevent the fraud, credit card
transactions are monitored and checked in near
real time.
 If the checks identify pattern inconsistencies and
suspicious activity, the transaction is identified for
review and scalation.

Fraud and Big Data
 Big Data technologies provide an optimal
technology solution based on the following three
Vs:
 High volume. Years of customer records and
transactions (150 billion1 records per year)
 High velocity. Dynamic transactions and social
media information
 High variety. Social media plus other unstructured
data such as customer emails, call centre
conversations, as well as transactional structured
data

Risk and Big Data
 The two most common types of risk management
are credit risk management and market risk
management.
 A third type of risk, operational risk management,
isn’t as common as credit and market risk.
 Credit risk analytics focus on past credit
behaviors to predict the likelihood that a borrower
will.
 Market risk analytics focus on understanding the
likelihood that the value of a portfolio will
decrease due to the change in stock prices,
interest rates, foreign exchange rates, and
commodity prices.

Credit Risk Management

Big Data and Advances in Health
Care

Any questions

Big data analytics

More Related Content

What's hot (20)

Similar to Big data analytics (20)

More from Vikram Nandini (20)

Recently uploaded (20)

Big data analytics