Big Data in Context

Big Data in Context
Data-intensive challenges: what’s hot and what’s hype?

The Register: Intel Academy
22nd November 2011

Martha Bennett
VP, Head of Strategy
Freeform Dynamics Ltd

www.freeformdynamics.com

Copyright 2011 Freeform Dynamics Ltd

Topics
 Data, data everywhere …
 The concept of value density – defining ‘Big
Data’
 The emerging ‘Big Data’ ecosystem
 A blended future
 Opportunities and challenges

2

What level of growth are you seeing in the following
types of data within your organisation?
0% 20% 40% 60% 80% 100%

Structured data (e.g. tabular
data in RDBMSs)

Unstructured data (e.g.
documents, messages,
multi-media, etc)

5 Extremely high growth 4 3 2 1 No growth Unsure
Online survey of Register readers, 122 respondents, first half of November 2011

In what form is your organisation’s most valuable or
critical data held (i.e. your crown jewels in
information terms)?
0% 20% 40% 60%

Exclusively in structured format

Mostly in structured format

Equal split

Mostly in unstructured format

Exclusively in unstructured format (No responses)


Considered overall, to what degree does your
organisation exploit its information assets for analysis
and decision making purposes?
0% 20% 40% 60% 80% 100%

Structured
data

Unstructured
data

5 Fully 4 3 2 1-Poorly Unsure

Do you make use of high volume streams or feeds of
data from external sources?

Yes, extensively
14%
Yes, in some
Unsure areas
11% 22%

No, and don't No, but expect
expect this to this to become a
change requirement
33% 20%


The concept of value density – Big Data today
Low value density High value density

Traditional BI and data
Structured Structured
warehousing

BIG
DATA

Traditional search and Unstructured
Unstructured
document management

Low value density High value density
7

The three Vs of Big Data

High physical
Volume volumes with low
Rule of thumb value density
Generally think
of Big Data Different sources and
Variety formats or information
when two of
these three
apply Rapid rate of data
Velocity movement, generation
or acquisition

8

Introducing ……

9

Source: The Register http://www.theregister.co.uk/2011/11/09/hadoop_kernel_distro/
10

Zooming out

Distributed data • For executing parallel distributed data analysis on
scale-out storage architectures
analytics

Distributed • For implementing fast distributed object storage and
indexing and retrieval on scale-out storage architectures
(including the so called ‘No SQL’ approach)
search
Scale-out • Storage systems able to scale linearly through
adding software, processing and storage elements
storage integrated into standard building blocks under the
architectures control of a common management layer

11

How much are the following used in your organisation
today, and how will the level of use change over the next 3
years?
CURRENT LEVEL OF USE CHANGE IN NEXT 3 YEARS
0% 20% 40% 60% 80% 100% -60% -30% 0% 30% 60%

Legacy databases & file systems

General purpose RDBMS
High performance RDBMS

OLAP multi-dimensional database

In memory databases
Specialist WORM databases

Stream processing engines

Scale-out storage architectures

Distributed indexing and search

Distributed data analytics engines

5 Extensive use 4 3 2 Lower Higher

How much do you agree or disagree with the
following statements?
0% 20% 40% 60% 80% 100%

Developments in advanced storage,
access and analytics are allowing us to
tackle problems today that were either
too hard or too expensive to deal with
in the past

Developments in advanced storage,
access and analytics are allowing us to
take different and better approaches to
tackling some key business
requirements

5-Totally agree 4 3 2 1-Totally disagree Unsure


A less prescriptive approach to analytics

Prescriptive Non-prescriptive
Traditional schema- Exploratory approach
based analysis based in which you don’t
on pre-defined know what you are
structures, reports, hy looking for until you
potheses, etc find it.

 Both analytical paradigms have their place

14

Blending approaches

Traditional
Exploratory
Structured data
‘Big Data’ derivative data set warehousing
style analysis
environment

 Big data feeding traditional BI systems
 But it can also go the other way
15

Do you see cloud based services helping with any of
the following in relation to more advanced
storage, access and analytical solutions?
0% 20% 40% 60% 80% 100%

Rapid and cost effective setup and disposal
of environment for familiarisation, training,
experimenting and piloting

Access to high-end configurations for
production systems that would not be cost
effective to install on premise

Rapid and cost effective setup and disposal
of environment for dealing with transient
needs within the business

Removing concerns about scalability, e.g.
running out of capacity for handling large data
sets and heavy processing loads

Big benefit Some benefit No benefit Don't know N/A - cloud not for us at all

Issues around rhetoric and perception
0% 20% 40% 60% 80% 100%

Regardless of substance and reality of
emerging technologies and
techniques, the term ‘big data’ is
currently being over-hyped by IT
vendors in an unhelpful way

The emergence of advanced storage,
access and analytics solutions means
the end of the traditional RDBMS



More practical challenges
0% 20% 40% 60% 80% 100%

Vendors and consulting firms are well
geared up to providing us with the
support and services we need to take
advanced storage, access and
analytics on board effectively

Database vendors are well geared up
to support their customers with
appropriate licensing and commercial
arrangements as data related needs
continue to evolve and become more
demanding



In conclusion
 Data volumes will continue to increase
 New architectures and technologies have an
important role
 Key challenges
Lack of skills
Security
Risk of ‘standards’ proliferation
 It’s not about “or”, it’s about “and”

19

Contact
Martha Bennett
11 The Willows
New Milton, Hampshire
BH25 7BE
Mobile +44 7944 663344
martha@freeformdynamics.com
Twitter: @martha_bennett

20

Appendix: Survey Demographics

21

What is the size of your organisation?

NA - Responding More than 50,000
based on employees
experience with 12%
multiple clients of
varying sizes
16%
5,000 to 50,000
employees
17%

Under 50
employees
14%

250 to 5,000
50 to 250 employees
employees 25%
16%


What industry sector does your organisation operate in?
0% 10% 20%

Financial services
Professional services
High tech
Media/entertainment (inc film, gaming, etc)
Research and/or educational establishment
Medical / health care
Telecommunications
Central government
Travel and transport
Aerospace and defence
Local government
Energy
Retail/distribution
Architecture, engineering or construction
Other
NA - working with multiple clients across a number of
industries


Sample by region
Rest of
World
12%

UK
55%
USA
16%

Rest of
Europe
17%


Contact
Martha Bennett
11 The Willows
New Milton, Hampshire
BH25 7BE
Mobile +44 7944 663344
martha@freeformdynamics.com
Twitter: @martha_bennett

25

Big Data in Context

More Related Content

What's hot (18)

Viewers also liked (7)

Similar to Big Data in Context (20)

More from Freeform Dynamics (11)

Recently uploaded (20)

Big Data in Context