Big Data Industry Insights 2015
14 likes9,666 views
This presentation by Gartner discusses big data industry insights and trends. It provides an overview of organizations' investments in big data technology, the challenges they face in adoption, the types of big data being analyzed now and planned for the future, and examples of how different industries are using big data to address key business problems.
Big Data Industry Insights 2015
- 1. This presentation, including any supporting materials, is owned by Gartner, Inc. and/or its affiliates and is for the sole use of the intended Gartner audience or other intended recipients. This presentation may contain information that is confidential, proprietary or otherwise legally protected, and it may not be further copied, distributed or publicly displayed without the express written permission of Gartner, Inc. or its affiliates. © 2015 Gartner, Inc. and/or its affiliates. All rights reserved. Big Data Industry Insights Lisa Kart @Kart_Lisa http://denreymer.com
- 2. 20% 10% 0% 30% 40% 50% 60% Has your organization already invested in technology specifically designed to address the big data challenge? Investments in Big Data Technology 100% 90% 80% 70% 2012 n=473 2013 n=720 2014 n=302 2015 n=437 Don't know No plans Plan within 2yrs Plan within 1yr Yes Big Data Investments Continue to Rise but Slowing Down 64% 73% Percentage investing or planning 58% 76% 3 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved.
- 3. Key Issues 4 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved. 1. What are the vertical industry trends around big data? 2. What business problems are top priority in different industries? 3. Where should I focus?
- 4. 5 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved. 55% 55% 53% 48% 47% 44% 42% 41% 39% 38% 26% 17% 15% 16% 14% 12% 14% 8% 23% 28% 23% 14% 7% 9% 13% 14% 6% 17% 21% 18% 33% 8% 21% 21% 19% 19% 24% 29% 17% 29% 18% 15% 33% 1% 1% 6% 8% 15% 5% Don’t know No plans at this time Plan to within 2 yrs Plan to within 1 yr Have invested Has your organization already invested in technology specifically designed to address the big data challenge? Total sample Big Data investment – industry n= 5 Percentage investing or planning Percentage investing Retail/ Svcs Insur- Trans- Health- Banking Edu Manu & Utilities Comm./ Gov Trade ance portation care N.Res. Media 29 78 32 21 17 59 24 80 18 13 57
- 5. 6 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved. Which of the following best describes your organization's stage of big data adoption? 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% n=437 n=720 n=302 Despite the Opportunities, Organizations Struggle with Getting to Deployment 2013 2014 2015 Deployed Piloting and experimenting Developing strategy Knowledge gathering No plans to invest at this time Don’t know 5% 31% 19% 18% 20% 8% 4 % 24% 13% 19% 27% 13% 4 % 21% 13% 18% 30% 14%
- 6. 7 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved. Which of the following best describes your organization’s stage of big data adoption? State of Big Data adoption- by industry 6% Services n=78 1% Manufacturing n=80 1% 29% 21% 19% 1% 12% 18% 6% Percentage piloting or deployed- Total sample 44% 14% 15% 31% 32% 36% 33% 19% 29% 21% 18% 10% 10% 7% Healthcare n=17 Retail/Trade n=29 Education n=24 Utilities n=18 Insurance n=32 Comm./Media n=13 Banking n=59 Government n=57 Don't know if currently investing Currently investing/planning- don't know adoption stage Developing strategy Deployed 5% 14% 33% 12% 18% 18% 21% 17% 2% 15% 12% 52% 5% Transportation n=21 No plans to invest Knowledge gathering Piloting and experimenting 7 Percentage full deployment- Total sample 14% 15% 15% 0% 31% 23% 15% 19% 6% 9% 47% 16% 22% 39% 22% 17% 29% 17% 17% 21% 17%
- 7. 8 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved. Rank 1 Rank 2 Rank 3 What are your organization’s top 3 hurdles or challenges with big data? The Top Big Data Challenge Remains the Same Sum Determining how to get value from big data 33% 13% 9% 55% Obtaining skills and capabilities needed 6% 16% 13% 36% Risk and governance issues (security, privacy, data quality) 11% 11% 11% 33% Funding for big data-related initiatives 9% 12% 10% 31% Defining our strategy 7% 11% 13% 31% Integrating multiple data sources 6% 11% 8% 26% Integrating big data technology with existing infrastructure 3% 8% 13% 25% Infrastructure and/or architecture 5% 7% 10% 22% Leadership or organizational issues 6% 4% 8% 18% Understanding what is "Big Data" 10% 3%2% 15% Other 3% 3%3% 9%
- 8. 9 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved. Big Data Challenges Are More Practical As You Adopt 31% 20% 20% 10% 17% 36% 37% 42% 16% 21% 62% 14% 22% 20% 25% 20% 33% 30% 43% 54% 9% 14% 23% 30% 34% 23% 23% 44% 39% 53% Understanding what is "Big Data" Leadership or organizational issues Infrastructure and/or architecture Integrating big data technology with existing infrastructure Integrating multiple data sources Funding for big data-related initiatives Defining our strategy Risk and governance issues (security, privacy, data quality) Obtaining skills and capabilities needed Determining how to get value from big data Have invested (n=192) Planning (n=138) No plans (n=86) 9
- 9. 67% 47% 26% 24% 23% 22% 18% 17% 12% 20% 33% 42% 39% 36% 40% 45% 41% 34% 87% 80% 68% 63% 59% 62% 64% 58% 46% Transactions Log data Geospatial/location data Social media profile data Emails/documents Social media chat/interaction data Sensor/machine-generated data (Internet of Things) Free-form text Images Currently Analyze (n=195) Not analyzing today but plan to analyze in the future (n=138) 10 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved. Which types of big data does your organization currently analyze and which do you plan to add to your analytics in the future? SUM Types of Big Data Analyzed- now vs. planned Multiple responses allowed Audio 9% 32% 41% Video 8% 33% 41% Other 10% 6% 16%
- 10. Key Issues 11 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved. 1. What are the vertical industry trends around big data? 2. What business problems are top priority in different industries? 3. Where should I focus?
- 11. Heatmap of Big Data Business Problems by Industry 12 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved. Multiple responses allowed Manu & N. Res. Media/ Comm Svcs Gov. Edu Retail Banking Insur- ance Health- care Trans- portation Utilities Enhanced customer experience 52% 78% 66% 43% 76% 83% 77% 77% 73% 69% 44% Process efficiency 45% 33% 35% 49% 65% 43% 41% 50% 73% 69% 78% More targeted marketing 43% 89% 53% 17% 41% 78% 66% 58% - 38% 17% Cost reduction 42% 33% 35% 37% 35% 30% 41% 31% 45% 56% 61% Improved risk management 14% 22% 29% 29% 35% 22% 52% 58% 55% 31% 61% New products 23% 67% 37% 14% 24% 35% 27% 50% - 19% 33% Developing information products 26% 33% 44% 31% 12% 22% 23% 19% 9% 19% 11% Enhanced security capabilities 17% 22% 21% 34% 29% 13% 27% 27% 9% 19% 28% Regulatory compliance 11% 22% 18% 23% 18% 9% 25% 23% 27% 31% 44% n= 65 9 62 35 17 23 44 26 11 16 18
- 12. Tackling the problem of game scheduling § Opportunity – Schedule NFL games to maximize profit § Data and Analytics – 20,000 variables and 50,000 constraints were analyzed using ‘FICO Xpress Optimization’ suite to come up with optimized schedule while evaluating 7000 game options – Best games schedules were selected which can fetch higher TV ratings and revenue opportunities § Results – NFL’s revenue and sponsorship grew substantially after using the solution in the last five years – Saves on time as new schedule can be produced in 24 hours, a task which could take months earlier 13 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved.
- 13. § Opportunity – A rapidly growing Turkish credit card business targeting lower value segments caused an increase in fraud § Data and Analytics – Replaced manual process of credit card application review with automated real-time scoring and flagging – Increase from 13% to 100% of applications reviewed – Implemented fraud modeling in 15 days using KXEN § Results – Increased number of identified actual fraudulent applications by 3x; 92% of fraud cases identified – Reduced number of fraud alerts from 300,000 to 30,000 per quarter by tuning and discovering new patterns – Saving $25,000 per day; ROI achieved in one week Finding Fraud Faster 14 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved.
- 14. § Opportunity – Improve citizen safety and save city resources § Data and Analytics – The New York City Fire Department algorithm analyzes 2400 factors from 330,000 commercial and public buildings – Determines a risk score that guides inspectors to prioritize certain buildings and their likely fire safety issues § Results (TBD) – 70% success in identifying fire hazards in buildings – Reduce fires and other safety related events; Save on personnel and firefighting resources; Reduce insurance claims Heat Mapping Potential Fire Risk Hotspots 15 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved.
- 15. § Opportunity – Offer credit to the underbanked – those without a credit history § Data and Analytics – A sophisticated self-learning scoring model – Up to 15,000 dynamic data points for each individual, including social networks, mobile usage, location, e-commerce data etc. § Results – Ability to lend to the 73% of those people with no traditional credit scores – <7% loss rate in established markets (lower than using traditional credit scores alone) – Loan payments in 15min vs 1-3 days Giving Credit When Credit is Due 16 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved.
- 16. § Opportunity – Improve health and well-being of dogs § Data and Analytics – Whistle: Wearable device and mobile app that tracks behavior of your dog and compares it to baselines of similar breeds & ages. – FitBark: Simply, FitBit for dogs. Its "BarkScore" tells you how much activity Rover has had each day. § Results – Alerts owners to possible health issues before they become evident – How much is your dog walking, playing, resting? – Is your dog walker doing her job? Big Data and Analytics Have Gone to the Dogs 17 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved.
- 17. § Opportunity – Increase the value, breadth, quality and consistency of company valuation information available to subscribers of the Credit Suisse Holt Platform § Data and Analytics – Generates and updates natural language narratives in real-time to ensure they reflect company’s latest performance using Narrative Science’s natural language generation platform – Applies proprietary investment-research methodology to describe the operational quality, valuation, and riskiness of companies using data, risk, valuation and operations data § Results – Increased company research available to subscribers by 250%; eliminated issues with quality and consistency of language Improving Quality of Financial Research, Naturally 18 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved.
- 18. § Opportunity – Understand activities within a building, without having to go inside § Data and Analytics – Indoor Atlas maps the interior of a building using the unique magnetic “fingerprint” of the structure caused by distortions of the Earth’s magnetic field. – Determines location of internal structures and people to within six feet, including which floor § Results – Emergency personnel can use a Google Map overlay to navigate a building – Competitors can see changes to layouts and foot traffic – Pay $99/mo to keep your building’s magnetic map private Buildings Have Magnetic Personalities Too 19 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved.
- 19. § Opportunity – Lack of consistent pricing, long negotiations and slow quote turnaround for deals § Data and Analytics – PROS pricing solution integrated millions of historical transactions along with attributes to create pricing segments and derive pricing recommendations – Enhanced pricing envelope for over 500K specific price points based on the product, customer and deal type available on demand for new quotes § Results – Customer-specific pricing based model led to 2% increase in profit margin – Reduced quote turnaround time by 50% enabling auto-approvals, leaving pricing analysts more time to spend on newer customers and strategic deals An Offer They Cannot Refuse 20 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved.
- 20. § Opportunity – Improve reliability of the electric grid and the utilization of energy to meet state renewables goals § Data and Analytics – Real-time visualization and analysis of 25,000 miles of power lines – Space-Time Insight, OSIsoft, Oracle – Hourly reforecasting of generation needs based on wind and solar estimates; real-time alerts for crisis conditions § Results – No system-wide outages since implementation – Enabled implementation of 4,000 pricing nodes (up from 5) to facilitate cost-effective local market pricing – 50% improvement in renewable forecast accuracy Keeping the Lights on in California 21 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved.
- 21. § Opportunity – Improve anti-money laundering investigation speed, cost and transparency § Data and Analytics – Automated data access, investigation and analytics via Pneuron in single integrated “fabric”. Data from internal and external sources are presented as one complete package without replacing or rewriting existing hardware and software investments – Avoid need for data centralization by pushing the analytics to the data & employing non-invasive late-binding dynamic scoring model § Results – Increased speed and reduced cost of investigation by 60% – All AML alerts are consistent, auditable and documented, and thresholds now can be set to zero A Fresh, Clean Approach to Optimized Anti-Money Laundering Performance Multinational Financial Ins titution 22 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved.
- 22. § Opportunity – Create defensible revenue forecast models to pass Comprehensive Capital Analysis and Review (CCAR) “stress test” § Data and Analytics – Correlated and analyzed 2600 macro-economic variables with revenue streams for dozens of business units using Ayasdi’s machine intelligence software – Uncovered variable permutations that were hard to identify using incumbent analytics approaches and shortened the variable selection process – from three months to two weeks § Results – Achieved the cleanest Federal Reserve test pass of top US banks – Citi stock added $9B in market capitalization the following day; and announced a dividend increase of 500% Financial Stress Tests: Stressed No More 23 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved.
- 23. 13% 4% 17% 35% 34% 38% 18% Don’t know Other ROI is/will be measured by the non-financial impact to the company. ROI is/will be measured by improvements to decision-making or process efficiency. ROI is/will be measured by improvements to organizational effectiveness. ROI is/will be measured by financial returns ROI is not being measured/We do not plan to measure ROI 41% of Organizations Don't Know if Big Data ROI Will Be Positive or Negative Multiple responses allowed Positive ROI 57% Negative ROI, 3% Don't know, 41% ROI Measurement 24 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved. Positive/negative ROI How is ROI being measured/How will ROI be measured for your organization’s big data investment? Please indicate whether your organization's big data investment has /is expected to have - a positive or negative ROI?
- 24. 25 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved. How is ROI being measured/How will ROI be measured for your organization’s big data investment? ROI measurement- by industry Retail/Trade n=23 Comm./Media n=9 Insurance n=26 Services n=62 Manufacturing n=65 Utilities n=18 Government n=35 Banking n=44 Transportation n=16 Education n=17 Healthcare n=11 ROI is not being measured/We do not plan to measure ROI ROI is/will be measured 25 Please indicate whether your organization's big data investment has /is expected to have - a positive or negative ROI? 36% 40% 47% 51% 54% 61% 61% 63% 66% 70% 89% 3% 2% 4% 6% 2% 6% 2% 9% 64% 53% 48% 42% 33% 36% 31% 32% 11%Comm./Media n=9 Retail/Trade n=23 Services n=62 Transportation n=16 Banking n=44 Utilities n=18 Insurance n=26 Manufacturing n=65 Education n=17 Government n=35 Healthcare n=11 Positive ROI Negative ROI Don't know 9% 87% 22% 78% 12% 77% 15% 73% 18% 72% 22% 67% 14% 66% 20% 64% 38% 63% 18% 59% 36% 36%
- 25. Key Issues 26 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved. 1. What are the vertical industry trends around big data? 2. What business problems are top priority in different industries? 3. Where should I focus?
- 26. It's All About Analytics Maturity Hybrid, integrated Unstructured, external Structured, internal, siloed Ad hoc, batch, offline analytics Pervasive, real-time, embedded analytics Data Increase analytics maturity by: • Analyzing new data sources • Broadening your portfolio of analytic capabilities • Applying analytics to more decisions, faster Descriptive Diagnostic Predictive Prescriptive 27 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved.
- 27. Build Your Portfolio of Analytics Capabilities Human Input Data Decision Predictive What will happen? Diagnostic Why did it happen? Descriptive What happened? Prescriptive What should I do? Analysis Action DecisionAutomation 28 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved. Decision Support
- 28. Future Past Apply Relevant Data and Analytics to Decision Making Create Awareness; a Decision Must Be Made Understand the Scope and Context of the Decision Identify Likely Outcomes Identify the Best Course of Action Report on the Results of the Action Descriptive PredictivePrescriptive Diagnostic 29 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved.
- 29. Three Core Skill Areas Are Needed Analytics Skills IT Skills Business Skills Ask Good Questions Decision Making Transparent Versus "Black Box" Which Analytics to Choose? Data Exploration High- Performance Computing Build, Buy, Outsource Feature Engineering Know the Constraints (E.g., Legal, Ethics, Market) Data Governance Deployment 30 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved.
- 30. 8% 5%5%6% 8% 10%10%10%10%11%11%12% 15%16% 31%32% 3% 5% 5% 5% 6% 7% 11% 8% 5% 8% 13% 15% 17% 16% 25% 37% 2015 survey (n=333) 2014 survey (n=218) 31 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved. Please indicate who initiated your organization’s big data initiatives Both IT and Business Should Drive Big Data Initiatives Multiple responses allowed
- 31. Art of the Possible Seek Big Ideas Beyond Your Borders I waited to see what leaders in our industry were doing with data I came up with some great ideas for using data on my own I worked with business partners to develop new ways to use data We adopted and adapted winning ideas from other industries for using data 32 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved.
- 32. Key Takeaways 33 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved. üBig data is maturing, moving away from the challenges of data volume and variety, toward getting value. üIncrease analytics maturity by: - Integrating and analyzing new relevant data sources. - Moving beyond basic BI to diagnostic, predictive and prescriptive analytics. - More closely tying insights to business decisions. üBuild your strategy around use cases, business goals and outcomes. The decisions will guide the required data and analytics.
- 33. Recommended Gartner Research 34 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved. è Survey Analysis: Practical Challenges Mount as Big Data Moves to Mainstream Nick Heudecker, Lisa Kart (G00289494) è Survey Analysis: Big Data Investment Grows but Deployments Remain Scarce in 2014 Nick Heudecker, Lisa Kart (G00263798) è Survey Analysis: Big Data Adoption in 2013 Shows Substance Behind the Hype Lisa Kart, Nick Heudecker, Frank Buytendijk (G00255160) è Toolkit: Big Data Business Opportunities From Over 100 Use Cases Frank Buytendijk and others (G00252112) è Extend Your Portfolio of Analytics Capabilities Lisa Kart and others (G00254653) For more information, stop by Gartner Research Zone.
- 34. 35 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved. Questions?
- 35. 36 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved. Appendix: Methodology & Respondent Profile
- 36. 37 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved. What Is "Big Data"? "Big data" is: • high-volume, high- velocity and high- variety information assets • that demand cost- effective, innovative forms of information processing • for enhanced insight and decision making. VOLUME VELOCITY VARIETY
- 37. Methodology 38 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved. 38 § This research was conducted via online survey in June 1-16, 2015 among Gartner Research Circle Members – a Gartner-managed panel comprised of IT and business leaders. § In total, 437 members participated by indicating their organization’s investment plans around technology to support big data: DEFINITION PRESENTENTED: Gartner defines “big data” as high volume, velocity and variety information assets that demand cost-effective, innovative forms of information processing for enhanced insight and decision-making. Ø 333 have invested, or plan to invest within the next two years Ø 91 do not have plans to invest at this time Ø 13 did not know § The survey was developed collaboratively by a team of Gartner analysts covering ITL Enterprise Software, and was reviewed, tested and administered by Gartner’s Research Data Analytics team. NOTE: The results of this study are representative of the respondent base and not necessarily the market as a whole.
- 38. 18% 18% 14% 13% 7% 5% 5% 5% 4% 4% 2% 1% 1% 2% Manufacturing & Natural Resources Services Banking Government Insurance Education Retail Transportation Utilities Healthcare Providers Communications Media Wholesale Trade Other Respondent profile – company characteristics n=437 Primary industry: Annual Revenue: DK, 12% Non profit, 11% # employees worldwide: DK, 0.00686498 9 <1000, 23% <$500M, 25% $500M - $1B - $3B $1B, 6% , 13% $3B - $10B, 13% $10B +, 19% n=437 1000-9999, 38% 39 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved. 39 10000 +, 39% n=437 MEAN=9016 MEAN=$4101M
- 39. Respondent profile – region 14% APAC 42% EMEA 40% N. America 5% LaHn America 40 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved. 40 n=437 Unknown 1%
- 40. Technology adoption profile Aggressive 16% 41 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved. 41 Mainstream 63% Conservative 21% n=437
- 41. 6% 43% 51% Primarily business- focused Blend of business and IT Primarily IT-focused 42 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved. 42 Respondent profile – job function IT roles: Business roles: n=437 Enterprise Architecture 42% IT Infrastructure and Operations 36% Applications: Development, Integration, CRM, ERP, SCM and Portals, Content and Collaboration 30% Business Intelligence and Information Management 30% C-level IT Executive Leadership 28% Security, Risk & Governance 27% Sourcing and Vendor Relationships 25% Business Process Improvement 24% Program and Portfolio Management 24% Other 5% Business Strategy 53% Business Unit/Executive Leadership 36% Product Development and Management 21% Marketing 11% Sales 8% Other 17%