Big data may 2012
- 1. Big Data opportunities for Market Research
- 2. Q. How big is Big Data ?
- 3. Computer Science 101 Byte B 100 1 Kilobyte KB103 1,000 Megabyte MB 106 1,000,000 Gigabyte GB 109 1,000,000,000 Terabyte TB 1012 1,000,000,000,000 Petabyte PB 1015 1,000,000,000,000,000 Exabyte EB 1018 1,000,000,000,000,000,000
- 4. A. Bigger than Shakespeare? t x 1000 = x 1000 = x5= 1B 1 KB 1 MB 5 MB
- 5. A. Bigger than your pocket? 5 MB x 100 = x2= x 60 = 500 MB 1 GB 60 GB
- 6. A. Bigger than the known universe? 60GB x 20 = x 140 = 1 TB 140TB
- 7. A. Bigger than a day at Google? x 11 = x 1.5 = 140TB 1.5PB 2.5PB x 5-10 = or 13PB/year 20PB/day
- 8. A. Bigger than the sum of human knowledge? 20PB/day x 250 = All words ever uttered by the human race since the beginning of time 5EB
- 9. A. Bigger than the Internet? All words ever uttered by the human race since the beginning of time 5EB x 100 = All data to flow across the Internet this year 500EB
- 10. Pause to think… These were the biggest data sets I could find statistics for and both would be good raw material for Market Research if we could find a big enough table to put them in
- 11. There is a simpler answer Q. How big is Big Data? A. Bigger than we can easily handle (and usually unstructured)
- 12. Why now? More activities are digital, creating “data exhaust” More sensor devices creating digital data: “chips with everything” More connectivity: data can be networked Storage is cheap and getting cheaper
- 13. Big Data means different things Scientists: new frontiers of knowledge IT industry: projects > 1 PB Investors: opportunity for growth Commerce: efficiency, decision-making Google: business as usual
- 14. Market leaders in commercial Big Data Data ownership Data Analytics Data Storage
- 15. Commercial applications for Big Data Micro-segmentation / mass customisation Predictive propensity modelling Digital marketing Pricing optimisation Operational performance improvement Forecasting Product improvement / development
- 16. The Big Data hypothesis for Market Research “The availability of large quantities of consumer data will allow us to generate new and/or lower cost consumer insights through analysis of that data”
- 17. Big Data sets for Consumer Insight Social media Web traffic Transactional Geodemographic &geolocation
- 18. And let’s not forget qual and ethnography Social media Blogs
- 19. A change in research process and mindset Data on real world outcomes Controllable sample Analytics Statistics Actionable insights Extendable conclusions Hypothesis-led / Fact-led / inductive Deductive
- 20. Transferable Research skills Understanding client needs Asking/framing the right questions Knowing what to look for Interpretation Synthesising insights
- 21. And researchers have a grasp of statistical techniques used in data analysis Pattern recognition Trend analysis Classification Cluster analysis Regression analysis
- 22. Big Data firms want a piece of our action Google Consumer Surveys Facebook research Big Data tells us what, but Dunnhumby entered the Honomichl 100 not why Nectar are launching an online panel
- 23. How can Researchers respond? 1) Find a friendly data scientist 2) Get involved: understand available data sets 3) Talk to clients: what data? what needs? 4) Get creative: how could the data meet client needs? 5) Experiment (with your friendly data scientist) 6) Complement data with traditional research
- 24. Success for Market Research in Big Data = + x Data Technology Researcher scientist
- 25. No-one is doing this well yet: there is an open goal for whoever gets it right