Data visualization
Download as PPTX, PDF0 likes214 views
This document discusses data visualization and big data. It defines big data as large data sets characterized by volume, velocity, and variety. Big data comes from various sources like business transactions, social media, sensors, and machine data. New technologies like Hadoop have helped store and analyze extremely large data sets to reveal patterns. The document emphasizes that visualizing data is important because humans do not understand raw data. Effective data visualization can show what is happening in the data, explain why, allow users to explore, and expose the data in a low-effort way for consumers. Examples are provided of visualizing religions in India and Australia, grocery sales data, movie ratings data, and birth data from the US and India to reveal patterns.
Data visualization
- 1. DATA VISUALIZATION INSIGHT BEHIND DATA Mukul Taneja Data Specialist, Gramener
- 2. WHAT IS BIG DATA ? Volume: Organizations collect data from a variety of sources, including business transactions, social media and information from sensor or machine-to-machine data. In the past, storing it would’ve been a problem – but new technologies (such as Hadoop) have eased the burden. Velocity: Data streams in at an unprecedented speed and must be dealt with in a timely manner. RFID tags, sensors and smart metering are driving the need to deal with torrents of data in near- real time. Variety: Data comes in all types of formats – from structured, numeric data in traditional databases to unstructured text documents, email, video, audio, stock ticker data and financial transactions.
- 3. BIG DATA also is… Extremely large data sets that may be analysed computationally to reveal patterns, trends, and associations, especially relating to human behaviour and interactions Which data is “BIG”? • Google Services • Social Media • E-Commerce • Geo-location and many more…
- 4. WHY CARE FOR DATA ? DATA IS EVERYWHERE. As users are continuously increasing, amount of data is getting massive.
- 5. HOW TO USE DATA ? To predict the user behavior…
- 6. HOW TO USE DATA ? To make connectivity better…
- 7. HOW TO USE DATA ? To provide better products to buy….
- 8. And the whole idea is …. To give user… Better experience Accuracy Relevancy Productivity Opportunity Reliability
- 9. WHAT TO DO WITH DATA ? IS EVERYWHERE ANALYSIS DATA VISUALIZE
- 10. WHY SHOULD WE VISUALIZE THE DATA? We humans do not understand the language of raw DATA
- 11. “ We have internal information. Getting information from outside is our challenge. There’s no way of doing that. – Senior Editor Leading Media Company
- 12. SHOW me what is happening with the data EXPLAIN to me why it’s happening Allow me to EXPLORE and figure it out Just EXPOSE the data to me Low effort High effort High effort Low effort Creator Consumer There are many ways to aid data consumption
- 13. SHOW me what is happening with the data EXPLAIN to me why it’s happening Allow me to EXPLORE and figure it out Just EXPOSE the data to me
- 17. LET’S TAKE TESCO’S GROCERIEScategory title kJ rate dairy Activia Pouring Natural Yogurt 1X950g 216 0.21 dairy Activia Pouring Strawberry Yogurt 1X950g 250 0.21 dairy Activia Pouring Vanilla Yogurt 1X950g 263 0.21 icecream Almondy Daim 400G 1804 0.75 icecream Almondy Toblerone 400G 1850 0.5 cereals Alpen 10 Pack Lite Summer Fruits Cereal Bars 210G 1222 1.57 cereals Alpen 10Pk Fruit Nut And Chocolate Cereal Bars 290G 1812 1.14 cereals Alpen Coconut And Chocolate Cereal Bars 5Pk 145G 1863 1.24 cereals Alpen Fruit And Nut With Chocolate Cereal Bar 5X29g 1812 1.24 cereals Alpen High Fruit 650G 1439 0.4 cereals Alpen Light Bars Chocolate And Orange 5X21g 1246 1.71 cereals Alpen Light Chocolate And Fudge Bar 5X21g 1264 1.71 cereals Alpen Light Sultana & Apple Bars 5Pk 105G 1197 1.71 cereals Alpen Light Summer Fruits Bars 5Pk 105G 1222 1.71 cereals Alpen No Added Sugar 1.3Kg 1488 0.31 cereals Alpen No Added Sugar 560G 1488 0.46 cereals Alpen Original 1.5Kg 1509 0.27 cereals Alpen Original Muesli 750G 1509 0.35 cereals Alpen Raspberry And Yoghurt Cereal Bars5x29g 1748 1.24 cereals Alpen Strawberry With Yoghurt Cereal Bar 5X29g 1756 1.24 dairy Alpro Natural Yofu 500G 0.28 dairy Alpro Raspberry Vanilla Yofu 4X125g 0.35 dairy Alpro Strawberry And Fof Soya Yofu 4X125g 0.35
- 20. The Shawshank Redepmption The Godfather The Dark Knight Titanic The Phantom Menace Twilight New Moon Wild Wild West Transformers The Good, The Bad, The Ugly 12 Angry Men 7 Samurai Taare Zameen Par Rang De Basanti Yojinbo MORE VOTES BETTER RATED Many unwatched movies Few unwatched movies Mix of watched & unwatched Few watched movies Many watched movies MOVIES ON THE IMDB 3 Idiots
- 21. SHOW me what is happening with the data EXPLAIN to me why it’s happening Allow me to EXPLORE and figure it out Just EXPOSE the data to me Simplifying access to data is a big win
- 22. SHOW me what is happening with the data EXPLAIN to me why it’s happening Allow me to EXPLORE and figure it out Just EXPOSE the data to me
- 23. EDUCATION PREDICTING MARKS What determines a child’s marks? Do girls score better than boys? Does the choice of subject matter? Does the medium of instruction matter? Does community or religion matter? Does their birthday matter? Does the first letter of their name matter?
- 24. 0 5,000 10,000 15,000 20,000 25,000 30,000 35,000 40,000 0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 TN CLASS X: ENGLISH
- 25. TN CLASS X: SOCIAL SCIENCE 0 5,000 10,000 15,000 20,000 25,000 30,000 35,000 40,000 0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100
- 26. TN CLASS X: MATHEMATICS 0 5,000 10,000 15,000 20,000 25,000 30,000 35,000 40,000 0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100
- 27. SHOW me what is happening with the data EXPLAIN to me why it’s happening Allow me to EXPLORE and figure it out Just EXPOSE the data to me … to make the hidden obvious
- 28. SHOW me what is happening with the data EXPLAIN to me why it’s happening Allow me to EXPLORE and figure it out Just EXPOSE the data to me
- 29. Let’s look at 15 years of US Birth Data This is a dataset (1975 – 1990) that has been around for several years, and has been studied extensively. Yet, a visualization can reveal patterns that are neither obvious nor well known. For example, • Are birthdays uniformly distributed? • Do doctors or parents exercise the C-section option to move dates? • Is there any day of the month that has unusually high or low births? • Are there any months with relatively high or low births? Very high births in September. But this is fairly well known. Most conceptions happen during the winter holiday season Relatively few births during the Christmas and Thanksgiving holidays, as well as New Year and Independence Day. Most people prefer not to have children on the 13th of any month, given that it’s an unlucky day Some special days like April Fool’s day are avoided, but Valentine’s Day is quite popular More births Fewer births … on average, for each day of the year (from 1975 to 1990)
- 30. The pattern in India is quite different This is a birth date dataset that’s obtained from school admission data for over 10 million children. When we compare this with births in the US, we see none of the same patterns. For example, • Is there an aversion to the 13th or is there a local cultural nuance? • Are holidays avoided for births? • Which months have a higher propensity for births, and why? • Are there any patterns not found in the US data? Very few children are born in the month of August, and thereafter. Most births are concentrated in the first half of the year We see a large number of children born on the 5th, 10th, 15th, 20th and 25th of each month – that is, round numbered dates Such round numbered patterns a typical indication of fraud. Here, birthdates are brought forward to aid early school admission More births Fewer births … on average, for each day of the year (from 2007 to 2013)
- 31. This adversely impacts children’s marks It’s a well established fact that older children tend to do better at school in most activities. Since many children have had their birth dates brought forward, these younger children suffer. The average marks of children “born” on the 1st, 5th, 10th, 15th etc. of the month tend to score lower marks. • Are holidays avoided for births? • Which months have a higher propensity for births, and why? • Are there any patterns not found in the US data? Higher marks Lower marks … on average, for children born on a given day of the year (from 2007 to 2013) Children “born” on round numbered days score lower marks on average, due to a higher proportion of younger children
- 32. SHOW me what is happening with the data EXPLAIN to me why it’s happening Allow me to EXPLORE and figure it out Just EXPOSE the data to me
- 35. HOW THE WORLD SEARCHED FOR TERROR ATTACKS ?
- 37. BIG DATA HANDY TOOLS Programming Languages Python R Octave
- 38. BIG DATA HANDY TOOLS Databases Hadoop Mondo DB TeraData
- 39. BIG DATA HANDY TOOLS Front End Javascript Libraries D3.JS Dimple Js
- 40. QUESTIONS?
- 41. THANK YOU