Introduction to data visualization 1
- 1. OBJECTIVES OF STATISTICSAND VISUALIZATION
- 2. • It's a revolution. We're really just getting under way. But the march of quantification, made possible by enormous new sources of data, will sweep through academia, business and government. There is no area that is going to be untouched. Gary King
- 3. All of us now are being blasted by information design. It's being poured into our eyes through the Web, and we're all visualizers now; we're all demanding a visual aspect to our information...And if you're navigating a dense information jungle, coming across a beautiful graphic or a lovely data visualization, it's a relief, it's like coming across a clearing in the jungle. David McCandless
- 4. There is more data available to us now than we can possibly process. Every minute, Internet users add the following to the big data pool (i): 204,166,667 email messages sent More than 2,000,000 Google searches 684,478 pieces of content added on Facebook $272,070 spent by consumers via online shopping More than 100,000 tweets on Twitter 47,000 app downloads from Apple 34,722 "likes" on Facebook for different brands and organizations 27,778 new posts on Tumblr blogs 3,600 new photos on Instagram 3,125 new photos on Flickr 2,083 check-ins on Foursquare 571 new websites created 347 new blog posts published on Wordpress 217 new mobile web users 48 hours of new video on YouTube http://www.ted.com/read/ted-studies/statistics/introductory-essay
- 5. Data data everywhere, how much sense does it make?
- 6. What good data presentation can do?
- 7. How the Recession Reshaped the Economy, in 255 Charts 'How the Recession Reshaped the Economy, in 255 Charts' is sharing the 'pièce de résistance de 2014' prize so far and probably stands up well against anything else produced in recent memory. It is a modern masterpiece, supremely conceived and executed interactive graphic produced by Jeremy Ashkanas and Alicia Parlapiano for TheUpshot (more later), showing how job numbers have fared across 255 different industries in the past 10 years. Pigeonholing it is hard: it is a small-multiple-scatterplotted area charts extravaganza. Some of the tool-tip interactivity detail is extraordinary, which kind of makes the single complaint - that of the colour scale failing on the red- green issue - a slightly surprising oversight, albeit the position and direction of the charts provides the main encoding.
- 10. Is it data or the visualization that we like?
- 19. 3D London maps, overlaid with air quality data http://www.londonair.org.uk/london/asp/virtualmaps.asp?view=maps
- 21. A modification of the basic histogram in order to display information for two variables. In this case, the example shows the distribution of some reaction times by gender. http://addictedtor.free.fr/graphiques/RGraphGallery.php?graph=136
- 22. Source: P. J. Rousseeuw, I. Ruts, and J. W. Tukey (1999). The bagplot: A bivariate boxplot. The American Statistician, 53:382-387. http://addictedtor.free.fr/graphiques/RGraphGallery.php?graph=112 A generalisation of the boxplot, this visualisation relies on the median (red star) and the representation of 50% of the observations (dark blue area).
- 24. Guardian US interactive team (News) http://www.guardian.co.uk/world/interactive/2012/may/08/gay-rights-united- states
- 25. Textual cross references found in the Bible. Chris Harrison's Website: page 18 (Local Area Agreement) http://www.chrisharrison.net/projects/bibleviz/index.html
- 26. Representation of all solar, lunar and interplanetary missions. http://books.nationalgeographic.com/map/map-day/2008/09/18 Obama's economic stimulus plan
- 29. 12 What is statistics? (and why is it so cool?) • The study of the collection, organization, analysis, and interpretation of data • Why bother? Principles provide a framework for • data collection • design of experiments and observational studies • drawing inferences about populations as a whole and of future events • Short story: Statistics is the science of using data to prove a point (hopefully forming a correct conclusion).
- 30. 20 3 Major Overarching Topics in the Course 1) Describing Data - Graphically - Numerically 2) Gathering Data - Experimental Design and Randomization - Observational Data and Random Sampling - Randomness and Probability Models 3) Inferring from Data - showing statistical significance and p-values - estimation (confidence intervals) and testing hypothesis
- 31. 11 Histograms can be sensitive to definition of the bins (IPS, Ex 1.4)
- 33. 19 Effect of Shape on Mean and Median • In a right skewed distribution, the mean is greater than the median • In a left skewed distribution, the mean is less than the median • In a symmetric distribution the mean is approximately (sometimes exactly) equal to the median
- 34. Measuring Spread (Variability) in Data Two common methods 1. Variance and standard deviation • Measure spread about the mean • Most often used, but also sensitive to large values in skewed distributions 2. Quantiles and percentiles • Median • Quartiles and more general percentiles
- 37. 36 Another Plot Type – Box plots • Box plots are design to show clearly the center, spread (especially IQR), and outliers • They are based on the five-number summary • Minimum, Q1, Median, Q3, Maximum • Easiest to explain with an example, using the tuition data.
- 38. 37 Box plot of Per Capita Income From Stata Documentation 99% 55755 55755 Kurtosis 5.07413 95% 46344 49852 Skewness 1.275147 90% 42392 46344 Variance 3.29e+07 75% 38712 45877 Largest Std. Dev. 5734.745 50% 34257 Mean 35470.67 25% 31891 29108 Sum of Wgt. 51 10% 29515 27935 Obs 51 5% 27935 27897 1% 26535 26535 Percentiles Smallest income . summarize income, detail 20,00030,00040,00050,00060,000 income
- 39. 42 Effects of linear transformation yi = a + bxi • mean (Y) = a + b × mean (X) • median (Y) = a + b × median (X) • variance (Y) = b2 × variance (X) • SD (Y) = |b| × SD (X) • IQR (Y) = |b| × IQR (X) In this notation, xi and yi are used for individual observation, and X and Y are used for the collection of xi’s and yi’s
- 40. 52 Density Curves • Always positive • Area under the curve equals 1 • Median is point where 50% of area is to the left and 50% is to the right. • Mean is “balance point”