Information Visualization for Large-Scale Data Workflows
7 likes5,075 views
The document discusses information visualization for large-scale data workflows. It describes principles for workflow visualization including making visualizations latent, modular, and using a consistent visual language. Specific tools are mentioned for workflow management, including Azkaban and White Elephant, as well as visualization tools like Tableau, RStudio Shiny, GoogleVis, and D3.js. Examples of visualizations described include topic modeling, model selection comparisons, and joint variable distributions.
Information Visualization for Large-Scale Data Workflows
- 1. Information Visualization for Large-Scale Data Workflows Michael Conover Senior Data Scientist, LinkedIn @vagabondjack reasonengine.wordpress.com Wednesday, October 9, 2013
- 2. Emergent Structure Wednesday, October 9, 2013
- 3. Elegant Complexity Pedro Cruz, University of Coimbra David Crandall, Indiana University John Nelson, IDV Solutions Credit Wednesday, October 9, 2013
- 4. Intellectual Dividends Realistic Mental Models Verification of Assumptions Shortened Iteration Cycles Improved Predictive Performance Product Insights Clarity of Communication Wednesday, October 9, 2013
- 5. Hypothesis Generation Wednesday, October 9, 2013
- 6. Wednesday, October 9, 2013
- 7. Color Commentary @whitehouse #RSVP Wednesday, October 9, 2013
- 8. Flock Together Wednesday, October 9, 2013
- 9. Political Polarization On Twitter Wednesday, October 9, 2013
- 10. Basic Workflow Structure Wednesday, October 9, 2013
- 11. aes_string() Basic Visualization Battery Wednesday, October 9, 2013
- 12. Feature Development Wednesday, October 9, 2013
- 13. Anscombe’s Quartet http://en.wikipedia.org/wiki/Anscombe's_quartet Wednesday, October 9, 2013
- 14. 0.0 0.1 0.2 0.3 0.4 −2.5 0.0 2.5 5.0 Standard Normal Density 0.0 0.1 0.2 0.3 0.4 −5.0 −2.5 0.0 2.5 5.0 Standard Normal Density 100,0001,000,000 Wednesday, October 9, 2013
- 15. A Lens On The Joint Distribution log(Connections) log(EndorsementPagerank) geom_point() Wednesday, October 9, 2013
- 16. A Lens On The Joint Distribution log(Connections) log(EndorsementPagerank) geom_point(alpha=1/5) Wednesday, October 9, 2013
- 17. A Lens On The Joint Distribution log(Connections) log(EndorsementPagerank) 25 50 75 100 count geom_bin2d(bins=35) Wednesday, October 9, 2013
- 18. A Lens On The Joint Distribution log(Connections) log(EndorsementPagerank) Class Negative Positive geom_point(alpha=1/5, aes(color=label)) Wednesday, October 9, 2013
- 19. A Lens On The Joint Distribution log(Connections) log(EndorsementPagerank) Class Negative Positive geom_density2d(aes(color=label), bins=10) Wednesday, October 9, 2013
- 20. A Lens On The Joint Distribution Marginal Histograms Wednesday, October 9, 2013
- 21. A Lens On The Joint Distribution Sepal.Length 6 7 8 5 6 7 8 Cor : −0.118 setosa: 0.743 versicolor: 0.526 virginica: 0.457 Cor : 0.872 setosa: 0.267 versicolor: 0.754 virginica: 0.864 Cor : 0.818 setosa: 0.278 versicolor: 0.546 virginica: 0.281 Sepal.Width 2.5 3 3.5 4 4.5 2 2.5 3 3.5 4 4.5 Cor : −0.428 setosa: 0.178 versicolor: 0.561 virginica: 0.401 Cor : −0.366 setosa: 0.233 versicolor: 0.664 virginica: 0.538 Petal.Length4 6 2 4 6 Cor : 0.963 setosa: 0.332 versicolor: 0.787 virginica: 0.322 Petal.Width 0.5 1 1.5 2 2.5 0 0.5 1 1.5 2 2.5 Species setosa versicolor virginica GGally (ggpairs) Wednesday, October 9, 2013
- 22. Model Fitting & Evaluation Wednesday, October 9, 2013
- 23. Model Selection Model A Model B Training Data I Training Data II Battery Battery Battery Battery Wednesday, October 9, 2013
- 24. stanford.edu/~jhuang11/ Homework At Scale Wednesday, October 9, 2013
- 26. Layercake Wednesday, October 9, 2013
- 27. Workflow Principles Latent, Pervasive Modular Consistent Visual Language Wednesday, October 9, 2013
- 28. Workflow Management Wednesday, October 9, 2013
- 32. Information Visualization for Large-Scale Data Workflows Michael Conover Senior Data Scientist, LinkedIn @vagabondjack reasonengine.wordpress.com Wednesday, October 9, 2013
- 33. Extended Toolbox Wednesday, October 9, 2013
- 38. kuler.adobe.com Adobe Kuler Wednesday, October 9, 2013
- 39. colorbrewer2.org Color Brewer Wednesday, October 9, 2013
- 40. d3js.org D3.js Wednesday, October 9, 2013
- 42. maps.stamen.com Stamen OpenStreetMap Tiles Wednesday, October 9, 2013
- 43. zipfianacademy.com/maps/h3/ SF Health Inspections Wednesday, October 9, 2013