Data Design
0 likes582 views
This document outlines a course on data design and creative research practices. The course covers topics like data mining, visualization, analysis techniques, and prototyping. It discusses methods for gathering meaningful data through surveys, crawling, and user experience design. Example analysis techniques covered include clustering, classification, regression, and association rules. Students are asked to consider what types of systems or products could be built by analyzing collective intelligence data related to topics like politics, entertainment, food, and more. For a homework assignment, students form groups to explore potential data sources and analysis techniques for a collective intelligence topic.
![Clustering
Input Data Output Clusters
Points Hard
OR
OR
Soft
Similarities OR
[ # of clusters ] Hierarchical](https://image.slidesharecdn.com/05-data-designr1-120406063244-phpapp01/85/Data-Design-9-320.jpg)
Data Design
- 1. Data Design 2114.409: Creative Research Practice HTTP://WWW.FLICKR.COM/PHOTOS/SERGIU_BACIOIU/4370021957/
- 2. Reflection Status Check Concerns Programming What can we build HTTP://WWW.FLICKR.COM/PHOTOS/FLOWER87/76719859/
- 3. Course Outline 1. Foundations 3. Prototyping Introduction Crawling Survey Methods / Data Mining Text Mining Visualization and Analysis To be determined (TBD) Social Mechanics Project Update 2. Methods 4. Refinement Creativity and Brainstorming TBD x3 Prototyping Project Presentations Project Management Reflection
- 4. Last Week: Building Blocks Clustering Classification & Regression Association Rules Outlier Detection HTTP://WWW.FLICKR.COM/PHOTOS/OGIMOGI/2253657555/
- 6. Data Mining Overview How do I see and Visualization, Storytelling communicate answers? What questions should Design, Data Exploration I ask of the data? How do I clean and Analysis Techniques process the data? How do I gather Crawling, Surveys, UX Design meaningful data?
- 7. Why might we prefer analysis? LABOR ACCURACY Too many pictures to look at. Can test for statistical significance, etc. Don’t know which are interesting. Some patterns don’t visualize easily. HTTP://WWW.FLICKR.COM/PHOTOS/STRIATIC/2144933705/
- 8. Clustering Find natural groupings in the data Organize data into classes: ‣ high intra-class similarity ‣ low inter-class similarity
- 9. Clustering Input Data Output Clusters Points Hard OR OR Soft Similarities OR [ # of clusters ] Hierarchical
- 10. Classification Regression Learn to map objects to Learn map objects to categories continuous variables
- 11. Classification Observations X Learn f(x) = y Labels Y Y = gender Male Female X = height
- 12. The Whole Process Data Set Featurization Featurized Random Split (e.g. 90/10) Training Data Test Data Training Model Evaluation Results
- 13. Association Rules Learn interesting relations in the data = proportion of events in which X occurs
- 14. Anomaly Detection Detect strange events in the data Simplest measure:
- 15. What Can We Build? HTTP://WWW.FLICKR.COM/PHOTOS/BPENDE/6736531173/
- 16. Collective Intelligence Clicks,) Likes,) Updates,) Ar,cles,) Scrolls,) Links,) Reviews,) Images,) Time) Checkins) Comments) Video) Collec,ve) How can we harness the Intelligence) activities of the world’s digital citizens to build new and useful consumer services? Community)
- 17. Politics The Korean elections are coming. How does the Internet tell us more than traditional polling ever could?
- 18. Politics What issues are important? Who are the influencers? How can we segment/characterize support groups? How do we spread our opinions more widely? Who will win the election?
- 19. How can we build this? “Can social media predict election outcomes?” HTTP://WWW.USATODAY.COM/TECH/ NEWS/STORY/2012-03-05/SOCIAL- SUPER-TUESDAY-PREDICTION/ 53374536/1
- 20. Tweet Insert Magic Author Date Here? Body Retweets Hashtags Prediction Candidate Location Classification & Author Clustering Regression Score Profile Confidence Tweets Favorites Following Followers Association Outlier Location Rules Detection
- 21. Workshop
- 22. Sentiment + Candidate System Overview Tweet Inputs Correction based Scoring on past elections Refinements Author Inputs RMSE Evaluation
- 23. Sentiment Detail Input Observation Feature Extractor Classifier Output Label Confusion Matrix Evaluation N-Gram Features Training Process Tweet + Label
- 24. Entertainment Food Movements HTTP://WWW.FLICKR.COM/PHOTOS/STUCKINCUSTOMS/2786154526/ HTTP://WWW.FLICKR.COM/PHOTOS/WILLIA4/2504379334/ HTTP://WWW.FLICKR.COM/PHOTOS/GILSONROME/6247208325/ Collaboration Shopping Travel HTTP://WWW.FLICKR.COM/PHOTOS/FIDELMAN/4640722483/ HTTP://WWW.FLICKR.COM/PHOTOS/ZOOBOING/4473219605/ HTTP://WWW.FLICKR.COM/PHOTOS/FELIPENEVES/5414239936/ Investing Medicine Trust HTTP://WWW.FLICKR.COM/PHOTOS/STUCKINCUSTOMS/2786154526/ HTTP://WWW.FLICKR.COM/PHOTOS/TRAVEL_AFICIONADO/2396819536/ HTTP://WWW.FLICKR.COM/PHOTOS/AGECOMBAHIA/6425101047/ HTTP://WWW.FLICKR.COM/PHOTOS/MARKETINGFACTS/6758968163/
- 25. Homework: Data Mining 1. Form groups! 2. Choose a Collective Intelligence topic from Lecture 1, or propose similar. 3. Make a list of data sources that might provide insights to that topic. 4. Propose a set of meaningful questions about the data based on your intuition. 5. How would you have to clean/process your data to start answering those questions? 6. Consider clustering, association rules, anomaly detection, classification. For each technique, how might you apply it to the data and what would it show? 7. Document your work and be prepared to present. HTTP://WWW.FLICKR.COM/PHOTOS/31907740@N00/4860840019/
- 26. Feedback