The data we want
- 1. THE DATA WE WANT: FRAMEWORKS AND TOOLS TO ENGAGE WITH DATA E L E N A S I M P E R L U N I V E R S I T Y O F S O U T H A M P TO N
- 2. WEB AND INTERNET SCIENCE @SOTON Large interdisciplinary lab, 120+ members, 25 academic staff Founding member of the Web ScienceTrust Founding partner of the Open Data Institute Developer of ePrints W3C office for UK and Ireland
- 3. “Data is infrastructure. It underpins transparency, accountability, public services, business innovation and civil society.”
- 6. How do we help people tell their data stories? What data stories do people share and why? How do we make data more engaging?
- 7. A story may start by looking for the relevant data
- 8. DATA SEARCH HOW DOES THE GOOGLE FOR DATA LOOK LIKE? • Who searches for data and why? • What sort of queries do people write? • Do they need query writing support? • How should results be displayed? • How do people pick the best results? • Do they need one or more search sessions to find what the user is looking for? • Is search exploratory?
- 9. FRAMEWORK FOR INTERACTING WITH DATA HELPS SYSTEM DESIGNERS IDENTIFY USER TASKS AND DEFINE RELEVANT FEATURES Goal or process oriented Web Data portals People FoI Relevance Usability Quality Visual scan Obvious errors Basic stats Headers Metadata Koesten, L.M., Kacprzak, E.,Tennison, J.F. and Simperl, E., 2017, May.The Trials and Tribulations of Working with Structured Data:-a Study on Information Seeking Behaviour. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (pp. 1277-1289).ACM.
- 10. SEARCH LOG ANALYSIS INFORMS THE DESIGN OF DATA SEARCH ENGINES ● Four national open governmental data portals, 2.2 million queries from 2013 to 2016 (Kacprzak et al., 2017) ● Shorter queries, include temporal and location information ● Explorative search ● Native and external queries topically different ● Data requests offer more context to user intent Kacprzak, E., Koesten, L.M., Ibáñez, L.D., Simperl, E. andTennison, J.,A Query Log Analysis of Dataset Search. In International Conference onWeb Engineering (pp. 429-436).Springer, 2017.
- 11. DATA NEEDS CONTEXT • Metadata • Summary • Provenance • Details on collection and use • Quality and reviews • Tools
- 12. DATA SUMMARIES HELP PEOPLE MAKE SENSE OF DATA EFFECTIVELY Dataset in one sentence Format, no. of rows and columns, machine readability Headers, groupings, key columns Value types and ranges for key headers Provenance Location information Temporal information Quality, known issues Usage, trends
- 13. Stories are meant to be shared
- 14. VIRAL DATA HELPS (ALTERNATIVE) FACTS SPREAD FASTERS •How does data travel? –E.g. on social media •What makes data go viral? –Visualisations? –Subject matter/topic? –“Transmission vectors”: journalists, celebrities, grassroots, botnets?
- 15. DATA SHARING HELPS (ALTERNATIVE) FACTS SPREAD FASTER • What evidence can we see of data sharing activities? – What form is data being shared in? – How are the various stages of the data science pipeline represented? • How common is data sharing? – Who is it done by? – How do they do it? • What kind of data is (not) being shared? – Does anyone share raw data? – Do narratives explicitly reference the data they are built on? • What makes data go viral? • Who makes use of the data for what purposes?
- 16. OFFICIAL DATA • 6 weekTwitter study of ons.gov.uk • 1186 original tweets made by 898 people, with 4906 subsequent retweets • 15 most active tweeters, half work for the ONS or are official accounts of the ONS • Most retweeted tweet (503 times) is by a BBC journalist mentioning an ONS data visualisation • One of the 64 separate tweets about this ONS data release
- 17. OPEN DATA • Six weekTwitter study of data.gov.uk • 113 original tweets made by 87 different accounts, with 258 subsequent retweets • No bias towards organisational affiliation is present in the set of active retweeters • The single most retweeted tweet (121 times) is by a Joint Nature Conservation Committee earth observation specialist. Mentions a crop map visualisation from environment.data.gov.uk
- 18. SPREADSHEETS • No XLSX, but Google sheets • 1475 original tweets from 1067 unique accounts with 6923 retweets • No bias towards organisational affiliation is present in the set of active re- tweeters • Most retweeted spreadsheet (1188 times) is a schedule for the timings of INKIGAYO broadcasts (famous Korean livestreamed pop music program with live voting)
- 19. SPREADSHEET CATEGORIES AND USE • Visual inspection of 100 highly retweeted sheets • sports statistics (including gambling analysis) • computer games statistics • catalogues of resources/assets (including artist’s videos or a series of TV episodes) • selling goods/artwork/services for a trader or fan group • coordinating donations/volunteers, political info • coordinating political activity • music voting • buying on behalf of an artist • monitoring cryptocurrency offerings Simple list 10% Rich data 40% Data analysis 10% Promoting action 15% Coordinating crowd action 20% Other 5%
- 20. USE OF CHARTS • 5% (29) of sheets contained charts • 4 charts intended to promote subsequent use and discussion • Survey of fanfic community from NYC festival attendees • A maths teacher who takes part in MathsTeaching discussion groups tweeted a Google form to record preferences for banana ripeness • A study on the citation of Registered Reports in Cognitive Neuroscience • Historic weather data collected by a local citizen offered to a “sports weather” journalist Games (trading, playing, curation) 7 Politics (monitoring, organising, arguing) 6 Surveys (attitudes, phenomena) 4 Financial investment analysis 3 Personal list of assets/achievements 2 TV/radio (voting/ratings) 2 Trading (orders) 1 Miscellaneous data collection - Historic weather data - Boeing 787 production data (hobbyist) - Google Analytics audit of Udemy - Academic citation analysis 4
- 21. USE OF CHARTS (2) • 2 charts support an argument or discussion • UN data on firearms. Discussion thread between pro- & anti- NRA positions. Sent by author, a senior technologist in Microsoft. • Use of the Physics GRE in N American University Physics admission processes. Sent by a delegate at the Conference for Undergraduate Underrepresented Minorities in Physics, not the spreadsheet author.
- 23. DECENTRALISATION EMPOWERS • Distributed ledgers give data owners new ways to manage and share their data • Terms of use can be defined and stored reliably without the need of a central authority • Data owners define rules about what they share and under which conditions and links to their data • Applications (e.g. surveys) use smart contracts to define the data needed, the aggregations performed, and the number of participants • Data transactions recorded on the blockchain
- 24. How to make data interactions more engaging?
- 25. NEW WAYS TO ENGAGE WITH DATA • Interfaces • Tasks • Tools • Interactions
- 26. COLLECTING DATA WITH VIRTUAL ASSISTANTS • Same interaction possibilities for all the participants; everyone can be interviewed by the same “interviewer” (Johnston et al. 2013) • Marginal additional cost per survey after the implementation of the system (Stent et al. 2007; Johnston et al. 2013) • A virtual assistant can conduct a single interview in several sessions • Virtual assistants have prior knowledge of their users schedule; they know when they are available to complete the survey
- 27. DESIGNING A SURVEY • All established guidelines are hand-coded in the app • The order of the questions is pre-defined • Some questions might lead to specific follow-ups based on the reply • Different types of questions are linked to different expected answers – Binary, categorical etc. Please answer with only a Yes or a No Select your age range: 16-18, 19-40, > 40
- 28. C H A L L E N G E S – E N G A G E M E N T
- 29. C H A L L E N G E S – E X P E C T E D A N S W E R S
- 30. C H A L L E N G E S – U N D E R S T A N D I N G A N S W E R S