Fail ir16 intro
Download as PPTX, PDF0 likes747 views
The document discusses a workshop focused on failures in social media research, aiming to collect examples of mishaps and learn from them across various research communities. Key challenges include engaging users, selecting appropriate methods for data analysis, dealing with unreliable tools, and managing the dynamic nature of social media content. The workshop emphasizes the need for best practices, improved evaluation standards, and an understanding of the complexities involved in social media research.
Fail ir16 intro
- 1. #FAIL! THINGS THAT DIDN‘T WORK OUT IN SOCIAL MEDIA RESEARCH - AND WHAT WE CAN LEARN FROM THEM Workshop at Internet Research 16, Phoenix, October 21st, 2015.
- 2. • Workshop hashtag: #fail2015b • Conference hashtag: #ir16 • Workshop website: http://failworkshops.wordpress.com • Etherpad: https://pad.okfn.org/p/fail WELCOME Luca Rossi @LR Karine Nahon @karineb Katrin Weller @kwelle
- 3. ABOUT #FAIL! WORKSHOPS • Traveling on to different conferences. First workshop was at WebSci15 (June 2015) • Aim: collect various examples for things that can go wrong and share them with different communities learn from experiences Connect different research communities
- 4. WHAT WE‘VE LEARNED SO FAR
- 5. 0 100 200 300 400 500 600 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 Twitter Facebook YouTube Blogs Wikis Foursquare LinkedIn MySpace Number of publications per year, which mention the respective social media platform‘s name in their title. Scopus Title Search. For details: http://kwelle.wordpress.com/2014/04/07/bibliometric-analysis-of-social-media-research/ SOCIAL MEDIA RESEARCH
- 6. 2008-2013 papers on Twitter and elections: data sources Weller, K. (2014). Twitter und Wahlen: Zwischen 140 Zeichen und Milliarden von Tweets. In: R. Reichert (Ed.), Big Data: Analysen zum digitalen Wandel von Wissen, Macht und Ökonomie (pp. 239-257). Bielefeld: transcript. 6 Data source number No information 11 Collected manually from Twitter website (Copy-Paste / Screenshot) 6 Twitter API (no further information) 8 Twitter Search API 3 Twitter Streaming API 1 Twitter Rest API 1 Twitter API user timeline 1 Own program for accessing Twitter APIs 4 Twitter Gardenhose 1 Official Reseller (Gnip, DataSift) 3 YourTwapperKeeper 3 Other tools (e.g. Topsy) 6 Received from colleagues 1 SOCIAL MEDIA RESEARCH
- 7. What we discussed at the first workshop… CURRENT PROBLEMS
- 8. Challenge 1: users • How to involve social media users in the research process? • Presentation by Elodie Crespel: “Extending data collection with web browser extension” – Participants may be creative in their use of technology – flexibility is needed.
- 9. Challenge 2: methods • Data analytics: which approaches should be chosen? • Taha Yasseri: “The double-edged sword of statistical significance” – Questions the p-value as a standard for data analytics. – “too much of attention and reliance on specific measures or methods without being aware of the logic behind them, can be misleading”
- 10. Challenge 3: tools • Many researchers use third party tools for data collection or analysis – which may not always work as expected. • Presentation by Michael Bossetta and Anamaria Dutceac Segesten: “Tracing Eurosceptic Party Networks via Hyperlink Network Analysis and #FAIL!ng: Can Web Crawlers Keep up with Web Design?” – Exemplary case: issuecrawler.
- 11. Challenge 4: content • Content analysis is heavily effected by the dynamic nature of social media. • Presentation by Marie Van Cranenbroeck: “Managing and Using Unstable Data in a Social Science Research about Museums and Audiences on Social Media” – Data collection and storage challenges
- 12. Specific details and additions • Researchers and users may have different ideas about the definition of social media / social networks • Lack of evaluation standards • Availability of data (also: not enough data) • Data may be corrupt (e.g. missing data) • Social media as a moving target (Karpf, D. (2012). Social science research methods in Internet time. Information, Communication & Society 15(5):639-661. )
- 13. Meta discussion • Social media research can have various forms. Different disciplines involved. • Best practices and pitfalls in social media research are mainly discussed informally. Few possibilities to share unsuccessful approaches.
- 14. WHAT WE‘D LIKE TO LEARN TODAY • Towards a categorization of challenges for social media research: what can go wrong? • Collection of more experiences • Structuring them into different categories
- 15. WHAT WE‘D LIKE TO LEARN TODAY Today: - 4 presentations - Think about your own experiences! - … in connection to each presentation - … in general
- 16. 9:00 Introduction: “What we’ve learnt from the first workshop and what we’d like to learn today” 9:15 Shawn Walker: “Complexity of collecting social media data in ephemeral contexts” 9:40 Cornelius Puschmann: “Why LIWC sucks (or: saner options for social media content analysis)” 10:05 Break 10:20 Luca Rossi: “The fourth deadly sin of social media researchers (or: scientific research and unstable socio-technical platforms)” 10:45 Marco Toledo Bastos, “Individual Behavior from Aggregate Social Media Data“ 11:10 Discussion & Conclusions 12:00 End PROGRAM
- 17. • Other experiences? Share your thoughts! • Main categories of #fail cases? • Top 3 take away messages for next workshop? DISCUSSION
- 18. WHERE TO GO FROM HERE? • Next steps – lessons learnt for future workshop organisation • Which additional conferences? • Publication? Guidebook?
- 19. • Archiving: – URLs may vanish (Question: linear rate of decay?) – Images missing – Platforms changing (moving target!) – not just about the interface! • Visualization of results – Word cloud (compare histograms) • Tools – sentiment140, Internet Archive, GNIP • Methods – Content Analysis: • replicability? Validation? • Context for social media contents (e.g. surrounding tweets). • LIWC, General Inquirer – Predictions – „Data Science“ • Lack of theory • Data Quality: – Can we still cite/use data and research published in 2007/2008`? – Baseline? (how to define for a moving target) • Theory: – Can we only do descriptive work for single platforms? – Look for the theory instead for the data? • Meta – Systematic review of existing literature is needed • Documentation – Timeframe generalizaion – Document time, cultures? – How long will my results be valid? – Have a general base for comparison
Editor's Notes
- #4: Social media platforms and users – a moving target… Best practices and pitfalls in social media research are mainly discussed informally. Few possibilities to share unsuccessful approaches. Researchers with lots of different disciplinary backgrounds enter the field. Different fields of expertise, few interdisciplinary exchange of approaches. Limited possibilities for data sharing / validation and reproduction of results.