Harvesting Data from Twitter Workshop: Hands-on Experience
Download as PPTX, PDF2 likes706 views
The document provides a comprehensive guide on harvesting data from Twitter, covering the Twitter API, available tools, and a comparison between R and Python for data analysis. It highlights the importance of Twitter as a vast information source used in various research fields and offers practical steps for accessing and utilizing Twitter data. Additionally, it outlines the installation processes for R and Python, along with resources for further learning and support.
Harvesting Data from Twitter Workshop: Hands-on Experience
- 1. Harvesting Data from Twitter: Hands on Experience Dr. Nora alTwairesh, Ms. Tarfa alBuhairi, Ms. Mawaheb alTuwaijri, and Ms. Afnan alMoammar
- 2. Content • Introduction about Twitter API • Some ready to use tools (no programming) • Comparison between R and Python • R • Python
- 4. Why Twitter • Twitter has become a mass information hub that can be used to study the evolution of any issue matter: revolutionary machine • Research disciplines that study Twitter data spanned the domains of computer science, information science, communications, business, economics, education, medicine, political science, and sociology.
- 5. • Recent studies show that %60 of daily Arabic tweets are from Saudi Arabia. Why Twitter Hamdy Mubarak and Kareem Darwish. 2014. Using Twitter to collect a multi-dialectal corpus of Arabic. ANLP 2014:1.
- 6. Twitter API • Free access to the tweets posted in the last 7 days within a certain rate-limit. • Any tweets posted earlier than 7 days are considered historical tweets and should be purchased through third party providers • The Twitter API provides three interfaces for tweet collection: Streaming API, REST API and Search API
- 7. Streaming API • The Streaming API provides real-time tweets in a live-poll fashion. • In a Streaming API, requested tweets will be constantly flowing as they are posted on Twitter. It is delivered in three bandwidths: “spritzer” :1%, “gardenhose”: 10% and “firehose”: 100% of all tweets posted on Twitter. • A regular user wanting to collect tweets will be granted spritzer access.
- 8. REST API • The REST API was specifically designed for programmatic access to read and write Twitter data. • Third party applications that interact with Twitter are provided with a large set of methods in the REST API to develop these applications. • The access of the REST API is also rate-limited, the limit is 150 requests per hour.
- 9. Search API • Similar to the REST API, the Search API is pull-based. It replicates the search functionality provided on the Twitter website. However, tweets retrieved are restricted to the past 7 days. • the Search API is not appropriate for high-throughput real-time data acquisition. As such Twitter Inc. discourages its use and plans to discontinue it in the future.
- 10. Create a Twitter App • To access the Twitter API you need to create a twitter app: follow this simple tutorial to do so: https://iag.me/socialmedia/how-to-create-a-twitter-app-in-8- easy-steps/ • you will use the OAUTH settings in both R and Python: • Consumer Key • Consumer Secret • OAuth Access Token • OAuth Access Token Secret
- 11. Tools to Collect Tweets • Nodexl: https://nodexl.codeplex.com/ • Tweet Archivist : https://www.tweetarchivist.com/ • Twitter Archiving Google Spreadsheet (TAGS): https://tags.hawksey.info/
- 13. What is R? •Roos & Robert. 16
- 16. R allows you to integrate with
- 20. Examples
- 21. Now .. Open your laptop, please
- 22. Steps to install R 1: install R: • https://cran.r-project.org/bin/windows/base/ ---- http://cran.r- project.org/bin/macosx/ 2: install RStudio (after installing R) • https://www.rstudio.com/products/rstudio/download3/ 3: Install these packages (see the user manual): • streamR/ ROAuth/ RJSONIO/ RTextTools/ e1071/ SparseM. User manual: • http://www.devchakraborty.com/RunningRJafroc.pdf R Packages list: • https://cran.r-project.org/web/packages/available_packages_by_date.html Developing Packages with RStudio: • https://support.rstudio.com/hc/en- us/articles/200486488?version=0.99.903&mode=desktop • https://cran.r-project.org/doc/manuals/R-exts.html
- 23. Useful URLs • https://www.r-bloggers.com • https://www.r-bloggers.com/how-to-learn-r-2/ • http://www.slideshare.net/ChiuYW/r-language-tutorial • https://www.rwaq.org/courses/introduction-r- programming • https://www.researchgate.net/publication/288485806_Hy brid_Sentiment_Analyser_for_Arabic_Tweets_using_R
- 24. Python • Two versions: 2.7 3.X • Twitter packages: twitter -- -tweepy • IDE :Anaconda: iPython notebook: Jupyter
- 25. Installing Python • Install Anaconda from here • https://www.continuum.io/downloads choose Python 2.7 version (only for this tutorial) • Install the twitter package: From the command line (terminal) type: pip install twitter
- 26. Comparison between R and Python • https://www.datacamp.com/community/tutorials/r-or-python-for- data-analysis#gs.GuXGfAc • http://blog.udacity.com/2015/01/python-vs-r-learn-first.html • http://www.dataschool.io/python-or-r-for-data-science/
- 27. Contact Us ASA Research Group Twitter: @ASA__IU Email: asa@imamu.edu.sa Website: http://asa.imamu.edu.sa/ IWAN Research Group Twitter: @IWAN_RG Email: iwan@ksu.edu.sa Website: http://iwan.ksu.edu.sa
- 28. Thank you, See you later … THE END ..
Editor's Notes
- #8: Gardenhose access is granted on special request from Twitter Inc. and the firehose access is granted to third-party business partners of Twitter Inc. which are considered third-party data providers.
- #9: An example of these applications is the inclusion of a Tweet share button on some websites that allows the reader of this website to share the link of the website on Twitter by posting it as a tweet; this is an example of writing Twitter data. An example of reading Twitter data is when websites display tweets of a certain hashtag or user account in a widget on their website’s pages.
- #10: With the Search API you can only sent 180 Requests every 15 min timeframe. With a maximum number of 100 tweets per Request this means you can mine for 4 x 180 x 100 = 72.000 tweets per hour.
- #14: The project was conceived in 1992, with an initial version released in 1994 and a stable beta version in 2000.
- #15: R is the leading tool for statistics, data analysis, and machine learning.
- #16: R is the leading tool for statistics, data analysis, and machine learning.
- #17: R allows you to integrate with other languages (C/C++, Java, Python)