Parsing Real-time
                             Tweets
                           Ramanujam Parthasarathy
                                 Monetate

                                   @ramanujam
                          http://github.com/ramanujam




Wednesday, May 30, 12
Why?


                   •    Freely available public intelligence

                   •    News reaches twitter super fast and spreads

                   •    250 million tweets per day (Oct ‘11)

                   •    Track keywords, sentiment analysis, data mining

                   •    Because you can!




Wednesday, May 30, 12
Twitter Streaming API
                        https://stream.twitter.com/1/statuses/filter.json




Wednesday, May 30, 12
Twitter Streaming API

                  curl https://stream.twitter.com/1/
                  statuses/filter.json?
                  track=SEARCH_PARAMETER -
                  uUSERNAME:PASSWORD




Wednesday, May 30, 12
Twitter Python Libraries

                 •      Tweepy
                 •      Python Oauth2
                 •      Python Twitter
                 •      Twython
                 •      Twitty Twister
                 •      Tweetstream

                               Use a library that has support for Oauth




Wednesday, May 30, 12
Get your auth keys




Wednesday, May 30, 12
Basic keyword/hashtag tracking
             Set the keywords to track


              Print the status




              File: tracker.py

Wednesday, May 30, 12
Save to MongoDB




                  Pymongo (http://api.mongodb.org/python/2.2/
                  index.html) is a recommended way to work with
                  MongoDB from Python



Wednesday, May 30, 12
Plot some data
             Pie chart of the devices used by people who tweet
             about Obama and Romney




             File: plotapps.py


Wednesday, May 30, 12
Play some music
                                #pugmusic




                        http://tinysong.com/6OAB

Wednesday, May 30, 12
Play some music




              File: playmusic.py




Wednesday, May 30, 12
Enqueue more songs



                   • Grooveshark javascript api
                   •    addSongsByID

                        •   window.Grooveshark.addSongsByID(13766524)




Wednesday, May 30, 12
Notify
         Twilio API to send a text or make a call based on an event
         trigger (eg: > 10 statuses in a 10 second window)

         setTerms = ['$AAPL', 'down', '530']
         setTerms = [‘aws’, ‘down’]
         setTerms = [‘phillies’, ‘win’]




      File: sendtext.py
Wednesday, May 30, 12
Thank You!


                        http://github.com/ramanujam/pugtalk




Wednesday, May 30, 12

More Related Content

PDF
Better than google?
PDF
WHY YOU SHOULD NEVER ASK, "HOW IS THIS BETTER THAN HADOOP?"
KEY
Effective Use of the Twitter Search API
PPTX
Practical Data Analysis in Python
PDF
pandas - Python Data Analysis
PDF
Twitter on Rails
PDF
Twitter streamingapi rubymongodbv2
PDF
Consuming the Twitter Streaming API with Ruby and MongoDB
Better than google?
WHY YOU SHOULD NEVER ASK, "HOW IS THIS BETTER THAN HADOOP?"
Effective Use of the Twitter Search API
Practical Data Analysis in Python
pandas - Python Data Analysis
Twitter on Rails
Twitter streamingapi rubymongodbv2
Consuming the Twitter Streaming API with Ruby and MongoDB

Similar to Parsing real-time data using Twitter Streaming API (20)

PDF
Data Access, Ownership and Control in Social Web Services: Issues for Twitter...
PDF
Evaluating the Utilization of Twitter Messages as a Source of Security Alerts
PDF
Dan node meetup_socket_talk
PDF
Collecting Twitter Data
PPTX
Language of Politics on Twitter - 02 Twitter
PDF
CSE5656 Complex Networks - Gathering Data from Twitter
PDF
Jordan Kay's Twitter API tour
PDF
Application Programming Interfaces
PPTX
Social Media Data
DOCX
Python report on twitter sentiment analysis
PDF
Twitter Platform 2013
PDF
Breaking HTML5 limits with Mobile JavaScript
PDF
Data forager
KEY
Visualising conversation around #c4thepromise
PDF
Realtime processing with storm presentation
PDF
20131008 - Wajug - TweetWall Pro
PDF
Filtering From the Firehose: Real Time Social Media Streaming
PDF
Data exchange formats
PDF
Chirp 2010: Scaling Twitter
PDF
@twitterapi at SocialApp Workshop
Data Access, Ownership and Control in Social Web Services: Issues for Twitter...
Evaluating the Utilization of Twitter Messages as a Source of Security Alerts
Dan node meetup_socket_talk
Collecting Twitter Data
Language of Politics on Twitter - 02 Twitter
CSE5656 Complex Networks - Gathering Data from Twitter
Jordan Kay's Twitter API tour
Application Programming Interfaces
Social Media Data
Python report on twitter sentiment analysis
Twitter Platform 2013
Breaking HTML5 limits with Mobile JavaScript
Data forager
Visualising conversation around #c4thepromise
Realtime processing with storm presentation
20131008 - Wajug - TweetWall Pro
Filtering From the Firehose: Real Time Social Media Streaming
Data exchange formats
Chirp 2010: Scaling Twitter
@twitterapi at SocialApp Workshop
Ad

Recently uploaded (20)

PDF
Zenith AI: Advanced Artificial Intelligence
PDF
Hybrid horned lizard optimization algorithm-aquila optimizer for DC motor
PPT
Geologic Time for studying geology for geologist
PDF
ENT215_Completing-a-large-scale-migration-and-modernization-with-AWS.pdf
DOCX
search engine optimization ppt fir known well about this
PDF
1 - Historical Antecedents, Social Consideration.pdf
PDF
Transform Your ITIL® 4 & ITSM Strategy with AI in 2025.pdf
PDF
NewMind AI Weekly Chronicles – August ’25 Week III
PDF
Developing a website for English-speaking practice to English as a foreign la...
PDF
DP Operators-handbook-extract for the Mautical Institute
PDF
WOOl fibre morphology and structure.pdf for textiles
PPTX
Benefits of Physical activity for teenagers.pptx
PDF
From MVP to Full-Scale Product A Startup’s Software Journey.pdf
PDF
sustainability-14-14877-v2.pddhzftheheeeee
PPT
Module 1.ppt Iot fundamentals and Architecture
PDF
Getting started with AI Agents and Multi-Agent Systems
PDF
A review of recent deep learning applications in wood surface defect identifi...
PDF
Five Habits of High-Impact Board Members
PPTX
observCloud-Native Containerability and monitoring.pptx
PPTX
O2C Customer Invoices to Receipt V15A.pptx
Zenith AI: Advanced Artificial Intelligence
Hybrid horned lizard optimization algorithm-aquila optimizer for DC motor
Geologic Time for studying geology for geologist
ENT215_Completing-a-large-scale-migration-and-modernization-with-AWS.pdf
search engine optimization ppt fir known well about this
1 - Historical Antecedents, Social Consideration.pdf
Transform Your ITIL® 4 & ITSM Strategy with AI in 2025.pdf
NewMind AI Weekly Chronicles – August ’25 Week III
Developing a website for English-speaking practice to English as a foreign la...
DP Operators-handbook-extract for the Mautical Institute
WOOl fibre morphology and structure.pdf for textiles
Benefits of Physical activity for teenagers.pptx
From MVP to Full-Scale Product A Startup’s Software Journey.pdf
sustainability-14-14877-v2.pddhzftheheeeee
Module 1.ppt Iot fundamentals and Architecture
Getting started with AI Agents and Multi-Agent Systems
A review of recent deep learning applications in wood surface defect identifi...
Five Habits of High-Impact Board Members
observCloud-Native Containerability and monitoring.pptx
O2C Customer Invoices to Receipt V15A.pptx
Ad

Parsing real-time data using Twitter Streaming API

  • 1. Parsing Real-time Tweets Ramanujam Parthasarathy Monetate @ramanujam http://github.com/ramanujam Wednesday, May 30, 12
  • 2. Why? • Freely available public intelligence • News reaches twitter super fast and spreads • 250 million tweets per day (Oct ‘11) • Track keywords, sentiment analysis, data mining • Because you can! Wednesday, May 30, 12
  • 3. Twitter Streaming API https://stream.twitter.com/1/statuses/filter.json Wednesday, May 30, 12
  • 4. Twitter Streaming API curl https://stream.twitter.com/1/ statuses/filter.json? track=SEARCH_PARAMETER - uUSERNAME:PASSWORD Wednesday, May 30, 12
  • 5. Twitter Python Libraries • Tweepy • Python Oauth2 • Python Twitter • Twython • Twitty Twister • Tweetstream Use a library that has support for Oauth Wednesday, May 30, 12
  • 6. Get your auth keys Wednesday, May 30, 12
  • 7. Basic keyword/hashtag tracking Set the keywords to track Print the status File: tracker.py Wednesday, May 30, 12
  • 8. Save to MongoDB Pymongo (http://api.mongodb.org/python/2.2/ index.html) is a recommended way to work with MongoDB from Python Wednesday, May 30, 12
  • 9. Plot some data Pie chart of the devices used by people who tweet about Obama and Romney File: plotapps.py Wednesday, May 30, 12
  • 10. Play some music #pugmusic http://tinysong.com/6OAB Wednesday, May 30, 12
  • 11. Play some music File: playmusic.py Wednesday, May 30, 12
  • 12. Enqueue more songs • Grooveshark javascript api • addSongsByID • window.Grooveshark.addSongsByID(13766524) Wednesday, May 30, 12
  • 13. Notify Twilio API to send a text or make a call based on an event trigger (eg: > 10 statuses in a 10 second window) setTerms = ['$AAPL', 'down', '530'] setTerms = [‘aws’, ‘down’] setTerms = [‘phillies’, ‘win’] File: sendtext.py Wednesday, May 30, 12
  • 14. Thank You! http://github.com/ramanujam/pugtalk Wednesday, May 30, 12