Twitter Streaming API
Download as KEY, PDF1 like959 views
The Twitter Streaming API allows developers to access tweets in real-time through a persistent HTTP connection. It provides a more efficient way to process large volumes of tweets compared to polling the REST API. Developers create a signed HTTP request, handle the chunked JSON response without a content length header, and continuously process tweets. The Streaming API can sample the full firehose of tweets, filter for specific topics, and power applications that interact with tweets in near real-time.
Twitter Streaming API
- 1. Do More, Faster The Twitter Streaming API
- 2. 1. Create a HTTP Request GET /1/statuses/sample.json HTTP/1.1 Authorization: OAuth realm="",oauth_no... Host: stream.twitter.com Accept: */* Sign it with OAuth
- 3. 2. Handle the response HTTP/1.1 200 OK Content-Type: application/json Transfer-Encoding: chunked Server: Jetty(6.1.25) {"in_reply_to_status_id_str":null,"geo":n ull,"text":"Hey fat boyyyyy!!... What’s missing? No Content-Length header! HTTP Libraries won’t know when to stop consuming
- 4. 3. Keep handling it • Twitter will keep sending you JSON chunks until you kill the connection • Normal HTTP libraries make this awkward • Try non-blocking / event-based I/O
- 5. The Streaming API lets you do cool stuff • Sample the firehose, free ~1% • Get interesting subsets of all tweets • By filtering, can get all tweets on a topic • Interact in near-realtime • Process more efficiently
- 6. Also... • User Streams • Site Streams
- 7. At Conversocial, we built a way to search twitter on behalf of our clients. There’s a stream client that receives tweets from the Streaming API
- 8. We monitor the stream client from a separate thread
- 9. We still have to poll the REST API to cover gaps between connections
- 10. Need to disconnect stream client from processing tasks. We do this via a queue.
- 11. Searches have to be matched back to the terms that produced them - the stream is multiplexed.
- 12. Thanks for listening! gareth@conversocial.com http://www.github.com/gareth-lloyd/twistedstream