![Saratoga (HTTP API)
{!
"metadata": {!
"name": “APIExample", "versions": [1]!
},!
"endpoints": [{!
! ! "endpoint": “example", "getProcessors": [!
{"versions": [1]}!
]}!
]}!
from saratoga.api import SaratogaAPI!
import json!
!
class APIExample(object):!
class v1(object):!
def example_GET(self, request, params):!
return {"hello": "world"}!
!
api = SaratogaAPI(APIExample, json.load(open("apidef.json")))!
api.run()](https://image.slidesharecdn.com/wtfistwisted-140522082332-phpapp01/85/WTF-is-Twisted-35-320.jpg)
WTF is Twisted?
- 1. WTF is Twisted? (or: Owl Amongst The Ponies)
- 2. About HawkOwl • hawkowl@atleastfornow.net • @hawkieowl on Twitter • hawkowl on GitHub, Keybase & ThoughtStreams • PGP Key Fingerprint: 2100 5027 897E C8AE 1D94 A905 2308 B479 D392 4A11 • Twisted Core Committer and Release Manager (13.2 & 14.0) • Is not actually an owl
- 3. Twisted in One Slide • Twisted is a framework for writing network applications in Python • Best known for “Deferreds”, Twisted’s async abstraction • Contains protocol implementations for HTTP, SMTP, IMAP, DNS, etc.
- 4. Why would you want to use Twisted? • Higher performance in some common situations • Stable, mature codebase • Protocol implementations for lots of things • High test coverage • Great community consisting of excellent individuals who do excellent work
- 5. What makes Twisted especially different from other projects? • Twisted has a reputation for being the “black sheep” in the Python community, mostly because we do things “weirdly” and “hard” • But, surprise! It turns out the Twisted developers were right all along (see: asyncio) • Different set of principals, from both the longevity of the project and their specific problems
- 6. The Tenets of Twisted (the time-tested, battle-hardened principals of the Twisted project, as interpreted by HawkOwl)
- 7. Subclassing is bad. Composition is the new inheritance!
- 8. Multiple inheritance is really bad. Handy hint: If you’ve written the word “super”, it’s already too late.
- 9. Time-based releases are king. For large, evolving frameworks, it’s better to gradually change than have a massive app migration effort.
- 10. Early APIs > No APIS You won’t know how your API works in the real world until other people use it. You can always deprecate it later.
- 11. All code needs tests. No exceptions. It doesn’t matter if you’re fixing an urgent release regression - an eye on coverage saves you a lot of time later.
- 12. All code needs reviewing. No exceptions. Code review is invaluable, even if it does descend into bikeshedding if they can’t find anything wrong.
- 13. Only bad APIs can’t be unit tested. Can’t unit test your API? Too bad, time to refactor it.
- 14. Build slaves are amazing. Test your code on everything. Virtual machines are cheap, compatibility problems down the line are not.
- 15. –David Reid, Glyph, et al “Call a function with some arguments and inspect some return values.”
- 16. How Twisted Works (a short, mostly-correct overview)
- 17. The Event Loop (Spin me right round, baby, right round)
- 18. The Reactor • Twisted, being an event-driven framework, provides an event loop, or “reactor”. • This is an infinite loop, wrapped around a blocking call that waits until some I/O is ready • Once the I/O is ready, this fires off an event to run the code that was waiting for it • Runs other tasks while I/O blocked tasks are waiting
- 19. Using the Reactor • To utilise the reactor, you use Deferreds. • Deferreds are an “IOU” for an actual result sometime in the future • A Deferred contains a callback chain, a series of functions run in sequence after it gets a result • Deferreds get their results from events occurring - such as I/O being received or some task being finished
- 20. Benefits • For I/O bound applications, allows you to efficiently use CPU resources, as the reactor runs other things in the meantime • Integration with common event loops - for example, GTK or QT, to use your Twisted code into your GTK or QT applications
- 21. The Callback Chain (Hey, I just met you, and this is crazy, but here’s a function, so call it maybe?)
- 22. The Callback Chain • The easiest way to understand the callback chain is to visualise it as a production line. • A production line is made up of discrete blocks - each doing a separate thing. • Let’s imagine a such a production line…
- 23. Lets say that we are building a toy car, and we need to do the following: 1. Create toy car body from materials 2. Paint racing stripes on car body 3. Add wheels to the car body 4. Put the completed car in a box
- 24. Here is how we would do that in psuedo-code Deferreds: ! ! d = Deferred()! !d.addCallback(buildCarBody)! d.addCallback(paintStripes)! d.addCallback(addWheels)! d.addCallback(placeInBox)! # Start the callback chain! d.callback(rawMaterials)
- 25. Deferreds • Each function in the chain is called with the result of the previous one as its first argument • Functions in the callback chain can return either a result or a Deferred that fires with a result sometime in the future
- 26. Lists of Deferreds • If you have several Deferred operations to do at the same time, and want your callback to fire only when they are all done, use DeferredList.! • Pass it a list of Deferreds, and it will return a Deferred that fires with a list containing the results of each when they have all finished.
- 27. Error Handling • Error handling is done by errbacks. • Errbacks are like callbacks, but are run when an exception is raised. • An errback will catch all errors that occur on the chain before it.
- 28. Deferred Benefits • Forces you to break up your logic into sections, broken up over external interface accesses • Encourages reuse, as these sections quite commonly do one thing to an input and return it • Can be cancelled (eg. if a client disconnected and you don’t care about the result anymore)
- 29. Deferred Benefits • Discourages global mutable state by having different execution threads share the same global state • No global mutable state means you can scale easier • If your global state is databases, just spin up more Twisted instances to use all your CPUs
- 30. Extending the Callback Chain (Because sometimes things like to do other things)
- 31. Deferreds returning Deferreds • Deferreds can contain their own callback chain - meaning that one Deferred returned may have several callbacks being called inside it • This is invisible to the “top level” of callbacks • You can extend your callback chain at any time
- 32. What can Twisted do? (Enough of the boring stuff!)
- 33. Web • Klein, a Flask-like API in Twisted • Saratoga, a HTTP API framework • Autobahn, a Websockets implementation • Hendrix, a Django deployment framework • Static file serving: twistd -n web --port tcp:8080 --path /tmp
- 34. Klein (Flask in Twisted) from klein import Klein! app = Klein()! ! @app.route('/')! def home(request):! return 'Hello, world!'! ! app.run("localhost", 8080)
- 35. Saratoga (HTTP API) {! "metadata": {! "name": “APIExample", "versions": [1]! },! "endpoints": [{! ! ! "endpoint": “example", "getProcessors": [! {"versions": [1]}! ]}! ]}! from saratoga.api import SaratogaAPI! import json! ! class APIExample(object):! class v1(object):! def example_GET(self, request, params):! return {"hello": "world"}! ! api = SaratogaAPI(APIExample, json.load(open("apidef.json")))! api.run()
- 36. Email • Complete SMTP and ESMTP server and clients • POP3 and IMAP4 servers and clients • Support for TLS-secured email out of the box For more info, see: http://twistedmatrix.com/documents/ current/mail/index.html
- 37. DNS • A caching, recursive DNS server out of the box twistd -n dns -c —port 10053 --recursive • DNS client APIs • DNS server toolkit For more info, see: http://twistedmatrix.com/documents/ current/names/howto/custom-server.html
- 38. Chat • IRC Server twistd -n words --irc-port tcp:8400 --group roomname --auth memory:usr:pass • IRC client (for making IRC bots) • XMPP client For more info, see: http://twistedmatrix.com/documents/ current/words/examples/
- 39. Want to know more? • Twisted Docs http://twistedmatrix.com/documents/current/ • Iffy’s Twisted FTW http://iffycan.com/twistedftw/ • HawkOwl’s Technicals http://technicals.atleastfornow.net/
- 40. Questions? (sane answers not guaranteed)