Async programming and python

Async
Programming and
Python
PyCon India, 2014
Chetan Giridhar

Basics
• Programming tasks:
o I/O bound
o CPU bound
• Say, you’re doing I/O
o Will it complete immediately? When will it be done?
o Wont they block you?

Blocking I/O: Example
import requests
r = requests.get(‘http://google.co.in’)
r.status_code
• What if the request takes a long time?
• Operation blocks until all the data is recieved from the server
Can we do something in the meanwhile?
Can we run another task, concurrently?

Non Blocking / Async
• Non-blocking means the ability to make continuous
progress at all times
• Resources needed for a response must not be
monopolized
• As such it can enable both lower latency, higher
throughput

Programming Models
Synchronous model
time
time
time
Threaded model
Asynchronous model
Task 1 Task 2 Task 3

Math
• Task = make a call to http://ip.jsontest.com
• Say, Task = Task1 = Task2 = Task 3 = 400ms
• Sync Model
o Time taken = Task1+ Task2 + Task3 = 1.2 sec
• Threaded Model
o Time taken = 510 ms
• Async Model
o Time taken = 460 ms
What’s the
magic here?

Async Paradigm
• Clients requests the event driven web server;
• requests are processed by event loop;
• event handlers cater to events with callbacks
Client Event driven server I/O loop
Event driven I/O loop
Request
IO loop handles
request
Event Handlers

Reactor Pattern
• Typical non blocking frameworks work on a
philosophy of single threaded event loop
o keeps polling for events
Reactor
Pattern
Waiting for Events
Handling Events

More Details!
• Framework typically maintains a list of file
descriptors(fd), events to monitor and
corresponding event handlers for each of the fd
• Listening to events on a fd is a kernel space task
o epoll, [kqueue/select] – libraries provide event notifications in a non-blocking
way
• Epoll watches file descriptors (sockets) and returns
needed (READ, WRITE & ERROR) events

Async way
• Async strategy aims for:
o Making I/O tasks non blocking
o I/O tasks would run independently
o generate an event when tasks are complete
o with help of callbacks
• Benefits
o No need to wait till blocking I/O tasks are complete
o More responsive real time applications
o Thread safety isn't an issue
• Can we solve any other Python problems with this
mechanism?
o Eliminating GIL?

Async in Python
• Frameworks
o Tornado
o Twisted
o Gevent
• Modules
o Tulip
o Asyncio

Asyncio
• Part of Python library
o The latest module for async application development
• Only for Python > 3.4
o Incompatible with prior versions
• A whole new way to development
o Let’s you write self contained, synchronous looking tasks
o Run two infinite loops at the same time on the same thread
• Works with other framework
o Tornado, Twisted, GEvent

Asyncio…
• Write single threaded concurrent code
• Principle of Interleaved execution of subroutines
• Co-operative scheduling
o Only one task at a time
• Based on libevent
o Select, kpoll, kqueue

Asyncio: Components
Event loop
Co-routines, Futures, Tasks
Transports, Protocols

Asyncio: Components
• Event loop
o Register, executing and cancelling calls
o Schedule execution of a task (co-routine)
o Creates transport (async client and server)
o Runs I/O callbacks (Watches file descriptors)
o Thread interface
o [BaseEventLoop.create_task()] or async()
o [asyncio.get_event_loop()]

Asyncio: Components
• Co-routine
o Generator (“yield from”)
o suspended at preset execution points, and
o resumed later by keeping track of local state
o @coroutine decorator

Asyncio: Components
• Task
o responsible for executing a coroutine
o If coroutine yields from a future, the task suspends the execution of the
coroutine and waits for the future
o coroutine restarts when future is done
o Subclass of class Future
o [async(coroutine)]
o BaseEventLoop.create_task(coro)

Asyncio: Components
• Future
o A class
o for results that are
available later
import asyncio
@asyncio.coroutine
def slow_operation(future):
yield from asyncio.sleep(1) <- Co-routine suspend
future.set_result('Future is done!')
def got_result(future):
print(future.result())
loop.stop()
loop = asyncio.get_event_loop() <- Event loop
future = asyncio.Future() <- Future object
asyncio.async(slow_operation(future)) <- Task
future.add_done_callback(got_result)
try:
loop.run_forever()
finally:
loop.close()

Asyncio: Components
• transport
o represent connections such as sockets, SSL connection and pipes
o Async socket operations
• Usually frameworks implement e.g. Tornado
• protocols
o represent applications such as HTTP client/server, SMTP, and FTP
o Async http operation
o [loop.create_connection()]

Example: Asyncio Redis
import asyncio
import asyncio_redis
@asyncio.coroutine
def my_subscriber(channels):
connection = yield from asyncio_redis.Connection.create(host='localhost', port=6379)
subscriber = yield from connection.start_subscribe()
yield from subscriber.subscribe(channels)
while True:
reply = yield from subscriber.next_published()
print('Received: ', repr(reply.value), 'on channel', reply.channel)
loop = asyncio.get_event_loop()
asyncio.async(my_subscriber('channel-1'))
asyncio.async(my_subscriber('channel-2'))
loop.run_forever()

Example: Asyncio ‘Tasks’
import asyncio
@asyncio.coroutine
def factorial(name, number):
f = 1
for i in range(2, number+1):
print("Task %s: Compute factorial(%s)..." % (name, i))
yield from asyncio.sleep(1)
f *= i
print("Task %s: factorial(%s) = %s" % (name, number, f))
tasks = [
asyncio.async(factorial("A", 2)),
asyncio.async(factorial("B", 3)),
asyncio.async(factorial("C", 4))]
loop.run_until_complete(asyncio.wait(tasks))
loop.close()

Tornado Async
• Event Loop => tornado.ioloop
• Coroutine => tornado.gen.coroutine
• Future => tornado.concurrent.future
• Transport/Protocol => tornado.iostream
• Bridge the gap => tornado.platform.asyncio –
Combines asyncio and tornado in same event loop

Tornado Async Http
import tornado.ioloop
from tornado.httpclient import AsyncHTTPClient
def handle_request(response):
'''callback needed when a response arrive'''
if response.error:
print("Error:", response.error)
else:
print(’Success')
print(response.body)
Before Event Loop Starts!
Success
b'{"ip": "117.192.252.80"}n'
Callback
http_client = AsyncHTTPClient() # initialize http client
http_client.fetch(” http://ip.jsontest.com/", handle_request)
print("Before Event Loop Starts!")
tornado.ioloop.IOLoop.instance().start() # start the tornado ioloop

Tornado Coroutine
import tornado.web
import tornado.gen
class GenAsyncHandler(tornado.web.RequestHandler):
@tornado.gen.coroutine
def get(self):
http_client = AsyncHTTPClient()
response = yield http_client.fetch("http://google.com")
print(response)
application = tornado.web.Application([ (r"/",
GenAsyncHandler), ])
if __name__ == "__main__":
application.listen(8888)
tornado.ioloop.IOLoop.instance().start()
gen.coroutine schedules the generator
to be resumed when the Future is
resolved
‘yield’ makes the function a generator
The generator in turn returns a Future
instance
In this case, response, will resolve with
response from fetch or an exception

Tornado Engine
class MainHandlerAsync(tornado.web.RequestHandler):
@tornado.web.asynchronous
@tornado.gen.engine
def get(self):
req = tornado.httpclient.HTTPRequest("http://127.0.0.1:8888/",)
client = tornado.httpclient.AsyncHTTPClient()
response = yield tornado.gen.Task(client.fetch, req)
self.finish()
application = tornado.web.Application([
(r"/async", MainHandlerAsync),
])
if __name__ == "__main__":
http_server = tornado.httpserver.HTTPServer(application)
http_server.listen(8888)

Let’s create our Future!
myfuture.py server.py
import time
import datetime
from tornado.concurrent import return_future
class AsyncHTTPClient(object):
@return_future
def fetch(self, url, callback=None):
print("In my fetch")
time.sleep(0.02)
result = str(datetime.datetime.utcnow())
callback(result)
import tornado.web
import tornado.gen
from myfuture import AsyncHTTPClient
def test(arg):
print('In test:' + arg)
class GenAsync(tornado.web.RequestHandler):
@tornado.gen.coroutine
def get(self):
http_client = AsyncHTTPClient()
r = yield http_client.fetch(“http://google.com”,test)
print(r)
application = tornado.web.Application([
(r"/", GenAsync),])
if __name__ == "__main__":
application.listen(8888)

Performance:
Blocking vs Async

Performance Results
ab -n 500 -c 10 http://localhost:8888/blocking
ab -n 500 -c 10 http://localhost:8888/async
6000
5000
4000
3000
2000
1000
0
Time per request
Async Blocking
Async
Blocking
200
150
100
50
0
Requests per second
Async Blocking
Async
Blocking

Learnings
• Async programming is an efficient, easy to
understand design and code
• Python asyncio module is comprehensive
• Has generic use cases for vast variety of
applications
o Responsive web applications
o Networking applications
• Requires a new way to program and design

Recommendations
• Async programming is not a holistic solution
• It has its own pros and cons
o Suitable for primarily I/O bound applications
o Needs enough tasks available to run
• asyncio module is only available for Python 3
applications
• Also explore other methods of concurrency:
o Eventlets
o STM
o Multiprocessing/threads
o Special languages e.g. GO, Scala
• Understand and use 

References
• asyncio – http://python.org
• Python asyncio –
o http://www.buzzcapture.com
o www.slideshare.net/saghul
• Tornado – http://tornadoweb.org
• Multithreading – www.drdobbs.com
• Event loop: https://docs.python.org/3/library/asyncio-eventloop.
html

Contact Us
• Chetan Giridhar
o www.technobeans.com
o https://github.com/cjgiridhar
• Vishal Kanaujia
o www.freethreads.wordpress.com
o https://github.com/vishalkanaujia

Asyncio: example
import asyncio
@asyncio.coroutine
def create():
yield from asyncio.sleep(3.0)
print("(1) create file")
@asyncio.coroutine
def write():
print("(2) write into file")
@asyncio.coroutine
def close():
print("(3) close file")
@asyncio.coroutine
def test():
asyncio.async(create())
asyncio.async(write())
asyncio.async(close())
loop.stop()
asyncio.async(test())
loop.run_forever()
print("Pending tasks at exit: %s" % asyncio.Task.all_tasks(loop))
loop.close()

Python Async Modules
• Asynccore
• Asyncchat
• Gevent
• Twisted
• Eventlets

Concurrency Techniques
• Multithreading/processing
• Green Threads
• STM

Tornado + AsyncIO
from tornado.platform.asyncio import AsyncIOMainLoop
import asyncio
AsyncIOMainLoop().install() -- # Tell Tornado to use the asyncio eventloop
loop = asyncio.get_event_loop() -- # get the loop
http_client = AsyncHTTPClient() -- # the Tornado HTTP client
def aio_fetch(client, url, **kwargs):
fut = asyncio.Future()
client.fetch(url, callback=fut.set_result, **kwargs)
return fut
@asyncio.coroutine
def main():
print("fetching my site")
mysite = yield from aio_fetch(http_client, "http://google.com/")
print("hello httpbin")
httpbin = yield from aio_fetch(http_client, "http://httpbin.org?q=%d" % mysite.code)
print(httpbin.body)
loop.run_until_complete(main())

Co-routine v/s Callback
import asyncio
def just_print_messages(loop):
print('Just print')
loop.call_later(1, just_print_messages, loop)
def main():
try:
loop.call_soon(just_print_messages, loop)
loop.run_forever()
finally:
loop.close()
if __name__ == '__main__':
main()
import asyncio
@asyncio.coroutine
def just_print_messages():
while True:
print('Just print')
yield from asyncio.sleep(1)
def main():
try:
loop.run_until_complete(just_print_messages())
finally:
loop.close()
if __name__ == '__main__':
main()

Async in NodeJS
request is an
var http = require(‘http’);
event
var server = http.createServer;
server.on(‘request’, function(request,response) {
response.writeHead(200);
response.end(‘Hello World’);
}).listen(8001);
Callback
console.log(‘Server running on port 8001’);

Async programming and python

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to Async programming and python (20)

More from Chetan Giridhar (7)

Recently uploaded (20)

Async programming and python

Editor's Notes