Gevent what's the point

Gevent
What’s the point?
Sean McQuillan
sean@bittorrent.com

Bottom Line

Gevent is fast (libev)
Pythonic style
Single threaded event loop

Gevent is Fast
application code

gevent

libev greenlet

kernel

Pythonic Style
def get_db():
return db.query(...)
vs
def _db_cb(result):
consumer.send(result)

def get_db():
x = db.query(...)
x.addCallback(_db_cb)

Single Threaded
Event Loop

Single threaded

Single Threaded
Event Loop

Single threaded
wait

Single Threaded
Event Loop

Single threaded
wait
what?

Execution Model

One OS thread
Only ONE thing happens at a time...
Switching constantly

More Execution Model

def example_function(sock):
r = sock.recv(4096)
count = len(r)
for c in xrange(count):
tokenizer.send(c)
r += sock.recv(4096)

More Execution Model

def get_db():
return db.query(...)

from flask import Flask
app = Flask(__name__)

@app.route('/')
def hello_world():
import time; time.sleep(1)
return 'Hello World!'

if __name__ == '__main__':
app.run()


import gevent.monkey
gevent.monkey.patch_all()

@app.route('/')
def hello_world():
import time; time.sleep(1)
return 'Hello World!'

if __name__ == '__main__':
app.run()


import gevent.monkey
gevent.monkey.patch_all()

Monkey Patching

Replace existing libraries
Not generally done in python

Monkey Patching

sys.modules[socket] = gevent.socket
sys.modules[time] = gevent.sleep
...

Part II: Event Loops

Low overhead
Single threaded
Handle LOTS of connections

poll, select, epoll, kqueue

“Given a list of file descriptors which are
ready for read or write.”

Lets dig in
application code

gevent

libev greenlet

kernel

Lets dig in
socket 1 write callback event socket 1 read callback
socket 2 write callback loop socket 2 read callback

socket 1, socket 2, ...

kernel
write buffer 1 read buffer 1
write buffer 2 read buffer 2

Example

Contrived event loop example
See t wisted event loop for real example
t wisted/interent/selectreactor.py

Example
rlist = []
wlist = []
callbacks = {}
pending = defaultdict(str)

def want_to_read(sock, cb):
callbacks[sock.fileno] = cb
global rlist
rlist = list(set(rlist + [sock.fileno]))

def want_to_write(sock, data):
pending[sock.fileno] += data
global wlist
wlist = list(set(wlist + [sock.fileno])))

Example

def event_loop():
while True:
reads, writes, _ = select(rlist, wlist, [])
for readfd in reads:
data = _do_actual_read(readfd)
callbacks[readfd](data)
for writefd in writes:
_do_actual_write(pending[writefd])

Part III: Sockets

Sockets
What magic lies here

patched socket.recv
def recv(self, *args):
sock = self._sock
while True:
try:
return sock.recv(*args)
except error:
ex = sys.exc_info()[1]
if ex.args[0] == EBADF:
return ''
if (ex.args[0] != EWOULDBLOCK or
self.timeout == 0.0 ):
raise
self._wait(self._read_event)

patched socket.recv
def recv(self, *args):
sock = self._sock
while True:
try:
return sock.recv(*args)
except error:
ex = sys.exc_info()[1]
if ex.args[0] == EBADF:
return ''
if (ex.args[0] != EWOULDBLOCK or
self.timeout == 0.0 ):
raise
self.want_to_read(sock,
self.mygreenthread.switch)

Part IV: Greenlets

“green” threads

Part IV: Greenlets

“green” threads
(that means fake threads)

Green Threads Good

Never switch accidentally
Never spend time blocked on IO
High performance switching

Green Threads Bad

Never preemptively interrupt
No SMP

Green Thread Execution

Python Initial Stack


Gevent Initialized



Green
Thread
Stack

Gevent Initialized


Green
Thread
Green Stack
Thread
Stack

Gevent Initialized


Green Green
Thread Thread
Green Stack Stack
Thread
Stack

Gevent Initialized


How does it switch?
(x86 assembly)
    void *ebp, *ebx;
    unsigned short cw;
    register int *stackref, stsizediff;
    __asm__ volatile ("" : : : "esi", "edi");
    __asm__ volatile ("fstcw %0" : "=m" (cw));
    __asm__ volatile ("movl %%ebp, %0" : "=m" (ebp));
    __asm__ volatile ("movl %%ebx, %0" : "=m" (ebx));
    __asm__ ("movl %%esp, %0" : "=g" (stackref));
    {
        SLP_SAVE_STATE(stackref, stsizediff);
        __asm__ volatile (
            "addl %0, %%espn"
            "addl %0, %%ebpn"
            :
            : "r" (stsizediff)
            );
        SLP_RESTORE_STATE();
    }
    __asm__ volatile ("movl %0, %%ebx" : : "m" (ebx));
    __asm__ volatile ("movl %0, %%ebp" : : "m" (ebp));
    __asm__ volatile ("fldcw %0" : : "m" (cw));
    __asm__ volatile ("" : : : "esi", "edi");
    return 0;

When to use Gevent

Replace thread-based ser vers
Replace thread-based clients
Lighter than multiprocessing

When not to use gevent

When SMP is required
High CPU load
Latency guarntees
Memory star ved systems

Twisted
Mature, async-baked, implementations
of most protocols
DNS Ser ver
SSH Server
AWS
AMQP
Web

Tornado

HTTP
Highly performant
Less protocol / library support

I’m not an expert

Erlang

Scales more vertically (dozens of cores)
Similar execution model
Less library support

Python Threading

I’ve never tried to write a “web-scale”
server with it, would love to hear your
experience

Python
Multiproccessing

Great SMP usage

Go

Never used, benchmarks look good

Gevent what's the point

More Related Content

What's hot (20)

Similar to Gevent what's the point (20)

Recently uploaded (20)

Gevent what's the point

Editor's Notes