PyConIT6 - MAKING SESSIONS AND CACHING ROOMMATES
0 likes777 views
Beaker is a framework for managing caching and sessions in web applications, created to address the dogpile effect. While it supports multiple backends and simplifies session management through cookie storage, challenges include issues with data serialization in Python 3 and design considerations around user expectations for function argument handling in caching decorators. The presentation highlights both the advantages and limitations of using Beaker, proposing solutions and improvements while reflecting on its development journey.
![HTTP Cookies
HTTP has allowed field content with text
in the ISO-8859-1 charset [ISO-8859-1],
supporting other charsets only through
use of [RFC2047] encoding. In practice,
most HTTP header field values use only a
subset of the US-ASCII charset [USASCII].
Newly defined header fields SHOULD limit
their field values to US-ASCII octets](https://image.slidesharecdn.com/pyconit62015-beaker-150418121301-conversion-gate01/85/PyConIT6-MAKING-SESSIONS-AND-CACHING-ROOMMATES-16-320.jpg)
PyConIT6 - MAKING SESSIONS AND CACHING ROOMMATES
- 1. BEAKER - MAKING SESSIONS AND CACHING ROOMMATES Alessandro Molina @__amol__ amol@turbogears.org
- 2. Who am I ● CTO @ AXANT.it, mostly Python company (with some iOS and Android) ● TurboGears2 dev team member ● Took over Beaker maintenace in 2015 ● Mostly contributed to web python libraries: Formencode, MING MongoDB ODM, ToscaWidgets2
- 3. What’s Beaker ● Framework for handling Caching and Sessions in web applications ● Used by many different frameworks: TurboGears, Bottle, Pyramid, etc... ● Created by Pylons Author to solve the dogpile effect.
- 4. DogPile!
- 5. The Good ● Handles the DogPile effect ● Provides many backends: Memcache, File, MongoDB, Redis, SQLAlchemy ● A single tool to handle both caching and sessions with common backends.
- 6. The Bad ● File based synchronization, what about distributed solutions? ● Session as a big BLOB of data doesn’t perform well in case of big BLOBs of data ● Updated for too long without breaking backward compatibility: Too many APIs
- 7. Getting on project: PY3 Support ● Beaker supported Python3 using 2to3 ● This lead to some bugs (not everything got properly converted) ● Made really hard to run the test suite and maintain the project.
- 8. So my first action as new maintainer...
- 9. Rewriting for Py3 a project that involves serializing/deserializing
- 10. Porting to PY3: #1 Snakes speak ? Beaker uses BASE64 to encode pickled data in sessions. >>> import base64, pickle >>> data = dict(key='value', otherkey='othervalue') >>> base64.b64encode(pickle.dumps(data)) 'KGRwMApTJ290aGVya2V5JwpwMQpTJ290aGVydmFsdWUnCnAyCnN TJ2tleScKcDMKUyd2YWx1ZScKcDQKcy4='
- 11. BASE64: rfc4648 Here are a few requirements that determine which alphabet should be used: o Handled by humans.
- 14. Human handle? BYTES! >>> import base64, pickle >>> base64.b64encode(pickle.dumps(‘HELLO’)) b'gANYBQAAAEhFTExPcQAu'
- 15. Porting to PY3: #2 Session Cookies? ● Session ID is stored in cookies ● CookieSession stores even the whole session in a cookie ○ Great idea, makes really simple to scale ○ HTTP Headers are plain text ○ We are pickling data and base64 so already ASCII ○ Just need to encrypt it to avoid people from messing with them.
- 16. HTTP Cookies HTTP has allowed field content with text in the ISO-8859-1 charset [ISO-8859-1], supporting other charsets only through use of [RFC2047] encoding. In practice, most HTTP header field values use only a subset of the US-ASCII charset [USASCII]. Newly defined header fields SHOULD limit their field values to US-ASCII octets
- 17. Encoding/Encripting data on Py3 ● Beaker expected to be able to perform text operations on the result of base64 ● Relied on AES to encrypt cookies, pbkdf2 to generates keys and base64/binascii to make it text. ● Reads the encrypt_key from config file
- 18. 4 lines of code == 10 doubts ● Even Python has no clear idea of what is text and what are bytes: ○ Both BASE64 and BINASCII work with ASCII ○ BINASCII accepts unicode (text) ○ BASE64 accepts bytes ○ Between Python3.2 and Python3.3 the binascii changed behaviour of accepting unicode vs bytes ○ ConfigParser returns text on py3, bytes on py2
- 19. Now crying in a corner...
- 20. Let’s go for something simpler... ● Review patch to caching decorator ● To cache a function, just apply a decorator that calls function and caches the result ● Caching decorator generates cache key ○ Cache Key from the function name ○ convert parameters to strings ○ add parameters to cache key
- 21. Getting the Cache Key def cache(self, *args, **kwargs): """Decorate a function to cache itself with supplied parameters :param args: Used to make the key unique for this function, as in region() above. :param kwargs: Parameters to be passed to get_cache(), will override defaults Example:: # Assuming a cache object is available like: cache = CacheManager(dict_of_config_options) def populate_things(): @cache.cache('mycache', expire=15) def load(search_term, limit, offset): return load_the_data(search_term, limit, offset) return load('rabbits', 20, 0) """ return _cache_decorate(args, self, kwargs, None)
- 22. Well, maybe...
- 23. Unknown parameters ● Decorator gets *args and **kwargs ● Makes sense... doesn’t know the real function arguments ● Not so much in fact… func(1) and func (a=1) end up with two different cache keys
- 24. Avoid the doubt ● Beaker solved this by accepting only positional arguments on cached functions .. note:: The function being decorated must only be called with positional arguments. ● Users not so happy, breaks people code when they introduce caching: ○ https://github.com/bbangert/beaker/issues/62
- 25. What to do? ● Leave it as is… As been like that for years ● Add **kwargs and just document the issue ● Add **kwargs and try to be smart using inspect.getargspec / inspect. getcallargs but that would be slow
- 26. Leave it as is… at least isn’t broken
- 27. Has been a great trip! ● Making sessions and caching in a single solution seemed simple, but has actually a lot of corner case ● Beaker Co-Author thrown in the towel and created dopgile.cache which is based on beaker code but is much simpler and doesn’t provide sessions
- 28. Still is incredibily convenient ● There are more shared features than differences (especially in cookie based marshalled sessions). ● It’s really convenient, write backends once and have support for both sessions and caching on them.
- 29. Questions?