Max euro python 2015
1 like361 views
MAX is a realtime messaging and activity stream engine that was originally designed for the Universitat Politècnica de Catalunya's social intranet. It provides a RESTful API with over 80 endpoints for multi-source user activity streams, asynchronous messaging and conversations. Key features include an activity stream, conversations, notifications, and the ability to aggregate external sources. It is fully deployable on-premises and addresses security and privacy concerns.
![MAX (Routing)
● Route definition
RESOURCES['avatar'] = dict(route='/people/{username}/
avatar', filesystem=True, category='User', name='User
avatar', traverse='/people/{username}')
● Endpoint definition
@endpoint(route_name='avatar', request_method='POST',
permission=modify_avatar)
def postUserAvatar(user, request):
"""
Upload user avatar
"""](https://image.slidesharecdn.com/maxeuropython2015-160105132201/85/Max-euro-python-2015-23-320.jpg)
![MaxClient
● Opinionated Wrapper for REST api’s
● Wraps endpoint resources based on endpoint list definition.
RESOURCES[‘activity’] = dict(route=’/people/{username}/activities’)
● Accesses endpoints in a pythonic way
>>> client.people[‘username’].activities.get(qs={‘limit’:2})
● Creates json bodies from “nested” kwargs (with optional
sensible defaults)
>>> client.activities.post(object_content=’Hello’)
{
“object”: {
“objectType”: “note”,
“content”: “Hello”
}}](https://image.slidesharecdn.com/maxeuropython2015-160105132201/85/Max-euro-python-2015-35-320.jpg)
Max euro python 2015
- 1. MAX Realtime messaging and activity stream engine Carles Bruguera (@sunbit) Víctor Fernández de Alba (@sneridagh)
- 2. Víctor ● Senior Python Developer and IT architect ● Plone Foundation member ● Plone core developer since 2010 ● Author of Plone 3 intranets book (2010, PacktPub) @sneridagh
- 3. Carles ● Python and JavaScript lover ● Working with python for the last 8 years ● Occasional Erlang coder (when on drugs) ● Regex freak @sunbit
- 4. Python @UPCnet and @BarcelonaTech
- 5. DEMO!
- 6. History ● First commit on August, 2011 ● Initially designed as the key feature for the Universitat Politècnica de Catalunya (BarcelonaTech) university concept of social intranet ● Today, MAX is used by more than 30.000 students and 8.000 university staff integrated in the online campus and the institutional collaboration tools
- 7. What is MAX? ● RESTful API ● 88 (and growing) endpoints ● Multi-source user and application activity stream ● Asynchronous messaging and conversations ● GPL Licensed
- 8. Old styled forums Forum Topic Post Post Post
- 9. What is a context? Context (unique URI) Forum Topic Post Posts Subscriptions Post Post
- 10. Contexts ● Identified by unique URIs ● Permissions per context o read o write o subscribe o unsubscribe ● Multiple context types based on permissions variations ● Granular permissions per user o Overriding the default ones defined by the context o Grant / Revoke o invite o kick o delete o flag
- 11. Real life examples Communities site Alumni Sell your stuff Institutional events Institutional news Online campus Compilers Faculty news Applied maths III Signal theory
- 12. Real life examples (II) Community types Open Closed Institutional Everyone can join and leave at will The owner should invite me to join and I can leave at will The site admin subscribes people, no one can leave
- 13. Features Activity Stream ● Stores activity from users and applications ● Usual social actions ○ Comments ○ Likes ○ Favorites ● Images and files support
- 14. Features Conversations ● Realtime conversations and private messaging ● One to one ● Groups ● Images and files support
- 16. Notifications ● Platform specific push notifications ● Internal notifications ○ Double check ○ others Features
- 18. Features Fully deployable on premises ● Addresses any security concerns ● Absolute customer data privacy and ownership ● “Corporate whatsapp”
- 20. Components overview iOs App Plone Moodle android App REST Api “MAX” OAuth Server “Osiris” Messaging “Maxbunny” NGINX max.ui.js RabbitMQ MongoDB LDAP Sync services “Hub” Twitter listener “MaxTweety”
- 21. Osiris ● Minimal OAuth2 server implementation ● Build on top of pyramid ● Resource Owner Credentials Flow ● Tokens stored on MongoDB ● /token endpoint to generate token for a user ● /checktoken endpoint to verify a token ● Base LDAP user storage implementation ● Pluggable repoze.who based alternative user storage implementations NGINX Pyramid + gevent MongoDB WSGI (Chausette)
- 22. MAX ● REST(ful) api ● Also build on top of pyramid ● Hybrid URL-dispatch + traversal routing ● ACL policy with fine-grained permissions per endpoint ● Customized venusian decorator to configure endpoints ● Tweens used for several tasks ● Per-exception catching to provide detailed JSON error messages ● Per-request caching of variables NGINX Pyramid + Gevent MongoDB WSGI (Chausette) RabbitMQ
- 23. MAX (Routing) ● Route definition RESOURCES['avatar'] = dict(route='/people/{username}/ avatar', filesystem=True, category='User', name='User avatar', traverse='/people/{username}') ● Endpoint definition @endpoint(route_name='avatar', request_method='POST', permission=modify_avatar) def postUserAvatar(user, request): """ Upload user avatar """
- 24. MAX (Tweens) ● exception catcher ● post tunneling ● compatibility check def compatibility_checker_factory(handler, registry): def compatibility_checker_tween(request): requested_compat_id = request.headers.get('X-Max-Compat-ID', None) if requested_compat_id is None: response = handler(request) return response expected_compat_id = str(request.registry.settings.get('max.compat_id')) if expected_compat_id == requested_compat_id: response = handler(request) return response else: return JSONHTTPPreconditionFailed( error=dict( objectType='error', error="CompatibilityIDMismatch", error_description='X-Max-Compat-ID header value mismatch, {} was expected'.format(expected_compat_id))) return compatibility_checker_tween
- 25. MAX (Exception handling) ● Known error use cases are raised as custom exceptions: raise ObjectNotFound("User {} doesn't have role {}".format(user, role)) ● And rendered as a JSON message @view_config(context=ObjectNotFound) def object_not_found(exc, request): return JSONHTTPNotFound(error=dict(objectType='error', error=ObjectNotFound.__name__, error_description=exc.message)) ● Non-handled exceptions are logged with request information
- 27. RabbitMQ & messaging ● Exchange-to-exchange routing ● STOMP over WS using rabbitmq plugins ● “Public” end-user stomp endpoints ● Message delivery and security through routing key bindings ● Oauth authentication via erlang plugin ● Easy plug-in of temp queues for debugging NGINX Queues & exchanges RabbitMQ Websockets STOMP AMQP Oauth Authentication Oauth2
- 29. MaxCarrot {"uuid": "005fab55bee84", "user": { "username": "johndoe", "displayname" : "John Doe" }, "action": "add", "object": "message", "data": { "text": "Hello world!" }, "source": "ios", "domain": "demo", "version": "4.0.1", "published": "2015-07-21"}
- 30. MaxCarrot ● JSON based message format ● Used on messages routed through RabbitMQ ● Packed and unpacked versions ● Metadata/debugging fields ● Purpose related fields ● Encapsulates messaging logic
- 31. MaxCarrot (Rules) ● Map field combinations to actions ● Pack messages following spec ● Ignore any message not matching any mapping "source": { "id": "s", "type": "char", "values": { "ios": { "id”: "c" }, (...) "max": { "id”: "m" } } "version": { "id": "v", "type": "string", }
- 32. MaxCarrot (human-readable) {"uuid": "005fab55bee84", "user": { "username": "johndoe", "displayname" : "John Doe" }, "action": "add", "object": "message", "data": { "text": "Hello world!" }, "source": "ios", "domain": "demo", "version": "4.0.1", "published": "2015-07-21"}
- 34. MaxBunny ● Pluggable multiprocess domain-aware queue consumer ● A multiprocess runner runs N process for each consumer defined. ● Each consumer binds to a queue and consumes messages ● Runner provides a shared pool of WSGI MaxClient instances, one for each domain.MongoDB RabbitMQ MAX MaxBunny Runner & Consumers WSGI MaxClient
- 35. MaxClient ● Opinionated Wrapper for REST api’s ● Wraps endpoint resources based on endpoint list definition. RESOURCES[‘activity’] = dict(route=’/people/{username}/activities’) ● Accesses endpoints in a pythonic way >>> client.people[‘username’].activities.get(qs={‘limit’:2}) ● Creates json bodies from “nested” kwargs (with optional sensible defaults) >>> client.activities.post(object_content=’Hello’) { “object”: { “objectType”: “note”, “content”: “Hello” }}
- 36. MaxClient ● Raises a custom RequestError exception on 5xx and 4xx responses ● Returns None on 404 responses ● Returns parsed JSON body on success responses
- 37. WSGI MaxClient ● WSGI version ● Subclassed MaxClient that makes calls to a “fake” wsgi server ● Actual MAX Codebase is run by the client ● Requests don’t stress main api servers, only database ● Limitations: o User must have privileges on “real” max server o Computer from where client is run must have access to storage backed
- 38. Twitter external aggregation ● Backend process listens twitter streaming service ● Selected tweets are queued and processed ● Valid tweets are injected into max activity stream ● Max users linked with twitter usernames ● Aggregation of content as a context
- 39. Current integrations UPCnet uLearn Communities UPCnet uLearn Campus iOS & Android apps
- 44. Potential integrations <place your web|app|whatever thingy name here> <place the screenshots here>
- 45. Whishlist ● Social interactions (Follow/share) ● Finish & polish documentation ● Microservices / Dockerization ● Redis backed cache ● RabbitMQ SSL without NGINX ● Python3 / asyncio ● Explore JSON-LD (HATEOAS) ● Encryption
- 46. Community building? Absolutely! Let’s do it! Contact us, PR are welcome!