SlideShare a Scribd company logo
WebCamp 2016: Python. Вячеслав Каковский: Real-time мессенджер на Python. Особенности разработки back-end.
Me!
@kakovskyi
Python Developer at SoftServe
Contributor of Atlassian HipChat — Python 2, Twisted
Maintainer of KPIdata — Python 3, asyncio
2
Agenda
● What is 'instant messenger'?
● Related projects from my experience
● Messaging protocols
● Life of messaging platform
● Lessons learned
● Summary
● Further reading
3
What is 'instant messenger'?
4
What is 'instant messenger'?
● online chat
● real-time delivery
● short messages
5
What is 'instant messenger'?
● history search
● file sharing
● mobile push notifications
● video calling
● bots and integrations
6
Related projects from my experience
● Hosted chat for teams and enterprises
● Founded in 2009 by 3 students
● 100 000+ connected users
● 100+ nodes
● REST API for integrations and bots
● Built with Python 2 and Twisted
7
Messaging protocols
Protocol is about:
● Message format
● Allowed types of messages
● Limitations
● Routine
○ How to encode data?
○ How to establish/close connection?
○ How to authenticate?
○ How to encrypt?
8
Messaging protocols
● OSCAR (1997)
● XMPP (1999)
● Skype (2003)
● WebSocket-based (2011)
● MQTT, MTProto, DHT-based, etc.
9
XMPP
● XMPP - signaling protocol
● BOSH - transport protocol
● Started from Jabber in 1999
● XML as a message format
● Stanza - basic unit in XMPP
● Types of stanzas:
○ Message
○ Presence
○ Info/Query
10
XMPP
● Extensions defined by XEPs (XMPP Extension
Protocols):
○ Bidirectional-streams Over Synchronous
HTTP (BOSH)
○ Serverless messaging
○ File transfer and etc.
11
XMPP: Establishing a connection
12
Client:
<?xml version='1.0'?>
<stream:stream to='example.com' xmlns='jabber:client'
xmlns:stream='http://etherx.jabber.org/streams'
version='1.0'>
Server:
<?xml version='1.0'?>
<stream:stream from='example.com' id='someid'
xmlns='jabber:client' xmlns:stream='http://etherx.jabber.
org/streams' version='1.0'>
XMPP: Sending a message
13
Client:
<message from='juliet@example.com' to='romeo@example.net'
xml:lang='en'>
<body>Art thou not Romeo, and a Montague?</body>
</message>
Server:
<message from='romeo@example.net' to='juliet@example.com' xml:
lang='en'>
<body>
Neither, fair saint, if either thee dislike.
</body>
</message>
XMPP: Closing a connection
14
Client:
</stream:stream>
Server:
</stream:stream>
XMPP: Pros
● Robust and standardized
● Extendable via XEPs
● Secured
● Native support of multi-sessions
● A lot of clients implementations
15
XMPP: Cons
● Overhead
○ Presence
○ Downloading the World on startup
● XML
○ Large documents
○ Expensive parsing
16
XMPP and Python
● Servers:
○ TwistedWords - good place to start
○ Tornado-based example
○ aioxmpp
○ XMPPFlask
○ Punjab - BOSH-server on Twisted
17
XMPP and Python
● Clients:
○ SleekXMPP - mature and solid
○ Slixmpp - asyncio-support
○ TwistedWords
○ Wokkel - Twisted-based
○ xmpp.py
● JS-client: Strophe.js
18
WebSocket-based solutions
● WebSocket - transport protocol
● Standardized in 2011 by W3C
● Full-duplex communication channel
● JSON as a message format
● Custom message types
19
WebSocket: Establishing a connection
20
Client:
GET /chat HTTP/1.1
Host: server.example.com
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==
Origin: http://example.com
Sec-WebSocket-Protocol: chat, superchat
Sec-WebSocket-Version: 13
Server:
HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=
WebSocket: Sending a message
21
Client:
{
"type": "message",
"ts": 1469563519,
"user": "kakovskyi",
"text": "Hello, @WebCamp!"
}
Server:
{
"type": "notification",
"ts": 1469563519,
"user": "WebCamp Bot",
"text": "Howdy @kakovskyi?"
}
WebSocket: Closing a connection
22
Client:
0x8
Server:
0x8
WebSocket: Pros
● Supported by majority of browsers
● Low latency
● Small bandwidth
● Easy to start development
23
WebSocket: Cons
● Needs development of signaling protocol
● Timeouts/reconnections should be additionally
handled
24
WebSocket and Python
● Servers:
○ Autobahn - Twisted and asyncio
implementations
○ aiohttp
○ Tornado
○ Flask-SocketIO
○ Flask-Sockets
25
WebSocket and Python
● Clients:
○ Autobahn
○ aiohttp
○ Tornado-based example
○ Vanilla websocket-client
● JS-client: SocketIO
26
Life of messaging platform
● Authentication
● Access control checks
● Delivery
○ Messages
○ User's presence
○ Push notifications
● History retrieval
● History search
27
Life of messaging platform
● Parsing
○ Protocol
○ Message content
● Dealing with file uploads
○ Security checks
○ Thumbnails distribution
● Multi-session support
● Reconnection handling
● Rate-limiting
28
Life of messaging platform
● Server keeps connections open for every client
● High amount of long-lived concurrent connections
● Multithreaded approach isn't efficient due to overhead
● Requires usage of a select implementation on backend:
○ poll
○ epoll
○ kqueue
● Usage of asynchronous Python frameworks is preferred
for high loaded solutions
29
Life of messaging platform
● Authentication
○ OAuth2
○ Run encryption operations in a separate Python thread
○ Cache users identities with Redis/Memcached
● Access-control checks
○ Make the checks lightweight and cheap
○ Raise an exception when operation isn't permitted
30
EAFP: Easier to ask for forgiveness than permission
Delivery
● Make message delivery fault-tolerant
● Limit size of a message
● Filter content of messages:
○ Users like to send chars that break all the things
● Reduce presence traffic, it could be a bottleneck for large chats
● Use asynchronous broker for delivery when a user is offline
(email or push)
○ Celery
○ RQ
○ Amazon Simple Queue Service
○ Huey
31
Life of messaging platform
● Push notifications
■ Vendors
● Amazon SNS
● APNS
● Google Cloud Messaging
● Firebase Cloud Messaging
■ Python tools
● PyAPNs
● Python-GCM
● Pusher
● Be careful with device registration
● Make delivery of pushes fault-tolerant
32
History retrieval
● Return last messages for every chat instantly
○ Use double writes
■ In-memory queue only for last messages
■ Persistent storage for all the things
● Majority of history retrievals is for the last days
○ Let's optimize the case
● Index messages by date
33
History search
● ElasticSearch is the default solution for full-
text search
● @a_soldatenko: What is the best full text
search engine for Python?
● Add timing for search requests
34
Parsing
● Protocol
○ Avoid to use Pure Python parsers
■ ujson
■ lxml
○ Run benchmarks against your typical cases
● Message content
○ Be careful with regular expressions
■ re2
■ pyre2
○ Alternative parsers in Python
35
Dealing with file uploads
● Security checks
○ File upload vulnerabilities
○ Image upload
■ Decompression bomb
■ Other vulnerabilities with Pillow
○ Amazon S3 as file storage
■ boto
■ aiobotocore
■ botornado
● Thumbnails distribution
○ Delegate that to S3
○ Requested by a client even if not needed
36
Life of messaging platform
● Multi-session support
○ Set expiration time
○ Be ready to handle up to 4x sessions per user simultaneously
■ Desktop
■ Mobile
■ Tablet
■ Laptop
● Reconnection handling
○ Spin a proxy layer between messaging server and clients
● Rate-limiting
○ Limit amount of operations per user/group for heavy stuff
○ Leaky bucket
○ Throttling
37
Lessons learned
● Bursty traffic
○ Load testing is a must, but not always enough
■ Locust
■ Yandex Tank
● Reconnect storm could be a big deal
○ We should handle that on platform and client-side
● AWS issues make bad customers experience
○ Put nodes in Multi-AZ
38
Lessons learned
● Incidents prevention is cheaper than resolution
○ Grab stats and metrics about your services and
storages
■ Redis for per-chat stats
■ StatsD
■ Grafana
○ Be notified when something starts going wrong
■ Elastalert
■ Monit
■ DataDog
39
Lessons learned
● Don't stick with one language/stack
○ Python is great, but for some cases Go, Ruby or
PHP are more suitable from product side
○ Avoid business logic duplication in several repos,
spin a service and just call the endpoint
● Releasing new features only for certain groups makes
product management easier
○ LaunchDarkly
40
Lessons learned
● Don’t F**k the Customer
○ Provide unit/integration tests with every PR
○ Have development environment same as prod
○ Have staging environment same as prod
○ Make deployments fast
○ Rollback faster
○ Have a fallback plan
41
Summary
42
Summary
● Select a messaging protocol which aligns with your needs
● WebSocket + JSON could be the thing for new projects
● Usage of asynchronous frameworks is preferred
● Execute blocking operations in a separate thread
● Collect metrics for common services operations
● Caching saves a lot of time
● Use C or Cython-based solutions for CPU-bound tasks
● Have fast release/deploy/rollback cycle
● Python is great, but don't hesitate to pick other tools
43
Further reading
● How HipChat Stores and Indexes Billions of Messages Using ElasticSearch
● @kakovskyi: Maintaining a high load Python project for newcomers
● HipChat: Important improvements to staging, presence & database storage
● HipChat and the little connection that could
● Elasticsearch at HipChat: 10x faster queries
● Atlassian: How IT and SRE use ChatOps to run incident management
● A Study of Internet Instant Messaging and Chat Protocols
● What Is Async, How Does It Work, And When Should I Use It?
● Leaky Bucket & Tocken Bucket - Traffic shaping
● A guide to analyzing Python performance
● Why Leading Companies Dark Launch - LaunchDarkly Blog
● WebCamp 2016: Asyncio-stack for web development (soon)
44
Questions?
45
Viacheslav Kakovskyi
viach.kakovskyi@gmail.com
@kakovskyi
Instant messenger with Python
Back-end development
Instant messenger with Python
Back-end development
Viacheslav Kakovskyi
WebCamp 2016

More Related Content

PDF
Last Month in PHP - November 2016
PDF
SSL certificate with mikrotik
PDF
Module: Welcome to Web 3.0
PDF
Encode club introduction_to_libp2p
PDF
WebCamp 2016: Python. Михаил Бегерский: Использование asyncio-стека для разра...
PDF
WebCamp 2016.PHP.Боднарчук Михаил.BDD на практике с Codeception
ODP
Kyua and Jenkins: Testing Framework for BSD
PDF
Python twisted
Last Month in PHP - November 2016
SSL certificate with mikrotik
Module: Welcome to Web 3.0
Encode club introduction_to_libp2p
WebCamp 2016: Python. Михаил Бегерский: Использование asyncio-стека для разра...
WebCamp 2016.PHP.Боднарчук Михаил.BDD на практике с Codeception
Kyua and Jenkins: Testing Framework for BSD
Python twisted

Viewers also liked (17)

PDF
Обзор фреймворка Twisted
PPTX
Asynchronous Python with Twisted
PDF
StrategicLeadership-EMBA%2520course-Bized
PPTX
presentatie Sentimo Engels
PPTX
PPT
Buy formal shirts for men
PPTX
PRESENTION
PDF
WebCamp 2016: DevOps. Николай Дойков: Опыт создания клауда для потокового вид...
PDF
WebCamp 2016: DevOps. Егор Лопатин и Юрий Железко: How we build and deploy ou...
PDF
WebCamp 2016: PM. Яна Пролис: "Вредные" советы по конфликтам в команде и комп...
PDF
WebCamp 2016: PHP.Денис Потапов.Рефакторим код не задумываясь
PDF
WebCamp2016:Front-End_Юрий Артюх_Современные подходы в верстке
PDF
WebCamp 2016: PM. Роман Кравченко: Разработка требований и управление рисками...
PDF
WebCamp 2016: DevOps. Ярослав Погребняк: Gobetween - новый лоад балансер для ...
PDF
WebCamp 2016: PM. Дмитрий Пискарев: Способы мотивации и продления LTV сотрудн...
PDF
WebCamp 2016: Front-end. Виталий Бобров: JavaScript для мобильной разработки.
PDF
WebCamp2016:Front-End_Роман Якобчук_Relay, GraphQL и остальные радости соврем...
Обзор фреймворка Twisted
Asynchronous Python with Twisted
StrategicLeadership-EMBA%2520course-Bized
presentatie Sentimo Engels
Buy formal shirts for men
PRESENTION
WebCamp 2016: DevOps. Николай Дойков: Опыт создания клауда для потокового вид...
WebCamp 2016: DevOps. Егор Лопатин и Юрий Железко: How we build and deploy ou...
WebCamp 2016: PM. Яна Пролис: "Вредные" советы по конфликтам в команде и комп...
WebCamp 2016: PHP.Денис Потапов.Рефакторим код не задумываясь
WebCamp2016:Front-End_Юрий Артюх_Современные подходы в верстке
WebCamp 2016: PM. Роман Кравченко: Разработка требований и управление рисками...
WebCamp 2016: DevOps. Ярослав Погребняк: Gobetween - новый лоад балансер для ...
WebCamp 2016: PM. Дмитрий Пискарев: Способы мотивации и продления LTV сотрудн...
WebCamp 2016: Front-end. Виталий Бобров: JavaScript для мобильной разработки.
WebCamp2016:Front-End_Роман Якобчук_Relay, GraphQL и остальные радости соврем...
Ad

Similar to WebCamp 2016: Python. Вячеслав Каковский: Real-time мессенджер на Python. Особенности разработки back-end. (20)

PDF
WebCamp Ukraine 2016: Instant messenger with Python. Back-end development
PDF
Storing your data in the cloud: doing right reversim 2018
PDF
MySQL X protocol - Talking to MySQL Directly over the Wire
PDF
Socket Programming with Python
PDF
AMQP with RabbitMQ
PDF
SPDY and What to Consider for HTTP/2.0
PDF
Sync IT Presentation 3.16
PDF
PHP at Density and Scale (Lone Star PHP 2014)
PDF
Using protocol analyzer on mikrotik
PDF
Open Chemistry, JupyterLab and data: Reproducible quantum chemistry
PDF
Big data @ Hootsuite analtyics
PDF
PyCon Poland 2016: Maintaining a high load Python project: typical mistakes
PDF
Applied Machine learning for business analytics
DOCX
CN 6131(15) Module IV.docx
PDF
CN 6131(15) Module IV.pdf
PDF
Log Management: AtlSecCon2015
PDF
Initial presentation of swift (for montreal user group)
PDF
Build real time stream processing applications using Apache Kafka
PDF
Cloud storage: the right way OSS EU 2018
PDF
Go at uber
WebCamp Ukraine 2016: Instant messenger with Python. Back-end development
Storing your data in the cloud: doing right reversim 2018
MySQL X protocol - Talking to MySQL Directly over the Wire
Socket Programming with Python
AMQP with RabbitMQ
SPDY and What to Consider for HTTP/2.0
Sync IT Presentation 3.16
PHP at Density and Scale (Lone Star PHP 2014)
Using protocol analyzer on mikrotik
Open Chemistry, JupyterLab and data: Reproducible quantum chemistry
Big data @ Hootsuite analtyics
PyCon Poland 2016: Maintaining a high load Python project: typical mistakes
Applied Machine learning for business analytics
CN 6131(15) Module IV.docx
CN 6131(15) Module IV.pdf
Log Management: AtlSecCon2015
Initial presentation of swift (for montreal user group)
Build real time stream processing applications using Apache Kafka
Cloud storage: the right way OSS EU 2018
Go at uber
Ad

More from WebCamp (19)

PDF
WebCamp 2016: Python.Максим Климишин.Типизированный Python
PDF
WebCamp2016:Front-End.Максим Климишин.Теоретические и практические концепции ...
PDF
WebCamp2016:Front-End.Катерина Поршнева.Эволюция CSS: от темных времен до CSS...
PDF
WebCamp 2016: Python.Павел Коломиец.Использование микросервисов при написании...
PDF
WebCamp 2016: Python_Кирилл Перевозчиков _Рецепты приготовления uWSGI
PDF
WebCamp 2016: PHP. Дмитрий Науменко: Рецепты для Yii2.
PDF
WebCamp 2016: PHP.Сергей Яковлев.Phalcon 3
PDF
WebCamp 2016: PHP.Алексей Петров.PHP at Scale: System Architect Toolbox
ODP
WebCamp2016:Front-End_Андрей Копёнкин_Оптимизируем мобильный веб полностью
PDF
WebCamp2016:BizDev_Алексей Иваница_Как построить и монетизировать мобильный п...
PDF
WebCamp 2016: BizDev. Андрей Моспан: Правда ли, что продукт стоит ровно столь...
PDF
WebCamp 2016: BizDev. Марина Никитчук : Искусство продажи мечты, а не сервиса.
PDF
WebCamp 2016: BizDev. Александр Борняков: Маркетинг в США: как привлечь к себ...
PDF
WebCamp 2016: BizDev. Кейт Голдберг: Agile business development: как применят...
PDF
WebCamp 2016: Front-end. Андрей Ситник: Stylelint — как и зачем линтить CSS
PDF
WebCamp 2016: PHP. Александр Шкарбалюк: SSO: Кому оно нужно и как его готовить?
PPTX
WebCamp 2016: PHP. Николай Паламарчук: PHP и микросервисы
PDF
WebCamp 2016: PM. Александр Демура: Откуда берутся тупые заказчики, и что с н...
PDF
WebCamp 2016: PM. Евгений Лабунский: Командная безответственность
WebCamp 2016: Python.Максим Климишин.Типизированный Python
WebCamp2016:Front-End.Максим Климишин.Теоретические и практические концепции ...
WebCamp2016:Front-End.Катерина Поршнева.Эволюция CSS: от темных времен до CSS...
WebCamp 2016: Python.Павел Коломиец.Использование микросервисов при написании...
WebCamp 2016: Python_Кирилл Перевозчиков _Рецепты приготовления uWSGI
WebCamp 2016: PHP. Дмитрий Науменко: Рецепты для Yii2.
WebCamp 2016: PHP.Сергей Яковлев.Phalcon 3
WebCamp 2016: PHP.Алексей Петров.PHP at Scale: System Architect Toolbox
WebCamp2016:Front-End_Андрей Копёнкин_Оптимизируем мобильный веб полностью
WebCamp2016:BizDev_Алексей Иваница_Как построить и монетизировать мобильный п...
WebCamp 2016: BizDev. Андрей Моспан: Правда ли, что продукт стоит ровно столь...
WebCamp 2016: BizDev. Марина Никитчук : Искусство продажи мечты, а не сервиса.
WebCamp 2016: BizDev. Александр Борняков: Маркетинг в США: как привлечь к себ...
WebCamp 2016: BizDev. Кейт Голдберг: Agile business development: как применят...
WebCamp 2016: Front-end. Андрей Ситник: Stylelint — как и зачем линтить CSS
WebCamp 2016: PHP. Александр Шкарбалюк: SSO: Кому оно нужно и как его готовить?
WebCamp 2016: PHP. Николай Паламарчук: PHP и микросервисы
WebCamp 2016: PM. Александр Демура: Откуда берутся тупые заказчики, и что с н...
WebCamp 2016: PM. Евгений Лабунский: Командная безответственность

Recently uploaded (20)

PDF
Peak of Data & AI Encore- AI for Metadata and Smarter Workflows
PDF
Review of recent advances in non-invasive hemoglobin estimation
PPTX
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
PDF
Unlocking AI with Model Context Protocol (MCP)
PDF
Network Security Unit 5.pdf for BCA BBA.
DOCX
The AUB Centre for AI in Media Proposal.docx
PPTX
20250228 LYD VKU AI Blended-Learning.pptx
PPT
Teaching material agriculture food technology
PPTX
Programs and apps: productivity, graphics, security and other tools
PDF
Reach Out and Touch Someone: Haptics and Empathic Computing
PPTX
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
PDF
Approach and Philosophy of On baking technology
PDF
A comparative analysis of optical character recognition models for extracting...
PPTX
MYSQL Presentation for SQL database connectivity
PDF
Per capita expenditure prediction using model stacking based on satellite ima...
PDF
Electronic commerce courselecture one. Pdf
PPTX
Digital-Transformation-Roadmap-for-Companies.pptx
PDF
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
PDF
Encapsulation theory and applications.pdf
PDF
Chapter 3 Spatial Domain Image Processing.pdf
Peak of Data & AI Encore- AI for Metadata and Smarter Workflows
Review of recent advances in non-invasive hemoglobin estimation
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
Unlocking AI with Model Context Protocol (MCP)
Network Security Unit 5.pdf for BCA BBA.
The AUB Centre for AI in Media Proposal.docx
20250228 LYD VKU AI Blended-Learning.pptx
Teaching material agriculture food technology
Programs and apps: productivity, graphics, security and other tools
Reach Out and Touch Someone: Haptics and Empathic Computing
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
Approach and Philosophy of On baking technology
A comparative analysis of optical character recognition models for extracting...
MYSQL Presentation for SQL database connectivity
Per capita expenditure prediction using model stacking based on satellite ima...
Electronic commerce courselecture one. Pdf
Digital-Transformation-Roadmap-for-Companies.pptx
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
Encapsulation theory and applications.pdf
Chapter 3 Spatial Domain Image Processing.pdf

WebCamp 2016: Python. Вячеслав Каковский: Real-time мессенджер на Python. Особенности разработки back-end.

  • 2. Me! @kakovskyi Python Developer at SoftServe Contributor of Atlassian HipChat — Python 2, Twisted Maintainer of KPIdata — Python 3, asyncio 2
  • 3. Agenda ● What is 'instant messenger'? ● Related projects from my experience ● Messaging protocols ● Life of messaging platform ● Lessons learned ● Summary ● Further reading 3
  • 4. What is 'instant messenger'? 4
  • 5. What is 'instant messenger'? ● online chat ● real-time delivery ● short messages 5
  • 6. What is 'instant messenger'? ● history search ● file sharing ● mobile push notifications ● video calling ● bots and integrations 6
  • 7. Related projects from my experience ● Hosted chat for teams and enterprises ● Founded in 2009 by 3 students ● 100 000+ connected users ● 100+ nodes ● REST API for integrations and bots ● Built with Python 2 and Twisted 7
  • 8. Messaging protocols Protocol is about: ● Message format ● Allowed types of messages ● Limitations ● Routine ○ How to encode data? ○ How to establish/close connection? ○ How to authenticate? ○ How to encrypt? 8
  • 9. Messaging protocols ● OSCAR (1997) ● XMPP (1999) ● Skype (2003) ● WebSocket-based (2011) ● MQTT, MTProto, DHT-based, etc. 9
  • 10. XMPP ● XMPP - signaling protocol ● BOSH - transport protocol ● Started from Jabber in 1999 ● XML as a message format ● Stanza - basic unit in XMPP ● Types of stanzas: ○ Message ○ Presence ○ Info/Query 10
  • 11. XMPP ● Extensions defined by XEPs (XMPP Extension Protocols): ○ Bidirectional-streams Over Synchronous HTTP (BOSH) ○ Serverless messaging ○ File transfer and etc. 11
  • 12. XMPP: Establishing a connection 12 Client: <?xml version='1.0'?> <stream:stream to='example.com' xmlns='jabber:client' xmlns:stream='http://etherx.jabber.org/streams' version='1.0'> Server: <?xml version='1.0'?> <stream:stream from='example.com' id='someid' xmlns='jabber:client' xmlns:stream='http://etherx.jabber. org/streams' version='1.0'>
  • 13. XMPP: Sending a message 13 Client: <message from='juliet@example.com' to='romeo@example.net' xml:lang='en'> <body>Art thou not Romeo, and a Montague?</body> </message> Server: <message from='romeo@example.net' to='juliet@example.com' xml: lang='en'> <body> Neither, fair saint, if either thee dislike. </body> </message>
  • 14. XMPP: Closing a connection 14 Client: </stream:stream> Server: </stream:stream>
  • 15. XMPP: Pros ● Robust and standardized ● Extendable via XEPs ● Secured ● Native support of multi-sessions ● A lot of clients implementations 15
  • 16. XMPP: Cons ● Overhead ○ Presence ○ Downloading the World on startup ● XML ○ Large documents ○ Expensive parsing 16
  • 17. XMPP and Python ● Servers: ○ TwistedWords - good place to start ○ Tornado-based example ○ aioxmpp ○ XMPPFlask ○ Punjab - BOSH-server on Twisted 17
  • 18. XMPP and Python ● Clients: ○ SleekXMPP - mature and solid ○ Slixmpp - asyncio-support ○ TwistedWords ○ Wokkel - Twisted-based ○ xmpp.py ● JS-client: Strophe.js 18
  • 19. WebSocket-based solutions ● WebSocket - transport protocol ● Standardized in 2011 by W3C ● Full-duplex communication channel ● JSON as a message format ● Custom message types 19
  • 20. WebSocket: Establishing a connection 20 Client: GET /chat HTTP/1.1 Host: server.example.com Upgrade: websocket Connection: Upgrade Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ== Origin: http://example.com Sec-WebSocket-Protocol: chat, superchat Sec-WebSocket-Version: 13 Server: HTTP/1.1 101 Switching Protocols Upgrade: websocket Connection: Upgrade Sec-WebSocket-Accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=
  • 21. WebSocket: Sending a message 21 Client: { "type": "message", "ts": 1469563519, "user": "kakovskyi", "text": "Hello, @WebCamp!" } Server: { "type": "notification", "ts": 1469563519, "user": "WebCamp Bot", "text": "Howdy @kakovskyi?" }
  • 22. WebSocket: Closing a connection 22 Client: 0x8 Server: 0x8
  • 23. WebSocket: Pros ● Supported by majority of browsers ● Low latency ● Small bandwidth ● Easy to start development 23
  • 24. WebSocket: Cons ● Needs development of signaling protocol ● Timeouts/reconnections should be additionally handled 24
  • 25. WebSocket and Python ● Servers: ○ Autobahn - Twisted and asyncio implementations ○ aiohttp ○ Tornado ○ Flask-SocketIO ○ Flask-Sockets 25
  • 26. WebSocket and Python ● Clients: ○ Autobahn ○ aiohttp ○ Tornado-based example ○ Vanilla websocket-client ● JS-client: SocketIO 26
  • 27. Life of messaging platform ● Authentication ● Access control checks ● Delivery ○ Messages ○ User's presence ○ Push notifications ● History retrieval ● History search 27
  • 28. Life of messaging platform ● Parsing ○ Protocol ○ Message content ● Dealing with file uploads ○ Security checks ○ Thumbnails distribution ● Multi-session support ● Reconnection handling ● Rate-limiting 28
  • 29. Life of messaging platform ● Server keeps connections open for every client ● High amount of long-lived concurrent connections ● Multithreaded approach isn't efficient due to overhead ● Requires usage of a select implementation on backend: ○ poll ○ epoll ○ kqueue ● Usage of asynchronous Python frameworks is preferred for high loaded solutions 29
  • 30. Life of messaging platform ● Authentication ○ OAuth2 ○ Run encryption operations in a separate Python thread ○ Cache users identities with Redis/Memcached ● Access-control checks ○ Make the checks lightweight and cheap ○ Raise an exception when operation isn't permitted 30 EAFP: Easier to ask for forgiveness than permission
  • 31. Delivery ● Make message delivery fault-tolerant ● Limit size of a message ● Filter content of messages: ○ Users like to send chars that break all the things ● Reduce presence traffic, it could be a bottleneck for large chats ● Use asynchronous broker for delivery when a user is offline (email or push) ○ Celery ○ RQ ○ Amazon Simple Queue Service ○ Huey 31
  • 32. Life of messaging platform ● Push notifications ■ Vendors ● Amazon SNS ● APNS ● Google Cloud Messaging ● Firebase Cloud Messaging ■ Python tools ● PyAPNs ● Python-GCM ● Pusher ● Be careful with device registration ● Make delivery of pushes fault-tolerant 32
  • 33. History retrieval ● Return last messages for every chat instantly ○ Use double writes ■ In-memory queue only for last messages ■ Persistent storage for all the things ● Majority of history retrievals is for the last days ○ Let's optimize the case ● Index messages by date 33
  • 34. History search ● ElasticSearch is the default solution for full- text search ● @a_soldatenko: What is the best full text search engine for Python? ● Add timing for search requests 34
  • 35. Parsing ● Protocol ○ Avoid to use Pure Python parsers ■ ujson ■ lxml ○ Run benchmarks against your typical cases ● Message content ○ Be careful with regular expressions ■ re2 ■ pyre2 ○ Alternative parsers in Python 35
  • 36. Dealing with file uploads ● Security checks ○ File upload vulnerabilities ○ Image upload ■ Decompression bomb ■ Other vulnerabilities with Pillow ○ Amazon S3 as file storage ■ boto ■ aiobotocore ■ botornado ● Thumbnails distribution ○ Delegate that to S3 ○ Requested by a client even if not needed 36
  • 37. Life of messaging platform ● Multi-session support ○ Set expiration time ○ Be ready to handle up to 4x sessions per user simultaneously ■ Desktop ■ Mobile ■ Tablet ■ Laptop ● Reconnection handling ○ Spin a proxy layer between messaging server and clients ● Rate-limiting ○ Limit amount of operations per user/group for heavy stuff ○ Leaky bucket ○ Throttling 37
  • 38. Lessons learned ● Bursty traffic ○ Load testing is a must, but not always enough ■ Locust ■ Yandex Tank ● Reconnect storm could be a big deal ○ We should handle that on platform and client-side ● AWS issues make bad customers experience ○ Put nodes in Multi-AZ 38
  • 39. Lessons learned ● Incidents prevention is cheaper than resolution ○ Grab stats and metrics about your services and storages ■ Redis for per-chat stats ■ StatsD ■ Grafana ○ Be notified when something starts going wrong ■ Elastalert ■ Monit ■ DataDog 39
  • 40. Lessons learned ● Don't stick with one language/stack ○ Python is great, but for some cases Go, Ruby or PHP are more suitable from product side ○ Avoid business logic duplication in several repos, spin a service and just call the endpoint ● Releasing new features only for certain groups makes product management easier ○ LaunchDarkly 40
  • 41. Lessons learned ● Don’t F**k the Customer ○ Provide unit/integration tests with every PR ○ Have development environment same as prod ○ Have staging environment same as prod ○ Make deployments fast ○ Rollback faster ○ Have a fallback plan 41
  • 43. Summary ● Select a messaging protocol which aligns with your needs ● WebSocket + JSON could be the thing for new projects ● Usage of asynchronous frameworks is preferred ● Execute blocking operations in a separate thread ● Collect metrics for common services operations ● Caching saves a lot of time ● Use C or Cython-based solutions for CPU-bound tasks ● Have fast release/deploy/rollback cycle ● Python is great, but don't hesitate to pick other tools 43
  • 44. Further reading ● How HipChat Stores and Indexes Billions of Messages Using ElasticSearch ● @kakovskyi: Maintaining a high load Python project for newcomers ● HipChat: Important improvements to staging, presence & database storage ● HipChat and the little connection that could ● Elasticsearch at HipChat: 10x faster queries ● Atlassian: How IT and SRE use ChatOps to run incident management ● A Study of Internet Instant Messaging and Chat Protocols ● What Is Async, How Does It Work, And When Should I Use It? ● Leaky Bucket & Tocken Bucket - Traffic shaping ● A guide to analyzing Python performance ● Why Leading Companies Dark Launch - LaunchDarkly Blog ● WebCamp 2016: Asyncio-stack for web development (soon) 44
  • 46. Instant messenger with Python Back-end development Viacheslav Kakovskyi WebCamp 2016