Realtime Content Delivery: Powering dynamic instant experiences
2 likes875 views
The document discusses real-time content delivery, emphasizing the need for efficient data dissemination methods such as event sourcing and publish/subscribe systems. It outlines the key components required for building a real-time platform: connection, subscription, and publishing mechanisms, highlighting different implementation technologies like WebSockets and EventSource. The performance metrics justify the scalability of the system, allowing for over 100,000 persistent connections per machine, making it suitable for dynamic and interactive applications.
Realtime Content Delivery: Powering dynamic instant experiences
- 1. Realtime Content Delivery: Powering dynamic instant experiences Akhilesh Gupta Realtime Engineer, LinkedIn
- 2. What is Realtime Content Delivery?
- 4. Live Hearts & Comments PERISCOPE
- 6. Realtime Driver Location How does Uber update the location of nearby cars? UBER
- 8. Polling • Empty responses are the majority • Frequency f of polling • High — Unnecessary traffic • Low — Delay in information dissemination Frontend Server t0 t1 t2 t3 f
- 10. Why did I end up in Auditorium 7? OLD LINKEDIN MESSAGING 3:19 AM Push notification based
- 11. Push Notifications • Slow • Unreliable • Throttled by APNS/GCM • Size constraints APNS/GCM Backend Processing AB
- 12. Realtime Driver Location Will this even work with push notifications? UBER
- 14. What do we need? Connection Topic Subscription to Topics Publish • A pipe between the client and the server • Identification of available data • Client informs the server what it is interested in • Deliver relevant data to the client in realtime
- 15. What do we need? Frontend Server Connection Subscriber
- 16. What do we need? Frontend Server Pub/Sub Server Connection Subscriber
- 17. What do we need? Frontend Server Pub/Sub Server Connection SubscriberSubscription
- 18. What do we need? Frontend Server Pub/Sub Server Server rack Connection SubscriberSubscription
- 19. What do we need? Frontend Server Pub/Sub Server Server rack Connection Subscriber Subscription
- 20. What do we need? Frontend Server Pub/Sub Server Connection Subscriber SubscriptionPublish
- 21. What do we need? Frontend Server Pub/Sub Server Connection Subscriber SubscriptionPublish
- 22. 4 Key Building Blocks Connection A persistent pipe between the client and the backend Subscriber Manage and store client information Subscription Allow clients to subscribe to topics. Manage & store subscriptions Publish Publish events on topics. Deliver events to subscribed clients.
- 24. EVENTSOURCE INTERFACE HTTP Long Poll with Server Sent Events GET /connect HTTP/1.1 Accept: text/event-stream
- 25. EVENTSOURCE INTERFACE HTTP Long Poll with Server Sent Events HTTP/1.1 200 OK - RESPONSE HEADERS - Content-Type: text/event-stream
- 26. EVENTSOURCE INTERFACE HTTP Long Poll with Server Sent Events HTTP/1.1 200 OK - RESPONSE HEADERS - Content-Type: text/event-stream - RESPONSE BODY - data: {“typingIndicator” object}
- 27. EVENTSOURCE INTERFACE HTTP Long Poll with Server Sent Events HTTP/1.1 200 OK - RESPONSE HEADERS - Content-Type: text/event-stream - RESPONSE BODY - data: {“typingIndicator” object} .. data: {“message” object}
- 28. EVENTSOURCE INTERFACE HTTP Long Poll with Server Sent Events HTTP/1.1 200 OK - RESPONSE HEADERS - Content-Type: text/event-stream - RESPONSE BODY - data: {“typingIndicator” object} .. data: {“message” object} .. data: {“location” object}
- 29. WEB EventSource Client Libraries var evtSource = new EventSource("https://www.linkedin.com/realtime/connect"); evtSource.onmessage = function(e) { var typingIndicator = JSON.parse(e.data); }
- 30. IE SUPPORT EventSource Client Libraries https://github.com/remy/polyfills/blob/master/EventSource.js
- 31. ANDROID & IOS SUPPORT EventSource Client Libraries Android: https://github.com/tylerjroach/eventsource-android Swift: https://github.com/inaka/EventSource Obj-C: https://github.com/neilco/EventSource
- 32. PLAY & AKKA Server: Accepting a EventSource connection // Client A connects to the server and is assigned connectionIdA public Result listen() { }
- 33. PLAY & AKKA Server: Accepting a EventSource connection // Client A connects to the server and is assigned connectionIdA public Result listen() { return ok(EventSource.whenConnected(eventSource -> { String connectionId = UUID.randomUUID().toString(); })); }
- 34. PLAY & AKKA Server: Accepting a EventSource connection // Client A connects to the server and is assigned connectionIdA public Result listen() { return ok(EventSource.whenConnected(eventSource -> { String connectionId = UUID.randomUUID().toString(); // construct an Akka Actor with the new EventSource connection identified by a random connection identifier Akka.system().actorOf( ClientConnectionActor.props(connectionId, eventSource), connectionId); })); }
- 35. Wait, what about Websockets?
- 36. WebSockets • Much better to establish an EventSource connection first. • Upgrade to WebSockets when possible. • Frameworks like socket.io can be used. • Generally not a good idea to rely on WebSockets as the first and only connection mechanism. • Proxies/Firewalls/Antivirus systems can block it. • Not available as an upgrade over HTTP/2.
- 37. Vanilla EventSource vs Socket.io • Server to client push only • Works over plain HTTP • Built-in support in most client, server frameworks • Clean, Fast & Easy for most use cases Vanilla EventSource • Bi-directional communication • Long Polling + WebSockets • Built-in delivery acks • Special socket.io libraries for most client/server frameworks • Advanced use cases Socket.io
- 38. 4 Key Building Blocks Connection A persistent pipe between the client and the backend Subscriber Manage and store client information Subscription Allow clients to subscribe to topics. Manage & store subscriptions Publish Publish events on topics. Deliver events to subscribed clients.
- 39. 4 Key Building Blocks Connection A persistent pipe between the client and the backend Subscriber Manage and store client information Subscription Allow clients to subscribe to topics. Manage & store subscriptions Publish Publish events on topics. Deliver events to subscribed clients.
- 42. Subscriber Frontend Server Pub/Sub Server Connection Id Frontend Server Address Play Java Frontend Java Backend Distributed K/V store B
- 43. 4 Key Building Blocks Connection A persistent pipe between the client and the backend Subscriber Manage and store client information Subscription Allow clients to subscribe to topics. Manage & store subscriptions Publish Publish events on topics. Deliver events to subscribed clients.
- 44. 4 Key Building Blocks Connection A persistent pipe between the client and the backend Subscriber Manage and store client information Subscription Allow clients to subscribe to topics. Manage & store subscriptions Publish Publish events on topics. Deliver events to subscribed clients.
- 45. Subscription Server rack User B Typing Indicators User B Topic Connection Id B
- 46. Subscription Frontend Server Pub/Sub Server Server rack Connection Id Frontend Server Address Connection IdTopic B
- 47. Subscription Frontend Server Pub/Sub Server Server rack Connection Id Frontend Server Address Topic Connection Id B
- 48. Subscription Frontend Server Pub/Sub Server Server rack Connection Id Frontend Server Address Topic Connection Id Distributed K/V store B
- 49. 4 Key Building Blocks Connection A persistent pipe between the client and the backend Subscriber Manage and store client information Subscription Allow clients to subscribe to topics. Manage & store subscriptions Publish Publish events on topics. Deliver events to subscribed clients.
- 50. 4 Key Building Blocks Connection A persistent pipe between the client and the backend Subscriber Manage and store client information Subscription Allow clients to subscribe to topics. Manage & store subscriptions Publish Publish events on topics. Deliver events to subscribed clients.
- 51. Publish Frontend Server Pub/Sub Server Connection Id Frontend Server Address Topic Connection Id Frontend Server B A C Messaging Backend (Publisher)
- 52. Publish Frontend Server Pub/Sub Server Connection Id Frontend Server Address Topic Connection Id Typing Indicators User B Topic Frontend Server C B A
- 53. Publish Frontend Server Pub/Sub Server Connection Id Frontend Server Address Topic Connection Id Topic A B Frontend Server C
- 54. Publish Frontend Server Pub/Sub Server Connection Id Frontend Server Address Topic Connection Id Topic A B Frontend Server C
- 55. Publish Frontend Server Pub/Sub Server Connection Id Frontend Server Address Topic Connection Id Topic A B Frontend Server C
- 56. Publish Frontend Server Pub/Sub Server Frontend Server Address Topic Connection Id Topic Connection Id A B Frontend Server C
- 57. Publish Frontend Server Pub/Sub Server Connection Id Frontend Server Address Topic Connection Id Topic Connection Id A B Frontend Server C
- 58. Publish Frontend Server Pub/Sub Server Connection Id Frontend Server Address Topic Connection Id Connection Id A B Frontend Server C Topic
- 59. Publish Frontend Server Pub/Sub Server Connection Id Frontend Server Address Topic Connection Id A B Frontend Server C
- 60. PLAY & AKKA Server: Publishing on EventSource eventSource.send(<typing indicator object>);
- 61. EVENTSOURCE INTERFACE FOR USER B Publish HTTP/1.1 200 OK - RESPONSE HEADERS - Content-Type: text/event-stream - RESPONSE BODY - data: {“User A is typing…” object}
- 62. 4 Key Building Blocks Connection A persistent pipe between the client and the backend Subscriber Manage and store client information Subscription Allow clients to subscribe to topics. Manage & store subscriptions Publish Publish events on topics. Deliver events to subscribed clients.
- 68. How many persistent connections can a single machine hold? Frontend Server
- 69. > 100,000 We can hold a million connections with just 10 machines
- 70. LINKEDIN ENGINEERING BLOG tinyurl.com/ linkedinscaling Akhilesh Gupta & Cliff Snyder >100K
- 71. LinkedIn Realtime Performance Persistent Connections/machine >100K E2E Publish Latency 75ms
- 74. LinkedIn Realtime Performance Persistent Connections/machine >100K E2E Publish Latency 75ms
- 75. Why does this system scale? Stateless, Horizontally Scalable System Frontend Server Pub/Sub Server Frontend Server Frontend Server Frontend Server Pub/Sub Server Pub/Sub Server Pub/Sub Server Subscriber Subscription Distributed K/V Storage System
- 76. Key takeaways
- 77. R E A LT I M E C O N T E N T D E L I V E R Y Enable dynamic instant experiences on your apps
- 78. S E M I R E A LT I M E T E C H N O L O G Y Polling & Push notifications are good for simple use cases Frontend t0 t1 t2 t3 f
- 79. 4 B U I L D I N G B L O C K S O F A R E A LT I M E P L A T F O R M Connection, Subscriber, Subscription, Publish
- 80. C O N N E C T I O N Eventsource/SSE great for most use cases
- 81. E V E N T S O U R C E / S S E Built-in support in most server frameworks Readily available client libraries
- 82. P U B L I S H / S U B S C R I B E R E A LT I M E P L A T F O R M Can be built on any server/storage technology Stateless, Horizontally scalable >100K
- 83. Active Status Online/Offline indicators LINKEDIN REALTIME TECHNOLOGY
- 84. P U B L I S H / S U B S C R I B E R E A LT I M E P L A T F O R M You can do it for your app!
- 85. R E A LT I M E E N G I N E E R , L I N K E D I N @agupta03 t i n y . c c / m o b i l e e r a - p p t