Scaling Push Messaging for Millions of Devices @Netflix
4 likes1,544 views
The document discusses the scaling of push messaging for millions of Netflix devices using a sophisticated architecture that includes Zuul servers, websockets, and a push message processing queue with Kafka. It outlines strategies for managing persistent connections, optimizing server performance, and auto-scaling based on user connections. The content emphasizes the advantages of push messaging in enhancing functionality while reducing polling overhead.
Scaling Push Messaging for Millions of Devices @Netflix
- 1. How we scaled push messaging for millions of Netflix devices Susheel Aroskar Cloud Gateway
- 2. InfoQ.com: News & Community Site • 750,000 unique visitors/month • Published in 4 languages (English, Chinese, Japanese and Brazilian Portuguese) • Post content from our QCon conferences • News 15-20 / week • Articles 3-4 / week • Presentations (videos) 12-15 / week • Interviews 2-3 / week • Books 1 / month Watch the video with slide synchronization on InfoQ.com! https://www.infoq.com/presentations/ neflix-push-messaging-scale
- 3. Presented at QCon New York www.qconnewyork.com Purpose of QCon - to empower software development by facilitating the spread of knowledge and innovation Strategy - practitioner-driven conference designed for YOU: influencers of change and innovation in your teams - speakers and topics driving the evolution and innovation - connecting and catalyzing the influencers and innovators Highlights - attended by more than 12,000 delegates since 2007 - held in 9 cities worldwide
- 4. Why do we need push?
- 6. How I spend my time in Netflix application...
- 7. ● What is push?
- 8. ● What is push? ● How you can build it
- 9. ● What is push? ● How you can build it ● How you can operate it
- 10. ● What is push? ● How you can build it ● How you can operate it ● What can you do with it
- 11. Susheel Aroskar Senior Software Engineer Cloud Gateway saroskar@netflix.com github.com/raksoras @susheelaroskar
- 25. Handling millions of persistent connections Zuul Push server
- 26. C10K challenge
- 27. Socket Socket Thread per Connection Thread-1 Thread-2 Read Write Write Read
- 28. Socket Socket Thread per Connection Thread-1 Thread-2 Read Write Write Read Async I/O Socket read callback write callback Socket Single Threadread callback write callback
- 30. protected void addPushHandlers(ChannelPipeline pl) { pl.addLast(new HttpServerCodec()); pl.addLast(new HttpObjectAggregator()); pl.addLast(getPushAuthHandler()); pl.addLast(new WebSocketServerCompressionHandler()); pl.addLast(new WebSocketServerProtocolHandler()); pl.addLast(getPushRegistrationHandler()); }
- 31. Authenticate by Cookies, JWT or any other custom scheme Plug in your custom authentication policy
- 32. Tracking clients’ connection Metadata in real-time Push Registry
- 33. public class MyRegistration extends PushRegistrationHandler { @Override protected void registerClient( ChannelHandlerContext ctx, PushUserAuth auth, PushConnection conn, PushConnectionRegistry registry) { super.registerClient(ctx, authEvent, conn, registry); ctx.executor().submit(() -> storeInRedis(auth)); } }
- 34. Push registry features checklist
- 35. ● Low read latency Push registry features checklist
- 36. ● Low read latency ● Record expiry Push registry features checklist
- 37. ● Low read latency ● Record expiry ● Sharding Push registry features checklist
- 38. ● Low read latency ● Record expiry ● Sharding ● Replication Push registry features checklist
- 40. What we use https://github.com/Netflix/dynomite Redis + Auto-sharding + Read/Write quorum + Cross-region replication Dynomite
- 42. We use Kafka message queues to decouple message senders from receivers
- 43. Fire and Forget
- 45. Different queues for different priorities
- 46. We run multiple message processor instances in parallel to scale our message processing throughput.
- 47. Operating Zuul Push Different than REST of them
- 48. Persistent connections make Zuul Push server stateful Long lived stable connections
- 49. Persistent connections make Zuul Push server stateful Long lived stable connections ○ Great for client efficiency
- 50. Persistent connections make Zuul Push server stateful Long lived stable connections ○ Great for client efficiency ○ Terrible for quick deploy/rollback
- 51. If you love your clients set them free... Tear down connections periodically
- 54. Ask client to close its connection.
- 55. Most connections are idle! How to optimize push server
- 56. BIG Server, tons of connections ulimit -n 262144 net.ipv4.tcp_rmem="4096 87380 16777216" net.ipv4.tcp_wmem="4096 87380 16777216"
- 60. Optimize for cost, NOT instance count ✓ $$ $$ ❌
- 62. How to auto-scale? RPS? CPU??
- 63. How to auto-scale? RPS? CPU?? Open Connections
- 64. Amazon Elastic Load Balancers cannot proxy WebSockets.
- 65. Solution - Run ELB as a TCP load balancer 7 Application 6 Presentation 5 Session 4 Transport 3 Network 2 Data link 1 Physical HTTP TCP IP Ethernet OSI 7 network layers (conceptual) HTTP over TCP/IP Layer 7 HTTP (WebSocket Upgrade Request) Layer 4 TCP
- 66. Managing push cluster - a quick recap ● Recycle connections after tens of minutes
- 67. Managing push cluster - a quick recap ● Recycle connections after tens of minutes ● Randomize each connection’s lifetime
- 68. Managing push cluster - a quick recap ● Recycle connections after tens of minutes ● Randomize connection’s lifetime ● More number of smaller servers >> few BIG servers
- 69. Managing push cluster - a quick recap ● Recycle connections after tens of minutes ● Randomize connection’s lifetime ● More number of smaller servers >> few BIG servers ● Auto-scale on number of open connections per box
- 70. Managing push cluster - a quick recap ● Recycle connections after tens of minutes ● Randomize connection’s lifetime ● More number of smaller servers >> few BIG servers ● Auto-scale on number of open connections per box ● WebSocket aware vs TCP load balancer
- 71. If you build it, They will push
- 73. Remote recovery
- 74. User messaging
- 75. WHAT WILL YOU USE IT FOR?
- 76. Call to action
- 77. PULL!
- 79. In conclusion, push can make you
- 80. In conclusion, push can make you rich (in functionality),
- 81. In conclusion, push can make you rich (in functionality), thin (by getting rid of polling)
- 82. In conclusion, push can make you rich (in functionality), thin (by getting rid of polling) and happy!
- 83. Thank you.
- 84. Questions? Susheel Aroskar Senior Software Engineer Cloud Gateway saroskar@netflix.com github.com/raksoras @susheelaroskar
- 85. Rich, exciting Apps More efficient systems Easy to customize Easy to operate Zuul Push Battle tested
- 86. Watch the video with slide synchronization on InfoQ.com! https://www.infoq.com/presentations/neflix- push-messaging-scale