Netflix Edge Engineering Open House Presentations - June 9, 2016
Download as PPTX, PDF7 likes4,810 views
The document discusses Netflix's edge engineering strategies, highlighting achievements such as over 42 billion hours streamed in 2015 and the challenges of scaling their various services. It outlines plans for system re-architecture, enhancements in developer experience, and improved playback performance to manage increasing user demand and maintain operational efficiency. Key emphasis is on leveraging caching strategies and optimizing service processing to enhance throughput amidst growing complexity and demand.
Netflix Edge Engineering Open House Presentations - June 9, 2016
- 1. Daniel Jacobson @daniel_jacobson Satish Gudiboina @sgudiboina Suudhan Rangarajan @suudhan Vasanth Asokan @vasanthasokan Edge Engineering Open House - June 9, 2016
- 2. 190 Countries (not China and a few others) 81+ Million Subscribers
- 3. 1000+ Different Device Types
- 4. Over 42 Billion Hours Streamed in 2015
- 5. Streaming Hours Per Year in Billions
- 6. Streaming Hours Per Year in Billions
- 7. Over 42 Billion Hours Streamed in 2015
- 9. Of Course, There Are Failures Too…
- 10. Two Primary Drivers Behind Our Successes
- 11. People Desire to Watch Netflix Two Primary Drivers Behind Our Successes
- 12. People Desire to Watch Netflix Systems Scale to Meet Desires Two Primary Drivers Behind Our Successes
- 14. Sign-Up
- 17. Edge Engineering provides data and functionality to support these three experiences
- 18. Designing APIs Enabling Playback Scaling Routing Insights DX Resiliency Tools Edge Engineering provides data and functionality to support these three experiences
- 20. D E V I C E S
- 23. D E V I C E S R O U T I N G A P I API API API API API API
- 24. D E V I C E S R O U T I N G A P I API API API API API API S E R V I C E S S2S2Recs S2S2Member S2S2Ratings S2S2Playback Lifecycle S2S2Authn/z S2S2A/B S2S2Search S2S2Identity S2S2 S2S2Playback Data S2S2DRMMetadata
- 25. D E V I C E S R O U T I N G A P I API API API API API API S E R V I C E S S2S2Recs S2S2Member S2S2Ratings S2S2S2S2Authn/z S2S2A/B S2S2Search S2S2Identity S2S2Metadata S2S2Playback Data S2S2DRM Owned by Edge Engineering Playback Lifecycle
- 26. D E V I C E S R O U T I N G A P I API API API API API API S E R V I C E S S2S2Recs S2S2Member S2S2Ratings S2S2S2S2Authn/z S2S2A/B S2S2Search S2S2Identity S2S2 S2S2Playback Data S2S2DRMMetadata Playback Lifecycle
- 27. D E V I C E S R O U T I N G A P I API API API API API API S E R V I C E S S2S2Recs S2S2Member S2S2Ratings S2S2S2S2Authn/z S2S2A/B S2S2Search S2S2Identity S2S2 S2S2Playback Data S2S2DRMMetadata Playback Lifecycle
- 28. D E V I C E S R O U T I N G A P I API API API API API API S E R V I C E S S2S2Recs S2S2Member S2S2Ratings S2S2S2S2Authn/z S2S2A/B S2S2Search S2S2Identity S2S2 S2S2Playback Data S2S2DRMMetadata Playback Lifecycle
- 29. D E V I C E S R O U T I N G A P I API API API API API API S E R V I C E S S2S2Recs S2S2Member S2S2Ratings S2S2S2S2Authn/z S2S2A/B S2S2Search S2S2Identity S2S2 S2S2Playback Data S2S2DRMMetadata Playback Lifecycle
- 30. API API API API API API S2S2S2S2Authn/z S2S2Playback Data S2S2DRM I N S I G H T S T O O L S D X Playback Lifecycle
- 33. The rest of Netflix’s AWS Cloud Footprint by %
- 34. Talking About the Future of Edge Engineering Satish Gudiboina API and Upcoming Re-Architecture Suudhan Rangarajan Playback Experience Vasanth Asokan Developer Tools, Velocity and Experience
- 35. The Netflix API Platform for Server- Side Scripting Current and The Future Satish Gudiboina
- 36. The Netflix API
- 37. Streaming Hours Per Year in Billions
- 38. Scale is multi-faceted Growing number of users ( → RPS) Growing number of device types Growing number of A/B tests Growing number of languages Growing number of countries
- 39. What we need to build for Velocity Resiliency Other requirements: Performance Great developer experience Operational insights Tooling
- 40. SERVICELAYER Js (mostly) java Client A Client B Client C Client A Client Y Client Z ... ... Netflix Microservices script script script script ... script script script script Network boundary API Server JVM Today’s architecture Resiliency with Hystrix
- 41. Developer Velocity: Decoupled deployments of versions n+3 Day 1 Day 2 Day 3 Day 4 Day 5 API device 1 device 2 device 3 device 4 i+4 i+1 i+2i+3 i n+2 n+1 n k+1 k j j+1 l
- 42. Changing risk profile Growing number of users ( → RPS) Growing number of devices Growing number of A/B tests Growing number of languages Growing number of countries Growing number and complexity of scripts (scripts → apps)
- 43. SERVICELAYER Js (mostly) java Client A Client B Client C Client A Client Y Client Z ... ... Netflix Microservices script script ... script script Network boundary API Server JVM Today’s system (T-3yrs) few, small scripts fewer uploads
- 44. SERVICELAYERJs (mostly) java Client A Client B Client C Client A Client Y Client Z ... ... Netflix Microservices script script script script ... script script script script Network boundary API Server JVM Today’s system (T) scripts scripts hundreds of more complex scripts, 10-50 uploads per day
- 46. Lack of process isolation is a growing risk.
- 47. Moving toward our ideal API: What will change Scripts will run in containers Scripts will call API remotely
- 48. SERVICELAYERJs (mostly) java Client A Client B Client C Client A Client Y Client Z ... ... Netflix Microservices node script node script ... node script node script Network boundary API Server JVM The (near) future node.js process isolation node for device teams
- 49. Why containers? Process isolation Fast startup Consistent developer experience across environments
- 50. Isolated failures: scripts don’t affect each other API device 1 device 2 device 3 device 4 Temporaril y unavailabl e!
- 51. Independent autoscaling API device 1 device 2 device 3 device 4
- 52. Fast startup New API server: minutes New container: seconds Fast rollout, fast rollback, fast MTTR
- 53. The Netflix API
- 54. Edge Developer Experience Translating developer productivity to Netflix customer delight
- 56. DEVELOP (rapidly) DEPLOY (reliably) OPERATE (effectively) Experimentation driven innovation ~700 apps, dozens of pushes a day 15+ client teams, ~200 developers ~50 direct services, 100s of AB tests, dozens of new features The Innovation Funnel API Devices Netflix Services Client Adaptor Applications
- 57. Why care about DevEx? Developer Productivity Product Innovation Tools Automation Insights Customer Satisfaction
- 58. App Development and Management DEVELOP (rapidly) DEPLOY (reliably) OPERATE (effectively)
- 59. SERVICELAYER Netflix Microservices app WAN Boundary API SERVER JVM js java Developer Ergonomics app ... app app CLIENTLIBRARIES Large / Complex SERVICELAYER
- 60. REMOTESERVICELAYER app API SERVER JVM Developer Ergonomics ... app ... app app CLIENTLIBRARIES js javajs DOCKER CONTAINERS WAN Boundary Netflix Microservices
- 61. Setup Canary SupportProd Push Pre-Prod Metrics Tracing Lifecycle Alerts Build Bootstrap API Discovery REPL Unit Test SDK Debug Logging Profiling Audits Security Custom Routing Dependency Management Client Application Development Critical Component! Dx Developer Experience
- 62. $ newt init Just bring your Javascript business logic NeWT: Netflix Workflow Toolkit Continuous Integration Deployment Pipelines Autoscaling Dashboards Alerting Logging Lifecycle Management Audits and Analytics Container tooling Canaries Dependency Management
- 63. Titus ATLAS NeWT: Netflix Workflow Toolkit
- 64. Edge PaaS UI
- 65. $ newt auto-deploy -d nodeJS project Docker Machine node-inspector Debugger File watcher / live reload trigger File watcher agent NeWT: Local Container Development Local Container docker build / run
- 66. $ newt auto-deploy -d Docker Machine NeWT: Local Container Development Local Container Cloud Microservices Cloud Proxy Terminate security DiscoveryAgent Service Discovery Local System Cloud
- 67. App Operations and Insights DEVELOP (rapidly) DEPLOY (reliably) OPERATE (effectively)
- 68. • Low Latency, High throughput, Highly Efficient • Handle bursty or large scale loads • Extensible programming model 600 jobs in production, 8M messages/sec at peak, 100Gbps network throughput Mantis - Stream Processing Platform
- 69. Monitoring facets of aggregate application health, globally Aggregate Insights
- 71. Analyze in real-time, requests matching a precise set of conditions Surgical Insights
- 72. Surgical Insights - Real-time Stream Queries
- 73. Surgical Insights - Real-time Stream Queries
- 74. Surgical Insights - Real-time Stream Queries
- 75. Monitoring server side calling pattern and internal application profile Session Tracing
- 76. Session Tracing
- 77. Session Tracing - Request Profile
- 78. Session Tracing - Per Node Profile
- 79. Automatic monitoring of high cardinality data across multiple dimensions Real-time Anomaly Detection
- 81. • Scaling developer productivity with business growth •Provide fully managed PaaS experience to client developers • Shift Left Insights to power smart development • Curated, blended visualizations that simplify devops In conclusion...
- 82. Tech Soup
- 83. Scaling Playback Services Suudhan Rangarajan Senior Software Engineer, Playback Features @suudhan
- 87. Authorize LICENS E ❏ Content usage / resolution policies ❏ Plan / device limits enforcement ❏ DRM / License generation
- 88. Collect & Learn Bookmarks & Hours Watched Streaming Errors and Metrics Quality Of Experience metrics 4
- 89. Lets look at Play Decisions DECIDE MANIFEST AUTHORIZE COLLECT & LEARN LICENS E SESSIO N
- 90. Huge number of Streams Resolutions - 720p, 1080p, 4K etc Codecs - H.264,HEVC etc Bitrates - 230, 780, 3000 etc Channels - Stereo, Surround Sound Languages - English, French etc Types - Subtitles, Closed Captions, Forced Narratives Languages - English, French etc
- 91. Streams to Tracks - H.264 Main Profile - English 5.1 Audio - No Subtitle - HEVC Dash Profile - French 2.0 Audio - English CC - HDR Dash Profile - Spanish AAC Audio - English Forced Narrative
- 94. Big Opportunity Rich playback experiences Tremendous increase in scale Customer growth
- 95. Challenge: Efficient Scaling Targeting sub-linear growth # of Requests Cloud Costs
- 96. Predictable Viewing Patterns Key Insight
- 98. Also.. Manifest Request for one title
- 99. Current: Completely Real-time Real-time manifest generation
- 100. With Caching Real-time manifest generation 80% Cached 20% Real-time
- 101. Challenges How do we determine the optimal combination of attributes to cache on?
- 102. Challenges Cache Considerations: ● When to populate? ● When to bust? ● How to scale for cache- miss or failures?
- 103. Potential Win 10x increase in requests with only 4x increase in costs
- 104. Optimize computation Can we re-imagine our service processing to dramatically increase throughput?
- 105. Anatomy of a Playback Manifest Request Metadata Access 27 % 36% Tracks Generation 16% Streams Filtering 21% Serialization
- 106. Potential Win 10x increase in requests with just 2x increase in service costs
- 107. Two-pronged Strategy to Scaling Cache Manifests Re-architect code to reduce processing time
- 108. Scaling Problems Across Services Decide Authorize Collect & Learn Playback Features Playback Access Playback Data Systems
- 109. Thanks! @suudhan Come Talk to Us!
- 110. Image Attribution All Images used are under creative commons or public domain license: ● Video icon - http://simpleicon.com/wp-content/uploads/video-camera-1.png ● Speaker icon - https://upload.wikimedia.org/wikipedia/commons/thumb/2/21/Speaker_Icon.svg/1024px- Speaker_Icon.svg.png ● Subtitle icon - https://thenounproject.com/term/subtitles/78795/ ● Uptrend image - https://pixabay.com/en/chart-line-line-chart-diagram-trend-148256/ ● Funnel image - https://commons.wikimedia.org/wiki/File:Funnel_Mech.svg ● Business Intelligence image - https://pixabay.com/static/uploads/photo/2015/04/14/23/17/it-business- 722950_960_720.png ● Key icon - https://pixabay.com/static/uploads/photo/2014/04/03/10/55/key-311738_960_720.png ● Person icon- https://pixabay.com/static/uploads/photo/2015/12/22/04/00/photo-1103596_960_720.png ● Mobile icon- https://upload.wikimedia.org/wikipedia/commons/thumb/1/14/Mobile_phone_font_awesome.svg/1024px- Mobile_phone_font_awesome.svg.png ● Globe image - https://upload.wikimedia.org/wikipedia/commons/thumb/6/60/Simple_Globe.svg/1024px- Simple_Globe.svg.png ● Devices icon- https://upload.wikimedia.org/wikipedia/commons/thumb/6/60/Simple_Globe.svg/1024px- Simple_Globe.svg.png ● wifi icon - https://pixabay.com/static/uploads/photo/2016/01/03/11/32/wireless-signal-1119306_960_720.png ● cell tower - https://pixabay.com/static/uploads/photo/2012/04/13/00/23/tower-31235_960_720.png