RedisConf18 - Video Experience Operational Insights in Real Time.
Download as PPTX, PDF1 like198 views
This document summarizes a video operational insights platform that provides real-time analytics on key metrics like quality of experience (QoE) and audience engagement for live video streams. The platform uses Redis to store streaming event data with HyperLogLog for unique counting. It partitions data across local and global Redis clusters and aggregates metrics near real-time to provide insights on millions of events with latency under 20 seconds.
RedisConf18 - Video Experience Operational Insights in Real Time.
- 1. Video Experience Operational Insights in Real Time. Aditya Vaidya - Sr Software Engineer
- 2. 2
- 3. 3 VDMS powers Oath Brands
- 4. 4 VDMS enables partner experiences Uplynk : 400 channels 35,000 live events | Edgecast: 45 Tbps total capacity
- 5. 5 Devices
- 6. Global Scale Live Event 6
- 7. 7 Video Player Event Lifecycle
- 8. 8 Video Operational Insights Platform - 2015 v/s 2017
- 9. 9 Redis ● In-memory db - High throughput & Low latencies ● Advanced data structures - SortedSet, HyperLogLog (probabilistic)
- 10. Operational Insights - Quality of Experience (QoE) 10 ● Rebuffer Ratio - % of viewers time spent watching stalled playback ○ Metrics ● Rebuffering duration ● Playback duration ○ Dimensions ● Content Delivery Networks (Multi CDN environment)
- 11. Operational Insights - Audience Engagement 11 ● Concurrent Views - Number of unique Concurrent Video Sessions ○ Metrics ● Unique Video Sessions ○ Dimensions ● Brands (Oath or Partner) e.g. yahoo sports, aol etc ● Devices e.g. Mobile, Desktop etc
- 12. 12 ● As close as possible to Real Time ● Near Real Time aggregations over millions of events Near Real Time Challenges
- 15. 15 Architecture
- 16. 1616 Partitioning ● Local Redis Cluster ○ video_session_id ● Global Redis Cluster ○ key -> DIMS:MEASURE
- 17. 17 HyperLogLog - Unique Counting ● Approximate data sketch ● Count upto 2^64 with std. error of .81% in 12KB
- 18. 18 Metrics - Concurrent Views ● Local Redis ○ PFADD key video_session_id (streaming) ○ DUMP key (once per interval) ● Remote Redis Cluster ○ RESTORE (once per interval) ○ PFMERGE keys_from_all_servers (once per interval)
- 19. 19 Metrics - Concurrent Views
- 20. 20 Metrics - Rebuff, Starts, Bitrate etc ● Local Redis ○ INCRBY key (streaming) ○ GET key (once per key per interval) ● Remote Redis Cluster ○ INCRBY key (once per interval) ○ GET key (once per interval)
- 25. 25 Results ● End-to-end latency < 20 secs ● Multi CDN failure detection during a global scale live event
- 26. Thank You