One year of measuring WebRTC service quality
2 likes2,092 views
The document discusses a year-long analysis of WebRTC service quality, highlighting key metrics such as browser usage, operational systems, participant behavior, and the challenges associated with connectivity disruptions. It notes that Chrome dominates WebRTC usage, with a majority of calls averaging around three participants and emphasizes the persistent issues caused by NAT/firewall failures. The document also introduces a simple API for setting up WebRTC communications and presents a vision for improving real-time communication systems.
One year of measuring WebRTC service quality
- 1. Performance in the Wild: Varun Singh, CEO 18.12.2015 Upperside Paris One year of measuring WebRTC service quality http://geek-and-poke.com/ 1
- 2. Analytics for WebRTC • the first cloud-based monitoring and management service for WebRTC (audio and video). • The team is contributing to the core standards that enable the WebRTC technology. 2
- 3. Monitoring What? • Annoyances • Transport quality • Per-stream media quality 3
- 6. Disruptions Disruption: loss of connectivity when network interfaces change, low available capacity,or high delay The light grey vertical lines show disruption, highlighted by the red bounding boxes. 6
- 7. Disruptions and user behaviour User Behaviour: The user tries to correct for the disruption by turning on and off video 7
- 8. Growth Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec purely web endpoints support: • TURN, • VideoBridge no mobile, yet 8
- 9. Browsers Chrome Firefox Opera Temasys 95% 5% 0.05% 9
- 10. Browsers Chrome Firefox Opera Temasys 95% 5% 0.05% 10
- 11. OSes 0 12,5 25 37,5 50 Android Windows Mac Linux 11 49% 34% 13% 4% using a browser
- 12. Number of Participants 0 10 20 30 40 50 60 70 80 90 100 2 3 4 5 6-8 >8 12 64% 30% 0,3%3%
- 13. Types of Relays 0 10 20 30 40 50 60 70 80 90 100 No Relay TURN/UDP TURN/TCP TURN/TLS 78% 13% 7% 2% 13
- 14. IPv6? 0 20 40 60 80 100 IPv4 IPv6 Not all is lost, for Europe alone 0 20 40 60 80 100 IPv4 IPv6 14 97% 83% 17%
- 15. how many ICE candidates? 15 0 20 40 60 80 100 0-4 5-8 8-20 >20 host stun turn v4 v6 multi homed 24 candidates: 8% 68% 8% 17% 7%
- 16. Setup times 0 10 20 30 40 50 <1s 1-2s 2-5s 5-10s >10s 16 47% 23% 14% 7% 9%
- 17. Failure Reasons~9% calls fail 5% 95%<1% ~20% of the setup calls have issues 17
- 18. Time to Failure 0 10 20 30 40 50 <10s 10-30s 30-60s >60s 18 People are very patient! perhaps because no mobile 50%33% 7% 10%
- 19. 0 20 40 60 80 100 no rejoins once twice thrice Churn* *participant rejoins the same call repeatedly Average joins per participant per conference 76% 19
- 21. Summary • Browsers: Chrome dominates WebRTC • OSes: Windows and Mac are pretty even • Participants: ~3 participants in a call on average • Relays: ~20% sessions need a TURN server • Failures: NAT/FW still causes most failures • Churn: 25% of sessions have a participant rejoining 21
- 22. A Very Simple API • 3 lines of code —> 5 minutes. 1. include <script> 2. initialize() —> needs registration keys 3. addNewFabric(pc, …) —> call started • optionally: send user events: audio muted/unmuted, video paused/resumed, call terminated, call held. • optionally: reportError() —> call failed to be setup due to SDP or Firewalls. • optionally: associateMstWithUserID() —> bridges using multiple media streams in a single peerconnection may want to correlate mediastream quality across endpoints. • optionally: collect and send user feedback http://www.callstats.io/api/22
- 23. Vision Gather Report Analyze FIX Deploy? webrtc-internal? via email? Hasthisbeen reportedbefore? Did it solve the issue? Until next report? Collect Diagnose Fix Deploy “ A w o r l d w h e r e r e a l - t i m e communication is both frictionless and effortless to set up, operate, and scale.” 23