Realtime video streaming the open source way
- 1. MADRID · NOV 27-28 · 2015 Realtime video streaming MADRID · NOV 27-28 · 2015 the opensource way Iván Belmonte
- 2. MADRID · NOV 27-28 · 2015 Github: ibelmonte Sysdivision Twitter: @sysdivision Website: http://sysdivision.com Twitter: @ibelmonte Iván Belmonte
- 3. MADRID · NOV 27-28 · 2015 Real time weather reporting system for outdoor sports. 2 main elements: - HD cameras in surfing spots (beaches) - Weather stations in surfing spots GOAL: Weather forecast typically fails. Spotfav gives you a window to your spot, to check the weather conditions before moving.
- 4. MADRID · NOV 27-28 · 2015 What forecast says the day before
- 5. MADRID · NOV 27-28 · 2015 What you expect when you take your car to go to the spot
- 6. MADRID · NOV 27-28 · 2015 What’s actually going on
- 7. MADRID · NOV 27-28 · 2015 IMPORTANT NOTE: I’m a backend developer/sysadmin I know nothing about video and audio processing In this project I am A VICTIM
- 8. MADRID · NOV 27-28 · 2015 Video manipulation and transmission is complex - Plenty of difficult concepts: encoding, muxing, transcoding, encapsulation, bitrates, fps… - Different media servers, different players, some codecs may not be supported - Several transmission protocols depending on your needs: RTP, RTSP, RTMP, HLS, HDS, Shoutcast, HTML5… Added complexity: LIVE streaming
- 9. MADRID · NOV 27-28 · 2015 Looking for a standard setup Most common case: H264 over RTSP Affiliates: get this if you want to get connected to our network
- 10. MADRID · NOV 27-28 · 2015 There are several options Commercial: Adobe media server, Wowza, Flussonic, Flumotion… SaaS: Ustream, Justin.tv (defunct)… Open source: ffmpeg, ffserver, libav, Live555, Red5, crtmpd, vlc, Erlyvideo…
- 11. MADRID · NOV 27-28 · 2015 OUR SCENARIO 1 Camera, H264 over RTSP 1. The goal Embed the camera in a website and also in a smartphone app 2. Nice to have Not permanently connected (connect on demand) Tolerant to network cuts 3. Science fiction Be able to filter by referral (avoid other websites to embed our cameras)
- 12. MADRID · NOV 27-28 · 2015 STEP 1: WEB player (RTMP)
- 13. MADRID · NOV 27-28 · 2015 ROUND 1 A javascript serving a Directx (wtf?) script.
- 14. MADRID · NOV 27-28 · 2015 ROUND 1 Cons: - The camera has a limited pool of connections available. Visitors exceeding this poll get rejected. Works? - In some browsers
- 15. MADRID · NOV 27-28 · 2015 ROUND 2 A VLC cli process reading the RTSP, transcoding and publishing an RTMP stream in a socket
- 16. MADRID · NOV 27-28 · 2015 ROUND 2 Works? - YES…
- 17. MADRID · NOV 27-28 · 2015 ROUND 2 Works? - YES… for 1 camera.
- 18. MADRID · NOV 27-28 · 2015 ROUND 2 Works? - YES… for 1 camera. What if more cameras? - 1 VLC process for each camera.
- 19. MADRID · NOV 27-28 · 2015 ROUND 2 Round 2 version with multiple cameras
- 20. MADRID · NOV 27-28 · 2015 ROUND 2 Cons: - A lot of processes in the server eating memory as hungry animals. - VLC is unable to recover from network cuts. - Each VLC process opens a socket in a different port, which in my case is a mess as I’m hosting the VLC processes inside a container and then routing the outside in. - VLC is permanently connected to the camera
- 21. MADRID · NOV 27-28 · 2015 AND THEN ONE DAY… Adobe updates flash player, breaks my streamings
- 22. MADRID · NOV 27-28 · 2015 BROKEN STREAMING Two possible reasons: 1. Something is wrong with my VLC command vlc-wrapper -vvv h,p://camera.ip.address/cgi-bin/faststream.jpg?stream=full&fps=16&size=640x480&preview" -sout '#transcode{vcodec=FLV1,vb=800,ab=32,fps=16,width=640,height=480,acodec=none}:duplicate{dst=std{access=h,p{mime=video/x- flv},mux=ffmpeg{mux=flv},dst=LINUX.BOX.IP.ADDRESS:8080/stream.flv},dst=display}' 2. Something is wrong with the new version of Flash
- 23. MADRID · NOV 27-28 · 2015 BROKEN STREAMING Two possible reasons: 1. Something is wrong with my VLC command vlc-wrapper -vvv h,p://camera.ip.address/cgi-bin/faststream.jpg?stream=full&fps=16&size=640x480&preview" -sout '#transcode{vcodec=FLV1,vb=800,ab=32,fps=16,width=640,height=480,acodec=none}:duplicate{dst=std{access=h,p{mime=video/x- flv},mux=ffmpeg{mux=flv},dst=LINUX.BOX.IP.ADDRESS:8080/stream.flv},dst=display}' 2. Something is wrong with the new version of Flash Of course option 2. I contacted Adobe: - ME: your upgrade has a bug and it broke my streamings - ADOBE: …(silence)
- 24. MADRID · NOV 27-28 · 2015 ROUND 3 A super cool media server capable to encapsulate H264 RSTP into RTMP for flash players: Erlyvideo.
- 25. MADRID · NOV 27-28 · 2015 ROUND 3 Pros: - Written in Erlang: amazingly light and fast - Connects to the source on demand - One process, several cameras - Tolerant to network cuts (due to the previous point) - Referrer filtering Cons: - Documented in russian (partially translated into english). Support forums also 80% russian. - HLS support (for smartphone apps) only supported in its new commercial version: Flussonic.
- 26. MADRID · NOV 27-28 · 2015 STEP 2: Smartphone app (HLS)
- 27. MADRID · NOV 27-28 · 2015 Nginx RTMP module An Nginx module capable to read streams in several formats as input, store them and then serve them encapsulated in other protocols (eg: HLS)
- 28. MADRID · NOV 27-28 · 2015 Nginx RTMP module Works? - YES, perfectly Cons: - Not very well documented - Not tolerant to network cuts - Unable to regain lost connectivity
- 29. MADRID · NOV 27-28 · 2015 Overall vision
- 30. MADRID · NOV 27-28 · 2015 Service schema A video source. A separate server for each streaming format. A website visitor and an smartphone user.
- 31. MADRID · NOV 27-28 · 2015 An automated process Using linux containers and some glue code everything can play together in an automated way
- 32. MADRID · NOV 27-28 · 2015 CONSLUSIONS
- 33. MADRID · NOV 27-28 · 2015 What I would do now if starting from scratch 1. Do it the opensource way: there are powerful tools 2. Look for a single solution for both web and mobile (probably libav, ffmpeg or Red5) 3. Spend time looking for a solution capable to serve also HTML5 formats (Mp4, Webm, Ogg)
- 34. MADRID · NOV 27-28 · 2015 THANK YOU!