CMAF Live Ingest Uplink Protocol
- 1. Live Ingest Protocol Summary of the CMAF and DASH profile of the live ingest protocol (DASH- IF) Rufael Mekuria (PhD), Senior R&D Engineer, Unified Streaming Demuxed The Conference for Video Engineers October 2019 San Francisco California
- 2. Streaming at Scale: some moving parts - A. Contribution/broadcast encoders - B. Live ABR encoders - C. Just in Time Packagers/origin servers/content stitching - D. Content delivery networks - E. Ad insertion (optional) - F. Players Standardization is more than just a player format
- 3. Live OTT at Scale Beyond the ABR encoder Media Source (Live) ABR encoder Packager/ Origin Server Server Stitching CDN Ad insertion Player Live ABR ingestSource ingest CDN Ingest/ Streaming protocol DASH/HLS I1 I2 Goal: bring industry together and define interface for point I1 and optionally I2! https://dashif-documents.azurewebsites.net/Ingest/master/DASH-IF-Ingest.html A B C D E F
- 4. Why a new protocol for live ingest ? - RTMP/Smooth becoming deprecated (no HEVC, VVC, HDR, CMAF, MPEG- H) - Fmp4/CMAF becoming the dominant media segment format for OTT -> CMAF ingest makes sense - Push Based DASH and HLS are common but under specified -> spec needed - Timed metadata & content splice information -> enable JITP & ad Insertion - Fault tolerance, low latency and redundancy for large scale streaming - Industry Convergence, document current best practices
- 5. Pull versus Push in streaming (Live) ABR encoder Origin Serverhttp post a.mpd http post rep1/rep1.init http post rep1/seg1.mp4 GET rep1/rep1.init DASH Ingest Pull Based DASH Client GET a.mpd (Live) ABR encoder Smart Server http post moov http post moof + mdat http post moof ..... Fmp4 Ingest DASH Client Just in Time Packager GET rep1/seg1.mp4 GET rep1/rep1.init GET a.mpd GET rep1/seg1.mp4 Common but Underspecified Common but needs Update to CMAF Push Based
- 6. DASH-IF Live Media Ingest Spec. Supporting companies (initial):
- 7. Project timeline Interop AWS, BBC, Unified, Harmonic, MediaExcel Informal IETF draft DASH-IF TF created Document for Community Review Aug 2019 Target publication July 2019Okt. 2018May 2018 Okt 2017
- 8. Two interfaces defined for live media ingest 1. Interface 1: CMAF Ingest -> for push no manifest is needed and it is created later 2. Interface 2: DASH/HLS Ingest -> manifest is also pushed, care needs to be taken Live ABR Encoder Origin Packager Stitching CDN CMAF ingest (I1) DASH/HLS ingest (I2) All are HTTP POST (or HTTP PUT) based
- 9. CMAF Ingest simplified (I1) Origin Packager ABR Encoder HTTP POST Video1.cmfv HTTP POST Video2.cmfv HTTP POST audio1.cmfa HTTP POST subt1.cmft HTTP POST meta.cmfm DASH/HLS/Smooth Long running POST (1 POST command per track) or short running POST (1 HTTP Post per fragment) Fragment boundary trigggers receiver, not POST command, at least one CMAF fragment http://pub_point/channel1
- 10. CMAF Ingest: basics (I1) - CMAF tracks: video, audio, timed text, subtitle, timed metadata - One TCP connection per track, short running fixed length post or chunked transfer encoding - Continuity + increasing fragment timestamp and or sequence number - Fmp4/CMAF fragment boundaries for retransmission and identification - New optional boxes introduced in CMAF and DASH - Support for Low Latency and Ad insertion based workflows
- 11. On sending CMAF Boxes styp prft emsgftyp moof mdatmoov tfdt tfhd Continuously increasing sequence number and BaseMediaDecodeTime optional boxes Defined in CMAF/DASH moof mdat CMAF header CMAF fragment CMAF fragment with optional boxes CMAF Track
- 12. CMAF Ingest security and network - HTTPS, TLS client certificates, Basic authentication, Digest Auth - Harmonized error response codes over HTTP with the other ingest interface for DASH/HLS - Typically the source resends the CMAF header and last fragment in case of failure - The protocol is kept simple as a design requirement, however many aspects that SHOULD be supported are included
- 13. CMAF Ingest: Switching Set constraints/signaling (I1) Implicit (mandatory) - CMAF Switching set constraints (implicit and mandatory) - Defined by CMAF specification, identical boxes + content can be used to identify tracks in switching sets Explicit (recommended/optional) Encoder generates switching ids for each unique switching set - Embed switching set id in post_uri by Switching(sw_id) keyword - Embed switching set id in kind box in udta - schemeIdUir urn:dashif:ingest:switchingset_id value = sw_id
- 14. Redundancy and hot failover (I1) Live Source 1 Live Source 2 Origin 1 Origin 2 CMAF ingestCMAF ingest SDISDI Clock Sync. Live Source 3# new 1. Redundant ingest/cross post 2. Procedures for starting new instance DASH/HLS/SMOOTH DASH/HLS/SMOOTH Contribution source
- 15. Timed Metadata CMAF Track (I1) - To enable fast processing fragmented timed metadata track is defined - ISOBMFF Samples carrying DashEventMessageBox - Avoid timeline problem with top level DashEventMessageBoxes - ISOBMFF metatrack + emerging mpeg specification - Separate track with SCTE-35/id3 and other timed metadata information, - schemeIdUri urn:scte:scte35:2013:bin (see SCTE-214) - Id3 in DashEventMessageBox as defined in aom by Microsoft and Apple - https://aomediacodec.github.io/av1-id3/ - Splice conditions set IDR frame at corresponding time point - Metadata track is technology an emerging MPEG specification
- 16. Timed Metadata CMAF Track (I1)
- 17. Splice conditioning - SCTE-35 urn:scte:scte35:2013:bin embedded in event message track, - Presentation time corresponds to SCTE-35 splice point - CMAF content conditioning for splice point: insert IDR frame or fragment boundary (splice conditioned encoding or packaging)
- 18. DASH/HLS Ingest (I2) Origin/ CDN ABR Encoder HTTP POST p.mpd Clients HTTP 200 OK HTTP POST p/rep1v/v.init HTTP 200 OK HTTP POST p/rep1v/seg1.mp4v HTTP POST p/rep1v/seg2.mp4v HTTP POST p/rep1a/a1.init HTTP POST p/rep1a/seg1.mp4a Mime types/ media types in file names HTTP POST p/rep1a/seg2.mp4a HTTP POST p/rep1t/seg1.cmft Segment Numbering and timestamp
- 19. DASH/HLS + CMAF Ingest (i1 + i2)=i3? - Both interfaces combined in one (under consideration) Origin/ CDN ABR Encoder Clients HTTP 200 OK HTTP POST p/rep1v/v.init HTTP 200 OK HTTP POST p/rep1v/seg1.cmfv HTTP POST p/rep1v/seg2.cmfv HTTP POST p/rep1a/a1.init HTTP POST p/rep1a/seg1.cmfa HTTP POST p/rep1a/seg2.cmfa HTTP POST p/rep1t/seg1.cmft Drop relative paths For repackaging Fragments comply With CMAF and i1 HTTP POST p.mpd CMAF DASH
- 20. Summary - CMAF ingest -> CMAF + HTTP POST - Fully aligned with CMAF specification - No manifest is needed, feed to JITP - Long or short running post (or in between) - Optimized for JITP and direct archiving - Distributed workflow (encoding, encryption + packaging) - DASH/HLS ingest -> DASH/HLS + HTTP POST, posting both - Possible to combine with CMAF ingest by using CMAF fragments - Target standard CDN, but for low latency still additional features might need to be added. - Mighty live encoder that needs to know it all - Receiver still needs some understanding to convert POST to GET and understand for example chunked CMAF pass through
- 21. Reference Software + demo - Live ingest of CMAF files using CMAF ingest profile - https://github.com/unifiedstreaming/fmp4-ingest/tree/master/ingest-tools - fmp4Ingest (ingest cmaf files using CMAF ingest) - Tools for converting timed metadatatrack (still in progress) - Demo with Unified origin and docker compose: - https://github.com/unifiedstreaming/cmaf-ingest-demo
- 22. Next Steps - Publish Document the final specification - Encourage more deployment in live encoders and work with them on this - Follow & promote the CMAF event message track in MPEG - Formally standardize the specification of live media ingest - Promote usage in other sectors: security camera, mobile device 5G uplink etc⦠- https://github.com/Dash-Industry-Forum/Ingest
- 23. Thank you for your attention! - Questions ? - Contact: Rufael@unified-streaming.com