Introduce native html5 streaming player
7 likes1,687 views
The document details the development of an HTML5 video streaming player, highlighting key processes such as video encoding, adaptive streaming, and digital rights management (DRM). It outlines the architecture of the player, focusing on features like manifest parsing, content protection, and playback of adaptive bitrate segments. Future goals include enhancing DRM support across various platforms and integrating additional functionalities like live streaming and ad support.
Introduce native html5 streaming player
- 1. HOW DID I CREATE A HTML5 VIDEO STREAMING PLAYER and what I’ve learned KKBOXVideo 陳建辰 jessechen
- 2. AGENDA • Some beforehand knowhow • Why developing this player • HTML5 extensions • Introduce our player • Future of this player
- 4. analog-to- digital adapter digital raw data PROCESS OF PLAYINGVIDEO 1. Make it digital analog signal
- 5. H.264 / H.265 Encode PROCESS OF PLAYINGVIDEO 2. Encode video file Raw data .mp4 / .ts file Easier for storage and transference
- 6. PROCESS OF PLAYINGVIDEO 3. Play it ! .mp4 / .ts file a/v sync codec decode and display extract content from container
- 8. WHAT PLAYER DOES .mp4 / .ts file a/v sync codec decode and display HERE extract content from container
- 9. THEY ARE ALREADY HANDLED .mp4 / .ts file HTML5 a/v sync codec decode and display extract content from container
- 10. SO WHY DEVELOPING THIS PLAYER?
- 11. OUR PLAYER FOCUS ON (1) extract content and a/v info from container .mp4 / .ts file get files in smart way a/v sync codec decode and display
- 12. HOW PLAYER GET FILE ?
- 13. server holds .mp4 content VINTAGE WAY Player get it and play single file slow and inflexible
- 14. server holds .mp4 content EFFICIENT WAY Player get part of video and start play progressive download Keep requesting rest part of video during playback still inflexible
- 15. prepare different qualities of content on server MODERN WAY adaptive streaming Player plays best quality depends on network status Keep requesting rest part of video during playback
- 16. HOW ABOUT WE SPLIT FILE • split file into segments, described by manifest • manifest tells - available bitrates - how file is fragmented - other information, e.g encryption + +
- 17. knows how to get fragments ADAPTIVE STREAMING IN SHORT fragments manifest file
- 18. ADAPTIVE STREAMINGTYPES SS Smooth Streaming HLS HTTP Live Streaming HDS
- 19. INTRODUCE DASH Dynamic Adaptive Streaming through HTTP
- 20. PROS OF DASH • open source, with industry standard, which means more universal usage • support 2 types of containers, the MPEG-4 file format or the MPEG-2Transport Stream
- 21. GET FILES IN SMART WAY • manifest parser - able to read different type of manifest • adaptive bitrate - able to decide which quality to load
- 22. OUR PLAYER FOCUS ON (2) .mp4 / .ts file get files in smart way protection logic extract content from container a/v sync codec decode and display
- 23. DRM ? (DIGITAL RIGHTS MANAGEMENT) Content provider / DRM provider likes it For end user
- 24. CLIENT SIDE PROTECTION • Client get a license instead of a key from DRM service • A blackbox or sandbox get the key by processing license • That blackbox / sandbox decrypt content and directly output it to display
- 25. H.264 / H.265 Encode HOWTO PROTECT CONTENT Raw data .mp4 / .ts file DRM implemented here
- 26. PROTECTION LOGIC • give what DRM server needs and retrieve license • negotiate with browser in order to implement protection on client side • deal with different browsers knows how to
- 27. NOW, LET’S STARTTALK ABOUT BROWSER <video>, MSE and EME
- 28. HTML5VIDEO ELEMENT • <video></video> support progressive download • For adaptive streaming, MSE is required
- 29. extract content from container WHAT ROLE MSE PLAYS .mp4 / .ts file MSE handles a/v sync codec decode and display MSE handles
- 30. INTRODUCE MSE • Given different bitrate segments and it helps html5 video element to play smoothly • SourceBuffer provide video and audio buffer condition Media Source Extension
- 32. HOW BROWSER ADOPT DRM • For browser, that blackbox called CDM (Content Decrypt Module) • Each browser support different DRM context - “a blackbox or sandbox get the key by processing license”
- 33. DRM ON BROWSER Widevine FairplayPlayready Primetime
- 34. INTRODUCE EME • Even though browser support its own DRM, W3C defines a EME spec, in order to expose same api for client • prefixed api was implemented on earlier version of chrome(smart tv) Encrypted Media Extension
- 35. EME CDM provide context from encrypted content get ‘challenge’ DRM license server request with challenge get license provide license for CDM to decrypt content player EME CDM PROTECTION LOGIC FLOW
- 36. WHAT ROLE EME PLAYS extract content from container .mp4 / .ts file a/v sync codec decode and display decrypt content in blackbox
- 38. OUR GOAL • play not single file but sequence of segments, with different bitrate, a.k.a adaptive streaming • play protected stuffs, for content providers’ goods a player able to
- 39. 影片 ya pi yapi.js
- 40. DEVELOP PROCESS make it work stable refactor
- 41. MAKE IT WORK FIRST only 2 files in the very beginning
- 42. BE STABLE • well structured • modularized • dependency management (dijon.js) • consistent code style
- 44. UTILS • Capabilities • Debug • ErrorHandler • EventBus • IntervalBus • UrlModel api adaptive extension fragment manifest protection stats stream utils
- 45. EXTENSION • MediaSourceExtension • SourceBufferExtension • VideoModel api adaptive extension fragment manifest protection stats stream utils
- 46. MANIFEST • ManifestLoader • ManifestExtension • ManifestModel api adaptive extension fragment manifest protection stats stream utils
- 48. FRAGMENT • FragmentCtrl • FragmentLoader • SegmentTemplate api adaptive extension fragment manifest protection stats stream utils
- 50. PROTECTION • ProtectionCtrl • ProtectionModel • ProtectionRequest • Playready / Widevine api adaptive extension fragment manifest protection stats stream utils
- 51. STATS • MetricsCtrl • Metrics • StatsCtrl • BitrateHistory api adaptive extension fragment manifest protection stats stream utils
- 52. REFACTOR • review flow • redefine api
- 53. APPLICATION FLOW
- 55. REDEFINE API • use jsdoc to generate spec document • define api, event and vo(value object) • spec • result
- 58. DEPENDENCY INJECTION • not necessary in the beginning, but became very important after it went much more complex • use dijon.js as di framework
- 59. INTRODUCE DIJON.JS • a system holds all dependencies, after mapping modules to it • an object get dependencies after injection
- 60. DEFINE DEPENDENCIES // A dep function A() {} // B requires A function B() { this.a = undefined; // how dijon knows where to inject } // instantiate dijon var system = new dijon.System();
- 61. 3 KINDS OF MAPPING var a = new A(); system.mapValue(‘aValue’, a); // system.getObject(‘a’) would return an ‘a’ instance system.mapSingleton(aSingleton,A); // system.getObject(‘aSingleton’) would return a singleton a system.mapClass(bClass, B); // system.getObject(‘bClass’) would return a new inited B
- 62. OUTLET MAPPING // map outlet to make dependency work // @param sourceKey {String} - the key mapped to system would be injected // @param targetKey {String|'global'} - the key outlet is assigned to // @param outletName {String|sourceKey} - the name of property used as an outlet system.mapOutlet('aSingleton', 'bClass',‘a'); var b = system.getObject(‘bClass’); // b.a is aSingleton
- 63. DIRECTLY INJECTION var b = new B(); system.injectInto(b); // b WOULDN’T have dependency singleton A // b/c we only mapOutlet to key bClass of system // map outlet to global system.mapOutlet(‘aSingleton’,‘global’,‘a’); system.injectInto(b); // this would work
- 64. AUTO MAP OUTLET function B(){ this.aSingleton = undefined; this.c = undefined; } function C(){} // auto mapped outlet would assign to ‘global’ // and outlet name is the same as mapped key (before mapping) system.autoMapOutlets = true; system.mapClass(‘bClass’, B); // map again b/c B is changed system.mapClass(‘c’, C); // system.getObject(‘b’) would have c dep
- 65. COMMUNICATE IN BETWEEN Now B has dependencies of A and C How would you do in this scenario: A is doing something when it’s done, invoke a method of B –this is a very common situation in player
- 66. MAP HANDLER AND NOTIFY function A() { this.say = function() { system.notify(‘aDoneSaying’); }; } function B() { this.a = undefined; this.afterAsays = function(){ // do something }; } system.mapSingleton(‘a’,A); system.mapSingleton(‘b’, B); system.mapHandler(‘aDoneSaying’, ’b’,‘afterAsays’); // system.getObject(‘b’).a.say() would invoke b.afterAsays
- 67. NOTIFYING CLASS system.mapSingleton(‘a’,A); system.mapClass(‘b’, B); // map class here var b1 = system.getObject(‘b’); // b1.say() would invoke a newly instantiated b.afterAsays // instead of b1.afterAsays system.mapValue(‘b1’, b1); system.unmapHandler(‘aDoneSaying’,‘b’,‘afterAsays’) system.mapHandler(‘aDoneSaying’,‘b1’,‘afterAsays’) // b1.say() would invoke b1.afterAsays
- 68. function E() { this.a = undefined; this.setup = function () {}; } var e = new E(); system.injectInto(e); // e.setup invoked CONVENTION setup method of module would be invoked after getObject or injectInto
- 69. INJECT SYSTEM ITSELF system.mapValue('system', system); function D() { this.system = undefined; } system.mapSingleton('d', D); // system.getObject(‘d’) has system as dependency
- 70. function A() {}; function B() {} function Dep() { return { system: undefined, setup: function () { this.system.autoMapOutlets = true; // map dep here this.system.mapSingleton('a',A); this.system.mapClass('b', B); } }; } function App() { var b; var system = new dijon.System(); system.mapValue('system', system); system.mapOutlet('system'); var dep = new Dep(); system.injectInto(dep); // inject system to dep and invoke setup function, which map all deps return { system: undefined, a: undefined, setup: function () { b = system.getObject('b'); }, init: function () { system.injectInto(this); // after init, app.a exists } }; } // after new App().init(), app get all dep setup in Dep
- 71. SUMMARY OF DIJONJS • 3kb • very little convention • focus on di and notify / handler
- 72. BUILD PROJECT • list all source files in index.html of testing page • grunt as task manager • concat and minify scripts with grunt-usemin • export to dist/player.min.js
- 73. ONE LAST NOTICE • ui should be totally separated • ui <-> player -> videoElement • ui order player by api, respond to player behavior by event player videoElement ui
- 74. FUTURE OFYAPI.JS • firefox / safari drm support • live • ads • smart tv / chromecast / nexus player