Interactive Content Authoring for A153 ATSC Mobile Digital Television Employing Open Source Tools
1 like581 views
The document discusses interactive content authoring for ATSC mobile digital television using open source tools. It provides an overview of the rich media environment defined in A/153 including the use of SVG Tiny 1.2 for presentation format, 3GPP DIMS for packaging format, and carousels for transportation format. It also describes using the open source GPAC software for prototyping applications, including authoring multimedia scenes, packaging in DIMS format, and streaming over a carousel. Key challenges discussed are the lack of return channel, need for a "push in advance of consumption" model, and experimental nature of current free software.
Interactive Content Authoring for A153 ATSC Mobile Digital Television Employing Open Source Tools
- 1. Interactive Content Authoring for A/153 ATSC Mobile Digital Television Employing Open Source Tools Brad Fortner – Ryerson University bfortner@ryerson.ca ATSC TechZone Pavilion Booth # N4039
- 2. interactive content authoring? Lack of books or information on subject that provide examples and clarification.
- 3. evolution
- 4. ATSC M/H content group Brad Fortner Tyler Pham Gregory Kon Steven James May From Left To Right: Many Ayromlou, Victor Nguyen, Alireza Tahmasbi, Raymond Padillo, Jeremy Littler
- 6. a decade later- standards A/153 part 5 set standards for the rich media environment for ATSC mobile dtv
- 7. a decade later- content Steve Bratt W3C Technical Plenary 2003 "The Semantic Web: common framework that allows data to be shared and reused across application, enterprise, and community boundaries."
- 8. a decade later- MPEG 4 MPEG-4 multimedia object based coding standard Today: way more than a video compression format H.264/MPEG-4 Part 10
- 9. a decade later- MPEG 4 MPEG-4 now 28 parts from which the principles of the rich media environment for A/153 was derived
- 10. a decade later- learned? = Rich Media Application Linked Data Multimedia Objects +
- 11. agenda Could you provide some background? • background • the three parts to rich media delivery • A/153’s rich media environment • what and why multimedia object coding is important • A/153’s “push in advance of consumption” content model • multimedia scenes • presentation format • SVG Tiny 1.2 • ECMAScript • packaging format • DIMS and carousels • open source software available for prototype development • pitfalls and presentation takeaways
- 12. rich media delivery Let’s do a traffic app! presentation format packaging format transportation format Rich Media Content Server(s) Browser with Rich Media Engine Let’s do a traffic app! Browser with Rich Media Engine
- 13. rich media environment Let’s do a traffic app! presentation format packaging format transportation format Browser with Rich Media Engine SVG Tiny 1.2 ECMA Script Dynamic Interactive Multimedia Scenes (DIMS) A/153 Let’s do a traffic app! Browser with Rich Media Engine
- 14. Scene Stream Server scenes in MPEG-4 Server A Server B How’s the Weather App progressing? Server C primitive multimedia objects How’s the Weather App progressing? rendered scene on browser
- 15. Scene Stream Server scenes in MPEG-4 Server A Server B How’s the Weather App progressing? Server C primitive multimedia objects How’s the Weather App progressing? rendered scene on browserscene description
- 16. scene description MPEG-4 primitive multimedia objects How’s the Weather App progressing? rendered scene on browserscene description Environmental Objects Office Desk Chair Businessman Objects Businessman Speech Bubble Fragment Fragment Fragment Fragment Fragment Fragment Fragment Fragment Hierarchy
- 17. scene description MPEG-4 primitive multimedia objects How’s the Weather App progressing? rendered scene on browserscene description • places (arranges) media objects on interface • transforms media objects attributes to alter appearance o scale, transparency, color, animation etc. • provides timing between media objects and streamed data • provides for user interactivity • called (microDOM) uDOM in SVG Tiny 1.2 + =
- 18. MPEG-4 object coding primitive multimedia objects How’s the Weather App progressing? rendered scene on browserscene description + = + = primitive multimedia objects rendered scene on browser scene description
- 19. presentation format packaging format transportation format SVG Tiny 1.2 ECMA Script Dynamic Interactive Multimedia Scenes (DIMS) A/153 pushing multimedia objects
- 20. push content model • return path not guaranteed in ATSC M/H • return path • device dependent • network availability dependent • forward, store, use content design model • applications must be designed accordingly “push content in advance of consumption model”
- 21. push content model “push content in advance of consumption model” • six “Multimedia Scenes” are used in the weather app • they are transmitted all at one time • a small number of SVG fragments can be reused over a number of scenes • judicious use of SVG fragments reduce transmission requirements
- 22. push content model “push content in advance of consumption model” • ECMAScript is used to program the interactive buttons to switch between multimedia scenes
- 23. presentation format So how does one create these multimedia objects?
- 24. SVG tiny 1.2 = • XML-based file format used for describing two-dimensional vector graphics • editable text • human-readable, machine-readable • can be searched, indexed, and scripted raindrop object (fragment) markup in SVG Tiny 1.2
- 25. SVG tiny 1.2 But what about my cute cat pictures? JPEG and PNG encoded to base64
- 26. SVG tiny 1.2 3X size: above rendered at 1262 * 727 – originally designed for 416 * 240 • traffic object is originally a 293 * 220 bitmap • does not scale with other SVG objects in scene
- 27. SVG tiny 1.2 scene creation
- 28. scene description SVG simultaneously creates multimedia objects and scene description as one authors content SVG Link to raindrop fragment Raindrop fragment screen positioning and sizing markup. This is used as scene description.
- 29. SVG tiny 1.2 scene creation SVG Tiny 1.2 rich media application displayed in Opera browser .svg
- 30. freely available software Show Me The Free!!! • any text editor will edit SVG and XSLT • integrated developer environment (IDE)
- 31. freely available software Show Me The Free!!! • SVG vector graphics editors “VISIO like” “Illustrator like”online
- 32. freely available software Show Me The Free!!! • SVG vector graphics libraries
- 33. freely available software SVG Tiny 1.2 rich media application displayed in Opera browser inkscape svg object text inside svg markup icon svg objects from openclipart.org objects chosen for display on client using ECMA script weather from rss feed inserted using XSLT inkscape svg object text inside svg markup event handler buttons are ECMAScript functions
- 34. software used weather app
- 35. packaging format Scene Stream Server scene description Server A Server B MPEG-4 Terminal Server C
- 36. 3GGP DIMS object packaging Server A Server B Server C Scene Stream Server and DIMS Packager packaged scene description and objects highly constrained network and devices
- 37. open source DIMS software What! Open Source DIMS software. WOW! “as is”
- 38. open source DIMS software Understood. GPAC is useful for prototypes. • GPAC Project on Advanced Content • implementation of the MPEG-4 systems standard • written in ANSI C (cross platform) • tools for media playback, vector graphics, MPEG-4 authoring and distribution • consists of o multimedia packager called MP4Box o some server functions (experimental) o multimedia player called Osmo4 • MP4Box o generates 3GPP DIMS packaging from SVG files • Osmo4 o displays SVG Tiny 1.2 scenes packaged in 3GPP DIMS files
- 39. 3GGP DIMS carousel types object 0 object 1 object 2 object 3 object 0 object 1 object 2 object 3 static carousel0 1 23 4 GPAC Experimental DIMS Stream Server • static carousel • repeatedly sends scene description and objects • repeats because devices tune in at different times
- 40. 3GGP DIMS carousel types • dynamic carousel • sends updates to screen elements 0 1 23 4 GPAC Experimental DIMS Stream Server dynamic carousel (update) object 0 object 1 object 2 object 3 object 4 object 0 object 1 object 2 object 3 object 4 object 5 object 5 changechangechange
- 41. GPAC DIMS static operation object 0 object 1 object 2 object 3 object 0 object 1 object 2 object 3 static carousel0 1 23 4 GPAC Experimental DIMS Stream Server GPAC DIMS server command-line interface instructions are as follows GPAC Software Packager Destination IP Address Time Between Carousel Sends Initiates DIMS Packaging Name Of File To Be Packaged And Sent Streams Scene Live
- 42. GPAC OSMOS4 player object 0 object 1 object 2 object 3 object 0 object 1 object 2 object 3 static carousel0 1 23 4 GPAC Experimental DIMS Stream Server GPAC OSMOS4 player command-line interface instructions are as follows GPAC OSMOS4 Player Looks for Session Description Protocol from DIMS server
- 43. GPAC dynamic updates Primary Stream GPAC Experimental DIMS Stream Server 0 1 23 4
- 44. Primary Stream GPAC Experimental DIMS Stream Server 0 1 23 4 GPAC dynamic updates Update Stream Primary Stream GPAC Experimental DIMS Stream Server 0 1 23 4
- 45. GPAC dynamic updates Update Main Carousel0 1 23 4 GPAC Experimental DIMS Stream Server Linked Data ServerLinked Data Server Update
- 46. GPAC dynamic updates Update Main Carousel0 1 23 4 GPAC Experimental DIMS Stream Server Linked Data ServerLinked Data Server Update
- 47. GPAC dynamic updates Linked Data Server • Linux BASH shell script to collect linked data and write the update file (doScript) • BASH: wget --output-document=camera.jpg http://www.toronto.ca/trafficimages/loc9.jpg • BASH/DIMS: a="<Replace ref="trafficImage" attributeName="xlink:href" value="data:image/ jpeg;base64,"$(base64 camera.jpg)"" />" • BASH: echo $a > updateFile.svg
- 48. Update Stream Primary Stream GPAC Experimental DIMS Stream Server With BASH Script, Update File, XSLT Program and Traffic Prototype SVG File 0 1 23 4 GPAC Software Packager Destination IP Address Time Between Carousel Sends Initiates DIMS Packaging Name Of File To Be Packaged And Sent Name Of Update File That GPAC Watches Streams Scene Live GPAC dynamic operation
- 49. freely available software Show Me The Free!!!
- 50. Pitfalls – SVG tiny 1.2 viewer GPAC OSMO4 Browser Render (Packaged DIMS File) OPERA Browser Render (SVG File) Lack of a fully implemented freely available SVG tiny 1.2 standard viewer
- 51. Update Stream Primary Stream GPAC Experimental DIMS Stream Server With BASH Script, Update File, XSLT Program and Traffic Prototype SVG File 0 1 23 4 GPAC – experimental nature • dynamic carousel • Prototype crashes when primary and update streams sent too close together
- 52. takeaways - moving objects presentation format packaging format transportation format SVG Tiny 1.2 ECMA Script Dynamic Interactive Multimedia Scenes (DIMS) A/153
- 53. takeaways - push • return path not guaranteed in ATSC M/H • return path • device dependent • network availability dependent • forward, store, use content design model • applications must be designed accordingly “push content in advance of consumption model”
- 54. SVG simultaneously creates multimedia objects and scene description as one authors content Takeaways - SVG
- 55. takeaways - carousels Primary Stream GPAC Experimental DIMS Stream Server 0 1 23 4 Update Stream GPAC Experimental DIMS Stream Server 0 1 23 4
- 58. Brad Fortner – Ryerson University bfortner@ryerson.ca ATSC TechZone Pavilion Booth # N4039 www.openmobiledtv.org Ryerson University Booth North Hall - #N524 International Research Park