HTML5 multimedia - browser-native video and audio - JSDay / Verona / 17 May 2012
1 like1,630 views
This document summarizes new multimedia capabilities in HTML5, including native support for video and audio playback without plugins. It describes how the <video> and <audio> tags can be used to embed video and audio into web pages directly in the browser. It also discusses the various video and audio formats that are supported, such as MP4, WebM, and Ogg formats. The document notes that HTML5 does not specify exact formats or controls, but provides JavaScript APIs to control playback programatically.
![canvas access to image data array
ctx = canvas.getContext("2d");
canvasData = ctx.getImageData(x,y,w,h);
[R,G,B,A,R,G,B,A,R,G,B,A,R,G,B,A, … ]](https://image.slidesharecdn.com/jsday17-05-2012-120517181005-phpapp02/85/HTML5-multimedia-browser-native-video-and-audio-JSDay-Verona-17-May-2012-45-320.jpg)
![camera access
// Opera
navigator.getUserMedia({video:true},success,error);
[…]
video.src = stream;
// WebKit
navigator.webkitGetUserMedia("video",success,error);
[…]
video.src = window.webkitURL.createObjectURL(stream);
gUM Shield: http://html5doctor.com/getusermedia/](https://image.slidesharecdn.com/jsday17-05-2012-120517181005-phpapp02/85/HTML5-multimedia-browser-native-video-and-audio-JSDay-Verona-17-May-2012-60-320.jpg)
HTML5 multimedia - browser-native video and audio - JSDay / Verona / 17 May 2012
- 1. HTML5 multimedia BROWSER-NATIVE VIDEO AND AUDIO Patrick H. Lauke / JSDay / Verona / 17 Maggio 2012
- 2. <video>
- 3. Adobe Flash currently most common video/audio delivery mechanism
- 4. <object width="425" height="344"> <param name="movie" value="http://www.youtube.com/v/9sEI1AUFJKw&hl=en &fs=1&"></param> <param name="allowFullScreen" value="true"></param> <param name="allowscriptaccess" value="always"></param> <embed src="http://www.youtube.com/v/9sEI1AUFJKw&hl=en&f s=1&" type="application/x-shockwave-flash" allowscriptaccess="always" allowfullscreen="true" width="425" height="344"></embed> </object>
- 5. plug-in is a black box
- 6. <video src="video.webm"></video> originally proposed by Opera in 2007 dev.opera.com/articles/view/labs-a-call-for-video-on-the-web
- 7. <video src="video.webm" controls autoplay muted loop preload="none|metadata|auto" poster="poster.jpg" width="320" height="240"> <a href="video.webm">Download movie</a> </video>
- 8. video as native object
- 12. <video> and JavaScript var v = document.getElementById('player'); v.play() v.pause() v.volume = … v.currentTime = … v.addEventListener('loadeddata', function() { … }, true) v.addEventListener('play', function() { … }, true) v.addEventListener('pause', function() { … }, true) v.addEventListener('timeupdate', function() { … }, true) v.addEventListener('ended', function() { … }, true)
- 15. HTML5 does not specify how browser controls should look
- 19. video formats the big debate?
- 20. HTML5 does not specify video container/codec (same as with images in HTML 4.01)
- 21. MP4/H.264 or Ogg Theora or WebM/VP8
- 22. MP4 / H.264 ubiquitous, licensing/royalties
- 23. Ogg Theora free and open, no licensing fees not many tools for it, not web optimised
- 24. WebM / VP8 open and royalty-free, web-optimised support by hardware and software vendors
- 26. Browsers vs codecs Chome Firefox IE Opera Safari Ogg Theora ✔ ✔ ✔ WebM / VP8 ✔ ✔ ? ✔ MP4 / H.264 ✔ ✔ ✔ mobile and devices slightly better...
- 27. providing multiple sources <video controls width="…" height="…"> <source src="movie.mp4" type="video/mp4"> <source src="movie.webm" type="video/webm"> <source src="movie.ogv" type="video/ogg"> <!-- fallback content --> </video>
- 28. specifying codecs <video controls width="…" height="…"> <source … type='video/mp4; codecs="avc1.42E01E,mp4a.40.2"'> <source … type='video/webm; codecs="vorbis,vp8"'> <source … type='video/ogg; codecs="theora,vorbis"'> <!-- fallback content --> </video>
- 29. understanding fallback <video controls width="…" height="…"> <source … type='video/mp4; codecs="avc1.42E01E,mp4a.40.2"'> <!-- fallback content → <p>Shown to browsers that don't support video element, and NOT just browsers that don't support MP4/H.264</p> </video>
- 31. feature-detection for codecs v.canPlayType('video/webm;codecs="vp8,vorbis"') "probably" | "maybe" | "" < WAT?
- 33. video and Responsive Web Design <source … media="(min-width:1000px)"> <source … media="(max-width:1000px)">
- 36. flash fallback for older browsers http://camendesign.com/code/video_for_everybody
- 37. <video controls width="…" height="…"> <source src="movie.mp4" type="video/mp4" /> <source src="movie.webm" type="video/webm" /> <source src="movie.ogv" type="video/ogg" /> <object width="…" height="…" type="application/x- shockwave-flash" data="player.swf"> <param name="movie" value="player.swf" /> <param name="flashvars" value=" … file=movie.mp4" /> <!-- fallback content --> </object> </video>
- 38. <audio>
- 39. audio...exactly the same as video <audio src=”music.mp3” controls autoplay … ></audio> <audio controls autoplay> <source src="music.mp3" type="audio/mpeg" /> <source src="music.oga" type="audio/ogg" /> <!-- fallback content --> </audio> formats: MP3 vs Ogg Vorbis (vs WAV)
- 41. no need for <audio> or <video> in your DOM var s = document.createElement('audio'); s.src = 'aaargh.oga'; s.play();
- 42. audio “sound sprites” to counter latency and overhead people.opera.com/~emoller/audio-demo
- 43. <canvas>
- 44. canvas drawing ready-made images ctx = canvas.getContext("2d"); var logo = new Image(); logo.src = 'logo.png'; ctx.drawImage(logo,x1,y1,w1,h1,x2,y2,w2,h2); or call in an existing image already on the page
- 45. canvas access to image data array ctx = canvas.getContext("2d"); canvasData = ctx.getImageData(x,y,w,h); [R,G,B,A,R,G,B,A,R,G,B,A,R,G,B,A, … ]
- 46. canvas also works with video ctx = canvas.getContext("2d"); v = document.getElementById('player'); ctx.drawImage(v,x1,y1,w1,h2,x2,y2,w2,h2); grab currently displayed frame (update as appropriate)
- 49. is it all safe to use, right now?
- 51. don't do browser sniffing http://www.flickr.com/photos/timdorr/2096272747/
- 53. feature-detection progressive enhancement, graceful degradation diveintohtml5.info/everything.html
- 54. feature-detection for audio/video if (!!document.createElement('video').canPlayType;) { … } if (!!document.createElement('audio').canPlayType;) { … }
- 55. sublimevideo.net
- 56. www.jplayer.org
- 60. camera access // Opera navigator.getUserMedia({video:true},success,error); […] video.src = stream; // WebKit navigator.webkitGetUserMedia("video",success,error); […] video.src = window.webkitURL.createObjectURL(stream); gUM Shield: http://html5doctor.com/getusermedia/
- 66. www.opera.com/developer patrick.lauke@opera.com