Vimeo and Open Source (SMPTE Forum 2015)
- 2. Tools & Techniques of the web at Vimeo
- 3. 2 Summary 1. Introduction 2. Architecture & Tools 3. Future 4. Conclusions Contact Derek Buitenhuis Video Encoding Engineer derekb@vimeo.com http://github.com/dwbuiten daemon404 FOSS
- 5. 4 Vimeo – About us Who are we? • Top 5 Video Site. • Content Creators. • Multimedia Hackers. • Creative Innovators. Introduction
- 6. 5 Vimeo – About us What do we do? • Transcoding and video hosting at a high level of quality, at a • Create educational and entertaining content. • Video On Demand and tools for professionals. Introduction
- 7. 6 Vimeo – About us Who uses us? • Independent artists. • Small to Large Pros. • Businesses. • Regular people! Introduction
- 9. 8Architecture Transcoding Pipeline Server 1 Gearman Daemon 1 Video Worker 1 Video Worker 2 Video Worker N Audio Worker Mux/Combine Worker Server 2 Video Worker 1 Video Worker 2 Video Worker M Audio Worker … Server N Gearman Daemon M Video Worker 1 Video Worker 2 Audio Worker Video Upload Metadata Collector (Caching) Job Scheduler Gearman Daemon 1 Audio Encode Chunk 1 Chunk 2 Combine and Multiplex … Chunk N Gearman Daemon 2 … Gearman Daemon N
- 10. 9 Tools of the Trade A Few Awesome Media Projects •Gearman – Great distributed job server with a simple protocol usable from any language. •FFmpeg/Libav – First class multimedia toolkit. •x264 – The Best H.264 encoder around. •L-SMASH – Our preferred library for MP4 creation and demuxing/info. Rigidly strict to the specs, nice upstream developers, practical and realistic, has fewer LoC than the space shuttle. Architecture
- 11. 10 Tools of the Trade Cont’d A Few Awesome Media Projects •ffms2 – Offers frame-accurate, and pre- indexed seeking using the libav* libraries, and has an easy-to-use API. •FDK-AAC – Fraunhofer AAC encoder. •MediaInfo – Very good probe tool. •Many packages for Go. •Many more! Architecture
- 12. 11 Open Source Policies Long Term Maintainability and Benefits • Going against the trend in the US, we do not use Open Source software in bad faith; we contribute bug fixes, features, and at- scale-testing back, and do not violate licenses. • 3 of 5 members of the transcode team are or were upstream FFmpeg developers, and all are active open source contributors. All are members of Open Source projects we use. • It is not viable in the long term to fork internally, maintenance-wise, or feature- wise. • You’re probably already using it without even realizing you are. FOSS
- 13. 12 European View Open Source Media Lives in Europe • The majority of FOSS multimedia developers are Europe-based (FFmpeg, VideoLAN, OBE, upipe, GPAC). • Much of the skilled US-based multimedia knowledge emigrated from Europe. Personal theory is that European investors are too risk-averse (old-style “pay it safe” thinking). e.g. VideoLAN people thinking of USA for VC funding. • EBU has Open Source initiatives. Nothing similar in USA? Hopefully leading to more uptake. • Unfortunately still a lot of FUD from European Big Corps about Open Source. Particularly in the UK. FOSS
- 14. 13 Current Situation • VP9 – Legitimate quality gains, library too slow to use without ridiculous amounts of hardware. Rate control utterly broken. No hardware support. • DASH – Makes streaming and GOP-distributed encoding easier than ever. Reasonable spec and implementations available server- side, not great libraries client- side. Stupid XML playlist; everyone is using MSE instead. • HEVC – No encoding library currently beats best-in-class AVC encoder (x264). Some hardware support. • AAC – Same old, same old. Architecture Bandwidth – Internet pipes aren’t getting better fast enough to support 4k. We either need to force change or workaround it with codecs.
- 15. 14 Future • Daala – Doing interesting things. Definitely worth keeping an eye on. • DASH – There’s always more to do, such as device support. • x265 – Shaping up well, still not useful in production yet. Confusing competing patent pools (WTF). • Opus – Soon available in ISOMBFF. Currently available in WebM/Matroska, MPEG-TS, and Ogg. Architecture
- 16. 15 Conclusions
- 17. 16 Conclusions • Do not be shy of admitting you use Open Source software or interacting with the community. • GOP-Distributed Encoding: Everyone’s been doing it for years, but it is easier than ever with DASH. • Still no real viable alternative to H.264 for mass consumption in 2015 for non- Google-like entities. Do we wait for slow change (devices, hardware support) or try and force it? • Internet pipes need to get better in many counties • Europe is well ahead of the North America in many of these aspects (Internet pipes, new tech adoption, etc.), but behind in keeping innovators in Europe. Conclusions
- 18. 17 Questions?