I Wrote an FFV1 Decoder in Go for Fun: What I Learned Going from Spec to Implementation
- 1. I Wrote an FFV1 Decoder in Go for Fun: What I Learned Going from Spec to Implementation derek@videolan.org @daemon404 Derek Buitenhuis 5 December 2019 Budapest, Hungary NTTW4
- 2. But… why? 15 December 2019 • Because why not? • It seemed fun. • I wanted to see how hard it was to write something while trying not to use anything but a spec. • Why? How do you spend your weekends? This is normal. • But mostly: To make the spec better and aid in adoption of free and open codecs and standards. • More implementations means a better spec. • Doing it while trying not to reference anything else would expose anything missing or confusing. NTTW4
- 3. You’ll end up reading sections out-of-order and multiple times 25 December 2019 NTTW4 • Specs are not written in the order you actually have to perform the steps in, or the order you’ll write them in. • A second monitor really helps with all the back-and-forth referencing between your code and the spec. • Pseudo-code in one section will use variables defined in other sections / scopes.
- 4. Third party implementation is important! 35 December 2019 NTTW4 • There can be many things missing or unclear in a spec until at least one non-author implements it. • Things that may be obvious or clear to the author may not be to everyone else. • The end result is a much more robust, tested spec.
- 5. Having background and context in the spec is extremely useful. 45 December 2019 NTTW4 • Not many ‘professional’ specs have context or background in them, so why something is the way it is, or important implementation consequences may not be obvious. • The FFV1 spec has lots of background, and it came in very useful.
- 6. Having contact with the spec author(s) is extremely useful. 55 December 2019 NTTW4 • It’s good to confirm you’re not going crazy trying to figure something out. • Lean on the knowledge of those who came before you.
- 7. Writing a spec based on an existing codebase makes for “interesting” specs 65 December 2019 NTTW4 • Lots of the spec bugs I found were assumptions based on FFmpeg’s encoder or decoder behavior. • Annoying implications like requiring 17-bit buffers for predicting 16-bit RGB, which are not spelled out in the spec at all. I hit this right as I finished up. • On the flip side, the spec also had advice for how to multithread based off of the slice footers, and other real-world tips.
- 8. A bunch of LaTeX math and a paper reference alone aren’t as good as pseudocode 75 December 2019 NTTW4 • Looking at you range coder. • Paper was OK, but implementation details were annoying. • Not gonna lie, I based my implementation off of Wikipedia + FFV1 spec constants.
- 9. Some Quick Notes on the Code 85 December 2019 NTTW4 • Not exactly idiomatic Go, in order to remain as close as possible to the spec. • Goal is be a good reference. • Everything is annotated with references to spec sections. • I used a bit of Perl for code generation…
- 10. Links 95 December 2019 NTTW4 • FFV1 Go Implementation: https://github.com/dwbuiten/go-ffv1 • godoc: https://godoc.org/github.com/dwbuiten/go-ffv1/ffv1 • Simple Matroska Go Package: https://github.com/dwbuiten/matroska • godoc: https://godoc.org/github.com/dwbuiten/matroska • FFV1 Spec: https://tools.ietf.org/id/draft-ietf-cellar-ffv1-10.html • FFV1 Repo: https://github.com/FFmpeg/FFV1/
- 11. 105 December 2019 NTTW4 Questions?