Thinking of Documentation as Code [YUIConf 2013]

Editor's Notes

• #2: Hi, everybody. I’m Evan Goer. It’s great to be back!
• #3: A little background:I’m a former technical writer, now a frontend engineer at Intuit, makers of fine products that help people deal with their financial lives, such as TurboTax, QuickBooks, and Mint.Oh, and I also wrote a book!
• #4: So! Thinking of documentation as code. Or the alternative title…
• #5: So I’m including this silly title for a serious reason. We all naturally and rightly want to make fun of the idea that there even is such a thing as a simple “tip” or “trick” for something as complex as software development, or writing documentation.As many of you know, I’m not a big fan of airy statements about writing like, “Be clear,” or “Write short sentences,” or airy statements about software engineering, for that matter. I mean, clear writing! Well-crafted software! Who doesn’t want that?But these are just platitudes. How exactly do I “be clear” when I’m writing? How do I write “well-crafted software”? What is that, anyway? These statements might make us feel good, but other than that, they’re not helpful.What would be helpful is a guideline, a heuristic, a razor. Like Occam’s Razor, “Among competing hypotheses, select the one with the fewest assumptions.”
• #6: So let’s start with a fable.You and a couple of teammates asked to take on a legacy project, and after investigatingit, you realize that it has no tests at all. Oh no! The three of you are a bit despondent over this, when your boss walks in.He says, “Don’t worry, we’ll get these tests done. In fact, I have this great unit testing solution for you. I’ve been talking to this vendor, and they have this awesome “unit test repository” for storing tests! It’s specially designed to hold tests. It comes with its own, internal specialized version control system. Isn’t that great?Oh, and the repository has all sorts of custom workflow rules and access rights we can setup. All built in!Also, they have this special proprietary tool for writing these tests. You have to edit tests using this tool, but don’t worry – it’s really really productive! The three of you are going to love it.”What would your reaction be? Imagine the three of you in the room hearing this spiel. Raise your hand if you think this sounds like a great idea.
• #7: This is how I imagine most software engineering teams reacting.You wouldn’t accept any of this for your tests, would you? It’s insane.And yet people are really quick to store their documentation in all kinds of strange places and deal with documentation source in all sorts of strange ways that would never fly for source code. Or tests or build scripts.
• #8: And so here’s our heuristic, our razor. Like Occam’s Razor, it’s not a platitude to make us feel good, it’s a tool to help make a decision when you have lots of messy, expensive options to choose from. “To first order, software documentation is like code.”I would love to take the credit for this, but I can’t, because I have no doubt that the idea is not original to me.Today we’re going to see what it implies (and where it doesn’t quite work). We have to be careful. No heuristic is perfect. They can be overly simple, or just wrong.I’ll start off by making it clear: if you have a specialized team of technical writers or support personnel who are fully responsible for writing docs, you can throw everything I’m about to say out the window. Everything I’m saying from here on out only makes sense if you have software engineers writing all of the docs, or at least some of the docs themselves.
• #9: So another way of stating this heuristic is, treat your documentation like any other “meta-code” related to your project.The great thing about this heuristic is that it’s easy to apply, because you have years of experience thinking about software code. We’ll see how it helps us quickly zero in on the right answer for documentation.
• #10: Start with an obvious one: where should we store our documentation source? Maybe a specialized knowledge base? Maybe a wiki sounds reasonable? Raise your hand if you store your documentation source somewhere like that.Let’s apply our heuristic. To first order, software documentation is like code. Where do we store code?We store our unit tests in a hosted git repo, for the same reasons we store our library code in a hosted git repo. We get very sophisticated version control of course, but also distribution, collaboration, backup.And the reason we do this is, version control systems for software developers have been evolving for decades, under intense scrutiny. A modern version control system is the beneficiary of this evolution. Specialized repositories for documentation generally have watered-down, enterprisey, poorly reinvented versions of these features. YOU, as software developers, don’t need or want that any stuff when you have real version control at your disposal.As soon as you leave documentation outside of your repo, not only do you degrade critical features – that’s not even the worst thing. The worst thing is that you destroy your normal everyday engineering workflow. Editing code is normal, editing docs becomes something special and weird. That, above all else, is what accelerates documentation rot.
• #11: Does it make sense to create documentation source in a binary format? It sounds a bit crazy, but yes, people do this all the time. Let’s apply our heuristic again. To first order, software documentation is like code.And once again, once you frame the decision that way, you know the answer immediately.Why do we write our code as plain text? So you can edit it in vim, emacs, Sublime Text, an IDE, whatever makes sense for you. So you can bring to bear all of the utilities we’ve developed over the last few decades for searching, inspecting, and manipulating text. So you can deal with it sanely in version control.If you choose a proprietary, binary format for your docs, once again, you blow apart your normal engineering workflow. Now everyone is forced to use a specialized tool to make changes, which means everyone is less productive (assuming they even have the right tool in the first place).
• #12: As software engineers, we know that knocking out reams and reams of code is not necessarily a good thing. Who’s looked under the hood of healthcare.gov? Raise your hand.We know that more code we write, the more our programs slow down, the more bugs they will have, the harder they will be to maintain. The same is true for documentation.In fact, it goes double for documentation. All along our heuristic has been, “To first order, documentation is like code.” Let’s shine some attention on the second-order effects, where the analogy breaks down. Documentation is like code… except that documentation is much, much harder to regression test. You can test a few things in an automated fashion: you can run spellcheck. Seriously, run spellcheck! You can factor out your code examples as separate files, and run them automatically. And documentation generators like Doxygen and Javadoc at least get the basic structure right. But the only way to keep documentation correct and up to date is manual review by humans, and lots of it. So documentation is more expensive than code. Which leads us to a corollary…
• #13: I used to run into this situation all the time back when I was a technical writer. I’d be writing the documentation for a new tool, and I’d wonder why there was so much work to do. I’d go over to the engineer and say, “This section is getting really huge. Are you sure we need to do all this?” And the engineer would say, “Oh. No, we could make this simpler.” And poof!, just like that, my job would get twice as easy.Writing lots of documentation is a code smell. Part of a technical writer’s job is to spot this and bring this to your attention. As a software engineer, you should be looking for it too.Are you finding yourself writing complicated install documentation? Well, why is your installation process so complicated? Maybe you could reduce it down to three steps? Or one?Are you finding yourself writing lots of API docs? Well, why does your API have such a huge public surface? Could you simplify it? Does the world really need all those methods?BTW, the absolute worst when it comes to documentation is dev environment setup docs. Holy crap those are bad. In fact, I’d like you to take a pledge. Everyone please raise your hand & repeat after me.“I, State Your Name, do solemnly swear that I will never write another dev setup wiki again. That I will use install automation. That I will use containers. That I will use VMs. And I will automate that shit! So help me, $DEITY.”
• #14: Among my friends, it’s become a bit of a running joke that I hate wikis. The truth is, I don’t have a problem with wikis as useful general purpose tool. My issues start with using wikis for technical documentation. Wikis are general purpose, which means they usually lack the semantics and features that a documentation nerd like me looks for. This isn’t to say that you couldn’t have a wiki that supports those features. But most don’t. Another problem is editing. Typing into a browser textarea all day sucks. And again, wikis could fix this. Today, we have full blown browser-based coding environments that are… actually decent. A wiki could use one of those, rather than some random jQuery plugin. Or the wiki could make offline editing easy. And some wikis do.The third and biggest problem with wikis, is that now docs are siloed outside of your repo, they use a different workflow. That is super, super bad. But again, you could imagine a wiki that tied its source very closely with your repo. Maybe the wiki is kind of alternative “view” on a big directory of Markdown or reStructuredText files.So in theory, you could build a wiki that answered my concerns. GitHub wikis are actually pretty close. They are tied to your repo! They support some extensions to Markdown that are suited for documentation. For lightweight docs, they’re… not bad. Now YUI 3 has never had a wiki, we’ve been doing it the right way since we launched the new YUIDoc and Selleck. A more interesting case is jQuery, they’ve been through this experience – they had a wiki up until about a year ago, but they threw it all away (literally, you can’t even find it on archive.org) and launched a new, very slick doc site that’s all on git. Go take a look at the jQuery doc site, the improvement in cohesion and quality is huge. And right at the bottom of each page, they have a button, “”Suggestions, comments, feedback? Send us a pull request or open an issue.” Exactly. That’s how you do it. Bravo to the jQuery doc team.
• #15: User comments. Don’t do this. Imagine if you treated your code that way. “We’re going to allow anyone in the world to append methods into our module.” Again: treat docs as a first class citizen. Harvest contributions the same way you do with code.You might think this is a niche-y issue, but smart people keep making this mistake over and over. Above is screenshot is of AngularJS. Angular is a really cool project, worth checking out. But the AngularJSdocumentation is not exactly the project’s strong suit, and the comments are not helping at all. They consist of confusion, of people complaining about the docs. Lots of noise, very little signal. That stuff belongs in issues or IRC, not in the final product.Or React.js. Really interesting new library from Facebook. Doing really innovative things with performance. Very smart people working on it. And of course, every doc page allows Facebook comments. These are even worse than AngularJS, which at least uses DISQUS.Old school. PHP.net. Longtime dedicated doc team. They have a hard job to do, and I think the core docs are really solid. But there’s lots of lousy, unsafe, deprecated advice in the user comments.Thinking about your docs holistically means you’ll never make this mistake. Don’t allow user comments. Ever.
• #16: Because documentation is so expensive, you want to add it judiciously. You need a balance. What to write, and what not to write.Your code covers the what. If your code is well written, it will illuminate the how. But even good code can fall down on covering “how”, and code rarely does a good job of explaining the why. That’s where documentation really shines. Code covers the stuff on the left. Documentation covers the stuff on the right.It’s still reasonable to write documentation that covers the “what”. A lot of people, particularly casual and new users, are never going to read your source. You’re going to need to write some docs to get them over the initial hump. That’s okay.But what you really want to focus on are the second two. “How do I do X with this library?” “How does Y work, exactly.” And even more than that, the “Why”. “Why would I call this method? What were the designers thinking when they wrote this, what did they intend?“Focusing on “Why” and “How” will help you avoid writing terrible API documentation. The worst documentation is API documentation that simply restates the name of the method. “getConnection(): gets the connection.” Shoot me now! Ask yourself: why and when do I call this? How does it work with other methods? Etc.
• #17: Given that we’re duplicating information when we cover the “what” and “how”, how can we put on our developer hats and apply the principle of DRY (do not repeat yourself) to documentation?One obvious example is Javadoc and other documentation generators. As I mentioned before, at the very least that gives you the basic object/method structure right, even if the comments turn out to be a pack of lies.Do you have code examples? Could those code examples be broken out as separate files? Could those separate example files be part of your unit test suite?What about docs for command line tools? I love it when command line tools generate their usage statements from their own argument parsing code. I’m sure some of you are allergic to fancy arg parsers, “that’s way more than I need,” but that ability to tie docs & code together is really powerful and good.Of course DRY concepts in documentation can be much simpler than any of that. A humble cross-reference (“for more about configuring widgets, refer to section …”) is a form of DRY documentation. Or a glossary entry that automatically links back to its definition. Simple little building blocks like these can reduce duplication and errors.Maybe you want to reuse some examples or procedures across different books, or maybe you have one build of your docs targeted at different OSes, where there are small variations between the two. Maybe you want to grab little bits of your docs and expose them as tooltips or other help constructs that are built into your GUI. You’re a developer! If you can parse it, the sky’s the limit.
• #18: And finally, speaking of builds, people want to consume your documentation in all sorts of different ways. I mean, man pages are awesome! How do we get there?HTML is a good target format for documentation. It’s not the best authoring format. Most of you know this – you write your READMEs as Markdown, not HTML. Writing docs in HTML is kind of a pain, even for frontend engineers!The key is to author in an abstract format that is designed to be built into other formats, like reStructuredText or Markdown with some extensions. Not only are these lightweight markup formats easier to read & author, but they are unopinionated about the end target format, whether it’s HTML or something else. For example: this construct isn’t an HTML hypertext link, this is a cross-reference. And if you render it as PDF, it’ll say, “refer to Page 117,” which means it’s still useable when printed. Or, this this construct isn’t a specially formatted red div, this is an admonition, a caution – which again, you can display differently in a web page vs. a man page vs. a printed page.
• #19: This is just a sample of the different ways you can use the heuristic. Not only can it help you avoid wasting lots of time on ratholes (like user comments), but it opens up possibilities. Documentation isn’t some alien thing; many of the concepts that we use every day in software engineering apply just as well to documentation.That’s it. You can reach me at my email address, or “evangoer” on Twitter or any channel you can think of. Thank you!