The Fine Manual: Documentation Processes at Elastic

More than any other company where I’ve worked, Elastic understands the value of clear, useful documentation. A year and a half ago, when I started work as Elastic’s first technical writer, I walked into a place where there was an established culture of engineers that considered documentation a vital part of the software, as attested by the Elasticsearch Definitive Guide, which was already in place before I started. Building on that culture as we grow the technical publications staff is one of the things that makes this job so exciting.

The Process

The core process is similar to how it’s done anywhere else: Product Management sets priorities, and Technical Publications talks to the engineers about the current projects to gather requirements and initial insight regarding the new features or improvements. Documentation tasks are tracked with issues on GitHub, just as any other request to improve the software’s usability or functionality. This unified process keeps documentation and engineering effort aligned.

When documentation is ready to review, Engineering reviews the material for technical accuracy, we make the necessary corrections, and when there’s a consensus that the material is ready, the source is merged into the repo.

But what do I mean by source with respect to technical documentation?

The Technical Details

Documentation at Elastic is written in AsciiDoc, a markup language similar to MarkDown, but with a broader feature set. A script invokes the tools that transform the AsciiDoc, first to DocBook XML, then to HTML. The script was created and is maintained by Clinton Gormley, and you can take a look at it yourself at the docs GitHub repository.

That repository also contains a configuration file that governs which repositories are polled for source AsciiDoc, which books to build, and how version selectors are laid out.

To build all the documentation that you see on our site, a cron job makes sure this happens every half hour:

  • For every repository specified in the configuration file, pull the current AsciiDoc source.
  • Transform the AsciiDoc to HTML.
  • Push the HTML to the webservers.

This process lets us be very responsive to documentation issues. Fixes typically go live to the Web less than an hour after we merge them into the relevant repository.

What this Means to You: Contributing to Documentation

Elastic is committed to open source, and one of the signal joys of open source is finding a problem and fixing it.

Just as with any other request on a project, you can always open an issue with the documentation on any of our public projects. We’ll assign the issue and get it taken care of. To contribute more directly, open a pull request with your proposed changes. To make the adoption of your changes smoother, here are some best practices:

Understand the Documentation Structure

The overview of our process earlier in this post is a good high-level description of how things work, but for a good documentation pull request, it helps to understand how documentation is structured at a more granular level. For example, we make use of AsciiDoc attribute declarations, which are defined with pairs of : characters, as in this example:

:version: 1.0.0-beta4

Call a defined variable by using the variable’s name in braces:

The current release of the software is {version}.

Typically, documentation for a given Elastic software release starts with the index.asciidoc file in the /docs directory for that software’s GitHub repository, and index.asciidoc is where any attributes used for that documentation are defined. The index.asciidoc file has include:: statements that incorporate the text of all the other files in the directory. Files in that directory that use attributes will give you errors about undefined attributes if you don’t build them from index.asciidoc. If your contribution relies on new attributes, be sure to specify them in index.asciidoc, or they won’t work outside of the one file you did specify them in.

Minimize Use of Tables

A lot of tables are just used to present two-columnar data, like lists of term definitions or parameter values. For that kind of data, the [horizontal] or [qanda]list styles work well.

Test Build Your Contribution

Go to the docs repo on GitHub, download the build script, and make sure your contribution builds cleanly. If you want to go the extra mile, post the rendered HTML online somewhere like Firebase so reviewers can look at it.