How to Write Documentation¶
Writing documentation is one of the most important but often overlooked tasks for increasing yt’s impact in the community. It is the way in which the world will understand how to use our code, so it needs to be done concisely and understandably. Typically, when a developer submits some piece of code with new functionality, the developer should also include documentation on how to use that functionality (as per Requirements for Code Submission). Depending on the nature of the code addition, this could be a new narrative docs section describing how the new code works and how to use it, it could include a recipe in the cookbook section, or it could simply be adding a note in the relevant docs text somewhere.
The documentation exists in the main code repository for yt in the
doc directory (i.e.
$YT_GIT is the path of
the yt git repository). It is organized hierarchically into the main
You will have to figure out where your new/modified doc fits into this, but browsing through the existing documentation is a good way to sort that out.
All the source for the documentation is written in Sphinx, which uses ReST for markup. ReST is very straightforward to markup in a text editor, and if you are new to it, we recommend just using other .rst files in the existing yt documentation as templates or checking out the ReST reference documentation.
New cookbook recipes (see The Cookbook) are very helpful for the community
as they provide simple annotated recipes on how to use specific functionality.
To add one, create a concise Python script which demonstrates some
functionality and pare it down to its minimum. Add some comment lines to
describe what it is that you’re doing along the way. Place this
source/cookbook/ directory, and then link to it explicitly in one
of the relevant
.rst files in that directory (e.g.
etc.), and add some description of what the script
actually does. We recommend that you use one of the
sample data sets in your recipe. When the full
docs are built, each of the cookbook recipes is executed dynamically on
a system which has access to all of the sample datasets. Any output images
generated by your script will then be attached inline in the built documentation
directly following your script.
After you have made your modifications to the docs, you will want to make sure that they render the way you expect them to render. For more information on this, see the section on Building the Documentation. Unless you’re contributing cookbook recipes or notebooks which require a dynamic build, you can probably get away with just doing a ‘quick’ docs build.
When you have completed your documentation additions, commit your changes to your repository and make a pull request in the same way you would contribute a change to the codebase, as described in the section on Making and Sharing Changes.
Building the Documentation¶
The yt documentation makes heavy use of the Sphinx documentation automation suite. Sphinx, written in Python, was originally created for the documentation of the Python project and has many nice capabilities for managing the documentation of Python code.
While much of the yt documentation is static text, we make heavy use of cross-referencing with API documentation that is automatically generated at build time by Sphinx. We also use Sphinx to run code snippets (e.g. the cookbook and the notebooks) and embed resulting images and example data.
Essential tools for building the docs can be installed alongside yt itself. From the top level of a local copy, run
$ python -m pip install -e .[doc]
Quick versus Full Documentation Builds¶
Building the entire set of yt documentation is a laborious task, since you
need to have a large number of packages in order to successfully execute
and render all of the notebooks and yt recipes drawing from every corner
of the yt source. As a quick alternative, one can do a
of the documentation, which eschews the need for downloading all of these
dependencies, but it only produces the static docs. The static docs do
not include the cookbook outputs and the notebooks, but this is good
enough for most cases of people testing out whether or not their documentation
contributions look OK before submitting them to the yt repository.
If you want to create the full documentation locally, then you’ll need
to follow the instructions for building the
full docs, so that you can
dynamically execute and render the cookbook recipes, the notebooks, etc.
Building the Docs (Quick)¶
In order to tell Sphinx not to do all of the dynamic building, you must set the
$READTHEDOCS environment variable to be
True by run from the command
line (using bash syntax for example), as
This variable is set for automated builds on the free ReadTheDocs service but can be used by anyone to force a quick, minimal build.
Now all you need to do is execute Sphinx on the yt doc source. Go to the documentation directory and build the docs:
cd $YT_GIT/doc make html
This will produce an html version of the documentation locally in the
$YT_GIT/doc/build/html directory. You can now go there and open
index.html or whatever file you wish in your web browser.
Building the Docs (Full)¶
As alluded to earlier, building the full documentation is a bit more involved than simply building the static documentation.
The full documentation makes heavy use of custom Sphinx extensions to transform recipes, notebooks, and inline code snippets into Python scripts, IPython notebooks, or notebook cells that are executed when the docs are built.
To do this, we use Jupyter’s nbconvert module to transform notebooks into HTML. to simplify versioning of the notebook JSON format, we store notebooks in an unevaluated state.
To build the full documentation, you will need yt, jupyter, and all dependencies needed for yt’s analysis modules installed. The following dependencies were used to generate the yt documentation during the release of yt 3.2 in 2015.
Rockstar halo finder 0.99.6
ffmpeg 2.7.1 (compiled with libvpx support)
You will also need the full yt suite of yt test data, including the larger datasets that are not used in the answer tests.
You will need to ensure that your testing configuration is properly configured and that all of the yt test data is in the testing directory. See How to Run the Answer Tests for more details on how to set up the testing configuration.
Now that you have everything set up properly, go to the documentation directory and build it using Sphinx:
cd $YT_GIT/doc make html
If all of the dependencies are installed and all of the test data is in the
testing directory, this should churn away for a while (several hours) and
eventually generate a docs build. We suggest setting
suppress_stream_logging = True in your yt configuration (See
The Configuration) to suppress large amounts of debug output from
To clean the docs build, use
Building the Docs (Hybrid)¶
It’s also possible to create a custom Sphinx build that builds a restricted set
of notebooks or scripts. This can be accomplished by editing the Sphinx
conf.py file included in the
source directory at the top level
of the docs. The extensions included in the build are contained in the
extensions list. To disable an extension, simply remove it from the
list. Doing so will raise a warning when Sphinx encounters the directive in the
docs and will prevent Sphinx from evaluating the directive.
As a concrete example, if one wanted to include the
notebook-cell directives, but not the
autosummary directives, one would just need to comment out the lines
that append these extensions to the
extensions list. The resulting docs
build will be significantly quicker since it would avoid executing the lengthy
API autodocumentation as well as a large number of Python script snippets in
the narrative docs.