How to Do a Release¶
Periodically, the yt development community issues new releases. Since yt follows
semantic versioning, the type of release can be read off
from the version number used. Version numbers should follow the scheme
MAJOR.MINOR.PATCH. There are three kinds of possible releases:
These releases are regularly scheduled and will optimally happen approximately once a month. These releases should contain only fixes for bugs discovered in earlier releases and should not contain new features or API changes. Bugfix releases should increment the
PATCHversion number. Bugfix releases should not be generated by merging from the
mainbranch, instead bugfix pull requests should be manually backported. Version
3.2.2is a bugfix release.
These releases happen when new features are deemed ready to be merged into the
stablebranch and should not happen on a regular schedule. Minor releases can also include fixes for bugs if the fix is determined to be too invasive for a bugfix release. Minor releases should not include backwards-incompatible changes and should not change APIs. If an API change is deemed to be necessary, the old API should continue to function but might trigger deprecation warnings. Minor releases should happen by merging the
mainbranch into the
stablebranch. Minor releases should increment the
MINORversion number and reset the
PATCHversion number to zero. Version
3.3.0is a minor release.
These releases happen when the development community decides to make major backwards-incompatible changes. In principle a major version release could include arbitrary changes to the library. Major version releases should only happen after extensive discussion and vetting among the developer and user community. Like minor releases, a major release should happen by merging the
mainbranch into the
stablebranch. Major releases should increment the
MAJORversion number and reset the
PATCHversion numbers to zero. If it ever happens, version
4.0.0will be a major release.
The job of doing a release differs depending on the kind of release. Below, we describe the necessary steps for each kind of release in detail.
Doing a Bugfix Release¶
As described above, bugfix releases are regularly scheduled updates for minor
releases to ensure fixes for bugs make their way out to users in a timely
manner. Since bugfix releases should not include new features, we do not issue
bugfix releases by simply merging from the development
main branch into
stable branch. Instead, we manually cherry-pick bugfixes from the from
main branch onto the
You may find the
pr_backport.py script located in the
scripts folder at
the root of the repository to be helpful. This script uses the github API to
find the list of pull requests made since the last release and prompts the user
to backport each pull request individually. Note that the backport process is
fully manual. The easiest way to do it is to download the diff for the pull
request (the URL for the diff is printed out by the backport script) and then
git apply to apply the patch for the pull request to a local copy of yt
stable branch checked out.
Once you’ve finished backporting push your work to Github. Once you’ve pushed to your fork, you will be able to issue a pull request containing the backported fixes just like any other yt pull request.
Doing a Minor or Major Release¶
This is much simpler than a bugfix release. All that needs to happen is the
main branch must get merged into the
stable branch, and any conflicts
that happen must be resolved, almost always in favor of the state of the code on
Incrementing Version Numbers and Tagging a Release¶
Before creating the tag for the release, you must increment the version numbers
that are hard-coded in a few files in the yt source so that version metadata
for the code is generated correctly. This includes things like
and the version that gets read by the Python Package Index (PyPI) infrastructure.
The paths relative to the root of the repository for the three files that need to be edited are:
releasevariables need to be updated.
VERSIONvariable needs to be updated
__version__variable must be updated.
Once these files have been updated, commit these updates. This is the commit we will tag for the release.
To actually create the tag, issue the following command from the
git tag <tag-name>
<tag-name> follows the project’s naming scheme for tags
yt-3.2.1). Once you are done, you will need to push the
tag to github:
git push origin --tags
This assumes that you have configured the remote
origin to point at the main
yt git repository. If you are doing a minor or major version number release, you
will also need to update back to the development branch and update the
development version numbers in the same files.
Uploading to yt-project.org¶
Before uploading the release to the Python Package Index (pypi.org) we will
first upload the package to yt-project.org. This facilitates building binary
wheels for pypi and binary conda packages on conda-forge before doing the
“official” release. This also ensures that there isn’t a period of time when
pip install yt and end up downloading the source distribution
instead of one of the binary wheels.
To create the source distribution, issue the following command in the root of the yt repository:
$ python setup.py sdist
This will generate a tarball in a
dist/ directory located in the root of the
Access to yt-project.org mediated via SSH login. Please contact one of the
current yt developers for access to the webserver running yt-project.org if you
do not already have it. You will need a copy of your SSH public key so that your
key can be added to the list of authorized keys. Once you login, use
scp to upload a copy of the souce distribution tarball to
https://yt-project.org/sdist, like so:
$ scp dist/yt-3.5.1.tar.gz firstname.lastname@example.org:yt-project.org/sdist
You may find it helpful to set up an ssh config for dickenson to make this command a bit easier to execute.
Updating conda-forge and building wheels¶
Before we finish the release, we need to generate new binary builds by updating yt’s conda-forge feedstock and the yt-wheels repository.
Binary wheels for yt are managed via the
multibuild project. For yt the main
point of access is at https://github.com/yt-project/yt-wheels. Take a look at
the pull requests from the previous few releases to get an idea of what to do,
but briefly you will need to update the multibuild and yt submodules to their
latest state and then commit the changes to the submodules:
$ cd multibuild $ git pull origin devel $ cd ../yt $ git pull origin stable $ cd .. $ git commit -am "updating multibuild and yt submodules"
Next you will need to update the
appveyor.yaml files to
build the latest tag of yt. You may also need to update elsewhere in the file if
yt’s dependencies changed or if yt dropped or added support for a Python
version. To generate new wheels you need to push the changes to GitHub. A good
process to follow is to first submit a pull request to test the changes and make sure
the wheels can be built. Once they pass, you can merge the changes into
and wait for the wheel files to be uploaded to
(note that the wheels will not be uploaded until the changes have been
main). Once the wheels are uploaded, download the
wheel files for the release and copy them to the
dist folder in the yt
repository so that they are sitting next to the source distribution
we created earlier. Here’s a
one-liner to download all of the wheels for the yt 3.6.1 release:
$ wget -r -nd -A 'yt-3.6.1-*whl' https://anaconda.org/multibuild-wheels-staging/yt/files
Uploading to PyPI¶
To actually upload the release to the Python Package Index, you just need to issue the following command:
twine upload dist/*
Please ensure that both the source distribution and binary wheels are present in
dist folder before doing this. Directions on generating binary wheels
are described in the section immediately preceding this one.
You will be prompted for your PyPI credentials and then the package should upload. Note that for this to complete successfully, you will need an account on PyPI and that account will need to be registered as an “owner” or “maintainer” of the yt package. Right now the following people have access to upload packages: Matt Turk, Britton Smith, Nathan Goldbaum, John ZuHone, Kacper Kowalik, and Madicken Munk. The yt package source distribution should be uploaded along with compiled binary wheel packages for various platforms that we support.
Conda-forge packages for yt are managed via the yt feedstock, located at https://github.com/conda-forge/yt-feedstock. When a release is pushed to PyPI a bot should detect a new version and issue a PR to the feedstock with the new version automatically. When this feedstock is updated, make sure that the SHA256 hash of the tarball matches the one you uploaded to dickenson and that the version number matches the one that is being released.
Should you need to update the feedstock manually, you will
need to update the
meta.yaml file located in the
recipe folder in the
root of the feedstock repository. Most likely you will only need to update the
version number and the SHA256 hash of the tarball. If yt’s dependencies change
you may also need to update the recipe. Once you have updated the recipe,
propose a pull request on github and merge it once all builds pass.