Getting Involved

There are lots of ways to get involved with yt, as a community and as a technical system – not all of them just contributing code, but also participating in the community, helping us with designing the websites, adding documentation, and sharing your scripts with others.

Coding is only one way to be involved!

Communication Channels

There are three main communication channels for yt:

  • Many yt developers participate in the yt Slack community. Slack is a free chat service that many teams use to organize their work. You can get an invite to yt’s Slack organization by clicking the “Join us @ Slack” button on this page: https://yt-project.org/community.html

  • yt-users is a relatively high-traffic mailing list where people are encouraged to ask questions about the code, figure things out and so on.

  • yt-dev is a much lower-traffic mailing list designed to focus on discussions of improvements to the code, ideas about planning, development issues, and so on.

The easiest way to get involved with yt is to read the mailing lists, hang out in IRC or slack chat, and participate. If someone asks a question you know the answer to (or have your own question about!) write back and answer it.

If you have an idea about something, suggest it! We not only welcome participation, we encourage it.

Documentation

The yt documentation is constantly being updated, and it is a task we would very much appreciate assistance with. Whether that is adding a section, updating an outdated section, contributing typo or grammatical fixes, adding a FAQ, or increasing coverage of functionality, it would be very helpful if you wanted to help out.

The easiest way to help out is to fork the main yt repository and make changes in it to contribute back to the yt-project. A fork is a copy of a repository; in this case the fork will live in the space under your username on github, rather than the yt-project. If you have never made a fork of a repository on github, or are unfamiliar with this process, here is a short article about how to do so: https://help.github.com/en/github/getting-started-with-github/fork-a-repo . The documentation for yt lives in the doc directory in the root of the yt git repository. To make a contribution to the yt documentation you will make your changes in your own fork of yt. When you are done, issue a pull request through the website for your new fork, and we can comment back and forth and eventually accept your changes. See Making and Sharing Changes for more information about contributing your changes to yt on GitHub.

Technical Contributions

Contributing code is another excellent way to participate – whether it’s bug fixes, new features, analysis modules, or a new code frontend. See Creating A New Code Frontend for more details.

The process is pretty simple: fork on GitHub, make changes, issue a pull request. We can then go back and forth with comments in the pull request, but usually we end up accepting.

For more information, see How to Develop yt, where we spell out how to get up and running with a development environment, how to commit, and how to use GitHub. When you’re ready to share your changes with the community, refer to Making and Sharing Changes to see how to contribute them back upstream.

Online Presence

Some of these fall under the other items, but if you’d like to help out with the website or any of the other ways yt is presented online, please feel free! Almost everything is kept in git repositories on GitHub, and it is very easy to fork and contribute back changes.

Please feel free to dig in and contribute changes.

Word of Mouth

If you’re using yt and it has increased your productivity, please feel encouraged to share that information. Cite our paper, tell your colleagues, and just spread word of mouth. By telling people about your successes, you’ll help bring more eyes and hands to the table – in this manner, by increasing participation, collaboration, and simply spreading the limits of what the code is asked to do, we hope to help scale the utility and capability of yt with the community size.

Feel free to blog about, tweet about and talk about what you are up to!

Long-Term Projects

There are some out-there ideas that have been bandied about for the future directions of yt – stuff like fun new types of visualization, remapping of coordinates, new ways of accessing data, and even new APIs to make life easier.

yt is an ambitious project. Let’s be ambitious together!

yt Community Code of Conduct

The community of participants in open source Scientific projects is made up of members from around the globe with a diverse set of skills, personalities, and experiences. It is through these differences that our community experiences success and continued growth. We expect everyone in our community to follow these guidelines when interacting with others both inside and outside of our community. Our goal is to keep ours a positive, inclusive, successful, and growing community.

As members of the community,

  • We pledge to treat all people with respect and provide a harassment- and bullying-free environment, regardless of sex, sexual orientation and/or gender identity, disability, physical appearance, body size, race, nationality, ethnicity, and religion. In particular, sexual language and imagery, sexist, racist, or otherwise exclusionary jokes are not appropriate.

  • We pledge to respect the work of others by recognizing acknowledgment/citation requests of original authors. As authors, we pledge to be explicit about how we want our own work to be cited or acknowledged.

  • We pledge to welcome those interested in joining the community, and realize that including people with a variety of opinions and backgrounds will only serve to enrich our community. In particular, discussions relating to pros/cons of various technologies, programming languages, and so on are welcome, but these should be done with respect, taking proactive measure to ensure that all participants are heard and feel confident that they can freely express their opinions.

  • We pledge to welcome questions and answer them respectfully, paying particular attention to those new to the community. We pledge to provide respectful criticisms and feedback in forums, especially in discussion threads resulting from code contributions.

  • We pledge to be conscientious of the perceptions of the wider community and to respond to criticism respectfully. We will strive to model behaviors that encourage productive debate and disagreement, both within our community and where we are criticized. We will treat those outside our community with the same respect as people within our community.

  • We pledge to help the entire community follow the code of conduct, and to not remain silent when we see violations of the code of conduct. We will take action when members of our community violate this code such as contacting confidential@yt-project.org (all emails sent to this address will be treated with the strictest confidence) or talking privately with the person.

This code of conduct applies to all community situations online and offline, including mailing lists, forums, social media, conferences, meetings, associated social events, and one-to-one interactions.

The yt Community Code of Conduct was adapted from the Astropy Community Code of Conduct, which was partially inspired by the PSF code of conduct.

How to Develop yt

yt is a community project!

We are very happy to accept patches, features, and bugfixes from any member of the community! yt is developed using git, primarily because it enables very easy and straightforward submission of revisions. We’re eager to hear from you, and if you are developing yt, we encourage you to subscribe to the developer mailing list. Please feel free to hack around, commit changes, and send them upstream.

Note

If you already know how to use the git version control system and are comfortable with handling it yourself, the quickest way to contribute to yt is to fork us on GitHub, make your changes, push the changes to your fork and issue a pull request. The rest of this document is just an explanation of how to do that.

See Coding Style Guide for more information about coding style in yt and Docstrings for an example docstring. Please read them before hacking on the codebase, and feel free to email any of the mailing lists for help with the codebase.

Keep in touch, and happy hacking!

Open Issues

If you’re interested in participating in yt development, take a look at the issue tracker on GitHub. You can search by labels, indicating estimated level of difficulty or category, to find issues that you would like to contribute to. Good first issues are marked with a label of new contributor friendly. While we try to triage the issue tracker regularly to assign appropriate labels to every issue, it may be the case that issues not marked as new contributor friendly are actually suitable for new contributors.

Here are some predefined issue searches that might be useful:

Submitting Changes

We provide a brief introduction to submitting changes here. yt thrives on the strength of its communities (https://arxiv.org/abs/1301.7064 has further discussion) and we encourage contributions from any user. While we do not discuss version control, git, or the advanced usage of GitHub in detail here, we do provide an outline of how to submit changes and we are happy to provide further assistance or guidance.

Licensing

yt is licensed under the BSD 3-clause license. Versions previous to yt-2.6 were released under the GPLv3.

All contributed code must be BSD-compatible. If you’d rather not license in this manner, but still want to contribute, please consider creating an external package, which we’ll happily link to.

How To Get The Source Code For Editing

yt is hosted on GitHub, and you can see all of the yt repositories at https://github.com/yt-project/. To fetch and modify source code, make sure you have followed the steps above for bootstrapping your development (to assure you have a GitHub account, etc.).

In order to modify the source code for yt, we ask that you make a “fork” of the main yt repository on GitHub. A fork is simply an exact copy of the main repository (along with its history) that you will now own and can make modifications as you please. You can create a personal fork by visiting the yt GitHub webpage at https://github.com/yt-project/yt/ . After logging in, you should see an option near the top right labeled “fork”. You now have a forked copy of the yt repository for your own personal modification.

This forked copy exists on the GitHub repository, so in order to access it locally you must clone it onto your machine from the command line:

$ git clone https://github.com/<USER>/yt ./yt-git

This downloads that new forked repository to your local machine, so that you can access it, read it, make modifications, etc. It will put the repository in a local directory of the same name as the repository in the current working directory.

$ cd yt-git

Verify that you are on the main branch of yt by running:

$ git branch

You can see any past state of the code by using the git log command. For example, the following command would show you the last 5 revisions (modifications to the code) that were submitted to that repository.

$ git log -n 5

Using the revision specifier (the number or hash identifier next to each changeset), you can update the local repository to any past state of the code (a previous changeset or version) by executing the command:

$ git checkout revision_specifier

You can always return to the most recent version of the code by executing the same command as above with the most recent revision specifier in the repository. However, using git log when you’re checked out to an older revision specifier will not show more recent changes to the repository. An alternative option is to use checkout on a branch. In yt the main branch is our primary development branch, so checking out main should return you to the tip (or most up-to-date revision specifier) on the main branch.

$ git checkout main

Lastly, if you want to use this new downloaded version of your yt repository as the active version of yt on your computer (i.e. the one which is executed when you run yt from the command line or the one that is loaded when you do import yt), then you must “activate” by building yt from source as described in Building from source.

How To Read The Source Code

The root directory of the yt git repository contains a number of subdirectories with different components of the code. Most of the yt source code is contained in the yt subdirectory. This directory itself contains the following subdirectories:

frontends

This is where interfaces to codes are created. Within each subdirectory of yt/frontends/ there must exist the following files, even if empty:

  • data_structures.py, where subclasses of AMRGridPatch, Dataset and AMRHierarchy are defined.

  • io.py, where a subclass of IOHandler is defined.

  • fields.py, where fields we expect to find in datasets are defined

  • misc.py, where any miscellaneous functions or classes are defined.

  • definitions.py, where any definitions specific to the frontend are defined. (i.e., header formats, etc.)

fields

This is where all of the derived fields that ship with yt are defined.

geometry

This is where geometric helpler routines are defined. Handlers for grid and oct data, as well as helpers for coordinate transformations can be found here.

visualization

This is where all visualization modules are stored. This includes plot collections, the volume rendering interface, and pixelization frontends.

data_objects

All objects that handle data, processed or unprocessed, not explicitly defined as visualization are located in here. This includes the base classes for data regions, covering grids, time series, and so on. This also includes derived fields and derived quantities.

units

This used to be where all the unit-handling code resided, but as of now it’s mostly just a thin wrapper around unyt.

utilities

All broadly useful code that doesn’t clearly fit in one of the other categories goes here.

If you’re looking for a specific file or function in the yt source code, use the unix find command:

$ find <DIRECTORY_TREE_TO_SEARCH> -name '<FILENAME>'

The above command will find the FILENAME in any subdirectory in the DIRECTORY_TREE_TO_SEARCH. Alternatively, if you’re looking for a function call or a keyword in an unknown file in a directory tree, try:

$ grep -R <KEYWORD_TO_FIND> <DIRECTORY_TREE_TO_SEARCH>

This can be very useful for tracking down functions in the yt source.

Building yt

If you have made changes to any C or Cython (.pyx) modules, you have to rebuild yt before your changes are usable. See Building from source.

Requirements for Code Submission

Modifications to the code typically fall into one of three categories, each of which have different requirements for acceptance into the code base. These requirements are in place for a few reasons – to make sure that the code is maintainable, testable, and that we can easily include information about changes in changelogs during the release procedure. (See YTEP-0008 for more detail.)

For all types of contributions, it is required that all tests pass, or that all non-passing tests are specifically accounted for.

  • New Features

  • Extension or Breakage of API in Existing Features

  • Bug fixes

    • Unit test is encouraged, to ensure breakage does not happen again in the future. (See Testing)

    • At a minimum, a minimal, self-contained example demonstrating the bug should because included in the body of the Pull Request, or as part of an indepedent issue.

When submitting, you will be asked to make sure that your changes meet all of these requirements. They are pretty easy to meet, and we’re also happy to help out with them. See Coding Style Guide for how to easily conform to our style guide.

How to Use git with yt

If you’re new to git, the following resource is pretty great for learning the ins and outs:

There also exist a number of step-by-step git tutorials to help you get used to version controlling files with git. Here are a few resources that you may find helpful:

The commands that are essential for using git include:

  • git <command> --help which provides help for any git command. For example, you can learn more about the log command by doing git log --help.

  • git add <paths> which stages changes to the specified paths for subsequent committing (see below).

  • git commit which commits staged changes (stage using git add as above) in the working directory to the repository, creating a new “revision.”

  • git merge <branch> which merges the revisions from the specified branch into the current branch, creating a union of their lines of development. This updates the working directory.

  • git pull <remote> <branch> which pulls revisions from the specified branch of the specified remote repository into the current local branch. Equivalent to git fetch <remote> and then git merge <remote>/<branch>. This updates the working directory.

  • git push <remote> which sends revisions on local branches to matching branches on the specified remote. git push <remote> <branch> will only push changes for the specified branch.

  • git log which shows a log of all revisions on the current branch. There are many options you can pass to git log to get additional information. One example is git log --oneline --decorate --graph --all.

We are happy to answer questions about git use on our IRC, slack chat or on the mailing list to walk you through any troubles you might have. Here are some general suggestions for using git with yt:

  • Although not necessary, a common development work flow is to create a local named branch other than main to address a feature request or bugfix. If the dev work addresses a specific yt GitHub issue, you may include that issue number in the branch name. For example, if you want to work on issue number X regarding a cool new slice plot feature, you might name the branch: cool_new_plot_feature_X. When you’re ready to share your work, push your feature branch to your remote and create a pull request to the main branch of the yt-project’s repository.

  • When contributing changes, you might be asked to make a handful of modifications to your source code. We’ll work through how to do this with you, and try to make it as painless as possible.

  • Your test may fail automated style checks. See Coding Style Guide for more information about automatically verifying your code style.

  • You should only need one fork. To keep it in sync, you can sync from the website. See Making and Sharing Changes for a description of the basic workflow and Working with Multiple GitHub Pull Requests for a discussion about what to do when you want to have multiple open pull requests at the same time.

  • If you run into any troubles, stop by IRC (see Go on Slack to ask a question), Slack, or the mailing list.

Making and Sharing Changes

The simplest way to submit changes to yt is to do the following:

  • Build yt from the git repository

  • Navigate to the root of the yt repository

  • Make some changes and commit them

  • Fork the yt repository on GitHub

  • Push the changesets to your fork

  • Issue a pull request.

Here’s a more detailed flowchart of how to submit changes.

  1. Fork yt on GitHub. (This step only has to be done once.) You can do this at: https://github.com/yt-project/yt/fork.

  2. Follow Building from source for instructions on how to build yt from the git repository. (Below, in How To Read The Source Code, we describe how to find items of interest.) If you have already forked the repository then you can clone your fork locally:

    git clone https://github.com/<USER>/yt ./yt-git
    

    This will create a local clone of your fork of yt in a folder named yt-git.

  3. Edit the source file you are interested in and test your changes. (See Testing for more information.)

  4. Create a uniquely named branch to track your work. For example: git checkout -b my-first-pull-request

  5. Stage your changes using git add <paths>. This command take an argument which is a series of filenames whose changes you want to commit. After staging, execute git commit -m "<Commit description>. Addresses Issue #X". Note that supplying an actual GitHub issue # in place of X will cause your commit to appear in the issue tracker after pushing to your remote. This can be very helpful for others who are interested in what work is being done in connection to that issue.

  6. Remember that this is a large development effort and to keep the code accessible to everyone, good documentation is a must. Add in source code comments for what you are doing. Add in docstrings if you are adding a new function or class or keyword to a function. Add documentation to the appropriate section of the online docs so that people other than yourself know how to use your new code.

  7. If your changes include new functionality or cover an untested area of the code, add a test. (See Testing for more information.) Commit these changes as well.

  8. Add your remote repository with a unique name identifier. It can be anything but it is conventional to call it origin. You can see names and URLs of all the remotes you currently have configured with:

    git remote -v
    

    If you already have an origin remote, you can set it to your fork with:

    git remote set-url origin https://github.com/<USER>/yt
    

    If you do not have an origin remote you will need to add it:

    git remote add origin https://github.com/<USER>/yt
    

    In addition, it is also useful to add a remote for the main yt repository. By convention we name this remote upstream:

    git remote add upstream https://github.com/yt-project/yt
    

    Note that if you forked the yt repository on GitHub and then cloned from there you will not need to add the origin remote.

  9. Push your changes to your remote fork using the unique identifier you just created and the command:

    git push origin my-first-pull-request
    

    Where you should substitute the name of the feature branch you are working on for my-first-pull-request.

    Note

    Note that the above approach uses HTTPS as the transfer protocol between your machine and GitHub. If you prefer to use SSH - or perhaps you’re behind a proxy that doesn’t play well with SSL via HTTPS - you may want to set up an SSH key on GitHub. Then, you use the syntax ssh://git@github.com/<USER>/yt, or equivalent, in place of https://github.com/<USER>/yt in git commands. For consistency, all commands we list in this document will use the HTTPS protocol.

  10. Issue a pull request at https://github.com/yt-project/yt/pull/new/main A pull request is essentially just asking people to review and accept the modifications you have made to your personal version of the code.

During the course of your pull request you may be asked to make changes. These changes may be related to style issues, correctness issues, or requesting tests. The process for responding to pull request code review is relatively straightforward.

  1. Make requested changes, or leave a comment indicating why you don’t think they should be made.

  2. Commit those changes to your local repository.

  3. Push the changes to your fork:

    git push origin my-first-pull-request
    
  4. Your pull request will be automatically updated.

Once your pull request is merged, sync up with the main yt repository by pulling from the upstream remote:

git checkout main
git pull upstream main

You might also want to sync your fork of yt on GitHub:

# sync my fork of yt with upstream
git push origin main

And delete the branch for the merged pull request:

# delete branch for merged pull request
git branch -d my-first-pull-request
git push origin --delete my-first-pull-request

These commands are optional but are nice for keeping your branch list manageable. You can also delete the branch on your fork of yt on GitHub by clicking the “delete branch” button on the page for the merged pull request on GitHub.

Working with Multiple GitHub Pull Requests

Dealing with multiple pull requests on GitHub is straightforward. Development on one feature should be isolated in one named branch, say feature_1 while development of another feature should be in another named branch, say feature_2. A push to remote feature_1 will automatically update any active PR for which feature_1 is a pointer to the HEAD commit. A push to feature_1 will not update any pull requests involving feature_2.

Coding Style Guide

Automatically checking and fixing code style

We use the pre-commit framework to validate and automatically fix code styling. It is recommended (though not required) that you install pre-commit on your machine (see their documentation) and, from the top level of the repo, run

$ pre-commit install

So that our hooks will run and update your changes on every commit. If you do not want to/are unable to configure pre-commit on your machine, note that after opening a pull request, it will still be run as a static checker as part of our CI. Some hooks also come with auto-fixing capabilities, which you can trigger manually in a PR by commenting pre-commit.ci autofix (see ` <https://pre-commit.ci/#features>`_).

We use a combination of black, ruff and cython-lint. See .pre-commit-config.yaml and pyproject.toml for the complete configuration details.

Note that formatters should not be run directly on the command line as, for instance

$ black yt

But it can still be done as

$ pre-commit run black --all-files

The reason is that you may have a specific version of black installed which can produce different results, while the one that’s installed with pre-commit is guaranteed to be in sync with the rest of contributors.

Below are a list of additional guidelines for coding in yt, that are not automatically enforced.

Source code style guide

  • In general, follow PEP-8 guidelines. https://www.python.org/dev/peps/pep-0008/

  • Classes are ConjoinedCapitals, methods and functions are lowercase_with_underscores.

  • Do not use nested classes unless you have a very good reason to, such as requiring a namespace or class-definition modification. Classes should live at the top level. __metaclass__ is exempt from this.

  • Avoid copying memory when possible. For example, don’t do a = a.reshape(3, 4) when a.shape = (3, 4) will do, and a = a * 3 should be np.multiply(a, 3, a).

  • In general, avoid all double-underscore method names: __something is usually unnecessary.

  • When writing a subclass, use the super built-in to access the super class, rather than explicitly. Ex: super().__init__() rather than SpecialGrid.__init__().

  • Docstrings should describe input, output, behavior, and any state changes that occur on an object. See Docstrings below for a fiducial example of a docstring.

  • Unless there is a good reason not to (e.g., to avoid circular imports), imports should happen at the top of the file.

  • If you are comparing with a numpy boolean array, just refer to the array. Ex: do np.all(array) instead of np.all(array == True).

  • Only declare local variables if they will be used later. If you do not use the return value of a function, do not store it in a variable.

API Style Guide

  • Internally, only import from source files directly – instead of:

    from yt.visualization.api import ProjectionPlot

    do:

    from yt.visualization.plot_window import ProjectionPlot

  • Import symbols from the module where they are defined, avoid transitive imports.

  • Import standard library modules, functions, and classes from builtins, do not import them from other yt files.

  • Numpy is to be imported as np.

  • Do not use too many keyword arguments. If you have a lot of keyword arguments, then you are doing too much in __init__ and not enough via parameter setting.

  • Don’t create a new class to replicate the functionality of an old class – replace the old class. Too many options makes for a confusing user experience.

  • Parameter files external to yt are a last resort.

  • The usage of the **kwargs construction should be avoided. If they cannot be avoided, they must be documented, even if they are only to be passed on to a nested function.

Docstrings

The following is an example docstring. You can use it as a template for docstrings in your code and as a guide for how we expect docstrings to look and the level of detail we are looking for. Note that we use NumPy style docstrings written in Sphinx restructured text format.

r"""A one-line summary that does not use variable names or the
function name.

Several sentences providing an extended description. Refer to
variables using back-ticks, e.g. ``var``.

Parameters
----------
var1 : array_like
    Array_like means all those objects -- lists, nested lists, etc. --
    that can be converted to an array.  We can also refer to
    variables like ``var1``.
var2 : int
    The type above can either refer to an actual Python type
    (e.g. ``int``), or describe the type of the variable in more
    detail, e.g. ``(N,) ndarray`` or ``array_like``.
Long_variable_name : {'hi', 'ho'}, optional
    Choices in brackets, default first when optional.

Returns
-------
describe : type
    Explanation
output : type
    Explanation
tuple : type
    Explanation
items : type
    even more explaining

Other Parameters
----------------
only_seldom_used_keywords : type
    Explanation
common_parameters_listed_above : type
    Explanation

Raises
------
BadException
    Because you shouldn't have done that.

See Also
--------
otherfunc : relationship (optional)
newfunc : Relationship (optional), which could be fairly long, in which
          case the line wraps here.
thirdfunc, fourthfunc, fifthfunc

Notes
-----
Notes about the implementation algorithm (if needed).

This can have multiple paragraphs.

You may include some math:

.. math:: X(e^{j\omega } ) = x(n)e^{ - j\omega n}

And even use a greek symbol like :math:`omega` inline.

References
----------
Cite the relevant literature, e.g. [1]_.  You may also cite these
references in the notes section above.

.. [1] O. McNoleg, "The integration of GIS, remote sensing,
   expert systems and adaptive co-kriging for environmental habitat
   modelling of the Highland Haggis using object-oriented, fuzzy-logic
   and neural-network techniques," Computers & Geosciences, vol. 22,
   pp. 585-588, 1996.

Examples
--------
These are written in doctest format, and should illustrate how to
use the function.  Use the variables 'ds' for the dataset, 'pc' for
a plot collection, 'c' for a center, and 'L' for a vector.

>>> a = [1, 2, 3]
>>> print([x + 3 for x in a])
[4, 5, 6]
>>> print("a\n\nb")
a

b
"""

Variable Names and Enzo-isms

Avoid Enzo-isms. This includes but is not limited to:

  • Hard-coding parameter names that are the same as those in Enzo. The following translation table should be of some help. Note that the parameters are now properties on a Dataset subclass: you access them like ds.refine_by .

    • RefineBy `` => `` refine_by

    • TopGridRank `` => `` dimensionality

    • TopGridDimensions `` => `` domain_dimensions

    • InitialTime `` => `` current_time

    • DomainLeftEdge `` => `` domain_left_edge

    • DomainRightEdge `` => `` domain_right_edge

    • CurrentTimeIdentifier `` => `` unique_identifier

    • CosmologyCurrentRedshift `` => `` current_redshift

    • ComovingCoordinates `` => `` cosmological_simulation

    • CosmologyOmegaMatterNow `` => `` omega_matter

    • CosmologyOmegaLambdaNow `` => `` omega_lambda

    • CosmologyHubbleConstantNow `` => `` hubble_constant

  • Do not assume that the domain runs from 0 .. 1. This is not true everywhere.

  • Variable names should be short but descriptive.

  • No globals!