Using code shelf#

Downloading code#

To VISA#

When you create an instance on VISA, the notebooks and scripts for your selected instrument are copied to your home directory automatically. You can simply use them from there or copy them to any directory that you work in.

To your computer#

To download notebooks and scripts from Code Shelf, click the corresponding notebook Jupyter button or script Python button on the shelves page.

Make sure to download from the deployment repository and not the a source repository. You can find the latter by clicking the big buttons on the shelves page.

Contributing code#

Setting up#

Pixi#

Code Shelf projects are managed with Pixi and Bibliotekar. See the official documentation for installing Pixi. Bibliotekar is installed automatically when using Pixi.

pre-commit#

Optionally, you can also set up pre-commit to run some checks before committing. This can avoid many common test failures later on.

The simplest option is to install pre-commit hooks with Pixi by running

pixi run pre-commit install

After this, every time you do a git commit, the pre-commit hooks will run automatically.

Note that you need to re-run the above install command if you delete the .pixi directory in your project.

This is the advanced option if you know what you are doing.

You can install pre-commit globally into your system. This way, you do not rely on Pixi to run hooks and you can share the installation between projects.

There are several ways to install pre-commit and you can find instructions online. E.g.,

  • With pixi as a “global package”.

  • With uv as a “tool”.

  • With your system package manager (Linux).

To install pre-commit as a global Pixi package, run

pixi global install [--with pre-commit-uv]

The code in brackets is optional but makes installing environments much faster.

And then install the hooks into your project:

pre-commit install

Adding a file#

To add a new file to Code Shelf, follow these steps:

  1. Copy the file to the repository.

  2. Run automatic tests.

  3. Add tracking metadata to the file.

  4. Commit the file to a new branch.

  5. Push the commit the GitLab.

  6. Create a merge request.

1. Copy the file to the repository#

If not already done, clone the instrument’s source repository. If you do not have write access to the repository, contact a maintainer or create your own fork.

Then copy the file to the notebooks or scripts folder or an appropriate subfolder of these.

2. Run automatic tests#

You can test the notebook or script using

pixi run pytest path/to/file.ipynb

Bear in mind that this needs to have access to any input data files. So you need to ensure that all paths are set up to work on GitLab, VISA, and your local machine. See the dedicated testing page for more information. And don’t hesitate to contact the Data Reduction Team (Scipp) at DMSC for help!

3. Add tracking metadata to the file#

All code submissions must contain a set of tracking metadata that allows users to find out where a notebook or script came from. See the dedicated tracking metadata page for more information.

You can add tracking metadata using

pixi run shelf patch notebooks/path/to/notebook.ipynb
# or
pixi run shelf patch scripts/path/to/script.py

and follow the instructions.

Then, you can check the metadata using

pixi run shelf check

If this produces no output, the metadata is ok.

4. Commit the file to a new branch#

Create a new branch for the file using

git checkout -b branch-name

and commit the file using

git add .
git commit -m "Explain the change"

5. Push the commit the GitLab#

Push the commit to GitLab using

git push

or, if that does not work,

git push origin branch-name

6. Create a merge request#

Create a merge request on GitLab from your new branch to main. If you know someone who would make a good reviewer, please request a review from them.

Once the merge request is open, a pipeline will start that checks the metadata of your code and runs the tests. The code can only be merged if this pipeline succeeds. You can reduce friction by making sure that the tests pass on your machine as described above.

Updating a file#

To update a file in a source repository, follow the same steps as for adding a file. But instead of creating a new file in the folder, replace the old one.

Make sure to check the tracking metadata! You need to increase the version number and (probably) add yourself as an author.