Packaging

Packaging#

Estimated reading time: 23 minutes

Pixi SPEC0

Pixi

注釈

We follow Scientific Python Ecosystem Coordination (SPEC) recommendation for Minimum Supported Dependencies (SPEC 0), and typically support the latest 2-3 distributions of python.

Reference:

  • pyproject.toml

Package management is orchestrated and performed by pixi.

Our pixi environments, features, and tasks are defined within the pyproject.toml manifest.

pixi offers fast, reproducible, cross-platform environment management that enables us to resolve and provision robust, consistent environments blended with packages from both conda and PyPI ecosystems.

We offer several similar collections of pixi environments for each supported distribution of python. Each environment within a collection belongs to the same solve-group i.e., environments within the same solve-group have their dependencies resolved at the same time, which means that all those environments share the exact same dependencies but may also include additional dependencies compatible within the solve-group.

Tip

We've adopted the following format convention to help clarify each type of pixi component:

  • environment-name

  • {feature-name}

  • task-name

  • solve-group-name

Pixi Solve-Groups#

Reference:

  • pyproject.toml

Our pixi environments are organized into collections by solve-group. See the [tool.pixi.environments] table defined in the pyproject.toml manifest.

The primary solve-group is called default and will always contain the latest supported distribution of python. This is denoted by the {py} feature which represents the latest distribution of python as recommended by SPEC 0.

Table 13: Pixi default Solve-Group#

Environment

Features

説明

default

{default}, {py}

This environment contains the core dependencies of geovista along with the latest supported distribution of python. See the [tool.pixi.dependencies] table in the pyproject.toml manifest.

devs

{default}, {devs}, {py}

As per the default environment plus additional development dependencies.

docs

{default}, {devs}, {docs}, {py}

As per the devs environment plus additional documentation dependencies.

geovista

{default}, {devs}, {docs}, {geovista}, {py}, {test}

This environment is the union of all environments in the default solve-group.

test

{default}, {devs}, {py}, {test}

As per the devs environment plus additional test dependencies.

参考

The pixi info command provides key information about the system, environments, solve-groups, dependencies and tasks in the workspace.

Given the above default solve-group we have the following inheritance relationship between its environments:

        ---
config:
   theme: base
   themeVariables:
      lineColor: "#F8B229"
      nodeBorder: "#2569E9"
      primaryColor: "#DCE7FC"
      secondaryColor: "#F4DDFF"
---
graph LR
   default(["`**default**`"])
   devs(["`**devs**`"])
   docs(["`**docs**`"])
   test(["`**test**`"])
   geovista(["`**geovista**`"])
   default -. "`*core*`" .-> devs
   devs ---> docs
   devs ---> test
   docs ---> geovista
   test ---> geovista

Fig. 7: Environment Hierarchy for default Solve-Group#

注釈

As its name suggests, the default environment is used by default when no explicit environment name is specified to pixi. e.g., the pixi shell command starts a shell with the default environment activated, whereas the pixi shell -e devs command will activate the devs environment instead.

Several secondary solve-groups are available, each of which are named after the distribution of python that they support e.g., py313.

The number of different python distribution solve-groups on offer at any given time is dictated by SPEC 0.

The generic py3xx solve-group is used here as a convenience to represent each of the different python distribution solve-groups available, all of which are identical in structure and content apart from the specific distribution of python that they support.

Also note that each py3xx solve-group will always contain the similarly named {py3xx} feature e.g., the py313 solve-group will contain the {py313} feature which in turn defines the python=3.13 dependency to be included in all py313 environments.

Table 14: Pixi py3xx Solve-Groups#

Environment

Feature

説明

devs-py3xx

{default}, {devs}, {py3xx}

As per the py3xx environment plus additional development dependencies.

docs-py3xx

{default}, {devs}, {docs} , {py3xx}

As per the devs-py3xx environment plus additional documentation dependencies.

geovista-py3xx

{default}, {devs}, {docs}, {geovista}, {py3xx}, {test}

This environment is the union of all environments in the same py3xx solve-group.

py3xx

{default}, {py3xx}

This environment contains the core dependencies of geovista along with python distribution py3xx e.g., py313.

test-py3xx

{default}, {devs}, {py3xx}, {test}

As per the devs-py3xx environment plus additional test dependencies.

注意

The generic py3xx solve-group does not exist, neither does the generic {py3xx} feature.

Given the above py3xx solve-group we have the following inheritance relationship between its environments:

        ---
config:
   theme: base
   themeVariables:
      lineColor: "#F8B229"
      nodeBorder: "#2569E9"
      primaryColor: "#DCE7FC"
      secondaryColor: "#F4DDFF"
---
graph LR
   py3xx(["`**py3xx**`"])
   devs(["`**devs-py3xx**`"])
   docs(["`**docs-py3xx**`"])
   test(["`**test-py3xx**`"])
   geovista(["`**geovista-py3xx**`"])
   py3xx --> |"`*core*`"| devs
   devs --> docs
   devs --> test
   docs --> geovista
   test --> geovista

Fig. 8: Environment Hierarchy for py3xx Solve-Group#

Pixi Features#

Reference:

  • pyproject.toml

A pixi environment is defined by combining one or more features. For further details see this pixi tutorial on how to create and use features in a multi-environment scenario.

Our features are defined under the [tool.pixi.feature] tables in the pyproject.toml manifest.

Each named feature table e.g., [tool.pixi.feature.devs], may contain various fields, such as dependencies, pypi-dependencies, pypi-options, system-requirements, activation, platforms, channels, channel-priority, target and tasks.

Table 15: Pixi Features#

Feature

TOML Table

説明

{default}

[tool.pixi.dependencies]

This feature is used to define the core dependencies and tasks of geovista. Note that the {default} feature is automatically included in all environments by pixi.

{devs}

[tool.pixi.feature.devs]

This feature is used to define the development dependencies and pypi-dependencies. Note that an editable install of geovista is performed by the {devs} feature.

{docs}

[tool.pixi.feature.docs]

This feature is used to define the documentation dependencies, pypi-dependencies and tasks.

{geovista}

[tool.pixi.feature.geovista]

This feature is only used to define tasks.

{py3xx}

[tools.pixi.feature.py3xx]

This feature is used to explicitly define the distribution of python supported e.g., py313. Note that the dependencies of this feature additionally includes the pip package. The number of different python distributions supported is governed by SPEC 0.

{test}

[tool.pixi.feature.test]

This feature is used to define the test dependencies and tasks.

Pixi Tasks#

Reference:

  • pyproject.toml

A pixi task is a custom cross-platform workflow command that is defined as part of a feature within an environment.

All our tasks are defined within the pyproject.toml manifest.

Tasks expose a convenient and easy to use entry-point to commands that allow streamlined and automated custom workflows.

The following tasks are defined for each of our features:

Table 16: Pixi Tasks#

Feature

Task

説明

{default}

download

Download and cache offline assets.

This task calls the geovista download command. Provide optional argument all, clean, doc-images, dry-run, images, list, natural-earth, operational, pantry, rasters, unit-images or verify. Defaults to all e.g.,

$ pixi run download operational

{docs}

clean

Purge all sphinx-autoapi, sphinx-gallery sphinx-tags, carousel, and other sphinx-build artifacts e.g.,

$ pixi run clean

This task is an alias for the make clean command.

clean-all

Perform both the clean and clean-cache tasks e.g.,

$ pixi run clean-all

This task is an alias for the make clean-all command.

clean-cache

Purge the myst-nb Jupyter cache. See myst-nb configuration for further details e.g.,

$ pixi run clean-cache

This task is an alias for the make clean-cache command.

doctest

Execute sphinx.ext.doctest to test code snippets within the documentation i.e.,

$ pixi run doctest

Note that the clean task is called prior to running this task.

This task is an alias for the make doctest command.

make

Build the documentation.

Provide optional argument html, html-docstring, html-docstring-inline, html-gallery, html-inline or html-noplot. Defaults to html-noplot e.g.,

$ pixi run make html-gallery

Note that the clean task is called prior to running this task.

This task is an alias for the make command.

serve-html

Build the documentation and start a local HTTP server on port 11000 to view the rendered documentation. This is necessary in order to support interactive scenes.

Note that the clean and make tasks are called prior to running this task.

Defaults to passing html-noplot to the make task. Override this behaviour by providing an alternative optional argument as per the make task e.g.,

$ pixi run serve-html html

This task is an alias for the make serve-html command.

{geovista}

tests-doc

Perform documentation image tests of pyvista-plot directive static scenes e.g.,

$ pixi run tests-doc

This task calls pytest --doc_mode to perform the documentation image tests using the pytest-pyvista plugin. Refer to the [tool.pytest.ini_options] table entry in the pyproject.toml manifest for default configuration options.

Note that the tests-clean, download doc-images and make html-docstring-inline tasks are called prior to running this task.

This task is only available in the geovista and geovista-py3xx environments.

注釈

Failed image tests are captured via the pytest-pyvista plugin option --failed_image_dir within the test_images_failed directory for analysis and investigation.

{test}

tests-clean

Purge both the documentation and unit test image caches, along with any images generated from previous test sessions e.g.,

$ pixi run tests-clean

tests-unit

Perform the unit tests.

This task calls the pytest command. Defaults to executing all unit tests discoverable from the geovista root directory.

Accepts a valid pytest marker expression as an optional argument. Refer to the [tool.pytest.ini_options] table entry in the pyproject.toml manifest for configured markers e.g.,

$ pixi run tests-unit "not image"

Note that the tests-clean task is called prior to running this task.

注釈

Failed Image Tests are captured via the pytest-pyvista plugin option --failed_image_dir (see Image Tests Generation) within the test_images_failed directory for analysis and investigation.

参考

The pixi task list command describes each task available within the workspace.

Whereas the pixi task list --summary command enumerates the tasks available per environment.

Continuous Integration#

Reference:

  • .github/workflows

The following packaging workflows are available:

Table 17: Packaging Workflows#

Workflow

説明

ci-locks

The ci-locks.yml cron based GHA workflow regularly schedules pixi to refresh the pixi.lock file based on the pyproject.toml manifest and the latest package versions available from the conda and PyPI ecosystems.

Only the latest python variant of the geovista-py3xx environment e.g., geovista-py313, will be exported to a conda explicit specification file (.txt) and also converted to an equivalent explicit environment YAML file (.yml). These resources are available in the requirements/locks directory.

Additionally, the geovista-py3xx environment will be exported to the conda environment YAML file requirements/geovista.yml containing only the top-level dependencies of that environment, as defined by its features.

ci-wheels

The ci-wheels.yml GHA workflow builds, tests and publishes the source distribution (sdist) and binary wheel of geovista.

We have adopted PyPI Trusted Publishing with OpenID Connect (OIDC) for secure deployments of assets.

Note that the sdist and wheel assets are automatically published to Test PyPI for each pull-request merged to the main branch, and PyPI for each tag release.

Python Package Index#

Reference:

  • pyproject.toml

  • requirements

PyPI package dependencies are configured in the pyproject.toml manifest under the [tool.setuptools.dynamic] and [tool.setuptools.dynamic.optional-dependencies] table entries.

Version management of our PyPI dependencies is orchestrated by Dependabot. Refer to the package-ecosystem: "pip" key entry in the .github/dependabot.yml file for further details.

The Dependabot service is regularly scheduled to check the versions of our dependencies and automatically increment them using maximum capping pins via a pull-request in response to the latest available package updates within the PyPI ecosystem.

注意

Updates to the PyPI maximum capping pins may require to be manually reflected in the pyproject.toml manifest for similar pixi managed conda dependencies.

Track native Dependabot support for pixi in dependabot/dependabot-core issue#2227 🤞

On this page