Testing

Estimated reading time: 22 minutes

Here we provide some high-level guidelines for contributors on how-to test geovista with pytest, along with a summary of the automated quality assurance infrastructure.

注釈

The unit tests are not bundled within the geovista package distributed on conda-forge or PyPI.

Continuous Integration

Reference:

• .github/workflows

• pyproject.toml

• requirements

• tests

注釈

We follow Scientific Python Ecosystem Coordination (SPEC) recommendation for Minimum Supported Dependencies (SPEC 0), and typically support the latest 2-3 distributions of python.

The following testing workflows are available:

Table 18: Testing Workflows

Workflow	説明
	The ci-tests.yml Action executes both the Image Tests and Unit Tests for each SPEC 0 supported distribution of python. A codecov report is generated only for the latest supported distribution of python. Also see the testing Pixi Workflow tests-unit task. 注釈 Failed Image Tests are captured via the pytest-pyvista plugin option --failed_image_dir (see Generation), and uploaded and archived as a Workflow Artifact for the CI job. The failed unit test images may then be downloaded for analysis and investigation.
	The ci-tests-lock.yml cron based Action regularly schedules the execution of both the Image Tests and Unit Tests for the latest SPEC 0 supported distribution of python. Prior to running the tests pixi performs an ephemeral refresh of the pixi.lock file based on the pyproject.toml manifest and the latest package versions available from the conda and PyPI ecosystems. The purpose of this Action is to preemptively test against the latest resolved python variant of the geovista-py3xx environment to uncover and highlight any issues prior to refreshing the pixi.lock file with the scheduled ci-locks.yml Action. See the packaging Continuous Integration section for further details. Any subsequent test failures will result in a bespoke Issue being raised for contributors to investigate referencing the Action run-id of the failed job.
	The ci-tests-pypi.yml cron based Action regularly schedules the execution of both the Image Tests and Unit Tests for the latest SPEC 0 supported distribution of python. Prior to running the tests pip installs the latest package dependencies from the PyPI ecosystem. See the packaging Python Package Index section for further details. The purpose of this Action is to preemptively test against the latest resolved PyPI environment to uncover and highlight any issues prior to the scheduled dependabot.yml orchestrated refresh of our PyPI package dependencies. Any subsequent test failures will result in a bespoke Issue being raised for contributors to investigate referencing the Action run-id of the failed job. 注意 pyvista performs a pull-request integration test against the main branch of geovista using the PyPI ecosystem. This Action provides a critical early warning of potential upstream issues that should be resolved as a priority.

Image Tests

Reference:

• pyproject.toml

• src/geovista/examples

• tests/plotting

This section outlines how to create and run our image tests, and also highlights the key infrastructure that underpins it.

Markers

Reference:

• pyproject.toml

The following pytest markers are configured for image unit tests:

Table 19: Image Unit Test Markers

Marker	説明
image	Generic marker to be used on all image unit tests.
example	Specific marker only to be used on 例 gallery unit tests.

参考

For further details see the markers array entry in the [tool.pytest.ini_options] table of the pyproject.toml manifest.

The following marker expressions can be used for finer-grained control of unit test selection for execution:

Table 20: Marker Expressions

Marker Expression	説明
pytest -m image	Execute all image unit tests.
pytest -m "not image"	Execute all unit tests except the image unit tests.
pytest -m example	Execute only the gallery unit tests.
pytest -m "not example"	Execute all unit tests except the gallery unit tests.
pytest -m "image and not example"	Execute all image unit tests that are not gallery unit tests.

参考

Refer to the Pixi Workflow tests-unit task.

注意

pytest must be executed from within the geovista root directory.

Fixtures

Reference:

• tests/conftest.py

All image unit tests require to use the following pytest fixtures:

Table 21: Image Unit Test Fixtures

Fixture	説明
plot_nodeid	This fixture generates the unique dotted nodeid to identify the unit test relative to the tests root directory e.g., geodesic.test_BBox.test_outline is the nodeid of the test_outline unit test defined within the test_BBox.py file located within the tests/geodesic directory. The fixture also ensures that the baseline image for the unit test is available in the image cache. See Caching for further details.
verify_image_cache	The pytest-pyvista pytest plugin is used to perform the image comparison between a unit test baseline image (expected) against its pyvista rendered image (actual). This fixture requires the unit test to call the show() method as a hook so that it can initiate the image comparison. 参考 For further details see [tool.pytest.ini_options] table in the pyproject.toml manifest.

Template

Reference:

• pyproject.toml

• tests/conftest.py

Image unit tests should adopt the following usage pattern of Markers and Fixtures e.g.,

Listing 1: Image Unit Test Template

import pytest

import geovsita as gv


@pytest.mark.image
def test(plot_nodeid, verify_image_cache):
    """Single line doc-string."""
    verify_image_cache.test_name = plot_nodeid
    p = gv.GeoPlotter()
    ...
    p.show()

Note that:

• line: 6 - The @pytest.mark.image marker must always decorate the unit test

• line: 7 - Both the plot_nodeid and verify_image_cache fixures are used

• line: 9 - The verify_image_cache fixure is configured with the name of the unit test baseline image

• line: 12 - The show() method of the plotter is called to render the unit test scene and then invoke image comparison via the pytest-pyvista plugin configured by its verify_image_cache fixture. Be aware that the show() method must only be invoked once within the unit test. However the test itself may be decorated with the @pytest.mark.parametrize marker to execute it multiple times.

Caching

Reference:

• pyproject.toml

• src/geovista/cache/__init__.py

• src/geovista/config.py

• src/geovista/tests/plotting/__init__.py

• src/geovista/tests/plotting/unit_image_cache

The pytest-pyvista plugin performs the image comparison between a unit test baseline image (expected) and rendered image (actual) via its verify_image_cache fixture.

Unit test baseline images are automatically fetched and cached by the pooch CACHE manager on request by the plot_nodeid fixture.

参考

The geovista download command may be used to cache offline assets.

The CACHE manager fetches assets for a specific DATA_VERSION release tag from the geovista-data repository and stores them according to the resource configuration or the GEOVISTA_CACHEDIR environment variable.

These cached assets are made available to the verify_image_cache fixture via the tests/plotting/unit_image_cache symbolic-link, as configured within the [tool.pytest.ini_options] table of the pyproject.toml manifest.

The tests/plotting/unit_image_cache symbolic-link to the CACHE manager image cache is automatically verified and managed by the tests.plotting module.

Generation

Reference:

• pyproject.toml

• src/geovista/cache/__init__.py

• src/geovista/cache/registry.txt

Unit test rendered image (actual) generation is managed through the pytest-pyvista plugin.

For example, to serizalize all unit test rendered images to file within the test_images directory e.g.,

$ pytest -m image --generated_image_dir test_images

To serialize both the baseline image (expected) and rendered image (actual) of all unit test failures to file within the test_images_failed directory e.g.,

$ pytest -m image --failed_image_dir test_images_failed

注意

pytest must be executed from within the geovista root directory.

Typically, we recommend the following:

• Use the --generated_image_dir option to capture the rendered image (actual) of any new or existing unit tests

• Use the --failed_image_dir option to capture and investigate any failed image unit tests

参考

Refer to pytest-pyvista for further CLI and configuration options.

Also see the tests-unit pixi task in the following Pixi Workflow section, as a convenience.

To register a new baseline image for a unit test:

1. Generate the baseline image using the --generated_image_dir option to the pytest-pyvista plugin

2. Add the new baseline image to the assets/tests/unit directory of the geovista-data repository via a pull-request

3. Once the pull-request has been reviewed and merged, a new geovista-data release will be tagged automatically

4. Update the DATA_VERSION within geovista.cache to the new geovista-data release version

5. Finally, update the registry.txt with the unit test baseline image name (relative to the assets directory) along with the hexadecimal SHA256SUM of the baseline image file contents

The pooch CACHE manager will automatically download and cache the new baseline image from the geovista-data repository when re-running the unit test. Alternatively, download all the new assets for the DATA_VERSION using the CLI e.g.,

$ geovista download --all --decompress

Pixi Workflow

Reference:

• pyproject.toml

Pixi Environment

The testing pixi tasks are available in the test, geovista and py3xx variant of those pixi environments e.g., geovista-py313. See Packaging for further details.

Testing may be performed using the convenience of pixi run tasks, which do not require to be executed from within geovista root directory e.g.,

$ pixi run <pixi-task>

Table 22: Pixi Tasks

Pixi Task	説明
download	Download and cache offline assets. This task calls the geovista download command. Provide optional argument all, clean, doc-images, dry-run, images, list, natural-earth, operational, pantry, rasters, unit-images or verify. Defaults to all e.g., $ pixi run download rasters
tests-clean	Purge both the documentation and unit test image caches, along with any images generated from previous test sessions e.g., $ pixi run tests-clean
tests-unit	Perform the unit tests. This task calls the pytest command. Defaults to executing all unit tests discoverable from the geovista root directory. Accepts a valid pytest marker expression as an optional argument. Refer to the [tool.pytest.ini_options] table entry in the pyproject.toml manifest for configured markers e.g., $ pixi run tests-unit "not image" Note that the tests-clean task is called prior to running this task. 注釈 Failed Image Tests are automatically captured via the pytest-pyvista plugin option --failed_image_dir (see Generation) and available within the test_images_failed directory for analysis and investigation. Additionally all generated images are captured via the --generated_image_dir plugin option and are available within the test_images directory.

Tip

Apply the --frozen option to avoid pixi checking and updating the pixi.lock file.

E.g., To execute all non-image based unit tests:

$ pixi run --frozen tests-unit "not image"

Unit Tests

Reference:

• pyproject.toml

• tests

The unit tests are located within the tests root directory and are organised into sub-directories, typically one for each geovista top-level module or sub-package.

The exception to this rule is tests/plotting which contains the Image Tests.

To execute all unit tests:

$ pytest

参考

Refer to the Pixi Workflow tests-unit task.

注意

pytest must be executed from within the geovista root directory.

Finer-grained control of unit test can be achieved by using our pytest Markers.