Use Spread to test commands in documentation¶

It’s challenging to keep documentation in sync with products as they evolve. This process is aided by Spread, a test distributor that can work through your documentation and report failures in GitHub workflows.

By using Spread tests, you can rely on the tests as the source of truth for commands in your documentation, enabling fully tested documentation at build time.

What you’ll need¶

Multipass installed on your machine
Spread installed on your machine

Warning

Spread requires elevated permissions to run as root. Use the Go install method recommended in the Spread README to install Spread.

Create a test suite¶

From the root of your project, create the file spread.yaml and insert the following contents:

project_name/spread.yaml¶

project: project_name

path: /project_name

Match the project name to your main directory’s name. The path designates the directory where the Spread materials exist.

So that Spread knows about your tests, add the following section to the end of spread.yaml:

project_name/spread.yaml¶

project: project_name

path: /project_name

suites:
  tests/spread/:
    summary: example test
    systems:
      - ubuntu-24.04-64

The suites section is how you tell Spread about the various Spread tests in your project along with the systems you want Spread to use. In this example, Spread looks for tests in the project_name/tests/spread directory and runs them on Ubuntu 24.04. If you create a new task.yaml file in a different directory, remember to add a corresponding suite for it in spread.yaml.

Set up the Multipass backend¶

Each job in Spread has a backend, or a way to obtain a machine for running your Spread tests. Spread can run on various backends, like Google, QEMU, or, as this guide sets up, Multipass.

Copy the following backends section of spread.yaml between the path and suites sections:

project_name/spread.yaml¶

project: project_name

path: /project_name

backends:
  multipass:
    type: adhoc
    allocate: |
      multipass_image=24.04
      instance_name="example-multipass-vm"

      # Launch Multipass VM
      multipass launch --cpus 2 --disk 10G --memory 2G --name "${instance_name}" "${multipass_image}"

      # Enable PasswordAuthentication for root over SSH.
      multipass exec "$instance_name" -- \
        sudo sh -c "echo root:${SPREAD_PASSWORD} | sudo chpasswd"
      multipass exec "$instance_name" -- \
        sudo sh -c \
        "if [ -d /etc/ssh/sshd_config.d/ ]
        then
          echo 'PasswordAuthentication yes' > /etc/ssh/sshd_config.d/10-spread.conf
          echo 'PermitRootLogin yes' >> /etc/ssh/sshd_config.d/10-spread.conf
        else
          sed -i /etc/ssh/sshd_config -E -e 's/^#?PasswordAuthentication.*/PasswordAuthentication yes/' -e 's/^#?PermitRootLogin.*/PermitRootLogin yes/'
        fi"
      multipass exec "$instance_name" -- \
        sudo systemctl restart ssh

      # Get the IP from the instance
      ip=$(multipass info --format csv "$instance_name" | tail -1 | cut -d\, -f3)
      ADDRESS "$ip"

    discard: |
      instance_name="example-multipass-vm"
      multipass delete --purge "${instance_name}"

    systems:
      - ubuntu-24.04-64:
          workers: 1

suites:
  tests/spread/:
    summary: example test
    systems:
      - ubuntu-24.04-64

The backends section contains the following pieces:

The backend is designated as type: adhoc as you must explicitly script the procedure to allocate and discard the Multipass VM.
The allocate section defines the image and name of the VM, launches the VM, and sets up the proper SSH permissions Spread then logs in to the VM with root permissions and inserts the Spread test. The last two lines tell Spread the IP address of the Multipass VM and set the environment variable ADDRESS.
The discard section deletes the Multipass VM once the Spread test has finished running.
The systems key notes which systems the backend uses. Note that this key must match the systems used by at least one test under suites.

Create a Spread task¶

Put your Spread files alongside your project’s existing tests. The rest of this guide assumes they’re in a top-level tests/spread directory.

Each Spread test requires a dedicated task.yaml file that contains all the commands you want to test. A single task.yaml can help you validate an entire assumed workflow, for instance, an end-to-end tutorial.

An example task.yaml file is shown below:

task.yaml¶

summary: Example Spread test

kill-timeout: 5m

prepare: |
  echo "Use this section to install any prerequisites"

execute: |
  echo "This is the first command that Spread will run"

  echo "This is the second command that Spread will run"

The summary section contains a brief description of the documentation you’re testing, the prepare section contains any initial setup your test needs, and the execute section contains your documentation’s commands. The kill-timeout option has a default of 10 minutes and doesn’t need to be included if you expect your test to complete in that time frame.

Note

For a real-world example, see task.yaml for the Rockcraft Go tutorial.

Include the tested commands in documentation¶

By using the literalinclude directive in Sphinx, you can insert the exact commands from task.yaml in your documentation file.

For example, consider the following task.yaml file:

task.yaml¶

summary: Clone and build the starter pack

kill-timeout: 5m

execute: |
  # [docs:clone-starter-pack]
  git clone https://github.com/canonical/sphinx-docs-starter-pack.git
  # [docs:clone-starter-pack-end]

  # [docs:build-documentation]
  cd sphinx-docs-starter-pack/docs
  make run
  # [docs:build-documentation-end]

Include the commands from task.yaml in your documentation with:

reStructuredText

Example literalinclude blocks¶

Clone the starter pack:

.. literalinclude:: relative-path-to/task.yaml
    :language: bash
    :start-after: [docs:clone-starter-pack]
    :end-before: [docs:clone-starter-pack-end]
    :dedent: 2

Enter the ``docs`` folder and build the project:

.. literalinclude:: relative-path-to/task.yaml
    :language: bash
    :start-after: [docs:build-documentation]
    :end-before: [docs:build-documentation-end]
    :dedent: 2

MyST

Example literalinclude blocks¶

Clone the starter pack:

```{literalinclude} relative-path-to/task.yaml
:language: bash
:start-after: [docs:clone-starter-pack]
:end-before: [docs:clone-starter-pack-end]
:dedent: 2
```

Enter the `docs` folder and build the project:

```{literalinclude} relative-path-to/task.yaml
:language: bash
:start-after: [docs:build-documentation]
:end-before: [docs:build-documentation-end]
:dedent: 2
```

By using the options :start-after: and :end-before:, the documentation file sources and includes all commands appearing in task.yaml between the specified lines.

Run tests locally¶

List all available Spread tests in the code repository:

spread --list

The terminal should respond with all the tests defined in spread.yaml. For example:

user@host:project_name$

spread --list

multipass:ubuntu-24.04-64:tests/spread/example_documentation_test

Run all Spread tests locally with spread. You can also run a single Spread test by specifying:

spread -vv -debug multipass:ubuntu-24.04-64:tests/spread/example_documentation_test

Depending on the complexity of your test, Spread can take several minutes to complete. The -vv -debug flags provide useful debugging information as the test runs.

Check the results¶

During runtime, the terminal outputs various messages about allocating the Multipass VM, connecting to the VM, sending the Spread test to the VM and executing the test. If the test is successful, the terminal will output something similar to the following:

2025-02-04 16:17:10 Successful tasks: 1
2025-02-04 16:17:10 Aborted tasks: 0

Another sign of a successful test is whether the Multipass VM was deleted as expected. Check by running multipass list, and if the Spread test was successful (and you have no other Multipass VMs created at the time), the terminal should respond with:

user@host:project_name$

multipass list

No instances found.

If the Spread test failed, then the -debug flag will open a shell into the Multipass VM so that additional debugging can happen. In that case, the terminal will output something similar to the following:

2025-02-04 16:17:10 Starting shell to debug...
2025-02-04 16:17:10 Sending script for multipass:ubuntu-24.04-64 (multipass:ubuntu-24.04-64:tests/spread/example_documentation_test):