tmt.steps.discover package

Submodules

tmt.steps.discover.fmf module

class tmt.steps.discover.fmf.DiscoverFmf(*, step: Step, data: StepDataT, workdir: Literal[True] | Path | None = None, logger: Logger)

Bases: DiscoverPlugin[DiscoverFmfStepData]

Discover available tests from fmf metadata.

By default all available tests from the current repository are used so the minimal configuration looks like this:

discover:
    how: fmf

Full config example:

discover:
    how: fmf
    url: https://github.com/teemtee/tmt
    ref: main
    path: /fmf/root
    test: /tests/basic
    filter: 'tier: 1'

If no ref is provided, the default branch from the origin is used.

Archive

By default url is treated as a git url to be cloned, but you can set url-content-type to archive to instead treat it as an archive url and download and extract it. For example:

discover:
    how: fmf
    url: https://github.com/teemtee/tmt/archive/refs/heads/main.tar.gz
    url-content-type: archive
    path: /tmt-main/fmf/root

Dist Git

For DistGit repo one can download sources and use code from them in the tests. Sources are extracted into $TMT_SOURCE_DIR path, patches are applied by default. See options to install build dependencies or to just download sources without applying patches. To apply patches, special prepare phase with order 60 is added, and prepare step has to be enabled for it to run.

It can be used together with ref, path and url, however ref is not possible without using url.

discover:
    how: fmf
    dist-git-source: true

Name Filter

Use the test key to limit which tests should be executed by providing regular expression matching the test name:

discover:
    how: fmf
    test: ^/tests/area/.*

tmt run discover --how fmf --verbose --test "^/tests/core.*"

When several regular expressions are provided, tests matching each regular expression are concatenated. In this way it is possible to execute a single test multiple times:

discover:
    how: fmf
    test:
      - ^/test/one$
      - ^/test/two$
      - ^/special/setup$
      - ^/test/one$
      - ^/test/two$

tmt run discover -h fmf -v -t '^/test/one$' -t '^/special/setup$' -t '^/test/two$'

The include key also allows to select tests by name, with two important distinctions from the test key:

The original test order is preserved so it does not matter in which order tests are listed under the include key.
Test duplication is not allowed, so even if a test name is repeated several times, test will be executed only once.

Finally, the exclude key can be used to specify regular expressions matching tests which should be skipped during the discovery.

The test, include and exclude keys use search mode for matching patterns. See the Regular Expressions section for detailed information about how exactly the regular expressions are handled.

Link Filter

Selecting tests containing specified link is possible using link key accepting RELATION:TARGET format of values. Regular expressions are supported for both relation and target part of the value. Relation can be omitted to target match any relation.

discover:
    how: fmf
    link: verifies:.*issue/850$

Multiple links can be provided as well:

discover:
    how: fmf
    link:
      - verifies:.*issue/850$
      - verifies:.*issue/1374$

Advanced Filter

The filter key can be used to apply an advanced filter based on test metadata attributes. These can be especially useful when tests are grouped by the tag or tier keys:

discover:
    how: fmf
    filter: tier:3 & tag:provision

tmt run discover --how fmf --filter "tier:3 & tag:provision"

See the pydoc fmf.filter documentation for more details about the supported syntax and available operators.

Modified Tests

It is also possible to limit tests only to those that have changed in git since a given revision. This can be particularly useful when testing changes to tests themselves (e.g. in a pull request CI).

Related keys: modified-only, modified-url, modified-ref

Example to compare local repo against upstream main branch:

discover:
    how: fmf
    modified-only: True
    modified-url: https://github.com/teemtee/tmt
    modified-ref: reference/main

Note that internally the modified tests are appended to the list specified via test, so those tests will also be selected even if not modified.

Adjust Tests

Use the adjust-tests key to modify the discovered tests’ metadata directly from the plan. For example, extend the test duration for slow hardware or modify the list of required packages when you do not have write access to the remote test repository. The value should follow the adjust rules syntax.

The following example adds an avc check for each discovered test, doubles its duration and replaces each occurrence of the word python3.11 in the list of required packages.

discover:
    how: fmf
    adjust-tests:
      - check+:
          - how: avc
      - duration+: '*2'
        because: Slow system under test
        when: arch == i286
      - require~:
          - '/python3.11/python3.12/'

Store plugin name, data and parent step

cli_invocation: 'tmt.cli.CliInvocation' | None = None

do_the_discovery(path: Path | None = None) → None: Discover the tests

get_git_root(directory: Path) → Path: Find git root of the path

go(*, logger: Logger | None = None) → None: Discover available tests

property is_in_standalone_mode: bool: Enable standalone mode when listing fmf ids

post_dist_git(created_content: list[Path]) → None: Discover tests after dist-git applied patches

process_distgit_source(distgit_dir: Path, sourcedir: Path) → None: Process dist-git source during the discover step.

tests(*, phase_name: str | None = None, enabled: bool | None = None) → list[TestOrigin]: Return all discovered tests

class tmt.steps.discover.fmf.DiscoverFmfStepData(name: str, how: str, order: int = 50, when: list[str] = <factory>, summary: str | None = None, where: list[str] = <factory>, dist_git_source: bool = False, dist_git_type: str | None = None, dist_git_download_only: bool = False, dist_git_install_builddeps: bool = False, dist_git_require: list['tmt.base.DependencySimple'] = <factory>, require_test: list[str] = <factory>, url: str | None = None, url_content_type: Literal['git', 'archive'] = 'git', ref: str | None = None, path: str | None = None, test: list[str] = <factory>, link: list[str] = <factory>, filter: list[str] = <factory>, include: list[str] = <factory>, exclude: list[str] = <factory>, modified_only: bool = False, modified_url: str | None = None, modified_ref: str | None = None, dist_git_init: bool = False, dist_git_remove_fmf_root: bool = False, dist_git_merge: bool = False, dist_git_extract: str | None = None, sync_repo: bool = False, fmf_id: bool = False, prune: bool = False, adjust_tests: list[tmt.base._RawAdjustRule] | None = <factory>, upgrade_path: str | None = None, repository: str | None = None, revision: str | None = None)

Bases: DiscoverStepData

adjust_tests: list[_RawAdjustRule] | None

dist_git_extract: str | None = None

dist_git_init: bool = False

dist_git_merge: bool = False

dist_git_remove_fmf_root: bool = False

exclude: list[str]

filter: list[str]

fmf_id: bool = False

include: list[str]

link: list[str]

modified_only: bool = False

modified_ref: str | None = None

modified_url: str | None = None

path: str | None = None

post_normalization(raw_data: _RawStepData, logger: Logger) → None: Called after normalization, useful for tweaking normalized data

prune: bool = False

ref: str | None = None

repository: str | None = None

revision: str | None = None

sync_repo: bool = False

test: list[str]

upgrade_path: str | None = None

url: str | None = None

url_content_type: Literal['git', 'archive'] = 'git'

tmt.steps.discover.fmf.normalize_ref(key_address: str, value: Any | None, logger: Logger) → str | None

tmt.steps.discover.shell module

class tmt.steps.discover.shell.DiscoverShell(*, step: Step, data: StepDataT, workdir: Literal[True] | Path | None = None, logger: Logger)

Bases: DiscoverPlugin[DiscoverShellData]

Use provided list of shell script tests.

List of test cases to be executed can be defined manually directly in the plan as a list of dictionaries containing test name and actual test script. It is also possible to define here any other test metadata such as the duration or a path to the test. The default duration for tests defined directly in the discover step is 1h.

Example config:

discover:
    how: shell
    tests:
      - name: /help/main
        test: tmt --help
      - name: /help/test
        test: tmt test --help
      - name: /help/smoke
        test: ./smoke.sh
        path: /tests/shell

For DistGit repo one can download sources and use code from them in the tests. Sources are extracted into $TMT_SOURCE_DIR path, patches are applied by default. See options to install build dependencies or to just download sources without applying patches. To apply patches, special prepare phase with order 60 is added, and prepare step has to be enabled for it to run.

discover:
    how: shell
    dist-git-source: true
    tests:
      - name: /upstream
        test: cd $TMT_SOURCE_DIR/*/tests && make test

To clone a remote repository and use it as a source specify url. It accepts also ref to checkout provided reference. Dynamic reference feature is supported as well.

discover:
    how: shell
    url: https://github.com/teemtee/tmt.git
    ref: "1.18.0"
    tests:
      - name: first test
        test: ./script-from-the-repo.sh

Store plugin name, data and parent step

cli_invocation: 'tmt.cli.CliInvocation' | None = None

fetch_remote_repository(url: str | None, ref: str | None, testdir: Path, keep_git_metadata: bool = False) → None: Fetch remote git repo from given url to testdir

go(*, logger: Logger | None = None) → None: Discover available tests

show(keys: list[str] | None = None) → None: Show config details

tests(*, phase_name: str | None = None, enabled: bool | None = None) → list[TestOrigin]

Return discovered tests.

Parameters:

phase_name –
if set, return only tests discovered by the phase of this name. Otherwise, all tests discovered by the phase are returned.
Note

This parameter exists to present unified interface with tmt.steps.discover.Discover.tests() API, but it has no interesting effect in case of individual phases:
- left unset, all tests discovered by the phase are returned,
- set to a phase name, tests discovered by that phase should be returned. But a phase does not have access to other phases’ tests, therefore setting it to anything but this phase name would produce an empty list.
enabled – if set, return only tests that are enabled (enabled=True) or disabled (enabled=False). Otherwise, all tests are returned.

Returns:

a list of phase name and test pairs.

class tmt.steps.discover.shell.DiscoverShellData(name: str, how: str, order: int = 50, when: list[str] = <factory>, summary: str | None = None, where: list[str] = <factory>, dist_git_source: bool = False, dist_git_type: str | None = None, dist_git_download_only: bool = False, dist_git_install_builddeps: bool = False, dist_git_require: list['tmt.base.DependencySimple'] = <factory>, require_test: list[str] = <factory>, tests: list[tmt.steps.discover.shell.TestDescription] = <factory>, url: str | None = None, ref: str | None = None, keep_git_metadata: bool = False)

Bases: DiscoverStepData

keep_git_metadata: bool = False

ref: str | None = None

tests: list[TestDescription]

to_spec() → _RawStepData: Convert to a form suitable for saving in a specification file

url: str | None = None

class tmt.steps.discover.shell.TestDescription(name: str, test: ~tmt.utils.ShellScript = <tmt.utils.ShellScript object>, summary: str | None = None, description: str | None = None, enabled: bool = True, order: int = 50, link: ~tmt.base.Links | None = None, id: str | None = None, tag: list[str] = <factory>, tier: str | None = None, adjust: list[~tmt.base._RawAdjustRule] | None = None, author: list[str] = <factory>, contact: list[str] = <factory>, component: list[str] = <factory>, path: str | None = None, framework: str | None = None, manual: bool = False, tty: bool = False, require: list[~tmt.base.DependencySimple | ~tmt.base.DependencyFmfId | ~tmt.base.DependencyFile] = <factory>, recommend: list[~tmt.base.DependencySimple | ~tmt.base.DependencyFmfId | ~tmt.base.DependencyFile] = <factory>, environment: ~tmt.utils.Environment = <factory>, check: list[~tmt.checks.Check] = <factory>, duration: str = '1h', result: str = 'respect')

Bases: SpecBasedContainer[dict[str, Any], dict[str, Any]], NormalizeKeysMixin, SerializableContainer

Keys necessary to describe a shell-based test.

Provides basic functionality for transition between “raw” step data representation, which consists of keys and values given by fmf tree and CLI options, and this container representation for internal use.

adjust: list[_RawAdjustRule] | None = None

author: list[str]

check: list[Check]

component: list[str]

contact: list[str]

description: str | None = None

duration: str = '1h'

enabled: bool = True

environment: Environment

framework: str | None = None

classmethod from_spec(raw_data: dict[str, Any], logger: Logger) → Self: Convert from a specification file or from a CLI option

id: str | None = None

link: Links | None = None

manual: bool = False

name: str

order: int = 50

path: str | None = None

recommend: list[DependencySimple | DependencyFmfId | DependencyFile]

require: list[DependencySimple | DependencyFmfId | DependencyFile]

result: str = 'respect'

summary: str | None = None

tag: list[str]

test: ShellScript = <tmt.utils.ShellScript object>

tier: str | None = None

to_spec() → dict[str, Any]: Convert to a form suitable for saving in a specification file

tty: bool = False

Module contents

class tmt.steps.discover.Discover(*, plan: Plan, data: _RawStepData | list[_RawStepData], logger: Logger)

Bases: Step

Gather information about test cases to be executed.

Store supported attributes, check for sanity

cli_invocation: 'tmt.cli.CliInvocation' | None = None

cli_invocations: list['tmt.cli.CliInvocation'] = []

go(force: bool = False) → None: Discover all tests

load() → None: Load step data from the workdir

property required_tests: list[TestOrigin]: The list of required tests gathered from all phases

save() → None: Save step data to the workdir

summary() → None: Give a concise summary of the discovery

tests(*, phase_name: str | None = None, enabled: bool | None = None) → list[TestOrigin]

Return discovered tests.

Parameters:

phase_name – if set, return only tests discovered by the phase of this name. Otherwise, tests discovered by all phases are returned.
enabled – if set, return only tests that are enabled (enabled=True) or disabled (enabled=False). Otherwise, all tests are returned.

Returns:

a list of phase name and test pairs.

wake() → None: Wake up the step (process workdir and command line)

class tmt.steps.discover.DiscoverPlugin(*, step: Step, data: StepDataT, workdir: Literal[True] | Path | None = None, logger: Logger)

Bases: GuestlessPlugin[DiscoverStepDataT, None]

Common parent of discover plugins

Store plugin name, data and parent step

classmethod base_command(usage: str, method_class: type[Command] | None = None) → Command: Create base click command (common for all discover plugins)

cli_invocation: 'tmt.cli.CliInvocation' | None = None

download_distgit_source(distgit_dir: Path, target_dir: Path, handler_name: str | None = None) → None

Download sources to the target_dir

distgit_dir is path to the DistGit repository

go(*, logger: Logger | None = None) → None: Perform actions shared among plugins when beginning their tasks

log_import_plan_details() → None: Log details about the imported plan

post_dist_git(created_content: list[Path]) → None: Discover tests after dist-git applied patches

abstractmethod tests(*, phase_name: str | None = None, enabled: bool | None = None) → list[TestOrigin]

Return discovered tests.

Parameters:

phase_name –
if set, return only tests discovered by the phase of this name. Otherwise, all tests discovered by the phase are returned.
Note

This parameter exists to present unified interface with tmt.steps.discover.Discover.tests() API, but it has no interesting effect in case of individual phases:
- left unset, all tests discovered by the phase are returned,
- set to a phase name, tests discovered by that phase should be returned. But a phase does not have access to other phases’ tests, therefore setting it to anything but this phase name would produce an empty list.
enabled – if set, return only tests that are enabled (enabled=True) or disabled (enabled=False). Otherwise, all tests are returned.

Returns:

a list of phase name and test pairs.

class tmt.steps.discover.DiscoverStepData(name: str, how: str, order: int = 50, when: list[str] = <factory>, summary: str | None = None, where: list[str] = <factory>, dist_git_source: bool = False, dist_git_type: str | None = None, dist_git_download_only: bool = False, dist_git_install_builddeps: bool = False, dist_git_require: list['tmt.base.DependencySimple'] = <factory>, require_test: list[str] = <factory>)

Bases: WhereableStepData, StepData

dist_git_download_only: bool = False

dist_git_install_builddeps: bool = False

dist_git_require: list[tmt.base.DependencySimple]

dist_git_source: bool = False

dist_git_type: str | None = None

require_test: list[str]

class tmt.steps.discover.TestOrigin(phase: str, test: Test)

Bases: object

Describes the origin of a test.

phase: str: Name of the discover phase that added the test.

test: Test: The test in question.