:tocdepth: 0 .. _/plugins/test-checks: Test checks Plugins ~~~~~~~~~~~~~~~~~~~ .. include:: test-checks-header.inc.rst .. _/plugins/test-checks/common-keys: Common Keys ----------- The following keys are accepted by all plugins of the ``test-checks`` step. enabled Whether the check is enabled or not. Default: ``true`` Environment variable: ``TMT_PLUGIN_TEST-CHECKS_AVC_ENABLED`` In plan metadata: .. code-block:: yaml enabled: true|false On command-line: .. code-block:: shell --enabled export TMT_PLUGIN_TEST-CHECKS_AVC_ENABLED=1|0 result How to interpret the check result. Default: ``respect`` Environment variable: ``TMT_PLUGIN_TEST-CHECKS_AVC_RESULT`` In plan metadata: .. code-block:: yaml result: info|respect|xfail On command-line: .. code-block:: shell --result 'info|respect|xfail' export TMT_PLUGIN_TEST-CHECKS_AVC_RESULT='info|respect|xfail' .. _/plugins/test-checks/avc: avc --- Check for SELinux AVC denials raised during the test. The check collects SELinux AVC denials from the audit log, gathers details about them, and together with versions of the ``selinux-policy`` and related packages stores them in a report file after the test. .. code-block:: yaml check: - name: avc .. note:: To work correctly, the check requires SELinux to be enabled on the guest, and ``auditd`` must be running. Without SELinux, the check will turn into no-op, reporting :ref:`skip` result, and without ``auditd``, the check will discover no AVC denials, reporting :ref:`pass`. If the test manipulates ``auditd`` or SELinux in general, the check may report unexpected results. .. versionadded:: 1.28 Configuration ^^^^^^^^^^^^^ See also :ref:`Common Keys` accepted by the plugin. delay-before-report How many seconds to wait before running ``ausearch`` after the test. Increasing it may help when events do reach logs fast enough for ``ausearch`` report them. Default: ``5`` Environment variable: ``TMT_PLUGIN_TEST-CHECKS_AVC_DELAY_BEFORE_REPORT`` In plan metadata: .. code-block:: yaml delay-before-report: SECONDS On command-line: .. code-block:: shell --delay-before-report SECONDS export TMT_PLUGIN_TEST-CHECKS_AVC_DELAY_BEFORE_REPORT=SECONDS ignore-pattern Optional list of regular expressions to ignore in AVC denials. If an AVC denial matches any of these patterns, it will be ignored and not cause a failure. Any other denials will still cause the test to fail. If no patterns are specified, any denial will cause a failure. Default: *not set* Environment variable: ``TMT_PLUGIN_TEST-CHECKS_AVC_IGNORE_PATTERN`` In plan metadata: .. code-block:: yaml ignore-pattern: PATTERN On command-line: .. code-block:: shell --ignore-pattern PATTERN export TMT_PLUGIN_TEST-CHECKS_AVC_IGNORE_PATTERN=PATTERN test-method Which method to use when calling ``ausearch`` to report new AVC denials. With ``checkpoint``, native ``--checkpoint`` option of ``ausearch`` is used, while ``timestamp`` will depend on ``--ts`` option and a date/time recorded before the test. Default: ``timestamp`` Environment variable: ``TMT_PLUGIN_TEST-CHECKS_AVC_TEST_METHOD`` In plan metadata: .. code-block:: yaml test-method: timestamp|checkpoint On command-line: .. code-block:: shell --test-method 'timestamp|checkpoint' export TMT_PLUGIN_TEST-CHECKS_AVC_TEST_METHOD='timestamp|checkpoint' ---- .. _/plugins/test-checks/coredump: coredump -------- .. warning:: Please, be aware that the documentation below is a work in progress. We are working on fixing it, adding missing bits and generally making it better. Also, it was originally used for command line help only, therefore the formatting is often suboptimal. Check for system crashes using coredump. The check monitors for any crashes caught by systemd-coredump during test execution. This includes segmentation faults and other crashes that produce core dumps. By default, any crash will cause the test to fail. Example config with optional ignore patterns: .. code-block:: yaml check: - how: coredump ignore-pattern: - 'Process.*\(sleep\).*dumped core' # Ignore sleep crashes - 'Package: ddcutil/2.1.2-2.fc41' # Ignore dumps of a specific package The patterns are matched against the full coredumpctl info output, which includes fields like Process, Command Line, Signal, etc. You can use 'coredumpctl info' to see the available fields and their format. Common pattern examples: - Ignore all crashes from a specific process: `'Process.*\(specific-process\)'` - Ignore crashes with a specific signal: `'Signal: .*\(SIGSEGV\)'` - Ignore crashes from a specific package: `'Package: package-name/.*'` - Ignore crashes during a specific command: `'Command Line: .*specific-command-pattern.*'` .. versionadded:: 1.46 Configuration ^^^^^^^^^^^^^ See also :ref:`Common Keys` accepted by the plugin. ignore-pattern Optional list of regular expressions to ignore in crash reports. If a crash report matches any of these patterns, it will be ignored and not cause a failure. Any other crashes will still cause the test to fail. If no patterns are specified, any crash will cause a failure. Default: *not set* Environment variable: ``TMT_PLUGIN_TEST-CHECKS_COREDUMP_IGNORE_PATTERN`` In plan metadata: .. code-block:: yaml ignore-pattern: PATTERN On command-line: .. code-block:: shell --ignore-pattern PATTERN export TMT_PLUGIN_TEST-CHECKS_COREDUMP_IGNORE_PATTERN=PATTERN is-availability-reason Default: *not set* Environment variable: ``TMT_PLUGIN_TEST-CHECKS_COREDUMP_IS_AVAILABILITY_REASON`` In plan metadata: .. code-block:: yaml is-availability-reason: On command-line: .. code-block:: shell --is-availability-reason ... export TMT_PLUGIN_TEST-CHECKS_COREDUMP_IS_AVAILABILITY_REASON=... is-available Default: *not set* Environment variable: ``TMT_PLUGIN_TEST-CHECKS_COREDUMP_IS_AVAILABLE`` In plan metadata: .. code-block:: yaml is-available: On command-line: .. code-block:: shell --is-available ... export TMT_PLUGIN_TEST-CHECKS_COREDUMP_IS_AVAILABLE=... ---- .. _/plugins/test-checks/dmesg: dmesg ----- Save the content of kernel ring buffer (aka "console") into a file. The check saves one file before the test, and then again when test finishes. .. code-block:: yaml check: - how: dmesg Check will identify patterns that signal kernel crashes and core dumps, and when detected, it will report as failed result. It is possible to define custom patterns: .. code-block:: yaml check: - how: dmesg failure-pattern: # These are default patterns - 'Call Trace: - '\ssegfault\s' # More patterns to look for - '\[Firmware Bug\]' .. versionadded:: 1.28 .. versionchanged:: 1.33 ``failure-pattern`` has been added. Configuration ^^^^^^^^^^^^^ See also :ref:`Common Keys` accepted by the plugin. failure-pattern List of regular expressions to look for in ``dmesg`` output. If any of patterns is found, ``dmesg`` check will report ``fail`` result. Default: ``Call Trace:``, ``\ssegfault\s`` Environment variable: ``TMT_PLUGIN_TEST-CHECKS_DMESG_FAILURE_PATTERN`` In plan metadata: .. code-block:: yaml failure-pattern: PATTERN On command-line: .. code-block:: shell --failure-pattern PATTERN export TMT_PLUGIN_TEST-CHECKS_DMESG_FAILURE_PATTERN=PATTERN ---- .. _/plugins/test-checks/internal/abort: internal/abort -------------- Check for test aborts during execution. This check fails when tests are aborted before completion. .. note:: This is an :ref:`internal check `, and it cannot be enabled or disabled by test metadata. .. versionadded:: 1.50 .. _/plugins/test-checks/internal/guest: internal/guest -------------- Check for guest errors during test execution. This check fails when guest related errors occur during test execution, such as reboot or reconnect timeouts. .. note:: This is an :ref:`internal check `, and it cannot be enabled or disabled by test metadata. .. versionadded:: 1.53 .. _/plugins/test-checks/internal/interrupt: internal/interrupt ------------------ Check for signal interruptions during test execution. This check fails when tests are interrupted by SIGINT or SIGTERM signals. .. note:: This is an :ref:`internal check `, and it cannot be enabled or disabled by test metadata. .. versionadded:: 1.50 .. _/plugins/test-checks/internal/invocation: internal/invocation ------------------- Check for uncategorized invocation errors during test execution. This check fails when tests encounter errors that are not covered by more specific checks. .. note:: This is an :ref:`internal check `, and it cannot be enabled or disabled by test metadata. .. versionadded:: 1.50 .. _/plugins/test-checks/internal/permission: internal/permission ------------------- Check for permission issues during execution. This check fails when tests encounter permission-related errors. .. note:: This is an :ref:`internal check `, and it cannot be enabled or disabled by test metadata. .. versionadded:: 1.50 .. _/plugins/test-checks/internal/timeout: internal/timeout ---------------- Check for test timeouts during execution. This check fails when tests exceed their maximum allowed duration. .. note:: This is an :ref:`internal check `, and it cannot be enabled or disabled by test metadata. .. versionadded:: 1.50 .. _/plugins/test-checks/journal: journal ------- .. warning:: Please, be aware that the documentation below is a work in progress. We are working on fixing it, adding missing bits and generally making it better. Also, it was originally used for command line help only, therefore the formatting is often suboptimal. Check messages in journal log recorded during the test. This check uses ``journalctl`` to capture log messages created during the test execution. It uses cursors to precisely pinpoint the start and end of the logging period. The check automatically configures systemd journal with persistent storage and compression to ensure reliable log capture during test execution. Configuration is applied with appropriate permission handling and fallback behavior. Example usage: .. code-block:: yaml check: - how: journal # Check only kernel messages dmesg: true .. code-block:: yaml check: - how: journal # Check messages from a specific systemd unit unit: httpd.service # Filter by priority priority: err .. code-block:: yaml check: - how: journal # Check messages from a specific syslog identifier identifier: sshd priority: warning Check will identify patterns that signal kernel crashes and core dumps, and when detected, it will report a failed result. It is possible to define custom patterns for failures and messages to ignore: .. code-block:: yaml check: - how: journal failure-pattern: # These are default patterns - 'Call Trace:' - '\ssegfault\s' # More patterns to look for - '\[Firmware Bug\]' ignore-pattern: - 'a known harmless error message' The check requires systemd to be available on the guest system. Journal configuration is automatically applied at the start of test execution, with fallback to default settings if configuration fails due to insufficient permissions. .. versionadded:: 1.54.0 Configuration ^^^^^^^^^^^^^ See also :ref:`Common Keys` accepted by the plugin. dmesg Check only kernel messages. Default: ``false`` Environment variable: ``TMT_PLUGIN_TEST-CHECKS_JOURNAL_DMESG`` In plan metadata: .. code-block:: yaml dmesg: true|false On command-line: .. code-block:: shell --dmesg export TMT_PLUGIN_TEST-CHECKS_JOURNAL_DMESG=1|0 failure-pattern List of regular expressions to look for in ``journal`` log. If any of patterns is found, ``journal`` check will report ``fail`` result. Default: ``Call Trace:``, ``\ssegfault\s`` Environment variable: ``TMT_PLUGIN_TEST-CHECKS_JOURNAL_FAILURE_PATTERN`` In plan metadata: .. code-block:: yaml failure-pattern: PATTERN On command-line: .. code-block:: shell --failure-pattern PATTERN export TMT_PLUGIN_TEST-CHECKS_JOURNAL_FAILURE_PATTERN=PATTERN identifier Check logs for a specific syslog identifier. Default: *not set* Environment variable: ``TMT_PLUGIN_TEST-CHECKS_JOURNAL_IDENTIFIER`` In plan metadata: .. code-block:: yaml identifier: On command-line: .. code-block:: shell --identifier ... export TMT_PLUGIN_TEST-CHECKS_JOURNAL_IDENTIFIER=... ignore-pattern Optional list of regular expressions to ignore in journal log. If a log entry matches any of these patterns, it will be ignored and not cause a failure. Default: *not set* Environment variable: ``TMT_PLUGIN_TEST-CHECKS_JOURNAL_IGNORE_PATTERN`` In plan metadata: .. code-block:: yaml ignore-pattern: PATTERN On command-line: .. code-block:: shell --ignore-pattern PATTERN export TMT_PLUGIN_TEST-CHECKS_JOURNAL_IGNORE_PATTERN=PATTERN priority Filter by priority (e.g. ``err``, ``warning``). Default: *not set* Environment variable: ``TMT_PLUGIN_TEST-CHECKS_JOURNAL_PRIORITY`` In plan metadata: .. code-block:: yaml priority: On command-line: .. code-block:: shell --priority ... export TMT_PLUGIN_TEST-CHECKS_JOURNAL_PRIORITY=... unit Check logs for a specific systemd unit. Default: *not set* Environment variable: ``TMT_PLUGIN_TEST-CHECKS_JOURNAL_UNIT`` In plan metadata: .. code-block:: yaml unit: On command-line: .. code-block:: shell --unit ... export TMT_PLUGIN_TEST-CHECKS_JOURNAL_UNIT=... ---- .. _/plugins/test-checks/watchdog: watchdog -------- Take various actions when guest becomes unresponsive. Watchdog runs selected probes every now and then, and when a given number of `probes` fail, watchdog would run one or more of the predefined `actions`. Check comes with two probes, "ping" and "SSH ping", and single action, "reboot". * "ping" uses the classic ICMP echo to check whether the guest is still up and running, * "SSH ping" tries to establish SSH connection, * "reboot" action issues a hard reboot of the guest. .. warning:: Be aware that this feature may be limited depending on how the guest was provisioned. See :ref:`/plugins/provision/hard-reboot`. Each probe has a "budget" of allowed failures, and when it runs out, the action is taken. A successful probe replenishes its budget to the original level. Multiple probes can be enabled at the same time, for the action to happen it's enough if just one of them runs out of its budget. .. code-block:: yaml check: - how: watchdog ping: true reboot: true .. code-block:: yaml check: - how: watchdog # Use only SSH ping. ping: false ssh-ping: true # Try every 5 minutes, allow 7 failed attempts, and reboot # the guest when we run out of attempts. interval: 300 reboot: true ssh-ping-threshold: 7 .. versionadded:: 1.32 Configuration ^^^^^^^^^^^^^ See also :ref:`Common Keys` accepted by the plugin. interval How often should the watchdog run. Default: ``60`` Environment variable: ``TMT_PLUGIN_TEST-CHECKS_WATCHDOG_INTERVAL`` In plan metadata: .. code-block:: yaml interval: SECONDS On command-line: .. code-block:: shell --interval SECONDS export TMT_PLUGIN_TEST-CHECKS_WATCHDOG_INTERVAL=SECONDS ping If enabled, watchdog would probe guest's responsiveness with ICMP packets. Default: ``false`` Environment variable: ``TMT_PLUGIN_TEST-CHECKS_WATCHDOG_PING`` In plan metadata: .. code-block:: yaml ping: true|false On command-line: .. code-block:: shell --ping export TMT_PLUGIN_TEST-CHECKS_WATCHDOG_PING=1|0 ping-packets How many ICMP packates to send as one probe. Default: ``1`` Environment variable: ``TMT_PLUGIN_TEST-CHECKS_WATCHDOG_PING_PACKETS`` In plan metadata: .. code-block:: yaml ping-packets: N On command-line: .. code-block:: shell --ping-packets N export TMT_PLUGIN_TEST-CHECKS_WATCHDOG_PING_PACKETS=N ping-threshold How many failed ping probes before taking any further action. Default: ``10`` Environment variable: ``TMT_PLUGIN_TEST-CHECKS_WATCHDOG_PING_THRESHOLD`` In plan metadata: .. code-block:: yaml ping-threshold: N On command-line: .. code-block:: shell --ping-threshold N export TMT_PLUGIN_TEST-CHECKS_WATCHDOG_PING_THRESHOLD=N reboot If enabled, watchdog would reboot the guest after enough failed probes. Default: ``false`` Environment variable: ``TMT_PLUGIN_TEST-CHECKS_WATCHDOG_REBOOT`` In plan metadata: .. code-block:: yaml reboot: true|false On command-line: .. code-block:: shell --reboot export TMT_PLUGIN_TEST-CHECKS_WATCHDOG_REBOOT=1|0 ssh-ping If enabled, watchdog would probe guest's responsiveness by connecting to its SSH port. Default: ``false`` Environment variable: ``TMT_PLUGIN_TEST-CHECKS_WATCHDOG_SSH_PING`` In plan metadata: .. code-block:: yaml ssh-ping: true|false On command-line: .. code-block:: shell --ssh-ping export TMT_PLUGIN_TEST-CHECKS_WATCHDOG_SSH_PING=1|0 ssh-ping-threshold How many failed SSH connections before taking any further action. Default: ``10`` Environment variable: ``TMT_PLUGIN_TEST-CHECKS_WATCHDOG_SSH_PING_THRESHOLD`` In plan metadata: .. code-block:: yaml ssh-ping-threshold: N On command-line: .. code-block:: shell --ssh-ping-threshold N export TMT_PLUGIN_TEST-CHECKS_WATCHDOG_SSH_PING_THRESHOLD=N