.. _/spec/policy:

Policy
======
*As an owner of a CI system or product CI workflow, I would like to
modify all component plans and tests to include phases and checks
that are deemed mandatory for the given CI workflow.*


.. note::

    As of now, the specification focuses and applies to
    :ref:`test metadata</spec/tests>` only. In the future, we plan
    to support plans and stories as well.

tmt policies offer a powerful and flexible way to dynamically
modify test metadata using templates. Instead of maintaining multiple
similar test versions, you can define a base test and then apply
policy to adjust its properties for different scenarios (e.g.,
different environments, feature flags, or CI runs).

A policy is stored in a YAML file, and consists of one or more
rules, each mapping a test key to a template that provides new
content for the said key.

.. code-block:: yaml

    test-policy:
      - <test key>: <template>
        <test key>: <template>
        ...

      - <test key>: <template>
        <test key>: <template>
        ...

A policy is applied by tmt after metadata are finalized. It is
applied on fully materialized tests, i.e. after reading fmf files,
``adjust`` evaluation, and after all command-line options were taken
into account. All values would also be normalized, e.g. ``tag`` or
``contact`` keys would always be lists of strings, and so on. At
this point, policies have access to the most final content of test
keys.

When a policy is applied:

* the template string is rendered as a
  :ref:`jinja2 template <custom_templates>`.
* the rendered output is treated as if it were
  :ref:`test metadata </spec/tests>` from an fmf file. Text is
  parsed and normalized as any other fmf input.
* finally, it replaces the original value of the specified metadata
  key.

.. important::

   New data **replaces** the original value of the key. If the
   original value needs to be reflected in the new value, it is the
   responsibility of the rule and its template to include it
   accordingly. There are no "magic" operators for addition or
   merging provided by the policy syntax, new value fully replaces
   the old one.

   .. code-block:: yaml

      test-policy:
        # The following will erase all checks tests may have defined,
        # and every test will gain just a single `dmesg` check instead:
        - check: |
            - how: dmesg

        # Use `VALUE` to propagate the original content:
        - check: |
            - how: dmesg

            {% for item in VALUE %}
            - {{ item | to_yaml | indent(2, first=False) }}
            {% endfor %}

.. important::

   Since the task of the template is to produce valid fmf data, the
   indentation of its content is crucial:

   .. code-block:: yaml

      test-policy:
        - check: |
            - how: dmesg

              # The following will not work because `check` is a list
              # of checks, and therefore `VALUE` will contain a list
              # which, if emitted into template this way, would produce
              # invalid fmf (a list nested in a mapping).
              {{ VALUE }}

.. important::

   For emitting original values more complex than strings or numbers,
   it is highly recommended to use ``to_yaml`` and ``indent`` filters.
   Otherwise, some Python data types may render incorrectly:

   .. code-block:: yaml

      # Incorrect outcome when `unit` key is not set
      - {{ check }}

      # [{"how": "journal", ..., "unit": "None"}]

      # On the other hand, `to_yaml` filter produces correct "value"
      # for the `unit` key:
      - {{ check | to_yaml | indent(2, first=False) }}

      # - how: journal
      #   unit:

      After all, the goal here is to emit an fmf snippet, and fmf
      is a YAML at its core, therefore ``to_yaml`` filter will lead
      to the most safe result.

Within the templates, additional variables are defined:

* ``VALUE``: the original value of the metadata key being modified.
* ``VALUE_SOURCE``: source of the original value: ``fmf`` when the
  value was set by fmf data, ``cli`` when the value was set via
  command line, ``default`` when the value is the default value of
  the key, i.e. no other explicit value was set, and ``policy`` when
  the value was set by previous policy instruction.
* ``TEST``: the entire test object, allowing you to access any of
  its metadata fields for conditional logic (e.g., ``TEST.contact``,
  ``TEST.component``).

Policy can be passed to tmt in two ways:

* as a file path, via ``--policy-file`` option or ``TMT_POLICY_FILE``
  environment variable,
* as a policy "name", via ``--policy-name`` option or
  ``TMT_POLICY_NAME`` environment variable. Policy name is
  translated into a file path, and the file path is expected to
  exist under a policy root:

  .. code-block:: shell

     --policy-name foo     => <policy root>/foo.yaml
     --policy-name foo/bar => <policy root>/foo/bar.yaml

.. important::

   Be aware that a policy root, specified via ``--policy-root``
   option or ``TMT_POLICY_ROOT`` environment variable, affects how
   policies are located.

   * Policies specified by their file path can be given either as
     absolute paths, or relative paths. Relative paths are
     interpreted either against the current working directory, or
     against the policy root if it is specified. In both cases, if
     policy root is specified, the final policy file path must be
     located under the policy root directory.
   * Policies specified by their name must be also located under the
     policy root directory, and defining the policy root is even
     mandatory as it serves as the base directory for locating such
     policies.

.. versionadded:: 1.50


**Examples:**


.. code-block:: yaml

   # To unconditionally change a key value, provide the new value
   # directly - the template does not need to use any variables or
   # advanced constructs.

   test-policy:
     # Sets the test duration to 24 hours for all tests.
     - duration: 24h



.. code-block:: yaml

   # Conditional modification: use flow controls (e.g., if/else) to
   # change values based on existing test metadata.

   test-policy:
     # Adds a specific command prefix only if the test is owned by
     # 'foo-sst@redhat.com'.
     - test: |
         {% if "foo-sst@redhat.com" in TEST.contact %}
         scl enable gcc-toolset-15 {{ VALUE }}
         {% else %}
         {{ VALUE }}
         {% endif %}



.. code-block:: yaml

   # Add items only if they (or similar items) don't already exist:

   test-policy:
     # Adds a default 'avc' check only if no 'avc' check is already
     # defined. Preserves all original checks.
     - check: |
         {# If no 'avc' check has been defined, inject the default one. #}
         {% if 'avc' not in VALUE | map(attribute='how') %}
         - how: avc
           result: respect
         {% endif %}

         {# Make sure to include checks already picked by the test #}
         {% for item in VALUE %}
         - {{ item }}
         {% endfor %}


**Status:** implemented and verified

* Implemented by `/tmt/policy.py <https://github.com/teemtee/tmt/tree/1.63.0-9-gcda563a/tmt/policy.py>`_


* Verified by `/tests/policy <https://github.com/teemtee/tmt/tree/1.63.0-9-gcda563a/tests/policy/main.fmf>`_