Test attestation Package

Conforma can verify test result attestations attached to images as in-toto statements. This package inspects the content of verified test-result predicates and produces violations for failed tests and warnings for warned tests. The package is a no-op when no test-result attestations are present.

Package Name

  • test_attestation

Rules Included

No failed test attestations

Produce a violation if any test result attestation has a result of "FAILED". Failed test names from the attestation predicate are included in the message when available.

Solution: Ensure all test attestations have a passing result. Review the failed tests listed in the attestation predicate.

  • Rule type: FAILURE

  • FAILURE message: Test attestation %q has a failed result, failed tests %s

  • Code: test_attestation.no_failed_tests

  • Source

No test attestation warnings

Produce a warning if any test result attestation has a result of "WARNED". Warned test names from the attestation predicate are included in the message when available.

Solution: Review the warned tests listed in the attestation predicate.

  • Rule type: WARNING

  • WARNING message: Test attestation %q has warnings, warned tests %s

  • Code: test_attestation.no_test_warnings

  • Source

No unsupported test attestation result values

Ensure the result field of each test result attestation is a recognized value. Valid values are PASSED, WARNED, and FAILED per the in-toto test-result predicate specification.

Solution: The test result attestation contains an unrecognized result value. Valid values are PASSED, WARNED, and FAILED.

  • Rule type: FAILURE

  • FAILURE message: Test attestation %q has an unsupported result value %q

  • Code: test_attestation.test_result_known

  • Source

Test attestation data includes result

Each test result attestation must include a result field in its predicate. Verify that the result field is present.

Solution: The test result attestation predicate must include a "result" field with a value of PASSED, WARNED, or FAILED.

  • Rule type: FAILURE

  • FAILURE message: Test attestation %q is missing the required result field

  • Code: test_attestation.test_data_found

  • Source