Skip to content

[CI] Fail system tests on any non-whitelisted error messages

Niels Alexander Buegel requested to merge 1179-ensure-ci-fails-if-there-are-error-messages-present-in-the-taped-frontend-logs into main

Description

The code is not the prettiest, but it should work and we can rewrite it once we get the Python system tests.

  • Add check that fails the system test when there are messages of the type ERROR or CRITICAL
  • Certain messages can be added to a whitelist as acceptable, due to the fact that our system tests might produce these errors intentionally.
  • The whitelist file for a test <test-script-name>.sh should reside in error_whitelists/<test-script-name>.txt
  • Added timeout to the stress test. The default is 2 hours, but we need more (unrelated, but quick fix)

In the future this should be done much nicer once we have the Python system tests: allow test cases to specify which error messages it might produce instead of keeping track of a central file.

Checklist

  • Documentation reflects the changes made.
  • Merge Request title is clear, concise, and suitable as a changelog entry. See this link

References

Closes #1179 (closed)

Edited by Niels Alexander Buegel

Merge request reports