Ensure CI fails if there are error messages present in the taped/frontend logs
We had an incident where a large number of errors were produced in production, while these were already present during the CI run as well. However, the system tests don't check for errors in the logs, so this went unnoticed. As part of the post-run checks, the system tests should check if there are any log messages of severity ERROR
and CRITICAL
.
The check for this is trivial to implement, but we also have errors being produced as part of the tests. We should review all of these errors:
- are these errors the result of a bug?
- should these errors not be warnings instead?
I will start by creating a check for the error messages and then we can inventorise all messages that are caught by this.