Clean misplaced ERROR/WARNING messages triggered by the queue cleanup runner
As noted by @jleduc in the issue https://gitlab.cern.ch/cta/operations/-/issues/937#note_6265136, the new queue cleanup runner (added on release 4.8.0
and executed on the maintenance process) is issuing a large number of WARNING messages.
These messages are most probably harmless and do not imply that the object store is performing badly. Therefore, they should be re-evaluated and, if justifiable, have their priority reduced or be removed.
References
-
The monitoring tool: https://meter-cta.web.cern.ch/d/ZxPmpXOWk/errors?orgId=1&from=now-7d&to=now&viewPanel=9&refresh=1m
-
Some of the messages:
In OStoreDB::getRetrieveQueuesCleanupInfo(): fetched a retrieve queue and that lasted more than 1 second.
In QueueCleanupRunner::runOnePass(): Unable to find the retrieve queue for cleanup. Queue may have already been deleted. Skipping it.
WARNING: In QueueCleanupRunner::runOnePass(): Cleaned up tape is not in a PENDING state. Unable to change it to its corresponding final state.
In QueueCleanupRunner::runOnePass(): Queue is has cleanup flag enabled but is not in the expected PENDING state. Skipping it.
Edited by Joao Afonso