Resolve "Tape server querying DB at very high rate"

Summary

This MR is to fix this specific cause of high DB queries: #155 (comment 6266313)

Requires manual tests in pre-production

NO. The high query rate is only evidenced when there are many tape servers and a high rate of retrieve requests. Best way to test this is in production:

1. Get an updated query rate from Nilo
2. Deploy the version of CTA containing this fix
3. Get a new query rate and it should hopefully be much lower

References

Closes #155 (closed)

added File catalogue Scheduler Tape server workflowIn Progress labels

requested review from @dhsmith

assigned to @mdavis

added 1 commit

• 1f4111ce - Adds separate cache of tape statuses

Compare with previous version

mentioned in issue #155 (closed)

added 1 commit

• 0c2bd45a - Also purge tape cache when purging retrieve stats cache

Compare with previous version

added 4 commits

• 017d0d64 - 1 commit from branch main
• ed9b9ce5 - Adds separate cache of tape statuses
• a4cd1eef - Also purge tape cache when purging retrieve stats cache
• e68409bd - Updates Release Notes

Compare with previous version

added 1 commit

• 28ac7968 - Handles corner case where second candidate vid gets evicted from the cache

Compare with previous version

added 5 commits

• 02c1b39c - 1 commit from branch main
• a1b1c9f8 - Adds separate cache of tape statuses
• 5f132c67 - Also purge tape cache when purging retrieve stats cache
• 20e6fcaf - Updates Release Notes
• be158c83 - Handles corner case where second candidate vid gets evicted from the cache

Compare with previous version

Hi Michael. Looks fine. In updateRetrieveQueueStatisticsCache I wondered about also faking the entry in g_tapeStatuses if one didn't exist, but as far as I can see usually the entries should exist, probably only a corner case of just-expired entry where it may not, in which case adding the entry g_tapeStatues in the usual way in selectBestRetrieveQueue should be a small number of queries.

That's all I can think of, I'll add the 'approve' mark to this MR.

approved this merge request

marked this merge request as ready

enabled an automatic merge when the pipeline for be158c83 succeeds

aborted the automatic merge because target branch was updated

added 5 commits

• 1bfc2590 - 1 commit from branch main
• 3e260e2e - Adds separate cache of tape statuses
• cd9ae5bd - Also purge tape cache when purging retrieve stats cache
• 2403464d - Updates Release Notes
• a738cce4 - Handles corner case where second candidate vid gets evicted from the cache

Compare with previous version

approved this merge request

enabled an automatic merge when the pipeline for a738cce4 succeeds

merged