Skip to content

Add revoke_tasks task in Celery beat

Maciej Pawel Szymanski requested to merge maszyman-beat-ttl into master

This MR sets up the beat service in celery app and adds revoke_tasks periodic task which runs every 3h to terminate the tasks running for more than 24h.

The message-ttl mechanism from !46 (merged) is not enough, because it covers only the case of the messages that stay unconsumed in the queue. However, it may well be that message was consumed by the worker, but never acked (this happens only after the task finished with late acknowledgement enabled).

This will work when we start the celery worker with beat service enabled: https://gitlab.cern.ch/lhcb-core/nightly-builds/lb-nightly-deployment/-/merge_requests/63

Note that it's based on !46 (merged) (this one requires the new setup of the queues set up there), so please merge that one first.

The actual diff is: 749fa8d5

Edited by Maciej Pawel Szymanski

Merge request reports