Periodic task max limit
Found an issue on preserve-qa where 4 periodic check_am_status
tasks were running for multiple days.
Possible solutions:
- Limit the number of repetitions for a periodic task (consider the scenario where we are waiting for AM to process a bigger batch - difficult to estimate - add possibility for manual trigger?)
- For admin users add a tab to see all periodic tasks (it's a postgresql table - oais_db.django_celery_beat_periodictask) and delete them potentially.
Manual workaround:
- Go to postgresql pod terminal
psql
-
\c oais_db
(connect to DB) -
select * from django_celery_beat_periodictask;
(list active periodic tasks) -
delete from django_celery_beat_periodictask;
(where ...) (remove them if they are not needed) - Restart celery pod