Skip to content

add priority, retries and expiration settings for tasks

Maciej Pawel Szymanski requested to merge maszyman-priority-retry into master

In this MR, I'm setting the priorities for the nightly tasks. I make use of https://gitlab.cern.ch/lhcb-core/LHCbNightlyConf/-/raw/master/cvmfs-priorities.yaml which is used to define the priorities with which slots are installed on cvmfs using lbtaskweb and lbtaskrun. I took and adapted some code from https://gitlab.cern.ch/lhcb-core/tasks/lbtaskweb/-/blob/master/src/lbtaskweb/hooks/build_ready.py to use the similar logic for calculating the priorities. The only exception is that we allow for 0 priority in the case of deployment (we do not deploy every slot), but of course, we want to build all the slots that we defined.

In essence, the task priority is obtained by multiplying priorities for slot, project, and platform (for checkout task it's only slot and project, and for start_slot task, it's just a slot priority). I also decrease slightly the priority for the test task in case it competes with the build task.

Besides, I added some settings for expiration (1 day) and retry policy (3 retries for checkout and build, 1 retry for test + some interval settings) which of course we may refine. Closes #13 (closed).

Note that this change has to be synchronised across worker_env, scheduler_env and frontend.

Edited by Maciej Pawel Szymanski

Merge request reports