Clean up test artifacts
We should have a daily clean up celery task to clean EOS.
- The locations that can contain test artifacts are here.
- Information about the files can be found with something like:
eos find -f -ctime +14 --size /eos/lhcb/wg/dpa/wp2/ci/
I suggest we chose a fairly simple tiered strategy, something like:
- Files which are more than 1GB are kept for 1 week
- Files which are more than 100MB are kept for a month
- Other files are kept for 6 months
It might be worth quickly estimating how much data this would corrospond to and tuning the thresholds.
We could do something more complex based upon if it's the most recent pipeline, if it's an intetmediate file, etc., however I don't think it's worthwhile.