Expand on DistComp processing plots of number of files or luminosity versus time
As the README now points out, we have plots of number of files or luminosity versus time available from the Computing Processing passes of Re-stripping cycles page for several Stripping campaigns, effectively showing how the processing went versus time.
This task collects feature requests in discussion with @cbozzi and others:
-
Produce the same plot for the WIP ReStripping campaign for 2017 and 2018 (2016 plot available already).
==> will be done as the production jobs start.
-
As we are only starting with Sprucing, it would be really nice to have similar plots for the campaigns done this year - the very first ones - and the future ones.
==> This will be implemented in the near future.
-
Currently, plots do not have the year displayed anywhere. Seems like a useful little piece of info to add. Also, the plots cannot be saved in any way, for example for a report. Only a screenshot + crop and save is possible, which is not great.
==> Proposal to address this in the following way given that the workflow is:
1) A cron job is prepared and a Python script is executed every night, which queries both the bookkeeping and the production databases; the cronjob is run from a service account (lbdirac). 2) The outcome is a JSON file that is copied over to an EOS area (/eos/project/l/lhcbwebsites/www/lhcbproject/lbdirac/Reprocessing/) that should be publicly accessible from lxplus. 3) The JSON is consumed by two PHP scripts that generate the plots on the fly by using a web browser.
In principle one can take the JSON file generated in 2) (filename is in the form data-re-strippingXXrYpZ.json) and consume it with whatever suits better (matplotlib?), adding info / saving / etc.
CC @fredi, @avenkate, @ngrieser, @nskidmor, @abertoli, @shunan.