Fixes for Stripping21rXp1 campaigns (stripping21-patches)
Repositories for Stripping21rXp1 campaigns (2011 & 2012 re-stripping campaigns) need to be "modernised" in order to have the same structure as Run 2 campaigns.
The branch is stripping21-patches. Please check out the corresponding package from this branch, create a new branch and work on it. The main tasks to carry out here are:
Distribute the line builders per WG
All line builders in Phys/StrippingSelections are mixed together, and this has to be sorted out. To do this, an idea would be to grep per WG and move the files to the corresponding WG folder (that has to be created). An example in bash (needs to be tested) to classify QEE, B&Q and RD builders (should be done for all WGs) would be (to be ran at python/StrippingSelections):
for wg in BandQ QEE RD
do
mkdir Stripping${wg} && grep -l -Z -r "\['${wg}'\]" . | xargs -0 -I{} mv {} Stripping${wg}/
doneAfter the classification is done, auxiliar python files have to be added to the WG folders as it is done for 2018-patches (please use this branch as a guideline). To know which files should be imported and which not, one can check the __init__ file in python/StrippingSelections and reproduce the same imports but per WG (each WG folder needs an __init__ file). More additional files might be needed, i.e. Utils.py, please use 2018-patches as a guideline on what put there (if you have doubts please let me know). Also, please be very careful and always compare to 2018-patches so you are sure you are not missing any file.
Addendum: it seems some of the builders do not use the current structure of default_config, hence they are not parseable by the testing scripts. This should be fixed. Also, some of the builders do not include a WG tag. Liaisons should be requested to classify these builders.
Fix the test scripts to test from Selections
There are two different test scripts, one for Run 1 and another one for Run 2. This is because the structure is different in Selections (see above). Once the Selections are fixed, these scripts need to be updated as well (for both users and liaisons). As before, use 2018-patches as a guideline. You might need to do changes as well to the StrippingStream logic (or any Configurables that affects the call of any line builder by the dictionary name), which is hosted in the Stripping project as well. Just by having a look to the test script, you will see which Configurables are used and therefore what needs to be updated in case there is a failure when running the script. Please be sure the tests are running for all WGs.
Fix the test scripts to test from Settings
Same as above, but running from Settings instead of Selections. Changes here should be minimal, if Settings needs to be changed, please let me know. You can use Stripping21(r1) settings for this test. Just by having a look to the test script, you will see which Configurables are used and therefore what needs to be updated in case there is a failure when running the script. Please be sure the tests are running for all WGs.
Final clean-up and assure consistency
Clean-up the old test and data files (add new db tags) and utils for liaisons, i.e. dictionaries collection. Also, do a global check to be sure everything is consistent.
Propagate changes to StrippingArchive
This is the most complicated part, ensure we can run previous Run 1 campaigns Stripping21(rXp0) from the archive and there are zero problems. A similar work as with Selections should be done here but in StrippingArchive (be careful of differences with default_config). Changes to some Configurables may be needed as well. This should be left for the end, as it only affects the AppConfig options and the coordinators tests.
Once a task is completed, please tick it below: