json loads fail in LocalCalibration + pedestals
When I tried to run scripts/HGCALPedestals.py I encountered an error regarding the loading of json.py.
It gets fixed by changing json.py to json_merger.py - @wangj I think you have noticed this
@ineuteli - there is now something I can't resolve which is modules that do import json
which json is it? the python module or the json_merger.py ? ;)
Then @lurda I made a bit of modifications to parallelize further the script and noticed that interface/helpers.h was missing- added here.
I took the chance to measure a bit the time performance (on a random lxplus9 node). Here is what I get
| #LD modules | #channels | Histo filling [s] | Correction maker [s] |
|---|---|---|---|
| 1 | 222 | 55.8 | 26.1 |
| 2 | 444 | 65.2 | 26.8 |
| 3 | 666 | 64.8 | 28.6 |
| 4 | 888 | 75.6 | 29.4 |
| 5 | 1110 | 86.87 | 32.4 |
| 6 | 1332 | 87.7 | 31.2 |
on a 284k events run, running the command
python3 scripts/HGCALPedestals.py -r 1710429303 -i /eos/cms/store/group/dpg_hgcal/tb_hgcal/2023/labtest/pro/ -o Pedestals/Run1710429303 --maxModules 6
we can optimize histo filling time (and also mem usage) using boost histograms. But as this will be changed to the generic "injection scan script" I would do it when merging the new script.