ATR-21622 implementation of the HLTMonitoringJSON module

Implementation of the HLTMonitoringJSON module. Related to ATR-21622

assigned to @ckaldero, @markowen, and @mark

Hi Everyone, I implemented a first version of the module HLTMonitoringJSON. You can also find a test file Mon_METSlice.json that I created for testing the module on just one Slice. I thought I integrated the call of the function in all the places where it was necessary, but when running the ART jobs I don't see any of the print statements that the module should produce. I already checked multiple times that I'm compiling the folders I modified. If any of you could find 5 min to browse the changes I made it would be very helpful.

Once I'll have this running over the test file, it would be good to define how to organize the input for configuring multiple Slices.

Thanks for the help,

Giani

Hi @gipezzul, I just tried to run your branch (test_trig_data_v1Dev_build.py) and it does seem that your changes are running - the test first died because it couldn't find the input Mon_METSlice.json. I hacked round that and then I get some real crash in the json code. How exactly are you setting up the release / compiling / testing?

Cheers

Mark

Hi @markowen , thanks for looking this up. I'm doing the following steps: (from the build dir)

setupATLAS
asetup Athena,master,2020-10-24T2101
cmake -DATLAS_PACKAGE_FILTER_FILE=../package_filters.txt ../athena/Projects/WorkDir

where the package file is:

+ Trigger/TrigConfiguration/TrigConfigSvc
+ Trigger/TriggerCommon/TriggerMenuMT
- .*

the from the /run directory, I'm running:

source ../build/x86_64-centos7-gcc8-opt/setup.sh
runTrigART.py -m -j4

then I look for the HLT Monitoring printout: find runTrigART -name *.log|xargs grep "Generating HLT Monitoring"

and it finds nothing.

If you could point me the error I'm making it would be great for me continuing this debugging.

Thanks again for the help,

Giani

I don't spot anything immediately wrong there. Running all the ART tests is maybe overkill anyway for first tests, can you try running just test_trig_data_v1Dev_build.py and then posting the log somewhere?

Cheers

Mark

added 2 commits

• c665eea5 - fixed structure of the json file
• 3afb3f5b - fixed bugs + added debug printout level

Compare with previous version

Hi @markowen , good news! I got the script working. I just committed the fixes. To run the test successfully, you need to copy the Mon_METSlice.json file in the dir where you run the script.

You can see the json the script make here: /afs/cern.ch/work/g/gipezzul/public/athena_Trigger/run/HLTMonitoring_LS2_v1_22.0.19.json

It might be great to have a quick chat for deciding:

• where should live the Mon_config file
• if we are OK with the structure of the HLTMonitoring json I made
• what should be loaded/uploaded from the DB

Thanks for the help,

Giani

Hi @gipezzul, sorry for taking so long to get to this. I put a few comments in the code. Then I have one other design comment, I think what we want to do is have the input in python, rather than this json file Mon_MetSlice.json. This has the benefit that e.g. if people have groups of chains with the same monitoring settings, they can use loops in the python, rather than listing every chain. Then these python functions can be imported into your json creating code & you output the json. Let me know if that makes sense.

Mark

added 1 commit

• 6dabc19d - changed function name to generateMonitoringJSON, and removed unused input...

Compare with previous version

added 2 commits

• 22fd17d6 - updates
• 495ac81c - test file

Compare with previous version

added 1 commit

• de7bae14 - crated new dir for the HLTMon script and config files

Compare with previous version

Hi @markowen , I integrated the modifications we discussed last time we chatted. I put in place as an example a few chains for the MET slice, but the script is already structured for being scaled-up to process multiple slices at the same time. Is there a place where I can get the config for the other slices to create other place-holders or you prefer to ask directly to the various sub-groups to implement their part themselves?

Cheers,

Giani

Thanks @gipezzul. Could you write a brief summary of what you have done so far & how it should work on the JIRA (i.e. signatures will update python files and then the code can create JSON files, checking for consistency with the menu)?

Thanks

Mark

added 1 commit

• 26b8d895 - improved structure for storing the chains within a given signature

Compare with previous version

added 4159 commits

• 26b8d895...0a56170c - 4158 commits from branch atlas:master
• c32ec321 - Merge branch 'master' of ssh://gitlab.cern.ch:7999/atlas/athena into offline_mon

Compare with previous version