Progress towards a working ZDC CA configuration (both calibration and reco)

Contains a fully-functional ZDC CA configuration, which works on calibration and physics data. Requires one small patch to RecoTransforms to re-enable ZDC.

Checking in with @tbold - I have the basic pieces set up (e.g. byte stream config, output containers) and I know what I want to do (configure a tool and give it to the ZdcRec algs, ZdcRecV3 for Run 2 and ZdcRecLucrodData for Run 3)

I tried that but I couldn’t get the syntax right. I’ll keep scouring other examples but I’d be happy to be pointed to one that works.

added 1 commit

• 4a4ebaf5 - seems to work after figuring out private tool configuration, after...reading the documentation

Compare with previous version

OK I read the documentation, and I seem to have the syntax right -- and it seemed to run without problems. @tbold can you have a look?

added 1 commit

• bf0e5c68 - major changes: restructured to resemble AFP config in ForwardRec but keeping...

Compare with previous version

added 1 commit

• 81b11789 - fixes to run with HI transform

Compare with previous version

marked this merge request as ready

changed title from Draft: steps towards a working ZDC calib CA config to Progress towards a working ZDC CA configuration (both calibration and reco)

changed the description

This merge request affects 2 packages:

• ForwardDetectors/ZDC/ZdcRec
• Reconstruction/RecJobTransforms

Affected files list will not be printed in this case

Adding @goetz as watcher

added 23.0 ForwardDetectors Reconstruction review-pending-level-1 labels

added Run3-DataReco-output-changed label

CI Result FAILURE (hash 81b11789)

	Athena
externals
cmake
make
tests

Full details available on this CI monitor view. Check the JIRA CI status board for known problems
Athena: number of compilation errors 0, warnings 0
For experts only: Jenkins output [CI-MERGE-REQUEST-CC7 70348]

added NewConfig label

This seems to be an actual problem

  21:59:35 Traceback (most recent call last):
  21:59:35   File "/build/ci-builds/23.0/Athena/build/Athena/AtlasTest/CITest/CMakeFiles/ciTestRun/RecoRun2Data/run_q442/runargs.RAWtoALL.py", line 49, in <module>
  21:59:35     fromRunArgs(runArgs)
  21:59:35   File "/build2/ci-builds/23.0/Athena/build/Athena/x86_64-centos7-gcc11-opt/python/RecJobTransforms/RAWtoALL_Skeleton.py", line 170, in fromRunArgs
  21:59:35     cfg = RecoSteering(flags)
  21:59:35   File "/build2/ci-builds/23.0/Athena/build/Athena/x86_64-centos7-gcc11-opt/python/RecJobTransforms/RecoSteering.py", line 208, in RecoSteering
  21:59:35     acc.merge(ZdcRecCfg(flags))
  21:59:35   File "/build2/ci-builds/23.0/Athena/build/Athena/x86_64-centos7-gcc11-opt/python/ZdcRec/ZdcRecConfig.py", line 110, in ZdcRecCfg
  21:59:35     acc.merge(ZdcRecRun2Cfg(flags))
  21:59:35   File "/build2/ci-builds/23.0/Athena/build/Athena/x86_64-centos7-gcc11-opt/python/ZdcRec/ZdcRecConfig.py", line 65, in ZdcRecRun2Cfg
  21:59:35     acc.merge(ByteStreamReadCfg(flags, type_names=['xAOD::TriggerTowerContainer/ZdcTriggerTowers',
  21:59:35 NameError: name 'ByteStreamReadCfg' is not defined

removed review-pending-level-1 label

added review-user-action-required label

This is "expected" I assume

INFO       xAOD::Init                INFO    Environment initialised for data access
INFO       Py:diff-root         INFO comparing tree [CollectionTree] in files:
INFO       Py:diff-root         INFO  old: [/cvmfs/atlas-nightlies.cern.ch/repo/data/data-art/WorkflowReferences/master/q449/v32/myAOD.pool.root]
INFO       Py:diff-root         INFO  new: [../RecoRun3Data/run_q449/myAOD.pool.root]
INFO       Py:diff-root         INFO branches of interest: set()
INFO       Py:diff-root         INFO ignore  leaves:       ['index_ref', '(.*)_mems(.*)', '(.*)_timings(.*)', '(.*)HLTNav_Summary_AODSlimmedAux(.*)', '(.*)TrigCompositeAuxContainer_v2_HLTNav_Summary_ESDSlimmedAux(.*)']
INFO       Py:diff-root         INFO enforce leaves:       ('BCID',)
INFO       Py:diff-root         INFO leaves prefix:
INFO       Py:diff-root         INFO hacks:                ('m_athenabarcode', 'm_token')
INFO       Py:diff-root         INFO entries:              60
INFO       Py:diff-root         INFO mode:                 semi-detailed
INFO       Py:diff-root         INFO error mode:           resilient
INFO       Py:diff-root         INFO order trees:          True
INFO       Py:diff-root         INFO exact branches:       True
INFO       Py:diff-root         INFO comparing over [60] entries...
INFO       Py:diff-root        ERROR the following variables exist only in the new file !
INFO       Py:diff-root        ERROR  - [ZdcModules]
INFO       Py:diff-root        ERROR  - [ZdcModulesAux.]
INFO       Py:diff-root         INFO comparing [2141] leaves over entries...
INFO       Py:diff-root         INFO Found [8577707] identical leaves
INFO       Py:diff-root         INFO Found [2] different leaves
INFO       Py:diff-root        ERROR files differ!

ping @jmaurer and @strichte .

added 1 commit

• 7c6b5c53 - fixes from MR comments

Compare with previous version

This merge request affects 2 packages:

• ForwardDetectors/ZDC/ZdcRec
• Reconstruction/RecJobTransforms

Affected files list will not be printed in this case

Adding @goetz as watcher

added review-pending-level-1 label and removed review-user-action-required label

added 1 commit

• bfecb846 - trying to add ByteStreamReadCfg in the right place, but it works for me in all setups without it

Compare with previous version

This merge request affects 2 packages:

• ForwardDetectors/ZDC/ZdcRec
• Reconstruction/RecJobTransforms

Affected files list will not be printed in this case

Adding @goetz as watcher

CI Result FAILURE (hash 7c6b5c53)

	Athena
externals
cmake
make
tests

Full details available on this CI monitor view. Check the JIRA CI status board for known problems
Athena: number of compilation errors 0, warnings 1
For experts only: Jenkins output [CI-MERGE-REQUEST-CC7 70356]

CI Result FAILURE (hash bfecb846)

	Athena
externals
cmake
make
tests

Full details available on this CI monitor view. Check the JIRA CI status board for known problems
Athena: number of compilation errors 0, warnings 1
For experts only: Jenkins output [CI-MERGE-REQUEST-CC7 70357]

Changing label to review-user-action-required until the discussions are resolved. Clara (L1)

added review-user-action-required label and removed review-pending-level-1 label

Hi Petr, The ByteStreamReadCfg issue is indeed confusing. Jobs fail because it is missing, yet the python sanitised says that it unused. I think you can just import it at the top of the script. BTW. you can test the failing job also locally following this instructions: https://gitlab.cern.ch/atlas/athena/-/tree/23.0/AtlasTest/CITest

added 1 commit

• 26d17490 - added ByteStreamReadCfg to global scope, and removed stray includes. Added...

Compare with previous version

This merge request affects 2 packages:

• ForwardDetectors/ZDC/ZdcRec
• Reconstruction/RecJobTransforms

Affected files list will not be printed in this case

Adding @goetz as watcher

added review-pending-level-1 label and removed review-user-action-required label

added Run2-DataReco-output-changed label

CI Result FAILURE (hash 26d17490)

	Athena
externals
cmake
make
tests

Full details available on this CI monitor view. Check the JIRA CI status board for known problems
Athena: number of compilation errors 0, warnings 0
For experts only: Jenkins output [CI-MERGE-REQUEST-CC7 70426]

Ok so not we need reference update I assume

ERROR    Please make sure this has been discussed in the correct meeting (RIG or Simulation) meeting and approved by the relevant experts.
ERROR    Copied '/build/ci-builds/23.0/Athena/build/Athena/AtlasTest/CITest/CMakeFiles/ciTestRun/RecoRun2Data/run_q442/myAOD.pool.root' to '/eos/atlas/atlascerngroupdisk/proj-sit/gitlabci/MR62435_26d174908707bd17a18e1560afabc666e6ac0cac/RecoRun2Data'
INFO       Py:diff-root        ERROR the following variables exist only in the new file !
INFO       Py:diff-root        ERROR  - [ZdcModules]
INFO       Py:diff-root        ERROR  - [ZdcModulesAux.]
INFO       Py:diff-root        ERROR files differ!
ERROR    Your change breaks the frozen tier0 policy in test q442.
ERROR    Please make sure this has been discussed in the correct meeting (RIG or Simulation) meeting and approved by the relevant experts.
ERROR    Copied '/build/ci-builds/23.0/Athena/build/Athena/AtlasTest/CITest/CMakeFiles/ciTestRun/RecoRun2Data/run_q442/myESD.pool.root' to '/eos/atlas/atlascerngroupdisk/proj-sit/gitlabci/MR62435_26d174908707bd17a18e1560afabc666e6ac0cac/RecoRun2Data'
INFO       Py:diff-root        ERROR the following variables exist only in the new file !
INFO       Py:diff-root        ERROR  - [DataVector<xAOD::ZdcModule_v1>_ZdcModules]
INFO       Py:diff-root        ERROR  - [xAOD::ZdcModuleAuxContainer_v2_ZdcModulesAux.]
INFO       Py:diff-root        ERROR files differ!

added review-user-action-required label and removed review-pending-level-1 label

Marking this as user action required until threads are resolved.

Kira (L1)

I have a new problem for experts - when running on a ZDC calibration file (i.e. a very extreme PEB file, with just ZDC+CTP), i'm apparently not getting proper metadata so it's getting the wrong Run2 or Run3 enum value. It seems to be defaulting to Run2, so worked before for 2015/2018 files, but not for 2022. Can I get some expert advice on this?

marked this merge request as draft

added 1 commit

• 652e1d6b - a few typos fixed, and standlone configured for Run 2 calibrations. Run 3...

Compare with previous version