Correct pixel delta ray threshold in pixel digitisation

Correct pixel delta ray threshold in pixel digitisation and some comments to highlight that there is a link between the range cut in simulation and the energy threshold in the digitisation.

@battagl @stsuno @gfacini @srettie @vcairo

Jenkins please retry a build

This merge request affects 2 packages:

• InnerDetector/InDetDigitization/PixelDigitization
• Simulation/G4Atlas/G4AtlasTools

Affected files list will not be printed in this case

Adding @jchapman ,@goetz ,@stsuno ,@lshan ,@battagl ,@sroe as watchers

added Digitization InnerDetector Simulation master labels

CI Result FAILURE (hash 3adcd226)

	Athena	AthSimulation	AthGeneration	AnalysisBase	AthAnalysis	DetCommon
externals
cmake
make
required tests
optional tests

Full details available on this CI monitor view
Athena: number of compilation errors 4, warnings 0
AthSimulation: number of compilation errors 3, warnings 0
AthGeneration: number of compilation errors 0, warnings 0
AnalysisBase: number of compilation errors 0, warnings 0
AthAnalysis: number of compilation errors 1, warnings 0
DetCommon: number of compilation errors 0, warnings 0
For experts only: Jenkins output [CI-MERGE-REQUEST-CC7 42078]

Hi @amorley, should this change also for Run 2 setup or is this only for Run 3 and beyond?

added 1 commit

• b75c6415 - update reference file for q221

Compare with previous version

added changes-digitization-output label

added Run2-MCOverlay-output-changed label

added Run2-DataReco-output-changed label

@tadej What is the procedure to update the reference files on CVMFS/EOS?

Copy them somewhere accessible to me and I'll update them.

Edit: sorry, I realised I was not clear enough. You should run RunTier0Test.py -o and then copy those files somewhere.

Hi @tadej,

I have moved the results into /afs/cern.ch/user/a/amorley/public/MR48161/ including the data overlay file which changes with this MR.

Cheers,

Anthony

added 254 commits

• b75c6415...c7471355 - 252 commits from branch master
• 24198893 - Correct cut value with a more accurate one directly from G4
• eef3f9e0 - Merge remote-tracking branch 'upstream/master' into PixelDeltaRayThreshold

Compare with previous version

marked this merge request as ready

This merge request affects 3 packages:

• InnerDetector/InDetDigitization/PixelDigitization
• Simulation/G4Atlas/G4AtlasTools
• Tools/PROCTools

Affected files list will not be printed in this case

Adding @jchapman ,@goetz ,@stsuno ,@lshan ,@battagl ,@sroe as watchers

added Tools review-pending-level-1 labels

FYI, The value of the cut has change slightly so the test results results will likely change, i am currently rerunning the T0 tests locally and will update when they are completed.

CI Result FAILURE (hash eef3f9e0)

	Athena	AthSimulation	AthGeneration	AnalysisBase	AthAnalysis	DetCommon
externals
cmake
make
required tests
optional tests

Full details available on this CI monitor view
Athena: number of compilation errors 0, warnings 0
AthSimulation: number of compilation errors 0, warnings 0
AthGeneration: number of compilation errors 0, warnings 0
AnalysisBase: number of compilation errors 0, warnings 0
AthAnalysis: number of compilation errors 0, warnings 0
DetCommon: number of compilation errors 0, warnings 0
For experts only: Jenkins output [CI-MERGE-REQUEST-CC7 42317]

I see the following error in OverlayTier0Test_required-test. Is this expected?

[CI_TEST_WRAPPER] minutes elapsed: 16 
ERROR    Py:diff-root         INFO  [InDetRawDataContainer_p2_PixelRDOs.m_collections.m_begin]: 15291 leaves differ

ERROR    Py:diff-root         INFO  [InDetRawDataContainer_p2_PixelRDOs.m_collections.m_end]: 15300 leaves differ

ERROR    Py:diff-root         INFO  [InDetRawDataContainer_p2_PixelRDOs.m_rawdata.m_rdoId]: 170553 leaves differ

ERROR    Py:diff-root         INFO  [InDetRawDataContainer_p2_PixelRDOs.m_rawdata.m_word]: 161686 leaves differ

ERROR    Py:diff-root         INFO  [InDetSimDataCollection_p3_PixelSDO_Map.m_simdata.first]: 50030 leaves differ

ERROR    Py:diff-root         INFO  [InDetSimDataCollection_p3_PixelSDO_Map.m_simdata.second.m_enDeposits]: 51113 leaves differ

ERROR    Py:diff-root         INFO  [InDetSimDataCollection_p3_PixelSDO_Map.m_simdata.second.m_links.m_barcode]: 23652 leaves differ

ERROR    Py:diff-root         INFO  [InDetSimDataCollection_p3_PixelSDO_Map.m_simdata.second.m_links.m_evtColl]: 523 leaves differ

ERROR    Py:diff-root         INFO  [InDetSimDataCollection_p3_PixelSDO_Map.m_simdata.second.m_links.m_mcEvtIndex]: 523 leaves differ

ERROR    Py:diff-root         INFO  [InDetSimDataCollection_p3_PixelSDO_Map.m_simdata.second.m_word]: 18273 leaves differ

ERROR    Your tag breaks the frozen tier0 policy in test dataoverlay-d1590. See run_dataoverlay-d1590/diff-root-dataoverlay-d1590.RDO.log file for more information.

while q221xAODDigestTest-test test is failing with:

Willing to acquire file master_q221_AOD_digest.ref from
/build2/ci-builds/master/Athena/build/Athena/x86_64-centos7-gcc11-opt/data/PROCTools/master_q221_AOD_digest.ref
copy master_q221_AOD_digest.ref
         run       event       nTopo   nIdTracks  nTauTracks       nTaus      nMuons       nElec   nTrueElec   nFakeElec       nPhot   nTruePhot   nFakePhot
5c5
<       284500    87473032          29          33          24           4           1          10           4           6           6           2           4
---
>       284500    87473032          29          32          24           4           1          10           4           6           6           2           4
8c8
<       284500    87473051         135         111          61          11           1          11           1          10          22          16           6
---
>       284500    87473051         135         110          60          11           1          11           1          10          22          16           6
15c15
<       284500    87473104          64          63          34           4           0           6           2           4           4           3           1
---
>       284500    87473104          64          63          35           4           0           6           2           4           5           4           1
18,21c18,21
<       284500    87473132          85          58          41           8           1          12           0          12           4           4           0
<       284500    87473137          90          71          58           7           3          15           0          15           7           7           0
<       284500    87473144          83          68          34           5           1           9           2           7           8           6           2
<       284500    87473154          92          88          46           6           0          11           2           9           8           4           4
---
>       284500    87473132          85          59          42           8           1          12           0          12           4           4           0
>       284500    87473137          90          70          57           7           3          15           0          15           7           7           0
>       284500    87473144          83          68          34           5           1           9           2           7           7           5           2
>       284500    87473154          92          88          46           6           0          12           3           9           8           4           4
23c23
<       284500    87473167          74          55          38           6           3          15           2          13          14           9           5
---
>       284500    87473167          74          56          39           6           3          15           2          13          15          10           5
25c25
<       284500    87473184          69          86          32           5           2           7           1           6           6           3           3
---
>       284500    87473184          69          86          32           5           2           7           1           6           5           3           2
==================END q221 Digest Diff with exit code: 1 Wed Nov 17 03:53:11 CET 2021

If expected can you please update the reference?

-- L1

added review-user-action-required label and removed review-pending-level-1 label

added 71 commits

• eef3f9e0...30839bc1 - 69 commits from branch master
• 2897a2dc - Merge remote-tracking branch 'upstream/master' into PixelDeltaRayThreshold
• 7a5398d6 - update reference file again after pixel digitistaion updates

Compare with previous version

This merge request affects 3 packages:

• InnerDetector/InDetDigitization/PixelDigitization
• Simulation/G4Atlas/G4AtlasTools
• Tools/PROCTools

Affected files list will not be printed in this case

Adding @jchapman ,@goetz ,@stsuno ,@lshan ,@battagl ,@sroe as watchers

added review-pending-level-1 label and removed review-user-action-required label

Hi @cvarni,

The change to the pixel SDO/RDO's are expected with this change as are some minor changes to the number of tracks The reference file that needs to be updated on cvmfs/EOS can be found here: /afs/cern.ch/user/a/amorley/public/MR48161/

Anthony

CI Result FAILURE (hash 7a5398d6)

	Athena	AthSimulation	AthGeneration	AnalysisBase	AthAnalysis	DetCommon
externals
cmake
make
required tests
optional tests

Full details available on this CI monitor view
Athena: number of compilation errors 0, warnings 0
AthSimulation: number of compilation errors 0, warnings 0
AthGeneration: number of compilation errors 0, warnings 0
AnalysisBase: number of compilation errors 0, warnings 0
AthAnalysis: number of compilation errors 0, warnings 0
DetCommon: number of compilation errors 0, warnings 0
For experts only: Jenkins output [CI-MERGE-REQUEST-CC7 42408]

Hi @amorley, can you update RunTier0TestsTools.py for overlay to v3 and for data overlay to v16?

added 1 commit

• 20f9ec7c - Update overlay reference file versions

Compare with previous version

This merge request affects 3 packages:

• InnerDetector/InDetDigitization/PixelDigitization
• Simulation/G4Atlas/G4AtlasTools
• Tools/PROCTools

Affected files list will not be printed in this case

Adding @jchapman ,@goetz ,@stsuno ,@lshan ,@battagl ,@sroe as watchers

CI Result FAILURE (hash 20f9ec7c)

	Athena	AthSimulation	AthGeneration	AnalysisBase	AthAnalysis	DetCommon
externals
cmake
make
required tests
optional tests

Full details available on this CI monitor view
Athena: number of compilation errors 0, warnings 0
AthSimulation: number of compilation errors 0, warnings 0
AthGeneration: number of compilation errors 0, warnings 0
AnalysisBase: number of compilation errors 0, warnings 0
AthAnalysis: number of compilation errors 0, warnings 0
DetCommon: number of compilation errors 0, warnings 0
For experts only: Jenkins output [CI-MERGE-REQUEST-CC7 42429]

@amorley, files still differ, any idea?

Hi @tadej, i reran and go the same results so i did a fresh build and have now got different results so something must have been corrupted in the original area. The files in ~amorley/public/MR48161/ have been updated. Hopefully this corrects the issue.

just for completeness, here are the error messages from OverlayTier0Test_required-test:

ERROR    Py:diff-root         INFO  [InDetSimDataCollection_p3_SCT_SDO_Map.m_simdata.first]: 3 leaves differ

ERROR    Py:diff-root         INFO  [InDetSimDataCollection_p3_SCT_SDO_Map.m_simdata.second.m_enDeposits]: 10 leaves differ

ERROR    Py:diff-root         INFO  [InDetSimDataCollection_p3_SCT_SDO_Map.m_simdata.second.m_links.m_barcode]: 1 leaves differ

ERROR    Py:diff-root         INFO  [InDetSimDataCollection_p3_SCT_SDO_Map.m_simdata.second.m_links.m_evtColl]: 1 leaves differ

ERROR    Py:diff-root         INFO  [InDetSimDataCollection_p3_SCT_SDO_Map.m_simdata.second.m_links.m_mcEvtIndex]: 1 leaves differ

ERROR    Py:diff-root         INFO  [InDetSimDataCollection_p3_SCT_SDO_Map.m_simdata.second.m_word]: 4 leaves differ

ERROR    Py:diff-root         INFO  [SCT_RawDataContainer_p4_SCT_RDOs.m_collections.m_begin]: 13109 leaves differ

ERROR    Py:diff-root         INFO  [SCT_RawDataContainer_p4_SCT_RDOs.m_collections.m_end]: 13111 leaves differ

ERROR    Py:diff-root         INFO  [SCT_RawDataContainer_p4_SCT_RDOs.m_sct3data.m_rowStrip]: 450801 leaves differ

ERROR    Py:diff-root         INFO  [SCT_RawDataContainer_p4_SCT_RDOs.m_sct3data.m_word]: 354085 leaves differ

ERROR    Your tag breaks the frozen tier0 policy in test overlay-d1726. See run_overlay-d1726/diff-root-overlay-d1726.RDO.log file for more information.

and

ERROR    Py:diff-root         INFO  [InDetSimDataCollection_p3_PixelSDO_Map.m_simdata.second.m_enDeposits]: 6 leaves differ

ERROR    Py:diff-root         INFO  [InDetSimDataCollection_p3_PixelSDO_Map.m_simdata.second.m_links.m_barcode]: 6 leaves differ

ERROR    Py:diff-root         INFO  [InDetSimDataCollection_p3_SCT_SDO_Map.m_simdata.second.m_enDeposits]: 29 leaves differ

ERROR    Py:diff-root         INFO  [InDetSimDataCollection_p3_SCT_SDO_Map.m_simdata.second.m_word]: 124 leaves differ

ERROR    Py:diff-root         INFO  [SCT_RawDataContainer_p4_SCT_RDOs.m_collections.m_begin]: 40224 leaves differ

ERROR    Py:diff-root         INFO  [SCT_RawDataContainer_p4_SCT_RDOs.m_collections.m_end]: 40233 leaves differ

ERROR    Py:diff-root         INFO  [SCT_RawDataContainer_p4_SCT_RDOs.m_sct3data.m_rowStrip]: 376453 leaves differ

ERROR    Py:diff-root         INFO  [SCT_RawDataContainer_p4_SCT_RDOs.m_sct3data.m_word]: 200948 leaves differ

ERROR    Your tag breaks the frozen tier0 policy in test dataoverlay-d1590. See run_dataoverlay-d1590/diff-root-dataoverlay-d1590.RDO.log file for more information.

-- L1

added review-user-action-required label and removed review-pending-level-1 label

Jenkins please retry a build

I updated the files, let's see

This merge request affects 3 packages:

• InnerDetector/InDetDigitization/PixelDigitization
• Simulation/G4Atlas/G4AtlasTools
• Tools/PROCTools

Affected files list will not be printed in this case

Adding @jchapman ,@goetz ,@stsuno ,@lshan ,@battagl ,@sroe as watchers

added review-pending-level-1 label and removed review-user-action-required label

Jenkins please retry a build

This merge request affects 3 packages:

• InnerDetector/InDetDigitization/PixelDigitization
• Simulation/G4Atlas/G4AtlasTools
• Tools/PROCTools

Affected files list will not be printed in this case

Adding @jchapman ,@goetz ,@stsuno ,@lshan ,@battagl ,@sroe as watchers

CI Result FAILURE (hash 20f9ec7c)

	Athena	AthSimulation	AthGeneration	AnalysisBase	AthAnalysis	DetCommon
externals
cmake
make
required tests
optional tests

Full details available on this CI monitor view
Athena: number of compilation errors 1, warnings 0
AthSimulation: number of compilation errors 0, warnings 0
AthGeneration: number of compilation errors 0, warnings 0
AnalysisBase: number of compilation errors 0, warnings 0
AthAnalysis: number of compilation errors 0, warnings 0
DetCommon: number of compilation errors 0, warnings 0
For experts only: Jenkins output [CI-MERGE-REQUEST-CC7 42499]

Jenkins please retry a build

This merge request affects 3 packages:

• InnerDetector/InDetDigitization/PixelDigitization
• Simulation/G4Atlas/G4AtlasTools
• Tools/PROCTools

Affected files list will not be printed in this case

Adding @jchapman ,@goetz ,@stsuno ,@lshan ,@battagl ,@sroe as watchers

CI Result SUCCESS (hash 20f9ec7c)

	Athena	AthSimulation	AthGeneration	AnalysisBase	AthAnalysis	DetCommon
externals
cmake
make
required tests
optional tests

Full details available on this CI monitor view
Athena: number of compilation errors 0, warnings 0
AthSimulation: number of compilation errors 0, warnings 0
AthGeneration: number of compilation errors 0, warnings 0
AnalysisBase: number of compilation errors 0, warnings 0
AthAnalysis: number of compilation errors 0, warnings 0
DetCommon: number of compilation errors 0, warnings 0
For experts only: Jenkins output [CI-MERGE-REQUEST-CC7 42522]

Hi, code changes look OK from the L1 perspective, but please address the comment about Run 2 setup for this MR to be approved.

Magdalena (L1)

added review-user-action-required label and removed review-pending-level-1 label

resolved all threads

added review-approved label and removed review-user-action-required label

added 1 commit

• 0b68aca6 - Revert cut for Run 2 configuration

Compare with previous version

This merge request affects 3 packages:

• InnerDetector/InDetDigitization/PixelDigitization
• Simulation/G4Atlas/G4AtlasTools
• Tools/PROCTools

Affected files list will not be printed in this case

Adding @jchapman ,@goetz ,@stsuno ,@lshan ,@battagl ,@sroe as watchers

added review-pending-level-1 label and removed review-approved label