Skip to content

Bug fixing before MR

Description

We would like to make an MR to bring functions from Robert's branch, mainly FTagDumper/bin/dump-fatjet and FTagDumper/bin/dump-single-btag-flow, into main. The source branch of this MR is based on Robert Les @rles, merged with latest changes from main.

Currently the output of dump-fatjet from merged framework differs from Robert's old branch. I would like to seek advice from experts.

@dguest suggested the difference within /flow might relate to this issue: #168, and the difference within /subjets from: atlas/athena!73569 (merged) .

The H5writer bug should be fixed by running build script from README.md, and is gone for variables in /flow, but remains in /subjets. This bug has a clear feature of making difference at index[0].

The half-precision issue is not fixed yet.

The comparison looks like:

[yuanda@lxplus934 atlas-flavor-tagging-tools]$ python3 check_h5_diff.py
📂 Comparing HDF5 file structure, content, and attributes...

✅ No attribute differences at /
    ✔️ Keys: matched
    ✔️ Shapes: matched
    ✔️ Values: matched

🔍 Comparing dataset: /flow
✅ No attribute differences at /flow
    ✔️ Keys: matched
    ✔️ Shapes: matched
    ✔️ Values: matched
✅ No differences in /flow/flow_pt
✅ No differences in /flow/flow_energy
✅ No differences in /flow/flow_eta
✅ No differences in /flow/flow_phi
✅ No differences in /flow/flow_deta
✅ No differences in /flow/flow_dphi
✅ No differences in /flow/flow_dr
✅ No differences in /flow/numberDoF
✅ No differences in /flow/theta
✅ No differences in /flow/qOverP
✅ No differences in /flow/chiSquared
✅ No differences in /flow/pt
✅ No differences in /flow/ptfrac
✅ No differences in /flow/deta
✅ No differences in /flow/z0RelativeToBeamspot
✅ No differences in /flow/qOverPUncertainty
❌ Differences in /flow/d0: 1 elements differ
   Example indices: [1380]
   merged/test/topjets.h5 values: [0.001]
   old/test/topjets.h5 values: [0.002]
❌ Differences in /flow/z0SinTheta: 3 elements differ
   Example indices: [1099 1126 1139]
   merged/test/topjets.h5 values: [ 0.125  -0.125  -0.0312]
   old/test/topjets.h5 values: [ 0.0625 -0.0625 -0.0156]
✅ No differences in /flow/eta
✅ No differences in /flow/phi
✅ No differences in /flow/d0Uncertainty
✅ No differences in /flow/z0SinThetaUncertainty
✅ No differences in /flow/phiUncertainty
✅ No differences in /flow/thetaUncertainty
❌ Differences in /flow/dphi: 6 elements differ
   Example indices: [ 821  821  861  861 1995]
   merged/test/topjets.h5 values: [-0.125  -0.125   0.25    0.25    0.0312]
   old/test/topjets.h5 values: [-0.0625 -0.0625  0.125   0.125   0.0625]
✅ No differences in /flow/dr
✅ No differences in /flow/z0RelativeToBeamspotUncertainty
❌ Differences in /flow/IP3D_signed_d0: 1 elements differ
   Example indices: [1380]
   merged/test/topjets.h5 values: [0.001]
   old/test/topjets.h5 values: [0.002]
❌ Differences in /flow/IP3D_signed_z0: 3 elements differ
   Example indices: [1099 1126 1139]
   merged/test/topjets.h5 values: [-0.125   0.125   0.0312]
   old/test/topjets.h5 values: [-0.0625  0.0625  0.0156]
✅ No differences in /flow/IP3D_signed_d0_significance
✅ No differences in /flow/IP3D_signed_z0_significance
✅ No differences in /flow/valid

🔍 Comparing dataset: /jets
✅ No attribute differences at /jets
    ✔️ Keys: matched
    ✔️ Shapes: matched
    ✔️ Values: matched
✅ No differences in /jets/pt
✅ No differences in /jets/eta
✅ No differences in /jets/energy
✅ No differences in /jets/mass
✅ No differences in /jets/phi
✅ No differences in /jets/KtDR
✅ No differences in /jets/Tau1_wta
✅ No differences in /jets/Tau2_wta
✅ No differences in /jets/Tau3_wta
✅ No differences in /jets/Tau4_wta
✅ No differences in /jets/Split12
✅ No differences in /jets/Split23
✅ No differences in /jets/ZCut12
✅ No differences in /jets/L2
✅ No differences in /jets/L3
✅ No differences in /jets/Qw
✅ No differences in /jets/Angularity
✅ No differences in /jets/Aplanarity
✅ No differences in /jets/PlanarFlow
✅ No differences in /jets/ECF1
✅ No differences in /jets/ECF2
✅ No differences in /jets/ECF3
✅ No differences in /jets/ThrustMaj
✅ No differences in /jets/FoxWolfram0
✅ No differences in /jets/FoxWolfram2
✅ No differences in /jets/truth_pt
✅ No differences in /jets/truth_eta
✅ No differences in /jets/truth_phi
✅ No differences in /jets/truth_m
✅ No differences in /jets/truth_Split12
✅ No differences in /jets/truth_Split23
✅ No differences in /jets/truth_GhostBHadronsFinalCount
✅ No differences in /jets/truth_GhostWBosonsCount
✅ No differences in /jets/truth_GhostZBosonsCount
✅ No differences in /jets/truth_GhostTQuarksFinalCount
✅ No differences in /jets/truth_GhostHBosonsCount
✅ No differences in /jets/deltaRToMatchedTruthJet
✅ No differences in /jets/jetPtRank
✅ No differences in /jets/R10TruthLabel_R22v1
✅ No differences in /jets/mcEventWeight
✅ No differences in /jets/eventNumber
✅ No differences in /jets/averageInteractionsPerCrossing
✅ No differences in /jets/actualInteractionsPerCrossing
✅ No differences in /jets/nPrimaryVertices

🔍 Comparing dataset: /subjets
✅ No attribute differences at /subjets
    ✔️ Keys: matched
    ✔️ Shapes: matched
    ✔️ Values: matched
❌ Differences in /subjets/pt: 1 elements differ
   Example indices: [0]
   merged/test/topjets.h5 values: [522262.7]
   old/test/topjets.h5 values: [26646.559]
❌ Differences in /subjets/eta: 1 elements differ
   Example indices: [0]
   merged/test/topjets.h5 values: [0.768]
   old/test/topjets.h5 values: [1.0774]
❌ Differences in /subjets/energy: 1 elements differ
   Example indices: [0]
   merged/test/topjets.h5 values: [684029.6]
   old/test/topjets.h5 values: [44022.895]
❌ Differences in /subjets/mass: 1 elements differ
   Example indices: [0]
   merged/test/topjets.h5 values: [3142.7651]
   old/test/topjets.h5 values: [5571.405]
❌ Differences in /subjets/phi: 1 elements differ
   Example indices: [0]
   merged/test/topjets.h5 values: [-1.0318]
   old/test/topjets.h5 values: [-1.2722]
❌ Differences in /subjets/DL1r_pb: 1 elements differ
   Example indices: [0]
   merged/test/topjets.h5 values: [0.7575]
   old/test/topjets.h5 values: [0.0478]
❌ Differences in /subjets/DL1r_pc: 1 elements differ
   Example indices: [0]
   merged/test/topjets.h5 values: [0.2144]
   old/test/topjets.h5 values: [0.2189]
❌ Differences in /subjets/DL1r_pu: 1 elements differ
   Example indices: [0]
   merged/test/topjets.h5 values: [0.0281]
   old/test/topjets.h5 values: [0.7334]
❌ Differences in /subjets/HadronConeExclTruthLabelID: 1 elements differ
   Example indices: [0]
   merged/test/topjets.h5 values: [5]
   old/test/topjets.h5 values: [0]
❌ Differences in /subjets/HadronConeExclExtendedTruthLabelID: 1 elements differ
   Example indices: [0]
   merged/test/topjets.h5 values: [5]
   old/test/topjets.h5 values: [0]
❌ Differences in /subjets/deta: 1 elements differ
   Example indices: [0]
   merged/test/topjets.h5 values: [-0.2275]
   old/test/topjets.h5 values: [0.3565]
❌ Differences in /subjets/dphi: 1 elements differ
   Example indices: [0]
   merged/test/topjets.h5 values: [0.0959]
   old/test/topjets.h5 values: [0.2808]
❌ Differences in /subjets/dr: 1 elements differ
   Example indices: [0]
   merged/test/topjets.h5 values: [0.2468]
   old/test/topjets.h5 values: [0.4538]
✅ No differences in /subjets/valid