Skip to content
Snippets Groups Projects

PyConf: Deterministic identity of components and data

Merged PyConf: Deterministic identity of components and data
rm-deterministic-hashes into master
Merged Rosen Matev requested to merge rm-deterministic-hashes into master

hash(alg/tool/data_handle) are non-deterministic since Algorithm, Tool and DataHandle use hash to construct their "identity" (which is returned by their __hash__). This makes the iteration order of sets of objects non-deterministic, which can be a problem since, at least currently, the names of Algorithms depends on instantiation order.

This is fixed here by using deterministic hashing.

Goes with Rec!3099 (merged)

See Moore!1737 (comment 5951216)

/cc @gunther

Edited by Rosen Matev

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • Rosen Matev added Configuration enhancement labels

    added Configuration enhancement labels

  • Rosen Matev assigned to @rmatev

    assigned to @rmatev

  • Rosen Matev added needs ref update label

    added needs ref update label

  • Software for LHCb added RTA label

    added RTA label

  • Rosen Matev added 1 commit

    added 1 commit

    • ba9ceea9 - Speed up tonic by protecting expensive debug messages

    Compare with previous version

  • Rosen Matev added 2 commits

    added 2 commits

    • 57c310eb - PyConf: Deterministic identity of components and data
    • b01aefd5 - Speed up tonic by protecting expensive debug messages

    Compare with previous version

  • Rosen Matev marked this merge request as ready

    marked this merge request as ready

  • Software for LHCb added ci-test-triggered label

    added ci-test-triggered label

  • L
    Software for LHCb @lhcbsoft ·
    Developer
    Edited by Software for LHCb
  • Rosen Matev mentioned in merge request Moore!1737 (closed)

    mentioned in merge request Moore!1737 (closed)

  • Gerhard Raven approved this merge request

    approved this merge request

  • Rosen Matev mentioned in commit Rec@79e615a7

    mentioned in commit Rec@79e615a7

  • Rosen Matev mentioned in merge request Rec!3099 (merged)

    mentioned in merge request Rec!3099 (merged)

  • Software for LHCb mentioned in commit Moore@72bd5ace

    mentioned in commit Moore@72bd5ace

  • Software for LHCb mentioned in commit Panoptes@2b2d1033

    mentioned in commit Panoptes@2b2d1033

  • Software for LHCb mentioned in merge request Moore!1744 (merged)

    mentioned in merge request Moore!1744 (merged)

    • L
      Software for LHCb @lhcbsoft ·
      Developer
      Resolved by Rosen Matev

      The RefBot pipeline created the following reference update MRs: Moore!1744 (merged), Panoptes!182 (closed)

      Click this to see encountered Moore warnings 
      Moore: Different test results for Hlt/Hlt2Conf/tests/refs/hlt2_noUT_trackefflines.ref on platforms ['x86_64_v2-centos7-gcc11-opt', 'x86_64_v2-centos7-gcc11-dbg', 'x86_64_v2-centos7-clang12-opt', 'x86_64_v3-centos7-gcc11-opt+g']
Moore: Comparing x86_64_v2-centos7-gcc11-opt to:
Moore:	 x86_64_v2-centos7-gcc11-dbg is identical
Moore:	 x86_64_v2-centos7-clang12-opt differs by:
Moore:	 Differences in Causes:
Moore:	x86_64_v2-centos7-gcc11-opt:
unexpected Wrong Counters
Moore:	x86_64_v2-centos7-clang12-opt:
unexpected standard error, Wrong Counters
Moore:	 x86_64_v3-centos7-gcc11-opt+g differs by:
Moore:	 Differences in Causes:
Moore:	x86_64_v2-centos7-gcc11-opt:
unexpected Wrong Counters
Moore:	x86_64_v3-centos7-gcc11-opt+g:
unexpected standard output, Wrong Counters
Moore:	 Differences in CountersMismatch:
Moore:	x86_64_v2-centos7-gcc11-opt:
    (ChargedProtoParticleAssociator new) Efficiency | 95 | 94 | ( 98.94737 +- 1.047077)%
    (ChargedProtoParticleAssociator new) MC particles per ProtoParticle | 94 | 94 | 1.0000
    (ChargedProtoParticleAssociator#2 new) Efficiency | 61 | 60 | ( 98.36066 +- 1.625851)%
    (ChargedProtoParticleAssociator#2 new) MC particles per ProtoParticle | 60 | 60 | 1.0000
    (PrTrackAssociator new) Efficiency | 95 | 94 | ( 98.94737 +- 1.047077)%
    (PrTrackAssociator new) MC particles per track | 94 | 94...
Moore:	x86_64_v3-centos7-gcc11-opt+g:
    (CaloAcceptanceBremAlg_Long new) #total tracks | 100 | 9292 | 92.920 | 46.719 | 9.0000 | 219.00
    (CaloAcceptanceBremAlg_Long new) #tracks in acceptance | 100 | 6777 | 67.770 | 33.489 | 7.0000 | 151.00
    (CaloAcceptanceBremAlg_Long ref) #total tracks | 100 | 9291 | 92.910 | 46.789 | 9.0000 | 220.00
    (CaloAcceptanceBremAlg_Long ref) #tracks in acceptance | 100 | 6775 | 67.750 | 33.530 | 7.0000 | 151.00
    (CaloAcceptanceEcalAlg_Long new) #total tracks | 100 | 9292 | 92.920 | 46.719 | ...
Moore:	 Key 'Output Diff' not found
Moore: Different test results for Hlt/Hlt2Conf/tests/refs/hlt2_trackefflines.ref on platforms ['x86_64_v2-centos7-gcc11-opt', 'x86_64_v2-centos7-gcc11-dbg', 'x86_64_v2-centos7-clang12-opt', 'x86_64_v3-centos7-gcc11-opt+g']
Moore: Comparing x86_64_v2-centos7-gcc11-opt to:
Moore:	 x86_64_v2-centos7-gcc11-dbg is identical
Moore:	 x86_64_v2-centos7-clang12-opt differs by:
Moore:	 Differences in Causes:
Moore:	x86_64_v2-centos7-gcc11-opt:
unexpected Wrong Counters
Moore:	x86_64_v2-centos7-clang12-opt:
unexpected standard error, Wrong Counters
Moore:	 x86_64_v3-centos7-gcc11-opt+g differs by:
Moore:	 Differences in Causes:
Moore:	x86_64_v2-centos7-gcc11-opt:
unexpected Wrong Counters
Moore:	x86_64_v3-centos7-gcc11-opt+g:
unexpected standard output, Wrong Counters
Moore:	 Differences in CountersMismatch:
Moore:	x86_64_v2-centos7-gcc11-opt:
    (ChargedProtoParticleAssociator new) Efficiency | 87 | 87 | ( 100.0000 +-  0.000000)%
    (ChargedProtoParticleAssociator new) MC particles per ProtoParticle | 87 | 89 | 1.0230
    (ChargedProtoParticleAssociator#1 new) Efficiency | 3183 | 2850 | ( 89.53817 +- 0.5424873)%
    (ChargedProtoParticleAssociator#1 new) MC particles per ProtoParticle | 2850 | 3246 | 1.1389
    (ChargedProtoParticleAssociator#2 new) Efficiency | 62 | 54 | ( 87.09677 +- 4.257497)%
    (ChargedProtoParticleAssociator...
Moore:	x86_64_v3-centos7-gcc11-opt+g:
    (BestTrackCreator_Downstream new) BadInput | 3628 | 0 | (  0.000000 +-  0.000000)%
    (BestTrackCreator_Downstream new) FitFailed | 3628 | 0 | (  0.000000 +-  0.000000)%
    (BestTrackCreator_Downstream new) FittedBefore | 3628
    (BestTrackCreator_Downstream ref) BadInput | 3590 | 0 | (  0.000000 +-  0.000000)%
    (BestTrackCreator_Downstream ref) FitFailed | 3590 | 0 | (  0.000000 +-  0.000000)%
    (BestTrackCreator_Downstream ref) FittedBefore | 3590
    (CaloAcceptanceBremAlg_Downstr...
Moore:	 Key 'Output Diff' not found
Moore: Warning, results for test: RecoConf.hlt2_protoparticles_baseline not found for platforms: {'x86_64_v2-centos7-gcc11-dbg', 'x86_64_v2-centos7-clang12-opt'}
Moore: Warning, results for test: RecoConf.hlt2_protoparticles_fastest not found for platforms: {'x86_64_v2-centos7-gcc11-dbg', 'x86_64_v2-centos7-clang12-opt'}

      Click this to see encountered Rec warnings 
      Rec/x86_64_v2-centos7-gcc11-opt: Test FunctorCore.test_functors contains nonzero exit code and/or warnings!
Rec/x86_64_v2-centos7-gcc11-opt: Test FunctorCore.test_vector_functors_1_4 contains nonzero exit code and/or warnings!
Rec/x86_64_v2-centos7-gcc11-opt: Test FunctorCore.test_vector_functors_2_4 contains nonzero exit code and/or warnings!
Rec/x86_64_v2-centos7-gcc11-opt: Test FunctorCore.test_vector_functors_3_4 contains nonzero exit code and/or warnings!
Rec/x86_64_v2-centos7-gcc11-opt: Test FunctorCore.test_vector_functors_4_4 contains nonzero exit code and/or warnings!
Rec/x86_64_v2-centos7-gcc11-opt: Test without ref failed: FunctorCore.test_functors!
Rec/x86_64_v2-centos7-gcc11-opt: Test without ref failed: FunctorCore.test_vector_functors_1_4!
Rec/x86_64_v2-centos7-gcc11-opt: Test without ref failed: FunctorCore.test_vector_functors_2_4!
Rec/x86_64_v2-centos7-gcc11-opt: Test without ref failed: FunctorCore.test_vector_functors_3_4!
Rec/x86_64_v2-centos7-gcc11-opt: Test without ref failed: FunctorCore.test_vector_functors_4_4!
Rec/x86_64_v2-centos7-gcc11-opt: Test without ref failed: FunctorCore.python!
Rec/x86_64_v2-centos7-gcc11-dbg: Test FunctorCore.test_functors contains nonzero exit code and/or warnings!
Rec/x86_64_v2-centos7-gcc11-dbg: Test FunctorCore.test_vector_functors_1_4 contains nonzero exit code and/or warnings!
Rec/x86_64_v2-centos7-gcc11-dbg: Test FunctorCore.test_vector_functors_2_4 contains nonzero exit code and/or warnings!
Rec/x86_64_v2-centos7-gcc11-dbg: Test FunctorCore.test_vector_functors_3_4 contains nonzero exit code and/or warnings!
Rec/x86_64_v2-centos7-gcc11-dbg: Test FunctorCore.test_vector_functors_4_4 contains nonzero exit code and/or warnings!
Rec/x86_64_v2-centos7-gcc11-dbg: Test without ref failed: FunctorCore.test_functors!
Rec/x86_64_v2-centos7-gcc11-dbg: Test without ref failed: FunctorCore.test_vector_functors_1_4!
Rec/x86_64_v2-centos7-gcc11-dbg: Test without ref failed: FunctorCore.test_vector_functors_2_4!
Rec/x86_64_v2-centos7-gcc11-dbg: Test without ref failed: FunctorCore.test_vector_functors_3_4!
Rec/x86_64_v2-centos7-gcc11-dbg: Test without ref failed: FunctorCore.test_vector_functors_4_4!
Rec/x86_64_v2-centos7-gcc11-dbg: Test without ref failed: FunctorCore.python!
Rec/x86_64_v2-centos7-clang12-opt: Test FunctorCore.test_functors contains nonzero exit code and/or warnings!
Rec/x86_64_v2-centos7-clang12-opt: Test FunctorCore.test_vector_functors_1_4 contains nonzero exit code and/or warnings!
Rec/x86_64_v2-centos7-clang12-opt: Test FunctorCore.test_vector_functors_2_4 contains nonzero exit code and/or warnings!
Rec/x86_64_v2-centos7-clang12-opt: Test FunctorCore.test_vector_functors_3_4 contains nonzero exit code and/or warnings!
Rec/x86_64_v2-centos7-clang12-opt: Test FunctorCore.test_vector_functors_4_4 contains nonzero exit code and/or warnings!
Rec/x86_64_v2-centos7-clang12-opt: Test without ref failed: FunctorCore.test_functors!
Rec/x86_64_v2-centos7-clang12-opt: Test without ref failed: FunctorCore.test_vector_functors_1_4!
Rec/x86_64_v2-centos7-clang12-opt: Test without ref failed: FunctorCore.test_vector_functors_2_4!
Rec/x86_64_v2-centos7-clang12-opt: Test without ref failed: FunctorCore.test_vector_functors_3_4!
Rec/x86_64_v2-centos7-clang12-opt: Test without ref failed: FunctorCore.test_vector_functors_4_4!
Rec/x86_64_v2-centos7-clang12-opt: Test without ref failed: FunctorCore.python!
Rec/x86_64_v3-centos7-gcc11-opt+g: Test FunctorCore.test_functors contains nonzero exit code and/or warnings!
Rec/x86_64_v3-centos7-gcc11-opt+g: Test FunctorCore.test_vector_functors_1_4 contains nonzero exit code and/or warnings!
Rec/x86_64_v3-centos7-gcc11-opt+g: Test FunctorCore.test_vector_functors_2_4 contains nonzero exit code and/or warnings!
Rec/x86_64_v3-centos7-gcc11-opt+g: Test FunctorCore.test_vector_functors_3_4 contains nonzero exit code and/or warnings!
Rec/x86_64_v3-centos7-gcc11-opt+g: Test FunctorCore.test_vector_functors_4_4 contains nonzero exit code and/or warnings!
Rec/x86_64_v3-centos7-gcc11-opt+g: Test without ref failed: FunctorCore.test_functors!
Rec/x86_64_v3-centos7-gcc11-opt+g: Test without ref failed: FunctorCore.test_vector_functors_1_4!
Rec/x86_64_v3-centos7-gcc11-opt+g: Test without ref failed: FunctorCore.test_vector_functors_2_4!
Rec/x86_64_v3-centos7-gcc11-opt+g: Test without ref failed: FunctorCore.test_vector_functors_3_4!
Rec/x86_64_v3-centos7-gcc11-opt+g: Test without ref failed: FunctorCore.test_vector_functors_4_4!
Rec/x86_64_v3-centos7-gcc11-opt+g: Test without ref failed: FunctorCore.python!

  • Software for LHCb mentioned in merge request Panoptes!182 (closed)

    mentioned in merge request Panoptes!182 (closed)

  • Rosen Matev mentioned in commit Moore@12687aa6

    mentioned in commit Moore@12687aa6

  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
Please register or sign in to reply