Skip to content
Snippets Groups Projects

Update ASG test inputs and enable CP algorithms in Athena

Merged Update ASG test inputs and enable CP algorithms in Athena
tadej/athena:asg/new-inputs into master
Merged Tadej Novak requested to merge tadej/athena:asg/new-inputs into master

Update ASG test inputs and enable CP algorithms in Athena.

Running full-unit-tests to check for potential issues.

/cc @krumnack @akraszna @lheinric

Edited by Tadej Novak

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • Tadej Novak added full-unit-tests label

    added full-unit-tests label

  • Tadej Novak
    Tadej Novak @tadej ·
    Author Developer

    Jenkins please retry a build

  • ATLAS Robot
    ATLAS Robot @atlasbot ·
    Developer

    This merge request affects 18 packages:

    • AsgExternal/Asg_Test
    • Control/xAODDataSource
    • Control/xAODRootAccess
    • Event/xAOD/xAODMissingET
    • PhysicsAnalysis/Algorithms/AsgAnalysisAlgorithms
    • PhysicsAnalysis/Algorithms/EgammaAnalysisAlgorithms
    • PhysicsAnalysis/Algorithms/FTagAnalysisAlgorithms
    • PhysicsAnalysis/Algorithms/JetAnalysisAlgorithms
    • PhysicsAnalysis/Algorithms/MetAnalysisAlgorithms
    • PhysicsAnalysis/Algorithms/MuonAnalysisAlgorithms
    • PhysicsAnalysis/Algorithms/TauAnalysisAlgorithms
    • PhysicsAnalysis/Algorithms/TriggerAnalysisAlgorithms
    • PhysicsAnalysis/AnalysisCommon/AssociationUtils
    • PhysicsAnalysis/AnalysisCommon/PileupReweighting
    • PhysicsAnalysis/ElectronPhotonID/ElectronEfficiencyCorrection
    • PhysicsAnalysis/ElectronPhotonID/PhotonVertexSelection
    • Projects/Athena
    • Reconstruction/MET/METUtilities

    Affected files list will not be printed in this case

    Could not map the following files to a domain:

    • AsgExternal/Asg_Test/cmt/TestFileData.txt
    • AsgExternal/Asg_Test/cmt/TestFileMCAFII.txt
    • AsgExternal/Asg_Test/cmt/TestFileMC.txt

    Adding @zmarshal ,@goetz ,@krumnack ,@pagessin ,@akraszna ,@mhodgkin ,@tadej ,@ssnyder ,@rbianchi as watchers

  • ATLAS Robot added Analysis Build Core EDM Egamma JetEtmiss Reconstruction Tau Trigger master labels

    added Analysis Build Core EDM Egamma JetEtmiss Reconstruction Tau Trigger master labels

  • ATLAS Robot
    ATLAS Robot @atlasbot ·
    Developer

    :negative_squared_cross_mark: CI Result FAILURE (hash 59769600)

    Athena AthSimulation AthGeneration AnalysisBase AthAnalysis
    externals :white_check_mark: :white_check_mark: :white_check_mark: :white_check_mark: :white_check_mark:
    cmake :warning: :white_check_mark: :white_check_mark: :white_check_mark: :white_check_mark:
    make :warning: :white_check_mark: :white_check_mark: :white_check_mark: :o:
    required tests :o: :o: :o: :o: :o:
    optional tests :white_check_mark: :white_check_mark: :white_check_mark: :white_check_mark: :white_check_mark:

    Full details available on this CI monitor view
    :warning: Athena: number of compilation errors 0, warnings 49
    :white_check_mark: AthSimulation: number of compilation errors 0, warnings 0
    :white_check_mark: AthGeneration: number of compilation errors 0, warnings 0
    :white_check_mark: AnalysisBase: number of compilation errors 0, warnings 0
    :o: AthAnalysis: number of compilation errors 1, warnings 38
    :pencil: For experts only: Jenkins output [CI-MERGE-REQUEST-CC7 25665]

  • Tadej Novak
    Tadej Novak @tadej ·
    Author Developer

    These seem to be valid issues which I do not know how to fix now:

    	108 - xAODRootAccess_ut_xaodrootaccess_remap_test_ctest (Failed)
	119 - xAODRootAccess_ut_xaodrootaccess_tevent_test_ctest (Failed)
	232 - EventLoopAlgs_gt_DuplicateChecker_ctest (Failed)
	238 - EventLoopTest_gt_LeakChecks_ctest (Failed)

    Will continue after the break.

  • Tadej Novak added 94 commits

    added 94 commits

    • 59769600...4cda231e - 88 commits from branch atlas:master
    • d9cd7316 - Enable CP algorithms in Athena
    • 59da7bb1 - Update ASG test inputs
    • feab4398 - Update xAODDataSource references
    • e625128c - Fix unit tests after test file update
    • f046dab4 - Fix AthAnalysis compilation warnings
    • fbada764 - Recompute MET name hash in standalone code if not available

    Compare with previous version

    Toggle commit list
  • Tadej Novak
    Tadej Novak @tadej ·
    Author Developer

    Jenkins please retry a build

  • ATLAS Robot
    ATLAS Robot @atlasbot ·
    Developer

    This merge request affects 18 packages:

    • AsgExternal/Asg_Test
    • Control/xAODDataSource
    • Control/xAODRootAccess
    • Event/xAOD/xAODMissingET
    • Event/xAOD/xAODMissingETAthenaPool
    • PhysicsAnalysis/Algorithms/AsgAnalysisAlgorithms
    • PhysicsAnalysis/Algorithms/EgammaAnalysisAlgorithms
    • PhysicsAnalysis/Algorithms/FTagAnalysisAlgorithms
    • PhysicsAnalysis/Algorithms/JetAnalysisAlgorithms
    • PhysicsAnalysis/Algorithms/MetAnalysisAlgorithms
    • PhysicsAnalysis/Algorithms/MuonAnalysisAlgorithms
    • PhysicsAnalysis/Algorithms/TriggerAnalysisAlgorithms
    • PhysicsAnalysis/AnalysisCommon/AssociationUtils
    • PhysicsAnalysis/AnalysisCommon/PileupReweighting
    • PhysicsAnalysis/ElectronPhotonID/ElectronEfficiencyCorrection
    • PhysicsAnalysis/ElectronPhotonID/PhotonVertexSelection
    • Projects/Athena
    • Reconstruction/MET/METUtilities

    Affected files list will not be printed in this case

    Could not map the following files to a domain:

    • AsgExternal/Asg_Test/cmt/TestFileData.txt
    • AsgExternal/Asg_Test/cmt/TestFileMCAFII.txt
    • AsgExternal/Asg_Test/cmt/TestFileMC.txt

    Adding @zmarshal ,@goetz ,@krumnack ,@pagessin ,@akraszna ,@tadej ,@ssnyder ,@rbianchi as watchers

    • Tadej Novak
      Tadej Novak @tadej ·
      Author Developer
      Resolved by Tadej Novak

      Hi @akraszna, could you help me understand the failures in

      	108 - xAODRootAccess_ut_xaodrootaccess_remap_test_ctest (Failed)
	119 - xAODRootAccess_ut_xaodrootaccess_tevent_test_ctest (Failed)

      I think it should be enough to just re-run the test with a new version of ASG_TEST_FILE_DATA from /cvmfs/atlas-nightlies.cern.ch/repo/data/data-art/ASG/DAOD_PHYS/v1/data.DAOD_PHYS.pool.root.

  • ATLAS Robot
    ATLAS Robot @atlasbot ·
    Developer

    :negative_squared_cross_mark: CI Result FAILURE (hash fbada764)

    Athena AthSimulation AthGeneration AnalysisBase AthAnalysis
    externals :white_check_mark: :white_check_mark: :white_check_mark: :white_check_mark: :white_check_mark:
    cmake :warning: :white_check_mark: :white_check_mark: :white_check_mark: :white_check_mark:
    make :white_check_mark: :white_check_mark: :white_check_mark: :white_check_mark: :white_check_mark:
    required tests :o: :o: :o: :o: :o:
    optional tests :white_check_mark: :white_check_mark: :white_check_mark: :white_check_mark: :white_check_mark:

    Full details available on this CI monitor view
    :white_check_mark: Athena: number of compilation errors 0, warnings 0
    :white_check_mark: AthSimulation: number of compilation errors 0, warnings 0
    :white_check_mark: AthGeneration: number of compilation errors 0, warnings 0
    :white_check_mark: AnalysisBase: number of compilation errors 0, warnings 0
    :white_check_mark: AthAnalysis: number of compilation errors 0, warnings 0
    :pencil: For experts only: Jenkins output [CI-MERGE-REQUEST-CC7 25768]

  • Tadej Novak
    Tadej Novak @tadej ·
    Author Developer

    TODO:

    • remove StandaloneAnalysisAlgorithms from the build
    • fix xAODRootAccess tests
    • fix EventLoop tests (@krumnack, any tips for those two?)
    	232 - EventLoopAlgs_gt_DuplicateChecker_ctest (Failed)
	238 - EventLoopTest_gt_LeakChecks_ctest (Failed)
    • check TauAnalysisTools_ut_TauAnalysisTools_test_ctest

    The rest look like they're also failing in the nightlies (or it's AthAnalysis which I'd rather tackle separately).

  • K
    Nils Erik Krumnack @krumnack ·
    Developer

    So, neither of the EventLoop tests is actually using the input files in a meaningful way. They are using them, but they should be completely insensitive to what's in them…

    EventLoopTest_gt_LeakChecks_ctest is essentially just running on all events in the file, not even looking at this inputs and instead just leaking a pre-determined amount of memory on each event and then checking if the leaked memory triggers the leak checker. Maybe there is something in the code that reads the input file that somehow releases extra memory to make up for the leak? Maybe the old files had more events, leading to a larger leak? One option to address this may be to increase the number of events run on (maybe by adding the same input file multiple times), another to increase the amount of memory leaked on each event to make the leak more pronounced.

    EventLoopAlgs_gt_DuplicateChecker_ctest does use the input files very indirectly, essentially not at all. It is just opening up the input file to be able to write an output file that then becomes the actual input to the test (TEvent doesn't allow to create output files unless you are at least pretending to be copying data from an input file). Not sure why that would fail, but my first instinct would be to fix all failures in xAODRootAccess before debugging this. The actual failure you see here seems extremely weird.

    As for StandaloneAnalysisAlgorithms, what are the issues with it that you want to remove it?

    Edited by Nils Erik Krumnack
  • Tadej Novak
    Tadej Novak @tadej ·
    Author Developer

    StandaloneAnalysisAlgorithms is just not built in Athena and just prints a warning. It's a set of very few standalone-only algorithms.

  • Tadej Novak mentioned in merge request !39467 (merged)

    mentioned in merge request !39467 (merged)

  • Tadej Novak added 30 commits

    added 30 commits

    • fbada764...fcbdffaa - 14 commits from branch atlas:master
    • 60b8d0a7 - Update ASG test inputs
    • 6d8c07ad - Update xAODDataSource references
    • c3c64605 - Fix unit tests after test file update
    • 4863f80e - Fix AthAnalysis compilation warnings
    • 5b94c42c - Recompute MET name hash in standalone code if not available
    • 56f1c860 - Minor fixes in TEvent
    • bcee30b5 - Update xAODRootAccess proxydict test
    • cf16b6ff - Adapt TauTruthMatchingTool for DAOD_PHYS
    • 43b7554a - Disable IOVDbSvc test in AthAnalysis
    • f1a5e0a5 - Fix FileMetaData writing test outside Athena
    • 1c8fab5b - Disable LumiBlockComps tests in AthAnalysis
    • d24a0d20 - Disable some e/gamma tests in AthAnalysis
    • d2d0bd9e - Disable some jet tests in AthAnalysis
    • 8fc6de85 - Fix EventBookkeeperTools tests in AthAnalysis
    • 62d8ae7f - Enable CP algorithms in Athena
    • a65f7711 - Tweak AthAnalysis package_filters.txt

    Compare with previous version

    Toggle commit list
  • Tadej Novak added full-build label

    added full-build label

  • Tadej Novak
    Tadej Novak @tadej ·
    Author Developer

    The next one will be full-build as I'll remove a package from AthAnalysis.

  • Tadej Novak added 15 commits

    added 15 commits

    • 0f723750 - Fix unit tests after test file update
    • edc1373d - Fix AthAnalysis compilation warnings
    • 9e92cc8f - Recompute MET name hash in standalone code if not available
    • 50794d18 - Minor fixes in TEvent
    • 6ceb2247 - Update xAODRootAccess proxydict test
    • aa306f83 - Adapt TauTruthMatchingTool for DAOD_PHYS
    • 605f186c - Disable IOVDbSvc test in AthAnalysis
    • 9a1eab38 - Fix FileMetaData writing test outside Athena
    • 591cbe30 - Disable LumiBlockComps tests in AthAnalysis
    • 037279e2 - Disable some e/gamma tests in AthAnalysis
    • a45bcc91 - Disable some jet tests in AthAnalysis
    • 7f6854b7 - Enable CP algorithms in Athena
    • 86b60773 - Remove TrigAnalysisTest from AthAnalysis
    • 6b0747d0 - Disable JobOptionsUnitTests in AthAnalysis
    • c02cae9f - Fix EventBookkeeperTools tests in AthAnalysis

    Compare with previous version

    Toggle commit list
  • Tadej Novak
    Tadej Novak @tadej ·
    Author Developer

    Jenkins please retry a build

  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
Please register or sign in to reply