Skip to content
Snippets Groups Projects

Hdf5 merger

Merged Dan Guest requested to merge hdf5-utils/athena:hdf5-merger into 21.2

It would be quite nice to merge HDF5 files on the grid. Fortunately @jburr has written a utility to do this!

NOTE: I've kept the history from Jon's original repository in this merge request, in case it helps to merge in any final changes. I suppose we should squash this when it's ready to be merged, though.

Edited by Dan Guest

Merge request reports

Pipeline #982282 passed

Pipeline passed for 76335561 on hdf5-utils:hdf5-merger

Merged by Attila KrasznahorkayAttila Krasznahorkay 5 years ago (Jul 17, 2019 11:44am UTC)

Merge details

  • Changes merged into with 7d18f560.
  • Deleted the source branch.

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • Dan Guest added 1 commit

    added 1 commit

    • a8b61867 - Remove stray CMakeLists file

    Compare with previous version

  • Dan Guest added 1 commit

    added 1 commit

    Compare with previous version

  • Author Developer

    @jburr, comments welcome. If you're ok with this I'll unmark it as a WIP.

  • Contributor

    I don't know if it matters but the ifndef statements should probably also be changed to stop referencing the original HDF5Writer package...

  • Dan Guest added 1 commit

    added 1 commit

    Compare with previous version

  • Author Developer

    Hi @jburr, changed. Anything else you can spot?

  • Dan Guest unmarked as a Work In Progress

    unmarked as a Work In Progress

  • This merge request affects 1 package:

    • PhysicsAnalysis/AnalysisCommon/HDF5Utils

    Adding @dguest as watcher

  • CI Result FAILURE

    AnalysisBase AnalysisTop AthAnalysis AthDerivation
    externals
    cmake
    make
    required tests
    optional tests

    Full details available at NICOS MR-24892-2019-07-16-01-44
    AnalysisBase: number of compilation errors 0, warnings 0
    AnalysisTop: number of compilation errors 0, warnings 0
    AthAnalysis: number of compilation errors 1, warnings 257
    AthDerivation: number of compilation errors 0, warnings 11
    📝 For experts only: Jenkins output [CI-MERGE-REQUEST 39906]

  • Author Developer

    Jenkins please retry a build

  • This merge request affects 1 package:

    • PhysicsAnalysis/AnalysisCommon/HDF5Utils

    Adding @dguest as watcher

  • Author Developer

    I can build this on lxplus and the AthAnalysis docker images, so whatever caused the failure above is specific to the CI.

  • CI Result FAILURE

    AnalysisBase AnalysisTop AthAnalysis AthDerivation
    externals
    cmake
    make
    required tests
    optional tests

    Full details available at NICOS MR-24892-2019-07-16-08-56
    AnalysisBase: number of compilation errors 0, warnings 0
    AnalysisTop: number of compilation errors 0, warnings 0
    AthAnalysis: number of compilation errors 1, warnings 83
    AthDerivation: number of compilation errors 0, warnings 1
    📝 For experts only: Jenkins output [CI-MERGE-REQUEST 39908]

  • Contributor

    The error also makes no sense... It appears to compile completely fine but then ends on

    08:42:08 resuts_dict_generator: Info: added Build["AthDerivation"] = "SUCCESS"
    08:42:08 Info: result SUCCESS, overall_result FAILURE 08:42:08 resuts_dict_generator: Info: added BuildTime["Start;AthDerivation"] = "1563258783.797408765"
    08:42:08 resuts_dict_generator: Info: added BuildTime["End;AthDerivation"] = "1563259328.950837516"
    08:42:08 resuts_dict_generator: Info: added BuildTime["Start"] = "1563258441.219391935"
    08:42:08 resuts_dict_generator: Info: added BuildTime["End"] = "1563259328.981692101"
    08:42:09 Info: recorded start build time: 1563258441.219391935, end time: 1563259328.981692101
    08:42:09 Info: OVERALL RESULT: FAILURE
    08:42:09 Build step 'Execute shell' marked build as failure
    08:42:09 Finished: FAILURE

    Edited by Jon Burr
  • Hi @dguest, @jburr, could you please add ATLAS Robot as developer to the fork? (!24892 (merged))

    Thanks, Tadej (L2)

  • Author Developer

    @jburr I think you're not looking at the whole log:

    08:32:51 [ 32%] Building CXX object Control/AthAllocators/CMakeFiles/AthAllocators.dir/src/Arena.cxx.o
    08:32:51 Scanning dependencies of target CxxUtils_make_unique_test
    08:32:51 In file included from /var/lib/jenkins/workspace/CI-MERGE-REQUEST/21.2/PhysicsAnalysis/AnalysisCommon/HDF5Utils/HDF5Utils/DefaultMerger.h:8:0,
    08:32:51                  from /var/lib/jenkins/workspace/CI-MERGE-REQUEST/21.2/PhysicsAnalysis/AnalysisCommon/HDF5Utils/util/hdf5-merge.cxx:6:
    08:32:51 /var/lib/jenkins/workspace/CI-MERGE-REQUEST/21.2/PhysicsAnalysis/AnalysisCommon/HDF5Utils/HDF5Utils/IH5Merger.h:62:15: error: 'H5::H5Location' has not been declared
    08:32:51            H5::H5Location& targetLocation,
    08:32:51                ^~~~~~~~~~
    08:32:51 /var/lib/jenkins/workspace/CI-MERGE-REQUEST/21.2/PhysicsAnalysis/AnalysisCommon/HDF5Utils/HDF5Utils/IH5Merger.h:71:15: error: 'H5::H5Location' has not been declared
    08:32:51            H5::H5Location& targetLocation,
    08:32:51                ^~~~~~~~~~
    08:32:51 In file included from /var/lib/jenkins/workspace/CI-MERGE-REQUEST/21.2/PhysicsAnalysis/AnalysisCommon/HDF5Utils/util/hdf5-merge.cxx:6:0:
    08:32:51 /var/lib/jenkins/workspace/CI-MERGE-REQUEST/21.2/PhysicsAnalysis/AnalysisCommon/HDF5Utils/HDF5Utils/DefaultMerger.h:68:15: error: 'H5::H5Location' has not been declared
    08:32:51            H5::H5Location& targetLocation,
    08:32:51                ^~~~~~~~~~

    this is likely a header mismatch since H5::H5Location is a "new" base class. My wild guess is that LCG is including HDF5 1.8 and cmake is building against that.

  • Contributor

    Ah damn - I missed that link at the top :(

    But why would it be building against something different than the analysis releases?

  • Dan Guest added 1 commit

    added 1 commit

    • 3ba431cc - Fix HDF5 header path in CMake

    Compare with previous version

  • This merge request affects 1 package:

    • PhysicsAnalysis/AnalysisCommon/HDF5Utils

    Adding @dguest as watcher

  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
Please register or sign in to reply
Loading