Hdf5 merger
It would be quite nice to merge HDF5 files on the grid. Fortunately @jburr has written a utility to do this!
NOTE: I've kept the history from Jon's original repository in this merge request, in case it helps to merge in any final changes. I suppose we should squash this when it's ready to be merged, though.
Merge request reports
Activity
@jburr, comments welcome. If you're ok with this I'll unmark it as a WIP.
Hi @jburr, changed. Anything else you can spot?
This merge request affects 1 package:
- PhysicsAnalysis/AnalysisCommon/HDF5Utils
Adding @dguest as watcher
added 21.2 Analysis review-pending-level-1 labels
CI Result FAILUREAnalysisBase AnalysisTop AthAnalysis AthDerivation externals cmake make required tests optional tests Full details available at NICOS MR-24892-2019-07-16-01-44
AnalysisBase: number of compilation errors 0, warnings 0
AnalysisTop: number of compilation errors 0, warnings 0
AthAnalysis: number of compilation errors 1, warnings 257
AthDerivation: number of compilation errors 0, warnings 11
For experts only: Jenkins output [CI-MERGE-REQUEST 39906]This merge request affects 1 package:
- PhysicsAnalysis/AnalysisCommon/HDF5Utils
Adding @dguest as watcher
CI Result FAILUREAnalysisBase AnalysisTop AthAnalysis AthDerivation externals cmake make required tests optional tests Full details available at NICOS MR-24892-2019-07-16-08-56
AnalysisBase: number of compilation errors 0, warnings 0
AnalysisTop: number of compilation errors 0, warnings 0
AthAnalysis: number of compilation errors 1, warnings 83
AthDerivation: number of compilation errors 0, warnings 1
For experts only: Jenkins output [CI-MERGE-REQUEST 39908]The error also makes no sense... It appears to compile completely fine but then ends on
08:42:08 resuts_dict_generator: Info: added Build["AthDerivation"] = "SUCCESS"
08:42:08 Info: result SUCCESS, overall_result FAILURE
08:42:08 resuts_dict_generator: Info: added BuildTime["Start;AthDerivation"] = "1563258783.797408765"
08:42:08 resuts_dict_generator: Info: added BuildTime["End;AthDerivation"] = "1563259328.950837516"
08:42:08 resuts_dict_generator: Info: added BuildTime["Start"] = "1563258441.219391935"
08:42:08 resuts_dict_generator: Info: added BuildTime["End"] = "1563259328.981692101"
08:42:09 Info: recorded start build time: 1563258441.219391935, end time: 1563259328.981692101
08:42:09 Info: OVERALL RESULT: FAILURE
08:42:09 Build step 'Execute shell' marked build as failure
08:42:09 Finished: FAILURE
Edited by Jon BurrHi @dguest, @jburr, could you please add ATLAS Robot as developer to the fork? (!24892 (merged))
Thanks, Tadej (L2)
@jburr I think you're not looking at the whole log:
08:32:51 [ 32%] Building CXX object Control/AthAllocators/CMakeFiles/AthAllocators.dir/src/Arena.cxx.o 08:32:51 Scanning dependencies of target CxxUtils_make_unique_test 08:32:51 In file included from /var/lib/jenkins/workspace/CI-MERGE-REQUEST/21.2/PhysicsAnalysis/AnalysisCommon/HDF5Utils/HDF5Utils/DefaultMerger.h:8:0, 08:32:51 from /var/lib/jenkins/workspace/CI-MERGE-REQUEST/21.2/PhysicsAnalysis/AnalysisCommon/HDF5Utils/util/hdf5-merge.cxx:6: 08:32:51 /var/lib/jenkins/workspace/CI-MERGE-REQUEST/21.2/PhysicsAnalysis/AnalysisCommon/HDF5Utils/HDF5Utils/IH5Merger.h:62:15: error: 'H5::H5Location' has not been declared 08:32:51 H5::H5Location& targetLocation, 08:32:51 ^~~~~~~~~~ 08:32:51 /var/lib/jenkins/workspace/CI-MERGE-REQUEST/21.2/PhysicsAnalysis/AnalysisCommon/HDF5Utils/HDF5Utils/IH5Merger.h:71:15: error: 'H5::H5Location' has not been declared 08:32:51 H5::H5Location& targetLocation, 08:32:51 ^~~~~~~~~~ 08:32:51 In file included from /var/lib/jenkins/workspace/CI-MERGE-REQUEST/21.2/PhysicsAnalysis/AnalysisCommon/HDF5Utils/util/hdf5-merge.cxx:6:0: 08:32:51 /var/lib/jenkins/workspace/CI-MERGE-REQUEST/21.2/PhysicsAnalysis/AnalysisCommon/HDF5Utils/HDF5Utils/DefaultMerger.h:68:15: error: 'H5::H5Location' has not been declared 08:32:51 H5::H5Location& targetLocation, 08:32:51 ^~~~~~~~~~
this is likely a header mismatch since
H5::H5Location
is a "new" base class. My wild guess is that LCG is including HDF5 1.8 and cmake is building against that.This merge request affects 1 package:
- PhysicsAnalysis/AnalysisCommon/HDF5Utils
Adding @dguest as watcher