Add a basic script to validate metadata content

In the context of recent examples of inconsistencies in the in-file metadata content due to interference of jobs running in containers (ATLASG-2686, AFT-711), this MR adds a script to check the sanity of the metadata:

• check if the number of events from EventStreamInfo equals to the number of entries in DataHeader
• check if /TagInfo metadata contains inconsistent information
• check presence of FileMetaData
• check uniqueness of run/lumiblock/event number per event and against the summary in the FileMetaData

This is of course non-exhaustive, so I'm open to suggestions regarding other checks.

I've added this to run in one of the ART tests, please let me know if this is sufficient and correct.

/cc @gemmeren @amete

Thanks @maszyman ,

I think its a very good idea to have such a script check for metadata sanity. As a first step we can add it to art tests (as you did) and then later (after we found no false positives) even to the official file validation in production.

Peter

marked this merge request as ready

This merge request affects 2 packages:

• Database/APR/RootStorageSvc
• PhysicsAnalysis/DerivationFramework/DerivationFrameworkART

Affected files list will not be printed in this case

Adding @gemmeren ,@mnowak ,@jcatmore ,@amete ,@ssnyder ,@emmat ,@maszyman as watchers

added Database Derivation main review-pending-level-1 labels

CI Result SUCCESS (hash 2e864e8a)

	Athena	AthSimulation	AthGeneration	AthAnalysis
externals
cmake
make
tests

Full details available on this CI monitor view. Check the JIRA CI status board for known problems
Athena: number of compilation errors 0, warnings 0
AthSimulation: number of compilation errors 0, warnings 0
AthGeneration: number of compilation errors 0, warnings 0
AthAnalysis: number of compilation errors 0, warnings 0
For experts only: Jenkins output [CI-MERGE-REQUEST-EL9 8211] (remote access info)

removed review-pending-level-1 label

added review-approved label

merged

mentioned in commit a1bd0bbc

That's a very nice script, but I think it should go to some other location than APR. Maybe AthenaPoolUtilities?

Yes, I'm not attached to this location. I chose it to live with other scripts serving similar purpose.

Right, copy-paste error. Will fix it in the following MR

Otherwise, it does not work.

mentioned in merge request !70717 (merged)