Skip to content

FileMetaDataCreatorTool: treat updating event and non-event information in FileMetaData separately, and treat dataType as non-event information

This MR emerged from https://its.cern.ch/jira/browse/ATEAM-882

In short, the issue is that in short jobs, where some workers do not process any events, file metadata (in particular dataType) is just copied from the input file, which may lead to e.g. incorrect dataType in the output.

FileMetaData contains non-event information (including dataType) and event information (runNumbers, lumiBlocks and mcProcID) which are set in FileMetaDataCreatorTool::postExecute() by workers (those which saw the events). SharedWriter uses outputs of workers in FileMetaDataTool::beginInputFile(), so it copies the first one, and later on it may warn about inconsistency, but in general does not change it (apart from accumulating runNumbers and lumiBlocks). Then, on endInputFile incidents (files being the outputs of workers in case of SharedWriter) FileMetaDataCreatorTool::updateFromNonEvent() is called where non-event metadata is set. To my understanding, SharedWriter never calls FileMetaDataCreatorTool::postExecute() (as it does not process events), but calls FileMetaDataCreatorTool::preFinalize(). Hence, currently it never updates metadata content (the method returns immediately if event information was not filled (which is always the case for SharedWriter) or non-event information was not filled).

This MR treats separately event and non-event information from FileMetaData in FileMetaDataCreatorTool::preFinalize(), so that SharedWriter can update non-event content in FileMetaData. It also moves setting dataType from FileMetaDataCreatorTool::postExecute() to FileMetaDataCreatorTool::updateFromNonEvent().

Merge request reports

Loading