Skip to content

Draft: ref: output: Rewrite sink functionality

This MR rewrites much of the output layer. It's meant to consolidate loose variables and redundant code into concise abstractions and support further changes beyond the traditional POSIX use case, like expanding support for DAX interfaces.

Main improvements:

  • OutputFile.cc source file
  • sink.h / sink.cc unit with dedicated FileSink class to concentrate synchronous metadata write operations and asynchronous commits, managing the preexisting close-and-rename queue in a separate thread. As such, only the data staging step remains within the scope of OutputFile
  • Metadata classes: RunMetadata, LumisectionMetadata and FileMetadata.
  • [potential bug fix] The run number used in the footers and filenames is now collected at the data intake step, not by consulting the control structure at the output step. This may prevent a race condition if the structure is changed in parallel before all slices make their way down the pipeline.

Some further changes are still planned; above all, clang-formatting has to be re-toggled. These updates will come as needed according to progress on DAX support in the next few days.

Edited by Giovanna Lazzari Miotto

Merge request reports