Skip to content

Draft: Only read necessary columns during convert parquet to root for finalfits

Jan Lukas Spah requested to merge reduce_postprocessing_memory_footprint into master

In the current state, the postprocessing can fail due to memory limitations of uproot since all columns are read by default in convert_parquet_to_root. I added functionality that only the ~bare minimum of fields are read to reduce the memory footprint during the conversion, making it less likely to fail for big conversions (especially on lxplus). This is not the prettiest solution and should be changed in the future so that people can specify which fields they would like to have read from merged parquet and saved to root.

Merge request reports