Skip to content

WIP: Reduce memory consumpion of plotting and module file combination

Konstantin Mauer requested to merge combine_module_files_memory into development

In plotting data is not loaded from the .h5 file during initialization anymore, but rather gets loaded sequentially the corresponding create_plot methods to minimize the amount of memory occupied. This does not influence the runtime significantly, but is required for quad modules with big histograms.

In combine_module_files.py the occupied memory is minimized, by only loading a histogram of one chip at any time and writing it to a combined histogram.

Now combining and plotting data of e.g. a common quad should not occupy really more than 1.5 to 2 GB of Memory.(Closes #484 (closed))

In addition Tot tuning results now should be able to be plotted again. The combine script failed since the Tot tuning has different scan parameters between multiple chips, but cannot be rebinned. Instead of being rebinned, scan parameters get concatenated, while a 'chip_id' column gets added. The histograms get concatenated (smaller histograms get extended to biggest shape by filling them up with zeros).

Edited by Konstantin Mauer

Merge request reports