Support multiple dataframe in preprocessing
Here are the changes implemented by this MR
-
The preprocessing was cleaned up (I forgot to remove a few things in the previous branch) -
A new subdirs
parameter in the preprocessing configuration, that can be used to have several dataframes as input -
A preprocessing step that loops over the input dataframes. -
A preprocessing step that uses groupby
to loop over events, which is faster than filtering the events at every iteration -
A new "sorted-wise edges" that links the hits by sorting the hits by increasing z
-
An updated "plane-wise edges" that links the hits that belong to adjacent planes within a particle -
A new training that is being slowly set up ( track-weighting
)
I tried to implement some multi-processing in the preprocessing and but seemed to be actually slower so I gave up on the idea.
Edited by Fotis Giasemis