Skip to content

Repair NN code update and move away from COOL

Katherine Pachal requested to merge kpachal/athena:21.0-repair-NN-update into 21.0

Hi all,

This is a replacement for MR 27787 which we reverted. Two ART tests were found to crash.

After investigation this was found to be due to the clustering NN trained networks being stored in the conditions database, such that each run was being reprocessed with the version of the NN in the corresponding conditions tag. We discussed in ATLIDTRKCP-223 and decided to move to using the latest version of the NNs instead and storing them in a CalibArea so they can be detached from the run conditions.

This MR:

  • Removes outdated NN functionality as in 27787
  • Sets up to read the trained NN configuration from a specified location using PathResolver, which can be either a CalibArea or a local directory for testing
  • Sets the current default trained NN to be the one used in mc16 reconstruction
  • Moves all instances of NnClusterizationFactory configuration to a single Python class in SiClusterizationFactory so that all settings are controlled from a single location
  • Removes some code redundancy in evaluating the cluster positions

I made one decision I am not sure of. I have kept this setting: correctLorShiftBarrelWithTracks = 0.030, for reconstruction of Run 1 data. I do not know where it comes from, and after testing various runs with this turned on and off, I see slightly fewer errors and slightly fewer clusters evaluated as shared when it is left on. If anyone knows the origin of this term and can clarify if it corresponds to detector or software conditions I'd appreciate it.

Tagging @gfacini, @rhowen, @goblirsc, @npetters, @pottgen for info.

Some notes:

  • This MR will result in different outcomes for ART tests which currently use old NNs. Particularly test_run00184169_egamma_collisions and test_Run00218048_HardProbes_Collisions_HIPReco but possibly others also. If the tests report failure we should update the reference files.
  • No CalibArea is available yet so the default file is being read directly from the COOL folders on cvmfs. This will be updated in a later MR
  • This is marked sweep:ignore since the NnClusterizationFactory is sufficiently different in master that the equivalent changes will need to be done by hand.

Merge request reports