FPGATrackSim: major optimization of memory usage in track fitter

The track fitter is given a road and, from that road, attempts to generate and fit all possible track combinations. A road is essentially a vector of hit vectors; there can be multiple hits per "layer" on a road object. The combinations are generated by first calling a helper function which generates a list of all possible combinations in the form of coordinate indices, and then processing that list of indices to generate FPGATrackSimTrack objects.

However, if the number of combinations is extremely high, this can attempt to allocate enormous amounts of memory. The makeTrackCandidates function attempts to allocate a vector of FPGATrackSimTrack objects equal to the size of the combinations vector in one go-- and if that is large, this can result in attempting to allocate multiple gigabytes of RAM at once.

This is further made problematic by the fact that some of the index combinations are not actually valid when running with spacepoints. Since spacepoints are represented as two FPGATrackSimHit objects, it isn't valid to attempt to pair up 'different' spacepoints on either side of the same stave. These combinations get marked as invalid and dropped immediately, but the memory for them has already been allocated-- which leads to problems.

There was a comment in the track fitter that this could be optimized by converting to a more iterator-like approach in which the tracks are generated on demand. This commit does exactly that. It adds a makeTrackCandidate function, which is given an index into the combinations vector and generates the next FPGATrackSimTrack object. if that track subsequently turns out to be invalid or gets rejected for any other reason, the memory is immediately freed and the track is not saved. This keeps the memory consumption flat when running over even very large numbers of combinations.

The fitter could be further optimized by reworking the coordinate arithmetic to know about spacepoints; but that will just speed things up a bit. The memory consumption should be significantly less problematic now when running over pileup.

There is a second optimization included here to try and disable more creation of output structures for the ROOT test vector file, when the creation of that file is disabled. I did this first; it should help a bit too, but the main change here is the one described above.

CC @jahreda @jsurduto @imaznas.

I believe the neural network track tool code path (F-610) is different enough that this shouldn't be an issue there, but I'm not sure.

Merge request reports

Loading