Skip to content

Faster alg states search

Benjamin Michael Wynne requested to merge bwynne/Gaudi_gaudi:FasterAlgStatesSearch into master

When there are a large number of algorithms in a job (>1k) the time taken to search the AlgsExecutionStates data becomes a bottleneck.

Profiling with VTune shows that the limiting factor is the custom iterator for retrieving all algorithms in a particular state: while this code is extremely neat, it does not scale well as the structure becomes large.

Master.ManySmall.cropped (Sorted by self-cost, entries associated with alg state iterator highlighted. Tested with GaudiHive/options/ManySmallAlgs.py)

Rather than searching for algs in particular states, I have added a bookkeeping structure that collects algorithm indices by their current state. In the ManySmallAlgs.py test this leads to a 10% performance improvement.

This issue was originally noticed in the ATLAS HLT (https://its.cern.ch/jira/browse/ATR-24078) where there are around 4000 algorithms in a job, and ~50 event views can be expected. A test job with these characteristics (although 0 algorithm execution time) shows a 2.5x speedup with this MR compared to master.

Edited by Benjamin Michael Wynne

Merge request reports