Skip to content

Improve scheduler stall debug output

Benjamin Michael Wynne requested to merge bwynne/Gaudi_gaudi:StallDebugOutput into master

When the scheduler stalls, additional information about CONTROLREADY algorithms helps to debug the stall.

For each algorithm in the CONTROLREADY state, dumpSchedulerState() will print out which data dependencies are unmet. Using a DataHandleHolderVisitor, identify if this dependency comes from a tool owned by the algorithm. Then print which algorithms might produce that data, and what their current state is.

Sub-slot/EventView debug output is made less verbose by only printing those PRGraph nodes downstream of the entry point.

For example, the stall output from GaudiHive/options/detectSimpleStall.py now looks like this:

[ slot: 0  event: 0 ]:

RootDecisionHub (0), w/ decision: UNDEFINED(-1)
  EarlyExitBranch (1), w/ decision: FALSE(0)
    A1 (2), w/ decision: FALSE(0), in state: EVTREJECTED
    A2 (3), w/ decision: UNDEFINED(-1), in state: INITIAL
  A3 (4), w/ decision: UNDEFINED(-1), in state: CONTROLREADY
  ========
  missing data: '/Event/a2'
  can be produced by alg(s): ( A2 in state: INITIAL )
  ========
Edited by Benjamin Michael Wynne

Merge request reports