Skip to content

TrigCost: Add summary of over/under flows in histograms

Aleksandra Poreba requested to merge aporeba/athena:cost-post-underflow-summary into master

Add additional summary to cost metadata about under and overflows in histograms after exceeding given threshold. The values are also printed in the logfile as warnings.

The threshold is defined as a (configurable) part of entries of histogram. Default values are 0,5 for underflow and 0,1 for overflow.

The purpose of this feature is to easily find possibly incorrect measurements and adjust the histograms boundaries. Some underflow is expected, but after exceeding certain number it can impact the results.

Example output (based on https://atlas-trig-cost.cern.ch/?dir=Technical-and-M-weeks&type=&tag=T18&run=393686 file)

      "Histogram under/overflows": [
...
        "Underflow of 79715208.0 bins (8721081.0 entries) in LumiBlock_00000_Global_HLT_LumiBlock_00030_AlgTime_perCall",
        "Underflow of 82428392.0 bins (9021772.0 entries) in LumiBlock_00000_Global_HLT_LumiBlock_00031_AlgTime_perCall",
        "Underflow of 10465760.0 bins (1144890.0 entries) in LumiBlock_00000_Global_HLT_LumiBlock_00032_AlgTime_perCall",
        "Underflow of 1758435712.0 bins (179005180.0 entries) in LumiBlock_00000_Algorithm_Class_HLT_PassFilter_Time_perCall",
        "Underflow of 1758435712.0 bins (179005180.0 entries) in LumiBlock_00000_Algorithm_Class_HLT_PassFilter_FirstTime_perEvent",
        "Underflow of 636893312.0 bins (81787288.0 entries) in LumiBlock_00000_Algorithm_Class_HLT_RoRSeqFilter_Time_perCall",
        "Underflow of 636893312.0 bins (81787288.0 entries) in LumiBlock_00000_Algorithm_Class_HLT_RoRSeqFilter_FirstTime_perEvent",
...
        {
          "Summary": [
            "Algorithm_HLT_Time_perCall: 1780 histogram over/under flows",
            "Algorithm_HLT_FirstTime_perEvent: 1780 histogram over/under flows",
            "Algorithm_HLT_Time_perEvent: 1723 histogram over/under flows",
            "Global_HLT_AlgTime_perCall: 31 histogram over/under flows",
            "Algorithm_Class_HLT_Time_perCall: 90 histogram over/under flows",
            "Algorithm_Class_HLT_FirstTime_perEvent: 90 histogram over/under flows",
            "Algorithm_Class_HLT_Time_perEvent: 53 histogram over/under flows",
            "Chain_HLT_Time_perCall: 102 histogram over/under flows",
            "Thread_Occupancy_HLT_FrameworkTime_perEventFractional: 14 histogram over/under flows",
            "Thread_Occupancy_HLT_UnmonitoredTime_perEvent: 14 histogram over/under flows",
            "Underflow threshold: 0.5",
            "Overflow threshold: 0.1"
          ]
        }

Tagging @tamartin

Edited by Aleksandra Poreba

Merge request reports