Skip to content

Backpressure cases

Created by: hsakulin

In order of usefulness (within a group, modules are exclusive or can trigger at the same time):

  • Raw Trigger rate > 200 kHz => Ask the trigger shifter to check where the high trigger rate is coming from #139 (closed)
  • Raw Trigger rate 100 to 200 kHz => Are we running with the correct pre-scale column ?=> talk to the trigger shifter and shift leader. #139 (closed)

Can fire independently (and are used by DAQ backpressure module)

  • CMSSW crashes (see #134 (closed)) => "Call the HLT DOC, mentioning the messages under you see under HLT Alerts in F3 Mon. Call the DAQ DOC. He might need to clean up the Filter Farm. "

To be done by the LM: Check if DAQ Backpressure from Filter Farm also fired and mention if it did. (This may appear after the condition has started)

  • HLT output rate too high (see #17 (closed)) => "Are we running with the correct pre-scale column ? => talk to the trigger shifter and shift leader. You may need to call the HLT DOC."

To be done by the LM: Check if DAQ Backpressure from Filter Farm also fired and mention if it did. (This may appear after the condition has started)

  • HLT CPU usage high (>90%, see also #44 (closed)) => "If you see high dead time and DAQ Backpressure: Are we running with the correct pre-scale column ? => talk to the trigger shifter and shift leader. You may need to call the HLT DOC."

To be done by the LM: Check if DAQ Backpressure from Filter Farm also fired and mention if it did. (This may appear after the condition has started)

  • Individual FED dead time >2% but no( < 2 % ) DAQ Backpressure (#155 (closed))
  • Individual FED dead time >2% and DAQ backpressure (with the 3 sub-cases below)
  • Temporary Special for uTCA: FED with >2% DAQ Backpressure (with 3 sub-cases)
  • TTS dead time > 2% =>look at CPM page
  • Total Dead time > 8%

These 3 cases should be mutually exclusive (see #142 (closed))

=> DAQ Backpressure coming from FEROL or FEDBuilder. Call the DAQ On-Call and mention this message.

=> DAQ Backpressure from the Event Builder. Call the DAQ DOC mentioning the backpressure comes from event building.

BETTER (too many requests on EVM here, I took next snapshot): http://daq-expert.cms/daq2view-react/index.html?setup=cdaq&time=2017-10-01-20:12:30 BETTER: http://daq-expert.cms/daq2view-react/index.html?setup=cdaq&time=2017-10-01-20:12:32

The sub-cases below are not needed since each sub-case will already trigger a LM with higher priority. [

Check if the HIGH HLT output rate or CMSSW crashing modules fired

=> DAQ Backpressure from the Filter Farm a) Because of High output rate (#17 (closed)) => Are we running with the correct pre-scale column ? => talk to the trigger shifter and shift leader. You may need to call the HLT DOC. b) Because of CMSSW processes crashing (#134 (closed)) => Call the HLT DOC, mentioning the messages under you see under HLT Alerts in F3 Mon. Call the DAQ DOC. He might need to clean up the Filter Farm. c) HLT CPU usage high (#44 (closed)) => Are we running with the correct pre-scale column ? => talk to the trigger shifter and shift leader. You may need to call the HLT DOC.. d) (none of a,b,c) => Unidentified problem => Call the DAQ DOC.

]