Skip to content
Snippets Groups Projects

Updates to support systematics without running out of memory

Merged Updates to support systematics without running out of memory
dguest/training-dataset-dumper:bsvc into main
Merged Dan Guest requested to merge dguest/training-dataset-dumper:bsvc into main

There are a few changes here, mostly to make sure we can run the systematic variations on the grid.

  • Use a service to share instances of the GNN. This is particularly needed when we run systematic variations.
  • Move the jet calibration stuff out of the dumpster Alg, thus preventing one calibration tool per systematic
    • Use public calibration tools which are shared between all the systematics
    • Add an option to run the calibration after the GNN (currently it runs before) to evaluate how much this matters
  • Add back more systematics
  • Add an option to raise the frequency of updates in eventLoopHeartBeat.txt

Reworking where the jet calibration happens made me realize that the jet systematics are a bit more subtle than I'd originally thought: we normally apply the flavor tagging on uncalibrated jets, but meanwhile jetmet provides systematics on calibrated jets. So to vary the inputs to flavor tagging we first have to calibrate the jets, which isn't what we do in production. My plan is to add an "uncalibrated" systematic (run the calibration after the GNN) to see how big an effect this is.

Might be interesting to @vhomsak.

This MR requires releases with with atlas/athena!71323 (merged), which should be 25.2.12 and 25.0.6. I expect the CI to fail until 25.2.12 is pushed to the gitlab registry.

Edited by Dan Guest

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
Please register or sign in to reply