Skip to content

track systematics through the SystematicsSvc

Nils Erik Krumnack requested to merge krumnack/athena:runtime_sys into master

I'm keeping this as a draft for now: It is incomplete in multiple ways, and I mostly made it to gather feedback from users and other developers. I also only updated the muon sequence so far, just so that there is an example of how things should look like, so that people can criticize it.

Main changes here:

  • Each algorithm reports its list of systematics from the tool it uses to the SystematicsSvc (during initialize) which uses that information to calculate the list of systematics to run and report to the users what systematics are availbale.
  • Each systematics handle now requests input systematics and reports output systematics to the SystematicsSvc (during initialize) which tracks them per-object.
  • This completely separates handling of temporary shallow copies from the systematics management. This can in certain situations mean that unnecessary shallow copies are created (i.e. shallow copies that don't introduce new systematics) as the configuration can no longer peek at the systematics to eliminate those.
  • Besides per-object systematics the SystematicsSvc also tracks per-decoration systematics.
  • All copies of systematic varied objects and objects with systematic varied decorations are now tracked through the SystematicSvc. This is a requirement to track per-decoration systematics when an object gets copied.
  • All the systematics handles now cache all the information they need during initialize instead of during the first event they are used.
  • All the systematics handles now need to be connected to the SysListHandle. Some already needed to, but now it's all of them.
  • I updated the handles to fail in initialize if they are used while empty. This can be overridden via SG::AllowEmpty.
  • I merged the no-Aux specialization of SysWriteHandle into the main template. I don't recall why I did that separately, but they have so much in common that having an explicit template specialization just doubles the work in maintaining them.
  • I removed SysDecorationHandle::preExecute().
  • In the old implementation decoration names wouldn't include the names of systematics that are already applied to the container. Now it includes all affecting systematics in the decoration name. Is that Ok?

More followup work to do, that I hope can wait for future MRs:

  • Since the main purpose of the sequence post-configuration was to track systematics, we should take another look at whether we can further simplify or eliminate the post-configuration.
  • Since the systematics handles now get completely configured in initialize it would be possible to update them to use Gaudi data handles as a backend, allowing to run them in AthenaMT. This may potentially need some sort of hack to reduce the number of handles reported to Gaudi (i.e. only create one Gaudi handle per systematics handle).
  • Update the PileupReweightingAlg to store its systematics as decorations instead of systematics copies.
  • Update SysListHandle to add a property that allows to filter out systematics that it explicitly knows it is not affected by. This is mostly in case a systematic affects an input, but in a way that doesn't matter for the output of the algorithm. In particular the issue with the lacking tracking in the SysDecorationHandle made me realize that using the EventInfo object created by the PileupReweightingAlg introduces pileup reweighting systematics to various algorithms that shouldn't care. I could potentially also do that in this MR if people care. However, somebody would have to go through the configured sequences and decide where to apply it.
  • Double check that we only use copy-handles in cases in which we actually want to modify the object, and are not just decorating it.

There are probably some other points I meant to mention, but I forgot. I'll update this if I remember.

Tagging the usual suspects: @jburr @tadej @lheinric @alister @vimartin

Edited by Nils Erik Krumnack

Merge request reports