Skip to content

Better CA merging diagnostics

Tomasz Bold requested to merge tbold/athena:add-ca-creation-context into master

This MR is meant to improve the dianostics when one tried to merge CAs. It is more and more likely that we will be merging large CAs. In such cases sometimes it becomes unclear where from the conflicting values values come. This specially the case in HLT where we collect CAs first, then build the HLT Control Flow structure and then merge CAs in appropriate areas. Example below. In this MR the creation context is added for each CA (i.e. sequence of calls that lead to its creation). This information is then used when there are merging issues giving context alike the other one below (this is from unit test, I edited it a bit to remove test related part of the callstack). Also, switched all strings formatting in exception raing to use format rather than the %. Tagging @wlampl

Improved context info:

Traceback (most recent call last):
  File "/srv/build/x86_64-centos7-gcc8-opt/python/AthenaConfiguration/ComponentAccumulatorTest.py", line 315, in runTest
    badMerge2()
  File "/srv/build/x86_64-centos7-gcc8-opt/python/AthenaConfiguration/ComponentAccumulatorTest.py", line 314, in badMerge2
    topCA.merge(someCA)
  File "/srv/build/x86_64-centos7-gcc8-opt/python/AthenaConfiguration/ComponentAccumulator.py", line 580, in merge
    mergeSequences(ourSeq, otherSeq)
  File "/srv/build/x86_64-centos7-gcc8-opt/python/AthenaConfiguration/ComponentAccumulator.py", line 556, in mergeSequences
    riseWithCurrentContext(exception)
  File "/srv/build/x86_64-centos7-gcc8-opt/python/AthenaConfiguration/DebuggingContext.py", line 35, in riseWithCurrentContext
    raise type(exception)(str(exception) + '\nwith the context\n {}'.format(Context.current()) )
ValueError: conflicting settings for property MyInt of hello: cannot merge values 7 and 8
with the context
 When merging ComponentAccumulator
This CA contains 0 service, 0 conditions algorithms, 1 event algorithms and 0 public tools
  Last component added: hello
  Created by:  >> ComponentAccumulatorTest.py:runTest:315 >> ComponentAccumulatorTest.py:badMerge2:310
to:
This CA contains 0 service, 0 conditions algorithms, 1 event algorithms and 0 public tools
  Last component added: hello
  Created by:  >> ComponentAccumulatorTest.py:runTest:298

The initial example:

Traceback (most recent call last):
  File "runHLT_standalone_newJO.py", line 92, in <module>
    acc.merge(triggerRunCfg(flags, seqName="AthMasterSeq", menu=generateHLTMenu))
  File "/cvmfs/atlas-nightlies.cern.ch/repo/sw/master_Athena_x86_64-centos7-gcc8-opt/2021-01-29T2101/Athena/22.0.26/InstallArea/x86_64-centos7-gcc8-opt/python/AthenaConfiguration/ComponentAccumulator.py", line 594, in merge
    self.addCondAlgo(condAlg) #Profit from deduplicaton here
  File "/cvmfs/atlas-nightlies.cern.ch/repo/sw/master_Athena_x86_64-centos7-gcc8-opt/2021-01-29T2101/Athena/22.0.26/InstallArea/x86_64-centos7-gcc8-opt/python/AthenaConfiguration/ComponentAccumulator.py", line 381, in addCondAlgo
    deduplicate(algo,self._conditionsAlgs) #will raise on conflict
  File "/cvmfs/atlas-nightlies.cern.ch/repo/sw/master_Athena_x86_64-centos7-gcc8-opt/2021-01-29T2101/Athena/22.0.26/InstallArea/x86_64-centos7-gcc8-opt/python/AthenaConfiguration/Deduplication.py", line 19, in deduplicate
    newComp.merge(comp)
  File "/cvmfs/atlas-nightlies.cern.ch/repo/sw/master_Athena_x86_64-centos7-gcc8-opt/2021-01-28T2101/AthenaExternals/22.0.26/InstallArea/x86_64-centos7-gcc8-opt/python/GaudiConfig2/_configurables.py", line 279, in merge
    self, type(self), getattr(other, name)))
  File "/cvmfs/atlas-nightlies.cern.ch/repo/sw/master_Athena_x86_64-centos7-gcc8-opt/2021-01-28T2101/AthenaExternals/22.0.26/InstallArea/x86_64-centos7-gcc8-opt/python/GaudiConfig2/_configurables.py", line 88, in __merge__
    return self.semantics.merge(self.__get__(instance, owner), value)
  File "/cvmfs/atlas-nightlies.cern.ch/repo/sw/master_Athena_x86_64-centos7-gcc8-opt/2021-01-29T2101/Athena/22.0.26/InstallArea/x86_64-centos7-gcc8-opt/python/AthenaConfiguration/AtlasSemantics.py", line 91, in merge
    return a.merge(b)
  File "/cvmfs/atlas-nightlies.cern.ch/repo/sw/master_Athena_x86_64-centos7-gcc8-opt/2021-01-28T2101/AthenaExternals/22.0.26/InstallArea/x86_64-centos7-gcc8-opt/python/GaudiConfig2/_configurables.py", line 264, in merge
    type(self).__name__))
TypeError: cannot merge instance of TileCondProxyFile<TileCalibDrawerFlt> into an instance of TileCondProxyCool<TileCalibDrawerFlt>
Py:ComponentAccumulator   ERROR This ComponentAccumulator was never merged!
Py:ComponentAccumulator   ERROR This CA contains 37 service, 69 conditions algorithms, 133 event algorithms and 120 public tools
  Last component added: EDMCreatorAlg

Py:ComponentAccumulator   ERROR This ComponentAccumulator was never merged!
Py:ComponentAccumulator   ERROR This CA contains 35 service, 68 conditions algorithms, 155 event algorithms and 7 public tools
  Last component added: LArBadFebCondAlg

Merge request reports