Skip to content
Snippets Groups Projects

WIP: make scheduler state available in eventcontext and add timing feature

Closed WIP: make scheduler state available in eventcontext and add timing feature
NN_makeSchedulerStateAvailable into master
Closed Niklas Stefan Nolte requested to merge NN_makeSchedulerStateAvailable into master

OBSOLETE: to be split into smaller MRs

made the node + algorithm states available in the evtcontext to be able to access the information in a monitoring algorithm. Also a prototype "decreportswriter" will be introduced using the information in the evtcontext extension.

added a feature to define when to start and stop the timing process + some on-the-fly minichanges.

@graven @chasse FYI

Edited by Niklas Stefan Nolte

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • Niklas Stefan Nolte added 1 commit

    added 1 commit

    Compare with previous version

  • Niklas Stefan Nolte
    Niklas Stefan Nolte @nnolte ·
    Author Contributor 
    *** Error in `python': double free or corruption (fasttop): 0x00007ffba8002190 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x81499)[0x7ffbc0354499]
/home/nnolte/software/stack/LHCb/build.x86_64-centos7-gcc7-opt/lib/libHLTScheduler.so(_ZN5boost3any6holderISt4pairISt6vectorI9NodeStateSaIS4_EES3_IsSaIsEEEED0Ev+0x20)[0x7ffbafe70f30]
/home/nnolte/software/stack/Gaudi/InstallArea/x86_64-centos7-gcc7-opt/lib/libGaudiKernel.so(_ZN9Algorithm10sysExecuteERK12EventContext+0x65)[0x7ffbb52377d5]
/home/nnolte/software/stack/Gaudi/InstallArea/x86_64-centos7-gcc7-opt/lib/libGaudiAlgLib.so(_ZN14GaudiAlgorithm10sysExecuteERK12EventContext+0x224)[0x7ffbb1d07a14]
/home/nnolte/software/stack/LHCb/build.x86_64-centos7-gcc7-opt/lib/libHLTScheduler.so(_ZNK9BasicNode7executeERSt6vectorI9NodeStateSaIS1_EERS0_IsSaIsEER12EventContextP16IAlgExecStateSvcR7SmartIFI9IPropertyE+0xc4)[0x7ffbafe795e4]
/home/nnolte/software/stack/LHCb/build.x86_64-centos7-gcc7-opt/lib/libHLTScheduler.so(+0x767f5)[0x7ffbafe667f5]
/home/nnolte/software/stack/LHCb/build.x86_64-centos7-gcc7-opt/lib/libHLTScheduler.so(+0x77ce9)[0x7ffbafe67ce9]
/home/nnolte/software/stack/LHCb/build.x86_64-centos7-gcc7-opt/lib/libHLTScheduler.so(+0x782ad)[0x7ffbafe682ad]
/cvmfs/lhcb.cern.ch/lib/lcg/releases/LCG_93/tbb/2018_U1/x86_64-centos7-gcc7-opt/lib/libtbb.so.2(+0x27256)[0x7ffbb44bc256]
/cvmfs/lhcb.cern.ch/lib/lcg/releases/LCG_93/tbb/2018_U1/x86_64-centos7-gcc7-opt/lib/libtbb.so.2(+0x1fe58)[0x7ffbb44b4e58]
/cvmfs/lhcb.cern.ch/lib/lcg/releases/LCG_93/tbb/2018_U1/x86_64-centos7-gcc7-opt/lib/libtbb.so.2(+0x1e94b)[0x7ffbb44b394b]
/cvmfs/lhcb.cern.ch/lib/lcg/releases/LCG_93/tbb/2018_U1/x86_64-centos7-gcc7-opt/lib/libtbb.so.2(+0x1abc5)[0x7ffbb44afbc5]
/cvmfs/lhcb.cern.ch/lib/lcg/releases/LCG_93/tbb/2018_U1/x86_64-centos7-gcc7-opt/lib/libtbb.so.2(+0x1ae09)[0x7ffbb44afe09]
/lib64/libpthread.so.0(+0x7e25)[0x7ffbc0db0e25]
/lib64/libc.so.6(clone+0x6d)[0x7ffbc03d1bad]
======= Memory map: ========
00400000-00401000 r-xp 00000000 00:35 278485881                          /cvmfs/lhcb.cern.ch/lib/lcg/releases/Python/2.7.13-b163d/x86_64-centos7-gcc7-opt/bin/python2.7
00600000-00601000 r--p 00000000 00:35 278485881                          /cvmfs/lhcb.cern.ch/lib/lcg/releases/Python/2.7.13-b163d/x86_64-centos7-gcc7-opt/bin/python2.7
00601000-00602000 rw-p 00001000 00:35 278485881                          /cvmfs/lhcb.cern.ch/lib/lcg/releases/Python/2.7.13-b163d/x86_64-centos7-gcc7-opt/bin/python2.7
00a71000-00f8e000 rw-p 00000000 00:00 0                                  [heap]
7ffb98000000-7ffb98021000 rw-p 00000000 00:00 0 
7ffb98021000-7ffb9c000000 ---p 00000000 00:00 0 
7ffb9c000000-7ffb9c021000 rw-p 00000000 00:00 0 
7ffb9c021000-7ffba0000000 ---p 00000000 00:00 0 
7ffba0000000-7ffba0021000 rw-p 00000000 00:00 0 
7ffba0021000-7ffba4000000 ---p 00000000 00:00 0 
7ffba4000000-7ffba4021000 rw-p 00000000 00:00 0 
7ffba4021000-7ffba8000000 ---p 00000000 00:00 0 
7ffba8000000-7ffba8021000 rw-p 00000000 00:00 0 
7ffba8021000-7ffbac000000 ---p 00000000 00:00 0 
7ffbaea19000-7ffbaea1a000 ---p 00000000 00:00 0 
7ffbaea1a000-7ffbaee1a000 rw-p 00000000 00:00 0                          [stack:4034]
7ffbaee1a000-7ffbaee1b000 ---p 00000000 00:00 0 
7ffbaee1b000-7ffbaf21b000 rw-p 00000000 00:00 0                          [stack:4033]
7ffbaf21b000-7ffbaf21c000 ---p 00000000 00:00 0 
7ffbaf21c000-7ffbaf61c000 rw-p 00000000 00:00 0                          [stack:4032]
7ffbaf61c000-7ffbaf61d000 ---p 00000000 00:00 0 
7ffbaf61d000-7ffbafa1d000 rw-p 00000000 00:00 0                          [stack:4031]
7ffbafa1d000-7ffbafae8000 r-xp 00000000 00:2b 3275262933                 /home/nnolte/software/stack/Gaudi/InstallArea/x86_64-centos7-gcc7-opt/lib/libGaudiCommonSvcLib.so
7ffbafae8000-7ffbafb02000 r--p 000ca000 00:2b 3275262933                 /home/nnolte/software/stack/Gaudi/InstallArea/x86_64-centos7-gcc7-opt/lib/libGaudiCommonSvcLib.so
7ffbafb02000-7ffbafb03000 rw-p 000e4000 00:2b 3275262933                 /home/nnolte/software/stack/Gaudi/InstallArea/x86_64-centos7-gcc7-opt/lib/libGaudiCommonSvcLib.so
7ffbafb03000-7ffbafd3b000 r-xp 00000000 00:2b 3275262934                 /home/nnolte/software/stack/Gaudi/InstallArea/x86_64-centos7-gcc7-opt/lib/libGaudiCommonSvc.so
7ffbafd3b000-7ffbafdee000 r--p 00237000 00:2b 3275262934                 /home/nnolte/software/stack/Gaudi/InstallArea/x86_64-centos7-gcc7-opt/lib/libGaudiCommonSvc.so
7ffbafdee000-7ffbafdf0000 rw-p 002ea000 00:2b 3275262934                 /home/nnolte/software/stack/Gaudi/InstallArea/x86_64-centos7-gcc7-opt/lib/libGaudiCommonSvc.so
7ffbafdf0000-7ffbafea4000 r-xp 00000000 00:2b 3238145002                 /home/nnolte/software/stack/LHCb/build.x86_64-centos7-gcc7-opt/lib/libHLTScheduler.so
7ffbafea4000-7ffbafebb000 r--p 000b3000 00:2b 3238145002                 /home/nnolte/software/stack/LHCb/build.x86_64-centos7-gcc7-opt/lib/libHLTScheduler.so
7ffbafebb000-7ffbafebc000 rw-p 000ca000 00:2b 3238145002                 /home/nnolte/software/stack/LHCb/build.x86_64-centos7-gcc7-opt/lib/libHLTScheduler.so
7ffbafebc000-7ffbb00bc000 rw-p 00000000 00:00 0 
7ffbb00bc000-7ffbb00d5000 r-xp 00000000 00:35 278625177                  /cvmfs/lhcb.cern.ch/lib/lcg/releases/tbb/2018_U1-d3621/x86_64-centos7-gcc7-opt/lib/libtbbmalloc.so.2
7ffbb00d5000-7ffbb02d4000 ---p 00019000 00:35 278625177                  /cvmfs/lhcb.cern.ch/lib/lcg/releases/tbb/2018_U1-d3621/x86_64-centos7-gcc7-opt/lib/libtbbmalloc.so.2
7ffbb02d4000-7ffbb02d5000 r--p 00018000 00:35 278625177                  /cvmfs/lhcb.cern.ch/lib/lcg/releases/tbb/2018_U1-d3621/x86_64-centos7-gcc7-opt/lib/libtbbmalloc.so.2
7ffbb02d5000-7ffbb02d7000 rw-p 00019000 00:35 278625177                  /cvmfs/lhcb.cern.ch/lib/lcg/releases/tbb/2018_U1-d3621/x86_64-centos7-gcc7-opt/lib/libtbbmalloc.so.2
7ffbb02d7000-7ffbb02fa000 rw-p 00000000 00:00 0 
7ffbb02fa000-7ffbb0342000 r-xp 00000000 00:35 278463822                  /cvmfs/lhcb.cern.ch/lib/lcg/releases/ROOT/6.12.06-51921/x86_64-centos7-gcc7-opt/lib/libThread.so
7ffbb0342000-7ffbb0542000 ---p 00048000 00:35 278463822                  /cvmfs/lhcb.cern.ch/lib/lcg/releases/ROOT/6.12.06-51921/x86_64-centos7-gcc7-opt/lib/libThread.so%

    strange errors here.. update: happens when going to more than 1 thread apparently.
    update: seems to happen only when going to more than ~40 evts, not determinsistic.. This might be the reason i did not see it in my earlier tests..
    update: also happens when using code in master and adding the single line evtContext->setExtension(m_NodeStates) in executeEvent()
    update: Gaudi/GaudiKernel/src/Lib/Algorithm.cpp the copy of the eventContext in Algorithm::sysExecute() causes the error. See gaudi/Gaudi#47 (closed)

    Edited by Niklas Stefan Nolte
  • Niklas Stefan Nolte
    Niklas Stefan Nolte @nnolte ·
    Author Contributor

    @cattanem I removed the label for now, there seems to be something quite wrong.. will ping you again when i understood this..

  • Niklas Stefan Nolte mentioned in issue gaudi/Gaudi#47 (closed)

    mentioned in issue gaudi/Gaudi#47 (closed)

  • Niklas Stefan Nolte added 2 commits

    added 2 commits

    • 4bc45be1 - added suggested comments + made scheduler and configurable dummy compatible with Gaudi/703
    • 2bf68beb - added compatibility to Gaudi/703

    Compare with previous version

  • Niklas Stefan Nolte
    Niklas Stefan Nolte @nnolte ·
    Author Contributor

    added compatibility to gaudi/Gaudi!703 (merged).
    This and the Gaudi MR now build up to Brunel (did not test further).
    They can be tested together when the time comes. @clemenci fyi

    Edited by Niklas Stefan Nolte
  • Niklas Stefan Nolte added 2 commits

    added 2 commits

    • 4ff6f4ee - refactor scheduler code
    • 8ccb1362 - added first prototype of a decreportswriter

    Compare with previous version

  • Niklas Stefan Nolte added 1 commit

    added 1 commit

    • 13fe5ed4 - added output writing to decreportswriter

    Compare with previous version

  • Gerhard Raven @graven started a thread on the diff
    • Hlt/HLTScheduler/HLTAlgStatesWriter.cpp 0 → 100644
    12 auto sc = Gaudi::Algorithm::initialize();
    13 if ( !sc ) return sc;
    14 m_outputHandle = std::make_unique<std::remove_reference_t<decltype(*m_outputHandle)>>( std::string(m_outKey), this );
    15 declareProperty(m_outKey, *m_outputHandle);
    16 return StatusCode::SUCCESS;
    17 }
    18
    19 StatusCode HLTAlgStatesWriter::start()
    20 {
    21 auto sc = Gaudi::Algorithm::start();
    22 if ( !sc ) return sc;
    23 m_schedulerPtr = dynamic_cast<HLTControlFlowMgr*>(&*service<IEventProcessor>("HLTControlFlowMgr", false));
    24 return m_schedulerPtr ? StatusCode::SUCCESS : StatusCode::FAILURE;
    25 }
    26
    27 StatusCode HLTAlgStatesWriter::execute(EventContext const & evtCtx) const
    • G
      Gerhard Raven @graven ·
      Developer

      You know that you can define a Transformer<std::vector<NodeState>(EventContext const&)>? See eg here for an example...

      Edited by Gerhard Raven
    • Niklas Stefan Nolte
      Niklas Stefan Nolte @nnolte ·
      Author Contributor

      it would rather be a producer, wouldn't it? The reason for this was btw because the functional stuff does not yet inherit from Gaudi::Algorithm in gaudi/Gaudi!703 (merged) by default, forgot that i can change this by adjusting the traits, though..

    • G
      Gerhard Raven @graven ·
      Developer

      You need to consume EventContext and produce DecReport... The functional stuff already has EventContext special-cased as input which it gets from the argument of execute instead of the TES. (hence it is the only special case, as that's the only extra thing it gets ;-)

    • Niklas Stefan Nolte
      Niklas Stefan Nolte @nnolte ·
      Author Contributor

      Transformer overrides execute(), but I need to override execute(EventContext const &) (to be able to use it with Gaudi::Algorithm)..

    • G
      Gerhard Raven @graven ·
      Developer

      sounds like a job for std::conditional and some std::experimental::is_detected to figure out whether the base class has an execute() or execute(EventContext const &) const and adapt appropriately...

    • Please register or sign in to reply
  • Niklas Stefan Nolte
  • Rosen Matev
    Rosen Matev @rmatev started a thread on the diff
    • Hlt/HLTScheduler/HLTAlgStatesWriter.cpp 0 → 100644
    1 #include "HLTAlgStatesWriter.h"
    2 DECLARE_COMPONENT( HLTAlgStatesWriter )
    • Rosen Matev
      Rosen Matev @rmatev ·
      Maintainer

      XExecutionReportsMaker

      • X - something to do with the name of the scheduler (which is not fixed yet, I think)
      • States is too generic -> ExecutionReports, ExecReports or even the more historical DecReports
      • Creator because Writer so far typically means creating a raw bank (i.e. serialization)
      Edited by Rosen Matev
    • Please register or sign in to reply
  • Niklas Stefan Nolte
    Niklas Stefan Nolte @nnolte ·
    Author Contributor

    I will wait for the merge of gaudi/Gaudi!703 (merged) before continuing with this, since marco is still actively changing stuff there.

  • Niklas Stefan Nolte changed the description

    changed the description

  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
    • Please register or sign in to reply