Skip to content
Snippets Groups Projects

ensure thread termination tools only run in initialized threads

Merged ensure thread termination tools only run in initialized threads
leggett/Gaudi:dev/checkThreadInit into master
1 unresolved thread
Merged Charles Leggett requested to merge leggett/Gaudi:dev/checkThreadInit into master
1 unresolved thread

no matter how many threads you configure a tbb thread pool to have, it seems that tbb will create NCORES-1 threads, where NCORES is the number of available cores on the system. (You can artificially limit this by using taskset -c.) Of these threads, tbb will only activate the number you have requested for the thread pool at any one time. This means that jobs that have thread local initialization need to proceed carefully. We check for thread initialization via the thread local variable Gaudi::Concurrency::ThreadInitDone which is set during the thread init at the start of the job and as well the first time AlgoExecutionTask::execute() is called for a newly activated thread. However, sometimes one of the tbb threads is activated after the last event has been processed, meaning that the Gaudi::Concurrency::ThreadInitDone is false. This means that IThreadInitTool::terminateThread() of the ThreadInitTools should NOT be called, as the initialization method has not been called (and it's a waste of resources to call it just to terminate it immediately afterwards).

This MR also removes the forced thread initialization during ThreadPoolSvc::initialize, and lets it happen on demand when the AlgoExecutionTask sees a new thread.

Also uses the new tbb::global_control class to limit concurrency (which seems to work better in recent versions of TBB than task_scheduler_init)

Edited by Charles Leggett

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • Marco Cattaneo
    Marco Cattaneo @cattanem ·
    Developer

    @leggett thanks, that seems to have fixed the problem

  • Marco Clemencic assigned to @clemenci

    assigned to @clemenci

  • Marco Clemencic changed milestone to %v32r0

    changed milestone to %v32r0

  • Marco Clemencic mentioned in issue #65 (closed)

    mentioned in issue #65 (closed)

  • Marco Clemencic resolved all discussions

    resolved all discussions

  • Marco Clemencic approved this merge request

    approved this merge request

  • Marco Clemencic merged

    merged

  • Marco Clemencic mentioned in commit aa5e184d

    mentioned in commit aa5e184d

  • Walter Lampl
    Walter Lampl @wlampl started a thread on the diff
    • GaudiHive/src/ThreadPoolSvc.h
    9 9 #include "boost/thread.hpp"
    10 10 #include "tbb/spin_mutex.h"
    11 11 #include "tbb/task_scheduler_init.h"
    12 #define TBB_PREVIEW_GLOBAL_CONTROL 1
    13 #include "tbb/global_control.h"
  • Charles Leggett mentioned in merge request atlas/athena!23054 (merged)

    mentioned in merge request atlas/athena!23054 (merged)

    • Please register or sign in to reply