Skip to content
Snippets Groups Projects

AthenaMP+AthenaServices: Support executing algorithms/sequences in AthenaMP during PreFork

Merged Alaettin Serhan Mete requested to merge amete/athena:master-athenamp-prefork-cond into master
All threads resolved!

This is an adaptation of @fwinkl's !55759 (merged) to generic AthenaMP. In a nutshell, this gives the possibility of executing certain algorithms/sequences during PreFork incident in AthenaMP. One can use this to execute the conditions algorithms before the first event, hence sharing the memory across workers even without FAFE.

Based on an 8 process, 10k event AODtoDAOD test w/ SharedWriter, vanilla MP uses about 2 GB/worker, executing conditions algorithms before the event loop this way reduces this to 1.5 GB/core, while FAFE still performs the best at 0.9 GB/worker:

Default default

ExecAtPreFork exec

FAFE fafe

We can take a look to see if there is any milage in adopting such an approach in other jobs such as derivation production.

cc: @tsulaia @fwinkl @rbielski @jcatmore @gemmeren

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • mentioned in merge request !56517 (merged)

  • Alaettin Serhan Mete resolved all threads

    resolved all threads

  • Please register or sign in to reply
    Loading