Draft: Prototype of hybrid MP/MT setup
Prototype of hybrid MP/MT setup. The main addition is a new --threadsPerProcess argument that should be set together with --multithreaded.
This MR also merges MT and MP handling to avoid too much duplicated code and potential inconsistencies. That's why I'd get this into main first and if we need it, we can backport to 24.0.
The main inconsistency that no longer works is using --multithreaded and --athenaopts='--nprocs=N' together successfully. Now always if --nprocs or --threads is used, the --multithreaded and --multiprocess flags are not used.
Also I fixed when all was passed after some substep arguments. While this would probably never happen in production it may avoid confusion when running locally.
I also extended the tests significantly.
TODO:
- Better handle disabled MT or MP.
- Error when
--threadsPerProcessis used without--multithreaded. - Error when
--multithreadedis used with--nprocsand the other way around. - Tests for concurrent events setting.
This MR may again be too ambitious but I did not like just hacking the MT parsing.
Tagging @amete, @fwinkl, @tsulaia, @akraszna, @jchapman, @jcatmore, @elmsheus.