Skip to content

AthenaMP(Tools)+ AthenaPoolCnvSvc+PyJobTransforms: Streamline toggling SharedWriter on through the job transform

Alaettin Serhan Mete requested to merge amete/athena:master-streamline-sharedwriter-toggles into master

This MR contains a few changes that'll allow us to consistently toggle SharedWriter on through the job transform. Now when the user asks for --sharedWriter True we actually pass this along to the AthenaMPFlags. There is now a new option, --parallelCompression, which if set to True (default) it'll toggle on the "new" SharedWriter (which only summarizes the MetaData but event data is compressed by the workers instead). With these changes, one no longer needs to pass a postExec. The "old" (i.e. Run-2) SharedWriter behavior can be recovered by setting --parallelCompression False.

Let me mark this as draft until @gemmeren gives me the green light, and let me cc @elmsheus since I think he'd be interested in this.

Edit: MR is un-drafted after resolving the open threads.

Edited by Alaettin Serhan Mete

Merge request reports