ATR-28398: Fix nested containers for transform substeps

Another follow up of !66655 (merged).
Cover case of a transform substep running within a container, when the transform itself runs within a container on the Grid. Up to now, nested containers worked locally and in CI, but not on the Grid, due to differences in the job setup. Thanks to @desilva, @walkerr, @aforti for helping understanding better the details of the container jobs setup. Tagging @tadej, @jcatmore, @jchapman, @nstyles.

Tested with the following workflows locally:

1. el9 node: RDOtoRDOTrigger invokes centos7 container
2. centos7 container on el9 node: RDOtoRDOTrigger does not invoke new container
3. el9 container on el9 node: RDOtoRDOTrigger invokes centos7 container in a way that should also work on the Grid
   The three above were also tested in the multicore mode.

This merge request affects 1 package:

• Tools/PyJobTransforms

This merge request affects 1 file:

• Tools/PyJobTransforms/python/trfExe.py

added Tools main review-pending-level-1 labels

WARNING: big files (>100K) are found in the changeset

116K in file Tools/PyJobTransforms/python/trfExe.py

CI Result FAILURE (hash 07e47893)

	Athena	AthSimulation	AthGeneration
externals
cmake
make
tests

Full details available on this CI monitor view. Check the JIRA CI status board for known problems
Athena: number of compilation errors 0, warnings 0
AthSimulation: number of compilation errors 0, warnings 0
AthGeneration: number of compilation errors 0, warnings 0
For experts only: Jenkins output [CI-MERGE-REQUEST-EL9 1543]

Hi @okuprash. There are several failed tests here. Can these be looked into?

-- James for L1

added review-user-action-required label and removed review-pending-level-1 label

Hi @jbeacham, I will take a look. Will mark as a draft for the time being.

marked this merge request as draft

added 1 commit

• 967df44a - member variables for the top class

Compare with previous version

marked this merge request as ready

Jenkins please retry a build

This merge request affects 1 package:

• Tools/PyJobTransforms

This merge request affects 1 file:

• Tools/PyJobTransforms/python/trfExe.py

added review-pending-level-1 label and removed review-user-action-required label

WARNING: big files (>100K) are found in the changeset

116K in file Tools/PyJobTransforms/python/trfExe.py

CI Result SUCCESS (hash 967df44a)

	Athena	AthSimulation	AthGeneration
externals
cmake
make
tests

Full details available on this CI monitor view. Check the JIRA CI status board for known problems
Athena: number of compilation errors 0, warnings 0
AthSimulation: number of compilation errors 0, warnings 0
AthGeneration: number of compilation errors 0, warnings 0
For experts only: Jenkins output [CI-MERGE-REQUEST-EL9 1574]

The changes look okay and there are no issues with the CI. Approving from L1.

added review-approved label and removed review-pending-level-1 label

There were some comments on the ticket, can you confirm this should still be merged in this state?

added review-user-action-required label

marked this merge request as draft

added 1 commit

• e66cdc99 - Launch container with setupATLAS rather than startContainer.sh

Compare with previous version

marked this merge request as ready

Jenkins please retry a build

This merge request affects 1 package:

• Tools/PyJobTransforms

This merge request affects 1 file:

• Tools/PyJobTransforms/python/trfExe.py

added review-pending-level-1 label and removed review-approved review-user-action-required labels

WARNING: big files (>100K) are found in the changeset

116K in file Tools/PyJobTransforms/python/trfExe.py

CI Result SUCCESS (hash e66cdc99)

	Athena	AthSimulation	AthGeneration
externals
cmake
make
tests

Full details available on this CI monitor view. Check the JIRA CI status board for known problems
Athena: number of compilation errors 0, warnings 0
AthSimulation: number of compilation errors 0, warnings 0
AthGeneration: number of compilation errors 0, warnings 0
For experts only: Jenkins output [CI-MERGE-REQUEST-EL9 1625]

resolved all threads

All threads are resolved and changes look good to me, approving.

Cheers, Philipp (L1)

added review-approved label and removed review-pending-level-1 label

mentioned in commit 93e3e51e

merged

mentioned in merge request !67391 (merged)