Skip to content

Migrating JobTransforms modules to Python 3 with futurize as a first step

Pascal Rene Baehr requested to merge pbahr/athena:master-jobtransforms-py2to3 into master

To improve the visibility of the functional changes in !22401 (closed) where all changes were squashed into one commit, a new topic branch will help make everything more clear and help to identify possible errors.

First various changes concerning type checks and a safer usage of comparison operators are made for all modules:

  • Generators/EvgenJobTransforms
  • PhysicsAnalysis/PATJobTransforms
  • Reconstruction/RecJobTransforms
  • Simulation/SimuJobTransforms
  • Tools/PyJobTransforms

After applying these changes running futurize will ensure the compatibility for both Python 2/3.

Progress:

  1. PyJobTransforms fully migrated + locally tested
  2. SimuJobTransforms fully migrated + locally tested
  3. RecJobTransforms fully migrated + locally tested
  4. PATJobTransforms fully migrated + locally tested
  5. EvgenJobTransforms fully migrated + locally tested

Tests:

  • Validated the changes with the JobTransforms unit tests (Tools/PyJobTransforms/test/runAllUnitTests.sh)
  • Checked the build with the manual Tier0 test (RunTier0Tests.py)
  • Checked against previously failed tests from the CI build:
    • Reco_tf.py --AMI q440 --maxEvents=5
    • Reco_tf.py --AMIConfig d1498 --inputHITSFile /cvmfs/atlas-nightlies.cern.ch/repo/data/data-art/OverlayMonitoringRTT/mc16_13TeV.424000.ParticleGun_single_mu_Pt100.simul.HITS.e3580_s3126/HITS.11330296._000376.pool.root.1 --inputRDO_BKGFile /cvmfs/atlas-nightlies.cern.ch/repo/data/data-art/OverlayMonitoringRTT/PileupPremixing/22.0/v1/RDO.merged-pileup.100events.pool.root --outputRDOFile myRDO.pool.root --imf False --athenaopts=" --pmon=sdmonfp" --maxEvents=10
    • Sim_tf.py --AMIConfig s3126 --inputEVNTFile /cvmfs/atlas-nightlies.cern.ch/repo/data/data-art/SimCoreTests/ttbar_muplusjets-pythia6-7000.evgen.pool.root --outputHITSFile myHITS.pool.root --postInclude 'EVNTtoHITS:PyJobTransforms/UseFrontier.py' --imf False --maxEvents=10

Result: AthSimulation doesn't seem to support the necessary libraries for a futurize migration (yet).

Edited by Pascal Rene Baehr

Merge request reports