Skip to content

Add support for running in MT and config using properties

Julien Esseiva requested to merge juesseiv/athena-celeritas:celeritas-mt into atlassim-6635-cuda

A preliminary implementation supporting MT and using properties and flags to configure celeritas::SetupOptions.

The following transform config has been tested on Perlmutter and ran to completion.

export ATHENA_PROC_NUMBER=8
export ATHENA_CORE_NUMBER=$ATHENA_PROC_NUMBER
# Adjust input file
IN_FILE="PG_pi_plus_eta0p7.pool.root"
Sim_tf.py \
  --CA True \
  --simulator 'FullG4MT_QS' \
  --physicsList 'FTFP_BERT' \
  --multithreaded True \
  --perfmon none \
  --detectors 'Calo' \
  --conditionsTag 'OFLCOND-MC23-SDR-RUN3-01' \
  --postInclude 'PyJobTransforms.TransformUtils.UseFrontier' \
  --preInclude 'AtlasG4Tf:Campaigns.MC23SimulationSingleIoV,SimulationConfig.disablePhotonRussianRoulette,SimulationConfig.disableNeutronRussianRoulette,SimulationConfig.disableFrozenShowersFCalOnly' \
  --geometryVersion 'ATLAS-R3S-2021-03-02-00' \
  --inputEVNTFile "$IN_FILE" \
  --outputHITSFile '1kpi_plus_eta0p7_AtlasG4_Athena_Celeritas_gpu_Calo.HITS.pool.root' \
  --maxEvents '1000' \
  --skipEvents '0' \
  --randomSeed '10' \
  --preExec 'flags.Sim.OptionalUserActionList +=[\"G4UserActions.G4UserActionsConfig.GPUOffloadToolCfg\"];flags.GeoModel.EMECStandard=True;from SimulationConfig.SimEnums import CalibrationRun;flags.Sim.CalibrationRun=CalibrationRun.Off;flags.Sim.GPU.StackSize=8192;flags.Sim.GPU.HeapSize=512*1024*1024' \
  --postExec 'AtlasG4Tf:cfg.getService("StandardField").UseSoleCurrent=0.;cfg.getService("StandardField").UseToroCurrent=0.' \
  --imf False

@bmorgan

Merge request reports