Skip to content

athena.py: implement native support for CA

Frank Winklmeier requested to merge fwinkl/athena:athena_ca into main

This MR is a refactor of the functionality of ThinCAWrapper.sh and CARunner.py into the main athena executable. After the initial shell-phase, the process remains in python. This is achieved using the runpy module to directly launch the CA script via the __main__ entry point.

This MR is the first step in making the --CA option obsolete.

User visible changes:

  • The interactive mode now requires an optional <stage> argument also for legacy athena. If this is directly followed by the job options, the arguments need to be terminated by --. E.g all of the following are equivalent:
> athena AthExHelloWorld/HelloWorldOptions.py -i
> athena -i -- AthExHelloWorld/HelloWorldOptions.py
> athena -i init AthExHelloWorld/HelloWorldOptions.py
  • If the user tries to run a CA script in legacy mode, the following will be printed:
AthenaCommon.Include.IncludeError: include file AthExHelloWorld/HelloWorldConfig.py can not be found
If the script is a ComponentAccumulator configuration, add a shebang (#!) to its first line or run with the --CA option.

Implementation details:

  • This is the last MR in a series to align the legacy and CA command line behavior: !69744 (merged) !69693 (merged) !69773 (merged) !70158 (merged) !70151 (merged) !70102 (merged) !70265 (merged) !70266 (merged) !70154 (merged) !70275 (merged) !70281 (merged)
  • Use the AthOptionsParser for both legacy athena and CA-based job/athena. If used within athena.py the athena-specific command line options are added.
  • The command line option groups have been re-arranged such that all options related to legacy athena, appear under "Legacy options".
  • The shell-based CLI-parsing is now restricted to what is necessary to setup the preloading.
  • Implement the file path resolution in Python using unixtools.FindFile. The "shebang" CA detection now also works for file paths that are resolved at runtime.
  • The CARunner.py use-case is replaced by directly loading the pkl file (legacy/CA pickles are automatically distinguished). I dropped the support of CA pkls without MainServicesCfg. Could be added again if really needed.
  • Migrate the --tracelevel feature into AthenaCommon.Debugging. We should probably unify this with the existing --trace option in a second iteration.

For the moment, we still support the --CA flag. But if the proper shebang is used, the following commands all work transparently without the need to use --CA:

athena.py AthExHelloWorld/HelloWorldOptions.py
athena.py --config-only legacy.pkl AthExHelloWorld/HelloWorldOptions.py
athena.py legacy.pkl

athena.py AthExHelloWorld/HelloWorldConfig.py
athena.py --config-only cfg.pkl AthExHelloWorld/HelloWorldConfig.py
athena.py cfg.pkl

cc @wlampl @tadej @ssnyder

Edited by Frank Winklmeier

Merge request reports