Skip to content

DMC-1223 Uniform python2&3 support with auto selection

Petr Vokac requested to merge vokac/gfal2-util:auto-python-dmc-1223 into develop

Make gfal2_util python sources compatible with python 2.4 - 3.9 (unit tests should be compatible only with python 2.5 - 3.x) and provide shell wrapper that automatically selects available interpreter in this order:

  • executable from GFAL_PYTHONBIN environment variable
  • python3 from PATH if import gfal2 succeeds
  • python2 from PATH if import gfal2 succeeds
  • python from PATH if import gfal2 succeeds
  • /usr/bin/python

Basic functionality was tested and works in these environments:

  • ATLAS SLC5, python 2.4.3, gfal 2.9.3, gfal2-bindings 1.8.3 (no xroot test - no gfal plugin)
  • ATLAS SLC6, python 2.6.6, gfal 2.16.1, gfal2-bindings 1.9.5
  • ATLAS CentOS7, python 2.7.5, gfal 2.17.3, gfal2-bindings 1.9.5
  • EMI UMD 4 CentOS7, OS python 2.7.5, gfal 2.18.1, gfal2-bindings 1.9.5
  • EMI UMD 4 CentOS7, OS python 3.6.8, gfal 2.18.1, gfal2-bindings 1.9.5
  • EMI UMD 4 CentOS7, local python 3.7.9, gfal 2.18.1, gfal2-bindings 1.9.5
  • Normal SL6, python 2.6.6, gfal 2.18.1, gfal2-bindings 1.9.5
  • Normal CentOS7, python 2.7.5, gfal 2.18.1, gfal2-bindings 1.9.5
  • Normal CentOS7, python 3.6.8, gfal 2.18.1, gfal2-bindings 1.9.5
  • Normal CentOS8, python 3.6.8, gfal 2.18.1, gfal2-bindings 1.9.5
  • Local python on CentOS7, python 2.6.9, gfal 2.18.1, gfal2-bindings 1.9.5
  • Local python on CentOS7, python 3.4.10, gfal 2.18.1, gfal2-bindings 1.9.5
  • Local python on CentOS7, python 3.6.6, gfal 2.18.1, gfal2-bindings 1.9.5
  • Local python on CentOS7, python 3.7.9, gfal 2.18.1, gfal2-bindings 1.9.5
  • Local python on CentOS7, python 3.8.6, gfal 2.18.1, gfal2-bindings 1.9.5
  • Local python on CentOS7, python 3.9.0, gfal 2.18.1, gfal2-bindings 1.9.5

Functionality of gfal-* wrappers was tested with sh (SL5, SL6, CentOS7, CentOS8) and bash, dash, ksh, zsh on CentOS7.

If you try to repeat these tests be aware that ATLAS and EMI UMD environments use davix 0.7.5 which has known issues related to TPC, so failures are independent to these gfal2-util updates.

Be aware that python3 can fail with verbose logging gfal2.set_verbose(gfal2.verbose_level.trace) if you don't use davix with recent patch davix@3c48eabb and to provide more robust logging compatible with utf-8 there was also recently merged gfal2@140bcf5f update. All successful tests mentioned above used davix and gfal2 with these patches.

Edited by Petr Vokac

Merge request reports