Avoid a race in DIRAC which causes job creation to fail
Testing productions can fail with this surprising stack trace:
File "/miniconda/envs/analysis-productions/lib/python3.8/site-packages/dirac_prod/classes/input_dataset.py", line 32, in replicas
for se, pfn in dw(Dirac().getReplicas([str(self)], active=True))[str(self)].items():
File "/miniconda/envs/analysis-productions/lib/python3.8/site-packages/DIRAC/Interfaces/API/Dirac.py", line 858, in getReplicas
repsResult = dm.getReplicas(lfns, active=active, preferDisk=preferDisk, diskOnly=diskOnly)
File "/miniconda/envs/analysis-productions/lib/python3.8/site-packages/DIRAC/DataManagementSystem/Client/DataManager.py", line 1742, in getReplicas
succPfn = seObj.getURL(se_lfn[se],
File "/miniconda/envs/analysis-productions/lib/python3.8/site-packages/DIRAC/Resources/Storage/StorageElement.py", line 969, in getURL
result = self.__executeMethod(lfn, protocols=protocols)
File "/miniconda/envs/analysis-productions/lib/python3.8/site-packages/DIRAC/Resources/Storage/StorageElement.py", line 1240, in __executeMethod
storageParameters = storage.getParameters()
File "/miniconda/envs/analysis-productions/lib/python3.8/site-packages/DIRAC/Resources/Storage/StorageBase.py", line 119, in getParameters
parameterDict['URLBase'] = self.getURLBase().get('Value', '')
File "/miniconda/envs/analysis-productions/lib/python3.8/site-packages/DIRAC/Resources/Storage/GFAL2_XROOTStorage.py", line 109, in getURLBase
return self.__addDoubleSlash(super(GFAL2_XROOTStorage, self).getURLBase(withWSUrl=withWSUrl))
TypeError: super(type, obj): obj must be an instance or subtype of type
It turns out this is due to DIRAC importing the storage class twice. To reproduce it more simply, create a file named reproducer.py
containing:
class A:
def example(self):
pass
class B(A):
def example(self):
super(B, self).example()
Then run
import imp
import reproducer
obj1 = reproducer.B()
obj1.example()
reproducer = imp.load_module("reproducer", *imp.find_module("reproducer"))
obj2 = reproducer.B()
obj2.example()
obj1.example()
This results in:
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-1-c14935bd165d> in <module>
9
10 obj2.example()
---> 11 obj1.example()
~/Development/DPA/LbAnalysisProductions/reproducer.py in example(self)
5 class B(A):
6 def example(self):
----> 7 super(B, self).example()
TypeError: super(type, obj): obj must be an instance or subtype of type
It turns out I already fixed this for DIRAC 7.3 as part of other refactoring.
Edited by Chris Burr