Correct way to force logSE-EOS to use xRootD and not HTTPS
Hi,
for the tool DownloadAndBuildStats used by all MC liaisons to get the statistics following line is called.
Previously there was no issue but now one receives warnings/errors of the form:
[Wed, 01.05.2024 @ 10:05:25] gfal2 - WARN: (SEToken) Could not retrieve any token for https://lhcb-dirac-logse.web.cern.ch:443/lhcb/MC/Dev/LOG/00214189
[Wed, 01.05.2024 @ 10:05:26] gfal2 - WARN: (SEToken) Could not retrieve any token for https://lhcb-dirac-logse.web.cern.ch:443/lhcb/MC/Dev/LOG/00214189/0000
2024-05-01 10:05:52 UTC None/GFAL2_HTTPSStorage/GFAL2_StorageBase._getSingleFile INFO: Trying to download https://lhcb-dirac-logse.web.cern.ch:443/lhcb/MC/Dev/LOG/00214189/0000/00000001.zip t
o /interactive_storage/jwagner/MC_stats/GenLogs-214189/00000001.zip
[Wed, 01.05.2024 @ 10:05:52] gfal2 - WARN: (SEToken) Could not retrieve any token for https://lhcb-dirac-logse.web.cern.ch:443/lhcb/MC/Dev/LOG/00214189/0000/00000001.zip
[Wed, 01.05.2024 @ 10:05:53] gfal2 - WARN: (SEToken) Could not retrieve any token for https://lhcb-dirac-logse.web.cern.ch:443/lhcb/MC/Dev/LOG/00214189/0000/00000001.zip
2024-05-01 10:05:53 UTC None/DIRAC.Resources.Storage.GFAL2_XROOTStorage/CERN-EOS/GFAL2_StorageBase._getSingleFile INFO: Trying to download root://x509up_u5202@eoslhcb.cern.ch//eos/lhcb/grid/p
rod/lhcb/logSE/lhcb/MC/Dev/LOG/00214189/0000/00000001.zip to /interactive_storage/jwagner/MC_stats/GenLogs-214189/00000001.zip
In the end I received the results I wanted because after the HTTPS
did not work it used XRootD
.
Still, I received a mail informing me that there were a huge amount of failed traffics:
Of course I want to avoid this so my question is how to best force to go to XRootD
directly.
@cburr showed here a way to do it in the second example
In [1]: import DIRAC; DIRAC.initialize()
...: from DIRAC.Resources.Storage.StorageElement import StorageElement
In [2]: se = StorageElement("LogSE-EOS")
...: se.getFile("/lhcb/MC/Dev/LOG/00214189/0000/00000001.zip", "/tmp/0001.zip")
2024-05-01 12:41:16 UTC Framework/GFAL2_HTTPSStorage/GFAL2_StorageBase._getSingleFile INFO: Trying to download https://lhcb-dirac-logse.web.cern.ch:443/lhcb/MC/Dev/LOG/00214189/0000/00000001.zip to /tmp/0001.zip/00000001.zip
2024-05-01 12:41:16 UTC Framework/DIRAC.Resources.Storage.GFAL2_XROOTStorage/CERN-EOS/GFAL2_StorageBase._getSingleFile INFO: Trying to download root://x509up_u1000@eoslhcb.cern.ch//eos/lhcb/grid/prod/lhcb/logSE/lhcb/MC/Dev/LOG/00214189/0000/00000001.zip to /tmp/0001.zip/00000001.zip
Out[2]:
{'OK': True,
'Value': {'Failed': {},
'Successful': {'/lhcb/MC/Dev/LOG/00214189/0000/00000001.zip': 144050}}}
In [3]: se = StorageElement("LogSE-EOS", protocolSections=["GFAL2_XROOT"])
...: se.getFile("/lhcb/MC/Dev/LOG/00214189/0000/00000001.zip", "/tmp/0001.zip")
2024-05-01 12:41:25 UTC Framework/DIRAC.Resources.Storage.GFAL2_XROOTStorage/CERN-EOS/GFAL2_StorageBase._getSingleFile INFO: Trying to download root://x509up_u1000@eoslhcb.cern.ch//eos/lhcb/grid/prod/lhcb/logSE/lhcb/MC/Dev/LOG/00214189/0000/00000001.zip to /tmp/0001.zip/00000001.zip
Out[3]:
{'OK': True,
'Value': {'Failed': {},
'Successful': {'/lhcb/MC/Dev/LOG/00214189/0000/00000001.zip': 144050}}}
But he is not sure if this is the right way to force XRootD
. It would be great if you could provide information on the correct way to force this.
Thanks a lot!