shared filesystem detection and redirection
I did some more testing of the feature described here and discussed here.
With the feature disabled, I tried XRD_LOGLEVEL=Dump xrdcp -f root://eos-mgm-0.wlcg-dev.uvic.ca//eos/wlcg-dev.uvic.ca/data/atlas/atlasscratchdisk/rptaylor/testdownloads/DAOD_EXOT27.23317046._000092.pool.root.1 /dev/null
and saw redirection to a FST server as expected:
[2024-12-19 22:26:04.961765 +0000][Dump ][XRootD ] [eos-mgm-0.wlcg-dev.uvic.ca:1094] Got kXR_redirect response to message kXR_open (file: /eos/wlcg-dev.uvic.ca/data/atlas/atlasscratchdisk/rptaylor/testdownloads/DAOD_EXOT27.23317046._000092.pool.root.1, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ): eos-fst-0.wlcg-dev.uvic.ca?&cap.sym=...
Then I did eos space config default space.policy.localredirect=1
and on a client with cephfs mounted, it redirected to the local file and worked successfully:
[2024-12-19 22:23:02.126893 +0000][Dump ][XRootD ] [eos-mgm-0.wlcg-dev.uvic.ca:1094] Got kXR_redirect response to message kXR_open (file: /eos/wlcg-dev.uvic.ca/data/atlas/atlasscratchdisk/rptaylor/testdownloads/DAOD_EXOT27.23317046._000092.pool.root.1, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ): file://localhost/eos-storage/eos-data/eos-fst-0/00000000/00000122, port -1
However on a client without the mounted cephfs, the same command failed with exit code 54. Full log: xrd.log
[2024-12-19 22:47:00.367729 +0000][Error ][File ] Open MkdirPath failed /eos-storage/eos-data/eos-fst-0/00000000/00000122: permission denied
[2024-12-19 22:47:00.367737 +0000][Debug ][XRootD ] [localhost] Handling error while processing kXR_open (file: /eos-storage/eos-data/eos-fst-0/00000000/00000122, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ): [ERROR] Local error: permission denied.
So I wonder how the detection of the shared filesystem works, to avoid redirecting when it is not available. I also tried making /eos-storage an empty writable directory, it slightly changed the error:
[2024-12-19 22:58:21.456166 +0000][Error ][File ] Open: open failed: /eos-storage/eos-data/eos-fst-0/00000000/00000122: no such file or directory
[2024-12-19 22:58:21.456187 +0000][Debug ][XRootD ] [localhost] Handling error while processing kXR_open (file: /eos-storage/eos-data/eos-fst-0/00000000/00000122
, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ): [ERROR] Local error: no such file or directory.
but still had the same overall result.