Misleading "Network is unreachable" error
The error "Network is unreachable" can be misleading, as it does not necessarily indicate a problem with the network.
I have seen several failure modes which give rise to this error. For example, trying to access a file which has tape copies but no disk copies:
$ eos cp /eos/users/test/a2 /tmp/z
[eos-cp] path=/eos/users/test/a2 size=603
[eos-cp] going to copy 1 files and 603 B
error: [ERROR] Server responded with an error: [3014] Unable to open file /eos/users/test/a2; Network is unreachable
error: target file was not created!
#WARNING [eos-cp] copied 0/1 files and 0 B in 0.03 seconds with 0 B/s
Another recent example was caused by a hardware failure in a FST disk server:
Mar 23 09:18:25.900815 tpsrv235 cta-taped: LVL="Error" PID="29317" TID="15099" MSG="Exception while reading a file" thread="DiskRead" unitName="I1JB0405" vid="I07585" mountId="27" threadID="6" blockID="1" exceptionMessage="In XrootReadFile::XrootReadFile failed XrdCl::File::Open() on root://ctapps-eossrv01.cern.ch//eos/ctapps/preprod/576702c6-59de-4f36-b2b9-86dff7f6f4d1/9/test09008658?eos.lfn=fxid:512b27a [ERROR] Server responded with an error: [3014] Unable to open file /eos/ctapps/preprod/576702c6-59de-4f36-b2b9-86dff7f6f4d1/9/test09008658; Network is unreachable code:400 errNo:3014 status:1"