-
Geoffray Adde authoredto allow ipc communication - move backend map to shared memory using boost to allow sharing among the forked processes - add support for autodiscovery (XROOTD and EOS specifics) - move to version 0.5Geoffray Adde authoredto allow ipc communication - move backend map to shared memory using boost to allow sharing among the forked processes - add support for autodiscovery (XROOTD and EOS specifics) - move to version 0.5
Code owners
Assign users and groups as approvers for specific file changes. Learn more.
README 8.25 KiB
OVERVIEW
--------
This is an XRootD DSI plugin for gridftp.
LICENSE
-------
Copyright (C) 2013 CERN/Switzerland
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
FEATURES
--------
So far, the supported operations are :
mkdir, rmdir, delete, rename, cksm (checksum), chmod
Any other operation is not implemented and will result in an error.
Some eos specifics can be enabled.
More generally, the configuration of the plugin is
written in the file /etc/sysconfig/xrootd-gridftp-server.
Three server setups are supported.
- standalone :
In this setup the gridftp server handles requests and serve the data transfers.
It works as a gateway to the underlying XROOTD filesystem.
All the traffic flows through the server.
This is the LEGACY mode.
- frontend/backend : in this setup, requests are handled by (one or several) "frontend" servers,
Transfers are served by the "backend" servers.
A same server can be simultaneously frontend and backend (also refered as "split").
A backend node can serve transfer requests from several frontend nodes.
In this mode, the frontend will try to make the transfer happen on a backend server.
( This feature only works in delayed passive connection which requires "-dp" to be set
when using globus-url-copy )
which hosts a local replica of the file avoiding intra-instance traffic.
In this mode, each frontend node nead to know which are the available backend nodes.
There is two way to specify the backend servers (they can be used both):
* a static list of servers in the config file.
It's specified by setting the env variable XROOTD_DSI_GSIFTP_BACKENDSERVERS
If the autodiscovery is not enabled, this is the only servers which will be used.
If the autodiscovery is enabled, this is the startup list which will (probably) grow
* dynamic autodiscovery of available gridftp backend service on accessed XRootD sevrers.
This feature is enable if the env variable XROOTD_DSI_GSIFTP_BACKENDDISCOVERTTL is set and >0
It needs at least one machine in the previous static list to start with.
When enabled, all the XROOTD servers mentionned in any transfer as hosting a replica are probed
to see if they also host a gridftp sevrer on port XROOTD_DSI_GSIFTP_BACKENDDEFAULTPORT.
If they do, they are added to the backend servers list.
Note that this process should work with any XROOTD server.
When using an EOS server on which the gridftp frontend servers are registered as gateways,
it is possible to speed-up the discovery process by setting XROOTD_DSI_EOS_NODELS.
IMPORTANT INFORMATION:
When running in frontend/backend mode, the backend information is stored in a shared memory map.
This map will disappear if the machine reboot or if a "service xroot-gridftp-server restart resetbackend"
is issued. Please note, the additionnal argument. In other cases, this information is not erased.
Possible setups:
- one (or a few) gridftp servers might run in standalone mode.
Theses servers are then only gateways and the data is thransfered twice.
This completely isolate the gridftp entry points and narrows-down open ports and machines
accessible from the outside.
- all the XROOTD storage servers might run in split mode.
The transfers are then optimal and data is transfered only once (at least for replica layout).
This flattens out the transfer load accross all the servers (if the requested file evenely spread)
All the XROOTD storage servers potentially need to be exposed to the outside which might be a serious
security concern.
- one (or a few) gridftp servers might run in frontend mode while all the XROOTD storage servers run in
backend mode.
This allows to combine optimal transfers and limited risk exposure.
For more information just search "globus gridftp server Split Process"
the testing for standalone is based on uberftp:
warning!! the test has been reported working with uberftp 2.6 but NOT working with uberftp 2.4
(take care that 2.4 is the version currently installed on slc5)
OS
--
In standalone mode, this plugin has been successfully tested
- on SLC6 with eos specifics
- on SLC6 with a pure XRootD server
- on SLC5 with a pure XRootD server
In frontend/backend mode, this plugin has been successfully tested
- on an SLC5/SLC6 mix with eos specifics
Because of CMake, it sould be possible to compile it without any significant change on a wide variety of OSes
provided with gcc > 4.4 or an equivalent compiler.
EXTERNAL DEPENDENCIES
---------------------
This plugin has been developped and tested with globus-gridftp-server-devel 7.25 and XRootD 4.1.2
To run this plugin, the following packages are required:
- xrootd-cl
- globus-gridftp-server-progs
To Build this plugin, the following packages are required:
- globus-gridftp-server-progs
- globus-gridftp-server-devel
- boost (boost148 on SLC5)
- xrootd-cl-devel
- gcc > 4.4
- perl
Minimum versions for these packages are not known.
The testing script requires uberftp >= 2.6
INSTALL
-------
Create an empty build directory anywhere then run cmake. It should look like
> mkdir -p /path/to/build/dir
> cd /path/to/build/dir
> cmake /path/to/cmake.install/.
Then there are several possible targets for make : all(default), clean, install, uninstall, dist, srpm, rpm
For instance, to build the binary rpm, you should type
> cd /path/to/build/dir
> make rpm
FILE TREE
---------
- src/ : source code
- cmake/ : cmake helpers for the build process
- etc/ : config and service files
- doxygen/ : the directory to build the doxy documentation
- test/ : a testing script to unitary test each feature and to stress test some of them
- CMakeLists.txt : the CMake source file to make the build process
- xroot-dsi.spec : the spec file to build the rpm
- README : the current file
DOCUMENTATION
-------------
The documentation is split out into several files, namely :
- the current file for some general information
- the doxy documentation (to build it, just go to the doxygen/ subdir and run doxygen)
- comments in the source code for code related informations
- the files in the subdir etc/, especially etc/sysconfig/xrootd-gridftp-server
which should be meaningful for the configuration of the plugin and the eos specifics
KNOWN BUGS AND LIMITATIONS
--------------------------
- because the Xrd Client is asynchronous, the code of the plugin is multithreaded.
then, the threading should be turned on in the gridftp server (see the provided init script)
- even if it has an offset argument, it seems that globus_gridftp_server_register_write()
cannot be called out-of-order. The implementation of the plugin for send op is asynchronous and
a simple mechanism has been implemented to turn it synchronous. If this limitation disappears,
the plugin can be turned asynchronous but it requires to recompile. It's on purpose.
- WARNING: the code dealing with frontend/back configuration uses stripped-out internal globusstructures
It is a dirty hack but it's the only way as globus API to DSI does not provide enough information for
this implementation.
The buils system performs this extraction automatically with the right version of the globus source code.
If it fails, the build fails.
TODO
----
- Striped read from multipe servers could probably be implemented
- Gsi Authentication Forwarding
CONTACT
-------
geoffray.adde@cern.ch