Skip to content
Snippets Groups Projects

Implementation of layered LCG software stacks

Merged Pere Mato Vila requested to merge SPI-1401 into master
3 files
+ 114
1
Compare changes
  • Side-by-side
  • Inline
Files
3
# How to create layered LCG software stacks
## Installation of of the `lcgcmake` command
1. Clone the lcgcmake repository and add the `lcgcmake/bin` into the PATH:
```
git clone [--branch SPI-1401] https://gitlab.cern.ch/sft/lcgcmake.git
export PATH=$PWD/lcgcmake/bin:$PATH
```
## Create a new toolchain file based on an existing toolchain or set of toolchains files
1. Edit a new file in `lcgcmake/cmake/toolchain` or in the build directory or anywhere else. The file should called heptools-<version>.cmake, where <version> will be the argument provided at configuration time.
2. An example of the contents of layered stack:
```
#---Base release---------------------------------------------------
include(heptools-97)
#---Define the top level packages for this stack-------------------
LCG_top_packages(DD4hep CMake ninja)
#---Removing unwanted [optional] packages--------------------------
LCG_remove_package(R)
LCG_remove_package(rpy2)
#---Overwrites-----------------------------------------------------
LCG_external_package(ROOT 6.18.00)
```
In this example we take an existing release (LCG_97), replace the version of ROOT, select a sub-set of packages and their dependencies, and remove optional packages. Note that this later posibility is only existing for packages that have been declared as optional.
## Build the layered stack
1. You need first to configure the new LCG stack in a temporary (build) area.
```
mkdir build
cd build
lcgcmake config --version <version> --prefix <install-area> --compiler <compiler>
```
- you can see the available compilers with the command `lcgcmake show compilers`
- similarly, you can see the available LCG stack versions with the command `lcgcmake show versions`
- once you have configured you can inspect the configuration with the command `lcgcmake show configuration`
- you can see other available configuration options with `lcgcmake config --help`
2. Building the full stack is done using the sub-command `install`. If no target is specified then the top level packages, declared with `LCG_top_packages`, and their dependencies will be downloaded if the binary already exists for this exact platform and exact list of version of the dependencies or build locally.
```
lcgcmake install
```
3. Running with this new stack can be done by sourcing generated view or by using the `run` sub-command.
```
lcgcmake run
```
The new stack can be tested by running some local tests.
## Deploying the layered stack
1. You need to configure with the option `--with-tarfiles` to create binary tarfiles for each locally build package.
```
lcgcmake config --with-tarfiles
```
They are deposited in the `./tarfiles` directory after a successful `lcgcmake install`
2. Copying or uploading the binary tarfiles for later deployment in CVMFS or else where
```
lcgcmake upload tarfiles
```
For this to work you will need to have valid credentials (password) for copying into `/eos/project/l/lcg/www/lcgpackages/tarFiles`
3. Install the new stack into CVMFS.
- Login to the CVMFS publishing node and open a transaction
- On a temporary area do an installation of the stack using the CVMFS as install prefix.
```
lcgcmake config --version <version> \
--target_platform <platform> \
--prefix /cvmfs/sft.cern.ch/lcg/releases \
[ -o LCG_ADDITIONAL_REPOS=http://... ] # eventually for additional binary tarfiles repositories
lcgcmake install
```
Note that no package should be build locally, only tarfiles should be downloaded and expanded.
- The result of the installation will be in `/cvmfs/sft.cern.ch/lcg/releases/<version>`
- Finalize the CVMFS transaction
## Building a layered stack for the development of one or more packages
Additional functionality has been added to the `lcgcmake` tool for supporting the use case of developing a package within the stack. In this case the user should be able to indicate the location of the new sources for the package being developed instead of using the official source tarfiles. This is done by either providing a git repository or by the full path to a source directory. The toolchain file can be modified as follows:
```
#---Base release---------------------------------------------------
include(heptools-dev-base)
#---Define the top level packages for this stack-------------------
LCG_top_packages(DD4hep CMake ninja)
#---Overwrites-----------------------------------------------------
LCG_external_package(ROOT master SRC=/data/sftnight/lcgsoft/root)
LCG_external_package(DD4hep mytag GIT=https://github.com/AIDASoft/DD4hep.git)
```
In this example we have taken ROOT and DD4hep but it can be any other package or set of packages. You need to base the development on a give full release, then you overwrite the desired package[s] pointing to your [forked] repository or local checkout area. The version is used a 'tag' for the GIT repository and is ignored for the local checkout area.
The configuration and installation commands are the same as previously. The only difference is to enable to option `--build-always` to force the building of all locally buit packages. This is to ensure that changes in the dependent packages are visible during the build of the full stack.
```
lcgcmake config --version <version> \
--compiler gcc8 \
--prefix <install prefix> \
--toolchain <modified toolchain file> \
--build-always
lcgcmake install
```
Note that you can specify the modified toolchain file with the option `--toolchain`.
Loading