Introduction

LCGCMake is the EP-SFT infrastructure to build the software stack for the LHC experiments containing both external software dependencies and projects developed within our group. The tool is based on the ExternalProject module of CMake.

Preparation

LCGCMake requires CMake installed in your system. Check the pre-requisites section for the complete list of requirements for each system.

On CVMFS (CERN) set PATH to use one of latest CMake versions (system default is 2.6 and even 3.6.X are too old)

ARCH=$(uname -m)
export PATH=/cvmfs/sft.cern.ch/lcg/contrib/CMake/3.14.2/Linux-${ARCH}/bin:${PATH}

Installation and very Quick Start

1. Install lcgcmake by just cloning the lcgcmake package from CERN GitLab and ensure that the PATH environment variable contains the the bin directory from the just cloned repository.

   git clone https://gitlab.cern.ch/sft/lcgcmake.git
   export PATH=$PWD/lcgcmake/bin:$PATH

2. Configure the software stack by selecting the compiler, the version of the stack to be used, and the installation prefix

   lcgcmake configure [--compiler=gcc62] [--version=hsf] [--prefix=...]

   • you can see the available compilers with the command lcgcmake show compilers
   • similarly, you can see the available LCG stack versions with the command lcgcmake show versions
   • Once you have configured you can inspect the configuration with the command lcgcmake show configuration
   • you can see other available configuration options with lcgcmake config --help

3. Installation of the required packages

   lcgcmake install <list of package names>

   • you can see the list of packages (targets) with lcgcmake show targets

4. Setup environment and run on a new shell with

   lcgcmake run

Low level interface

This interface (legacy) is needed only if you do not find the option you want to the command lcgcmake. The advantage of the high level interface is that it remembers your configuration, it hides the options required in the cmake step, it chains several commands together, etc. But sometimes you want to do special configurations or actions for which the lcgcmake command has not been coded, in this case you should use the low level interface described in the following lines.

1. Checkout the lcgcmake package from lcgsoft GIT repository:

git clone https://gitlab.cern.ch/sft/lcgcmake.git

Notice: to build a package from release LCG_XX, append '-b LCG_XX' to the command to checkout the LCG_XX branch.

2. Create a workspace area on which to perform the builds

mkdir lcgcmake-build
cd lcgcmake-build

3. You may need at this moment to define the compiler to use if different from the native compiler

source /cvmfs/sft.cern.ch/lcg/contrib/gcc/<version>/platform/setup.(c)sh

4. Configure the build of all externals with cmake

cmake -DCMAKE_INSTALL_PREFIX=../lcgcmake-install ../lcgcmake

5. In order to build against an existing external repository use the option

-DLCG_INSTALL_PREFIX=/cvmfs/sft.cern.ch/lcg/releases

to tell the system to look for packages in the LCG area. This is much faster since it will be an incremental build. Several LCG installation areas can be searched by concatenating several areas separated with ';'. Other available options are:

• -DLCG_VERSION=XX to select a given LCG configuration version. The XX is the key of one the files in lcgcmake/cmake/toolchain (e.g. latest, 86beta, experimental)
• -DLCG_IGNORE='package1;package2;...' to ignore packages that are already existing in LCG area and force a re-build.
• -DLCG_ADDITIONAL_REPOS='http://...;http://...' to add extra web repositories with binary tarfiles.
• -DCMAKE_BUILD_TYPE=Debug for a debug build (default is optimized).

6. Build and install all external packages

make -jN

Or to build a single external package (make help shows all the possible targets)

make -jN <package> 

You may need to restart de build of a package from beginning in case of obscure errors. The best is to clean a specific package

make clean-<package>

7. Building a view and sourcing the setup.sh of the view to set an environment to test the build

make view
source ../lcgcmake-install/<platform-tag>/setup.[c]sh

Technical details

To add or modify existing packages, the user may edit the build recipes in the file <package-group>/CMakeLists.txt and the actual versions of the package in files cmake/toolchain/heptools-XX.cmake.

The LCG_Package_Add command is the basic command to encapsulate the instructions on how to build a package. This is commands wraps the ExternalProject_Add() command form ExternalProject module with additional options as well as pre-configuring a number of standard actions
(e.g. installing the log files and sources in /share). A fairly complete example is the following one:

LCGPackage_Add(
  pythia8
  URL http://lcgpackages.web.cern.ch/lcgpackages/tarFiles/sources/MCGeneratorsTarFiles/pythia8<NATIVE_VERSION>.tgz
  CONFIGURE_COMMAND ./configure --prefix=<INSTALL_DIR>
                                --with-hepmc=${HepMC_home}
                                --enable-shared
  BUILD_IN_SOURCE 1
  CONFIGURE_EXAMPLES_COMMAND <SOURCE_DIR>/examples/configure
                             --with-pythia8=<INSTALL_DIR>
                             --with-lhapdf=${lhapdf_home}
  BUILD_EXAMPLES_COMMAND make -C <SOURCE_DIR>/examples main01 main03
  INSTALL_EXAMPLES_COMMAND ${CMAKE_COMMAND} -E make_directory <INSTALL_DIR>/bin
                   COMMAND ${CMAKE_COMMAND} -E chdir <SOURCE_DIR>/examples
                           cp bin/main01.exe bin/main03.exe main03.cmnd <INSTALL_DIR>/bin
  TEST_COMMAND ${CMAKE_COMMAND} -E chdir <INSTALL_DIR>/bin ./main01.exe
  DEPENDS HepMC lhapdf
)

The parameters CONFIGURE_EXAMPLES_COMMAND, BUILD_EXAMPLES_COMMAND, INSTALL_EXAMPLES_COMMAND are additional parameters used by LCGPackage_Add to create new steps (configure_examples, build_examples, install_examples). They can be extended with additional commands with the keyword COMMAND.

The variables of the form are evaluated internally with some delay. This allows to use internally defined variables for a multi-versioned package.

The sources are usually taken from source tarballs locally stored at /eos/project/l/lcg/www/lcgpackages/tarFiles and made available via http://lcgpackages.web.cern.ch/lcgpackages/tarFiles/sources/

Configuration Parameters

A number of configuration options are available for the user to customize the LCGCmake build.

LCGPackage_Add Arguments

This is the full list of arguments available for writing the recipes using the function LCGPackage_Add(). Some arguments are interpreted by LCGPackage_Add itself others are passed directly to the underlying ExternalProject_Add() function from the ExternalProject module of CMake.

Multi-versioned packages

It is possible to build several versions of the same package at the build time. The number of versions and the parameters controlling their configuration are listed in the heptools-XX.cmake file in the toolchain directory. The build instructions encoded in the LCGPackage_Add() command need to be coded with the variable as parameter, which will be replaced to each concrete version in the list of versions when processing the CMakeLists.txt file. Additional parameters controlling the author version, other dependent package version, etc. can be added as extra parameters in the LCG_external_package() command in the toolchain file. These parameter values appear in the CMakeLists.txt file as variables of the form: (package)_(version)_(parameter_name). For example:

LCG_external_package(herwig++ 2.6.3 ${MCGENPATH}/herwig++ thepeg=1.8.3)

the value for the parameter thepeg can be accessed as ${herwig++_2.6.3_thepeg}

Here is a more complete example:

// In heptools-XX.cmake --------------------------------
LCG_external_package(herwig++ 2.6.1b ${MCGENPATH}/herwig++ thepeg=1.8.1)
LCG_external_package(herwig++ 2.6.3 ${MCGENPATH}/herwig++ thepeg=1.8.3)

//In CMakeLists.txt-----------------------------------------
LCGPackage_Add(
  herwig++
  URL http://cern.ch/service-spi/external/.../Herwig++-<VERSION>.tar.bz2
  CONFIGURE_COMMAND ./configure --prefix=<INSTALL_DIR>
                                --with-gsl=${GSL_home}
                                --with-thepeg=<<herwig++_<VERSION>_thepeg>_home>
                                --with-fastjet=${fastjet_home}
                                --with-boost=${Boost_home}
  BUILD_IN_SOURCE 1
  DEPENDS GSL Boost thepeg-<herwig++_<VERSION>_thepeg> fastjet
)

Pre-requisites

The page lists the prerequisite packages that need to be installed on the different platforms to be able to configure and to build basic ROOT. If more advanced ROOT plugins are required look at the cmake or ./configure output and add the desired third party packages before configuring again.

• RedHat family
• Ubuntu family
• Mac OS X family

Fedora; Scientific Linux; CentOS

• Source code management: git, subversion
• Build tools: cmake, make, gcc, gcc-c++, gcc-gfortran
• System tools: rsync
• LCG CMake dependencies:
  • libcap-devel needed for prctl

Use yum install <package>.

Ubuntu

• Source code management: git, subversion
• Build tools: cmake, make, gcc, g++, gfortran
  • On Ubuntu 18.04, there's also gcc-8, g++-8, gfortran-8
• System tools: rsync
• LCG CMake dependencies:
  • freeglut3 and freeglut3-dev for OpenGL for building Coin3D
  • libncurses5 and libncurses5-dev for building MySQL
  • libtool needed for building libunwind
  • uuid-dev needed for building xapian
  • libxml2-dev and libxslt1-dev needed for building all Python required modules
  • libbz2-dev needed for building Boost
  • bison and flex for doxygen
  • tk, tk-dev, tcl and tcl-dev for Python's tkiter module
  • byacc needed by yacc for ftjam
  • libmotif-common and libmotif-dev needed by Geant4
  • libxmu-headers needed by Geant4
  • build-dep r-base needed by R
  • libatlas-base-dev and libatlas3-base as implementation of blas and lapack for numpy (temporary solution until SPI-1206 is fixed)
  • libcap-dev needed for prctl

Use apt-get install <package> or apt-get build-dep <package> (for r-base).

MacOSX

• Xcode application for make, g++, gcc, ld, etc.

Xcode can found at the Mac App Store. Install Xcode command line tools by running at the terminal xcode-select --install

Source Repository

All package sources (tarfiles) have been collected into an EOS repository, which is web accessible. The full list of sources is available from this URL. To copy new sources into the repository is done using the folowing command:

xrdcp -f <file> root://eosuser.cern.ch//eos/project/l/lcg/www/lcgpackages/tarFiles/sources

or

scp <file> <username>@lxplus.cern.ch:/eos/project/l/lcg/www/lcgpackages/tarFiles/sources

For generators use:

xrdcp -f <file> root://eosuser.cern.ch//eos/project/l/lcg/www/lcgpackages/tarFiles/sources/MCGeneratorsTarFiles/