Introduction

LCGCMake is the EP-SFT infrastructure to build the software stack for the LHC experiments containing both external software dependencies and projects developed within our group. The tool is based on the ExternalProject module of CMake.

Preparation

LCGCMake requires CMake installed in your system. Check the pre-requisites section for the complete list of requirements for each system.

On CVMFS (CERN) set PATH to use one of latest CMake versions (system default is 2.6 and is too old)

    ARCH=$(uname -m)
    export PATH=/cvmfs/sft.cern.ch/lcg/contrib/CMake/3.6.0/Linux-${ARCH}/bin:${PATH}

A Quick Start:

1. Checkout the lcgcmake package from lcgsoft GIT repository:

   git clone https://gitlab.cern.ch/sft/lcgcmake.git

   Notice: to build a package from release LCG_XX, append '-b LCG_XX' to the command to checkout the LCG_XX branch.

2. Create a workspace area on which to perform the builds

   mkdir lcgcmake-build
   cd lcgcmake-build

3. You may need at this moment to define the compiler to use if different from the native compiler

   source /cvmfs/sft.cern.ch/lcg/contrib/gcc/version/platform/setup.(c)sh

4. Configure the build of all externals with cmake

   cmake -DCMAKE_INSTALL_PREFIX=../lcgcmake-install ../lcgcmake

5. In order to build against an existing external repository use the option

   -DLCG_INSTALL_PREFIX=/cvmfs/sft.cern.ch/lcg/releases

   to tell the system to look for packages in the LCG area. This is much faster since it will be an incremental build. Other available options are:

   • -DLCG_VERSION=XX to select a given LCG configuration version. The XX is the key of one the files in lcgcmake/cmake/toolchain (e.g. latest, 86beta, experimental)
   • -DLCG_IGNORE='package1;package2;...' to ignore packages that are already existing in LCG area and force a re-build.

6. Build and install all external packages

   make -jN

   Or to build a single external package (make help shows all the possible targets)

   make -jN <package> 

   You may need to restart de build of a package from beginning in case of obscure errors. The best is to clean a specific package

   make clean-<package>

7. Building a view and sourcing the setup.sh of the view to set an environment to test the build

   make view
   source ../lcgcmake-install/<platform-tag>/setup.[c]sh

Technical details

To add or modify existing packages, the user may edit the build recipes in the file <package-group>/CMakeLists.txt and the actual versions of the package in files cmake/toolchain/heptools-XX.cmake.

The LCG_Package_Add command is the basic command to encapsulate the instructions on how to build a package. This is commands wraps the ExternalProject_Add() command form ExternalProject module with additional options as well as pre-configuring a number of standard actions
(e.g. installing the log files and sources in /share). A fairly complete example is the following one:

LCGPackage_Add(
  pythia8
  URL http://lcgpackages.web.cern.ch/lcgpackages/tarFiles/sources/MCGeneratorsTarFiles/pythia8<NATIVE_VERSION>.tgz
  CONFIGURE_COMMAND ./configure --prefix=<INSTALL_DIR>
                                --with-hepmc=${HepMC_home}
                                --enable-shared
  BUILD_IN_SOURCE 1
  CONFIGURE_EXAMPLES_COMMAND <SOURCE_DIR>/examples/configure
                             --with-pythia8=<INSTALL_DIR>
                             --with-lhapdf=${lhapdf_home}
  BUILD_EXAMPLES_COMMAND make -C <SOURCE_DIR>/examples main01 main03
  INSTALL_EXAMPLES_COMMAND ${CMAKE_COMMAND} -E make_directory <INSTALL_DIR>/bin
                   COMMAND ${CMAKE_COMMAND} -E chdir <SOURCE_DIR>/examples
                           cp bin/main01.exe bin/main03.exe main03.cmnd <INSTALL_DIR>/bin
  TEST_COMMAND ${CMAKE_COMMAND} -E chdir <INSTALL_DIR>/bin ./main01.exe
  DEPENDS HepMC lhapdf
)

The parameters CONFIGURE_EXAMPLES_COMMAND, BUILD_EXAMPLES_COMMAND, INSTALL_EXAMPLES_COMMAND are additional parameters used by LCGPackage_Add to create new steps (configure_examples, build_examples, install_examples). They can be extended with additional commands with the keyword COMMAND.

The variables of the form are evaluated internally with some delay. This allows to use internally defined variables for a multi-versioned package.

The sources are usually taken from source tarballs locally stored at /eos/project/l/lcg/www/lcgpackages/tarFiles and made available via http://lcgpackages.web.cern.ch/lcgpackages/tarFiles/sources/

Configuration Parameters

A number of configuration options are available for the user to customize the LCGCmake build.

option name	default value	description
LCG_VERSION	dev2	This parameter selects the file listing the versions of all packages in cmake/toolchain
LCG_INSTALL_PREFIX		Location of existing LCG releases. Used for incremental builds
LCG_IGNORE		List of packages to be ignored from LCG_INSTALL_PREFIX (';' separated)
LCG_INSTALL_POLICY	separate	Select the 'separate' or 'join' installation of packages
LCG_SAFE_INSTALL	OFF	Ensure that no overwrites occur at the installation area of packages
LCG_TARBALL_INSTALL	OFF	Turn ON/OFF creation/installation of tarballs
LCG_SOURCE_INSTALL	ON	Turn ON/OFF installation of sources in \${CMAKE_INSTALL_PREFIX}///share
VALIDATION	OFF	Enable validation settings
POST_INSTALL	ON	Enable post install step
CMAKE_INSTALL_PREFIX		Installation destination directory
CMAKE_CXX_COMPILER	${CXX}	Full path to the C++ compiler
CMAKE_CC_COMPILER	${CC}	Full path to the C compiler
CMAKE_Fortran_COMPILER	${FC}	Full path to the Fortran compiler
PDFsets	--all	The PDFsets to be downloaded for the 'lhspdfsets' package

LCGPackage_Add Arguments

This is the full list of arguments available for writing the recipes using the function LCGPackage_Add(). Some arguments are interpreted by LCGPackage_Add itself others are passed directly to the underlying ExternalProject_Add() function from the ExternalProject module of CMake.

argument name	default value	description
name		Name for package target
DEPENDS package...		Packages on which the package depends (package or package-version)
DEST_NAME dest		Destination package name where this package will be installed
BUNDLE_PACKAGE bool	0	Indication that this package is an agregation of several packages
BINARY_PACKAGE bool	0	The tar file is directly the binaries
REVISION rev		Revision number of the configuration (recipe instructions). It participates to the hash value calculation
CVS_REPOSITORY url		CVSROOT of CVS repository
SVN_REPOSITORY url		URL of Subversion repository
SVN_REVISION rev		Revision to checkout from Subversion repository
SVN_USERNAME john		Username for Subversion checkout and update
SVN_PASSWORD doe		Password for Subversion checkout and update
SVN_TRUST_CERT 1		Trust the Subversion server site certificate
GIT_REPOSITORY url		URL of git repository
GIT_TAG tag		Git branch name, commit id or tag
HG_REPOSITORY url		URL of mercurial repository
HG_TAG tag		Mercurial branch name, commit id or tag
URL /.../src.tgz		Full path or URL of source
URL_HASH ALGO=value		Hash of file at URL
URL_MD5 md5		Equivalent to URL_HASH MD5=md5
TLS_VERIFY bool		Should certificate for https be checked
TLS_CAINFO file		Path to a certificate authority file
UPDATE_COMMAND cmd...		Source work-tree update command
PATCH_COMMAND cmd...	*	Command to patch downloaded source. Apply path if file patches/package-version.patch exists
CONFIGURE_COMMAND cmd...		Build tree configuration command. Use if empty
CONFIGURE_EXAMPLES_COMMAND		Configure command for configuring the examples
CMAKE_COMMAND /.../cmake		Specify alternative cmake executable
CMAKE_GENERATOR gen		Specify generator for native build
CMAKE_ARGS args...		Arguments to CMake command line
CMAKE_CACHE_ARGS args...		Initial cache arguments, of the form -Dvar:string=on
BUILD_COMMAND cmd...	make	Command to drive the native build
BUILD_EXAMPLES_COMMAND		Build command for the examples
BUILD_IN_SOURCE bool	0	Use source dir for build dir
INSTALL_COMMAND cmd...	make install	Command to drive install after build
INSTALL_EXAMPLES_COMMAND		Install command for the examples
TEST_COMMAND		Command to test the build and installation
COMMAND		Continuation command for any argument of the type XXXX_COMMAND
ENVIRONMENT		List of environment variables in the for NAME=VALUE
FIND_PACKAGE		Name of the package to be found in the system (using a Find[name].cmake module). If found, the package is not build and the system installation is used instead.

Multi-versioned packages

It is possible to build several versions of the same package at the build time. The number of versions and the parameters controlling their configuration are listed in the heptools-XX.cmake file in the toolchain directory. The build instructions encoded in the LCGPackage_Add() command need to be coded with the variable as parameter, which will be replaced to each concrete version in the list of versions when processing the CMakeLists.txt file. Additional parameters controlling the author version, other dependent package version, etc. can be added as extra parameters in the LCG_external_package() command in the toolchain file. These parameter values appear in the CMakeLists.txt file as variables of the form: (package)_(version)_(parameter_name). For example:

LCG_external_package(herwig++ 2.6.3 ${MCGENPATH}/herwig++ thepeg=1.8.3)

the value for the parameter thepeg can be accessed as ${herwig++_2.6.3_thepeg}

Here is a more complete example:

// In heptools-XX.cmake --------------------------------
LCG_external_package(herwig++ 2.6.1b ${MCGENPATH}/herwig++ thepeg=1.8.1)
LCG_external_package(herwig++ 2.6.3 ${MCGENPATH}/herwig++ thepeg=1.8.3)

//In CMakeLists.txt-----------------------------------------
LCGPackage_Add(
  herwig++
  URL http://cern.ch/service-spi/external/.../Herwig++-<VERSION>.tar.bz2
  CONFIGURE_COMMAND ./configure --prefix=<INSTALL_DIR>
                                --with-gsl=${GSL_home}
                                --with-thepeg=<<herwig++_<VERSION>_thepeg>_home>
                                --with-fastjet=${fastjet_home}
                                --with-boost=${Boost_home}
  BUILD_IN_SOURCE 1
  DEPENDS GSL Boost thepeg-<herwig++_<VERSION>_thepeg> fastjet
)

Pre-requisites

The page lists the prerequisite packages that need to be installed on the different platforms to be able to configure and to build basic ROOT. If more advanced ROOT plugins are required look at the cmake or ./configure output and add the desired third party packages before configuring again.

• Fedora family
• Ubuntu family
• Mac OS X family

Fedora 18, 19 and 20; Scientific Linux 5, 6; CentOS 6, 7

• git for /usr/bin/git
• cmake for cmake tools
• make for /usr/bin/make
• gcc-c++ for /usr/bin/g++
  
• gcc for /usr/bin/gcc

Use yum install <package> or use the graphical "Add/Remove Software" program.

Ubuntu 12 and 14

• git for git command
• cmake for cmake tools
• make for make
• freeglut3 and freeglut3-dev for opengl for building coin3d
• libncurses5 and libncurses5-dev for building mysql
• libtool needed for building libunwind
• uuid-dev needed for building xapian
• libxml2-dev and libxslt1-dev needed for building all python required modules
• libbz2-dev needed for building boost
• bison and flex for doxygen
• python-tk for python tkiter module
• byacc needed by yacc for ftjam
• libmotif-common and -dev needed by Geant4
• libxmu-headers needed by Geant4
• build-dep r-base needed by R Use sudo apt-get install <package> or use the graphical "Synaptic Package Manager" program.

MacOSX

• Xcode application for make, g++, gcc, ld, etc. Xcode can found at the Mac App Store.

Install Xcode command line tools by running at the terminal xcode-select --install

Source Repository

All package sources (tarfiles) have been collected into an EOS repository, which is web accessible. The full list of sources is available from this URL. To copy new sources into the repository is done using the folowing command:

xrdcp -f <file> root://eosuser.cern.ch//eos/project/l/lcg/www/lcgpackages/tarFiles/sources
or 
scp <file> <username>@lxplus.cern.ch:/eos/project/l/lcg/www/lcgpackages/tarFiles/sources

For generators use:

xrdcp -f <file> root://eosuser.cern.ch//eos/project/l/lcg/www/lcgpackages/tarFiles/sources/MCGeneratorsTarFiles/