Skip to content
Snippets Groups Projects
README.md 12.1 KiB
Newer Older
# Introduction
LCGCMake is the EP-SFT infrastructure to build the software stack for the LHC experiments 
containing both external software dependencies and projects developed within our group. 
The tool is based on the [ExternalProject module](http://www.kitware.com/media/html/BuildingExternalProjectsWithCMake2.8.html) 
of [CMake](https://cmake.org).
 
Dmitri Konstantinov's avatar
Dmitri Konstantinov committed
# Preparation
LCGCMake requires CMake installed in your system. 
Check the [pre-requisites](#pre-requisites) section for the complete list of requirements for each system. 

On lxplus (CERN) set PATH to use one of latest CMake versions (system default is 2.6 and is too old) 
```
    ARCH=$(uname -m)
    export PATH=/afs/cern.ch/sw/lcg/contrib/CMake/3.3.2/Linux-${ARCH}/bin:${PATH}
```
# A Quick Start:
1. Checkout the lcgcmake package from lcgsoft GIT repository:
  ```
  git clone https://gitlab.cern.ch/sft/lcgcmake.git
  ```
Dmitri Konstantinov's avatar
Dmitri Konstantinov committed
  Notice: to build a package from release LCG_XX, append '-b LCG_XX' to the command to checkout the LCG_XX branch.
Pere Mato Vila's avatar
Pere Mato Vila committed
2. Create a workspace area on which to perform the builds 
  ```
  mkdir lcgcmake-build
  cd lcgcmake-build
  ```
3. You may need at this moment to define the compiler to use if different from the native compiler
  ```   
  source /afs/cern.ch/sw/lcg/external/gcc/version/platform/setup.(c)sh
  ```
4. Configure the build of all externals with cmake 
  ```
  cmake -DLCG_VERSION=XX -DCMAKE_INSTALL_PREFIX=../lcgcmake-install ../lcgcmake
  ```
5. In order to build against an existing external repository use the option 
  ```
  -DLCG_INSTALL_PREFIX=/afs/cern.ch/sw/lcg/external
  ```
  to tell the system to look for packages in the LCG area. 
  Other available options are: 
  - `-DLCG_VERSION=XX` to select a given LCG configuration version
  - `-DLCG_IGNORE='package1;package2;...'` to ignore packages that are already existing in LCG area and force a re-build.
6. Build and install all external packages 
  ```
  make -jN
  ```
  Or to build a single external package (make help shows all possibilities)
  ```
  make -jN <package> 
  ```
  You may need to restart de build of a package from beginning in case of obscure errors. 
  The best is to clean a specific package
  ```
  make clean-<package>
Pere Mato Vila's avatar
Pere Mato Vila committed
  ```

# Technical details

To add or modify existing packages, the user may edit the build recipes in the file `<package-group>/CMakeLists.txt` and 
the actual versions of the package in files `cmake/toolchain/heptools-XX.cmake`.

The `LCG_Package_Add` command is the basic command to encapsulate the instructions on how to build a package. 
This is commands wraps the ExternalProject_Add() command form ExternalProject module with additional 
Pere Mato Vila's avatar
Pere Mato Vila committed
options as well as pre-configuring a number of standard actions  
(e.g. installing the log files and sources in /share). 
A fairly complete example is the following one:
``` 
LCGPackage_Add(
  pythia8
  URL http://service-spi.web.cern.ch/service-spi/external/tarFiles/pythia8<NATIVE_VERSION>.tgz
  CONFIGURE_COMMAND ./configure --prefix=<INSTALL_DIR>
                                --with-hepmc=${HepMC_home}
                                --enable-shared
  BUILD_IN_SOURCE 1
  CONFIGURE_EXAMPLES_COMMAND <SOURCE_DIR>/examples/configure
                             --with-pythia8=<INSTALL_DIR>
                             --with-lhapdf=${lhapdf_home}
  BUILD_EXAMPLES_COMMAND make -C <SOURCE_DIR>/examples main01 main03
  INSTALL_EXAMPLES_COMMAND ${CMAKE_COMMAND} -E make_directory <INSTALL_DIR>/bin
                   COMMAND ${CMAKE_COMMAND} -E chdir <SOURCE_DIR>/examples
                           cp bin/main01.exe bin/main03.exe main03.cmnd <INSTALL_DIR>/bin
  TEST_COMMAND ${CMAKE_COMMAND} -E chdir <INSTALL_DIR>/bin ./main01.exe
  DEPENDS HepMC lhapdf
)
```
The parameters `CONFIGURE_EXAMPLES_COMMAND`, `BUILD_EXAMPLES_COMMAND`, `INSTALL_EXAMPLES_COMMAND` are 
additional parameters used by `LCGPackage_Add` to create new steps (configure_examples, build_examples, 
install_examples). They can be extended with additional commands with the keyword COMMAND.

The variables of the form <variable> are evaluated internally with some delay. 
This allows to use internally defined variables for a multi-versioned package.

The sources are usually taken from source tarballs locally stored at `/afs/cern.ch/sw/lcg/externals/tarFiles` 
and made available via `http://service-spi.web.cern.ch/service-spi/external/tarFiles`

Pere Mato Vila's avatar
Pere Mato Vila committed
## Configuration Parameters
A number of configuration options are available for the user to customize the LCGCmake build.

| option name  | default value  | description |
|----------------|-----------------|---------------|
| LCG_VERSION | dev2 | This parameter selects the file listing the versions of all packages in `cmake/toolchain` |
| LCG_INSTALL_PREFIX | | Location of existing LCG releases. Used for incremental builds |
| LCG_IGNORE | | List of packages to be ignored from LCG_INSTALL_PREFIX (';' separated) |
| LCG_INSTALL_POLICY | separate | Select the 'separate' or 'join' installation of packages |
| LCG_SAFE_INSTALL | OFF | Ensure that no overwrites occur at the installation area of packages |
| LCG_TARBALL_INSTALL | OFF | Turn ON/OFF creation/installation of tarballs |
| LCG_SOURCE_INSTALL | ON | Turn ON/OFF installation of sources in \${CMAKE_INSTALL_PREFIX}/<package>/<version>/share |
Pere Mato Vila's avatar
Pere Mato Vila committed
| VALIDATION  | OFF | Enable validation settings |
| POST_INSTALL | ON | Enable post install step |
| CMAKE_INSTALL_PREFIX | | Installation destination directory |
| CMAKE_CXX_COMPILER | ${CXX} | Full path to the C++ compiler |
| CMAKE_CC_COMPILER | ${CC} | Full path to the C compiler |
| CMAKE_Fortran_COMPILER | ${FC} | Full path to the Fortran compiler |
| PDFsets  | --all |  The PDFsets to be downloaded for the 'lhspdfsets' package |


## LCGPackage_Add Arguments
This is the full list of arguments available for writing the recipes using the function LCGPackage_Add(). 
Some arguments are interpreted by LCGPackage_Add itself others are passed directly to the underlying ExternalProject_Add() 
function from the ExternalProject module of CMake.

| argument name      | default value  | description |
|--------------------|-----------------|---------------|
| name               |  | Name for package target |
| DEPENDS package... |  | Packages on which the package depends (package or package-version) |
| DEST_NAME dest     |  | Destination package name  where this package will be installed |
| BUNDLE_PACKAGE bool| 0| Indication that this package is an agregation of several packages |
| BINARY_PACKAGE bool| 0| The tar file is directly the binaries |
| REVISION rev       |  | Revision number of the configuration (recipe instructions). It participates to the hash value calculation |
| CVS_REPOSITORY url |  | CVSROOT of CVS repository |
| SVN_REPOSITORY url |  | URL of Subversion repository |
| SVN_REVISION rev   |  | Revision to checkout from Subversion repository |
| SVN_USERNAME john  |  | Username for Subversion checkout and update |
| SVN_PASSWORD doe   |  | Password for Subversion checkout and update |
| SVN_TRUST_CERT 1   |  | Trust the Subversion server site certificate |
| GIT_REPOSITORY url |  | URL of git repository |
| GIT_TAG tag        |  | Git branch name, commit id or tag |
| HG_REPOSITORY url  |  | URL of mercurial repository |
| HG_TAG tag         |  | Mercurial branch name, commit id or tag |
| URL /.../src.tgz   |  | Full path or URL of source |
| URL_HASH ALGO=value|  | Hash of file at URL |
| URL_MD5 md5        |  | Equivalent to URL_HASH MD5=md5 |
| TLS_VERIFY bool    |  | Should certificate for https be checked |
| TLS_CAINFO file    |  | Path to a certificate authority file |-
| UPDATE_COMMAND cmd... | | Source work-tree update command |
| PATCH_COMMAND cmd... | * | Command to patch downloaded source. Apply path if file patches/package-version.patch exists |  
| CONFIGURE_COMMAND cmd... |  | Build tree configuration command. Use <VOID> if empty |
| CONFIGURE_EXAMPLES_COMMAND | | Configure command for configuring the examples |
| CMAKE_COMMAND /.../cmake |  | Specify alternative cmake executable |
| CMAKE_GENERATOR gen | | Specify generator for native build |
| CMAKE_ARGS args... | | Arguments to CMake command line |
| CMAKE_CACHE_ARGS args... | | Initial cache arguments, of the form -Dvar:string=on |
| BUILD_COMMAND cmd... | make | Command to drive the native build |
| BUILD_EXAMPLES_COMMAND | | Build command for the examples |
| BUILD_IN_SOURCE bool | 0 | Use source dir for build dir |
| INSTALL_COMMAND cmd... | make install | Command to drive install after build |
| INSTALL_EXAMPLES_COMMAND | | Install command for the examples |
| TEST_COMMAND | | Command to test the build and installation |
| COMMAND | | Continuation command for any argument of the type XXXX_COMMAND |
| ENVIRONMENT | | List of environment variables in the for NAME=VALUE 
## Multi-versioned packages
It is possible to build several versions of the same package at the build time.
The number of versions and the parameters controlling their configuration are listed in the `heptools-XX.cmake` 
file in the toolchain directory. 
The build instructions encoded in the `LCGPackage_Add()` command need to be coded with the variable <VERSION> 
as parameter, which will be replaced to each concrete version in the list of versions when processing the 
CMakeLists.txt file. Additional parameters controlling the author version, other dependent package version, etc. 
can be added as extra parameters in the `LCG_external_package()` command in the toolchain file. 
These parameter values appear in the CMakeLists.txt file as variables of the form: 
`(package)_(version)_(parameter_name)`. For example:
```
Pere Mato Vila's avatar
Pere Mato Vila committed
LCG_external_package(herwig++ 2.6.3 ${MCGENPATH}/herwig++ thepeg=1.8.3)
```  
the value for the parameter `thepeg` can be accessed as `${herwig++_2.6.3_thepeg}`

Here is a more complete example:
```
// In heptools-XX.cmake --------------------------------
LCG_external_package(herwig++ 2.6.1b ${MCGENPATH}/herwig++ thepeg=1.8.1)
LCG_external_package(herwig++ 2.6.3 ${MCGENPATH}/herwig++ thepeg=1.8.3)

//In CMakeLists.txt-----------------------------------------
LCGPackage_Add(
  herwig++
  URL http://cern.ch/service-spi/external/.../Herwig++-<VERSION>.tar.bz2
  CONFIGURE_COMMAND ./configure --prefix=<INSTALL_DIR>
                                --with-gsl=${GSL_home}
                                --with-thepeg=<<herwig++_<VERSION>_thepeg>_home>
                                --with-fastjet=${fastjet_home}
                                --with-boost=${Boost_home}
  BUILD_IN_SOURCE 1
  DEPENDS GSL Boost thepeg-<herwig++_<VERSION>_thepeg> fastjet
)
```

# <a name="pre-requisites"></a>Pre-requisites
The page lists the prerequisite packages that need to be installed on the different platforms to be able to configure and to build basic ROOT. If more advanced ROOT plugins are required look at the  `cmake` or `./configure` output and add the desired third party packages before configuring again.

- [Fedora family](#fedora)
- [Ubuntu family](#ubuntu)
- [Mac OS X family](#macosx)

## <a name=fedora></a>Fedora 18, 19 and 20; Scientific Linux 5, 6; CentOS 6, 7
- **git** for /usr/bin/git
- **cmake** for cmake tools
- **make** for /usr/bin/make
- **gcc-c++** for /usr/bin/g++  
- **gcc** for /usr/bin/gcc

Use `yum install <package>` or use the graphical "Add/Remove Software" program.

## <a name=ubuntu></a>Ubuntu 12 and 14
- **git** for git command
- **cmake** for cmake tools
- **make** for make
- **freeglut3 and freeglut3-dev** for opengl  for building `coin3d`
- **libncurses5 and libncurses5-dev** for building `mysql`
- **libtool** needed for building `libunwind`
- **uuid-dev** needed for building `xapian` 
- **libxml2-dev** and **libxslt1-dev** needed for building all `python` required modules
- **libbz2-dev** needed for building `boost`
- **bison** and **flex** for `doxygen`
- **python-tk** for `python tkiter` module
- **byacc** needed by yacc for `ftjam`
- **libmotif-common and -dev** needed by Geant4
- **libxmu-headers** needed by Geant4
Use `sudo apt-get install <package>` or use the graphical "Synaptic Package Manager" program.

## <a name=macosx></a>MacOSX
Pere Mato Vila's avatar
Pere Mato Vila committed
- **Xcode** application for make, g++, gcc, ld, etc. 
  Xcode can found at the Mac App Store. 
  
Install Xcode command line tools by running at the terminal `xcode-select --install`