StatAnalysis

StatAnalysis is an assembly of statistical analysis software, leveraging the ATLAS build-configuration system (AtlasCmake) to manage the compilation.

The current active branches are:

From any machine with cvmfs, such as lxplus, using setupATLAS

StatAnalysis is available anywhere that ATLAS software it available through the asetup command. This includes lxplus. After you have made the asetup command available (ATLAS users on lxplus can run setupATLAS, for example) you can do:

asetup StatAnalysis,<version ref> # For a specific full release version, OR:
asetup StatAnalysis,0.2,latest # For the latest available nightly of the 0.2 branch

Note that (only) the 0.2 branch will not work on lxplus machines that have switched to EL9 (default) as it was built for centos7. For 0.2, please either connect to lxplus7.cern.ch, or use setupATLAS -c centos7.

Using docker images:

To start an interactive session just do:

docker run --pull always -e DISPLAY=host.docker.internal:0 -it gitlab-registry.cern.ch/atlas/statanalysis:<branch/or/tag>

To get x11 windows working you may need to also run: xhost +${hostname}

Building from source

Fast start

Some parts of StatAnalysis are not located in public repositories. The default access is through KRB5 authentication which may require you to kinit first. You can control the access method using the CERN_GITLAB_PREFIX option e.g. add -DCERN_GITLAB_PREFIX="ssh://git@gitlab.cern.ch:7999" to the cmake command below to use ssh access instead of krb5 auth.

Then just build and install like a regular cmake project - can take up to an hour to fully build:

mkdir build; cd build
cmake -DCMAKE_INSTALL_PREFIX=../install ../StatAnalysis
make -j install

If this succeeds you can then whenever you want to use the project just do:

source /path/to/install/setup.sh

Prerequisites

You must have a modern version of cmake and a relatively modern version of gcc compiler. E.g on cvmfs you can do the following to get cmake 3.21 with gcc 11.2:

asetup none,gcc11,cmakesetup

on a mac you can do:

xcode-select --install

which ensures you have compilers. You will also need to install cmake (google this, there are lots of ways).

You also need git. During the build, this will also be used to checkout code of the tools included in StatAnalysis. If the build appears to freeze it may be that git is asking for your username and/or password/ssh passphrase to perform the checkout. You might then need to cache your username and password:

git config --global credential.helper  "cache --timeout=86400"
git config --global credential.https://gitlab.cern.ch.username <yourUserName>

As the build takes a long time, it is beneficial to add the longer timeout for the cache. The default is 15 minutes. After calling that, you may encounter the build asking for your password/ssh passphrase (may be unclear this has happened, just hit enter to see the prompt again then enter password).

If you have ssh set up, you can also compile with the option -DCERN_GITLAB_PREFIX="ssh://git@gitlab.cern.ch:7999" to use ssh instead.

Checkout the project (ensure this is done with recursive option because project has submodules) and create a build directory.

git clone --recurse-submodules ssh://git@gitlab.cern.ch:7999/atlas/StatAnalysis.git

Decide where you would like the release installed to as well. Then do:

mkdir build; cd build
cmake -DCMAKE_INSTALL_PREFIX=../install ../StatAnalysis
make -j install

Depending on what software you have installed on your machine you may need or otherwise choose to disable the building of some parts of the release if you encounter errors at this stage. Do this with the following options (in the form of -D<OPTION>=OFF):

List all relevant options with:

cmake -LAH ./ | grep ATLAS_BUILD_

Additionally, the versions of the software built in the release are controlled by further options:

cmake -LAH ./ | grep STATANA_

A note for mac users: Python will require openssl in order to build the ssl module that will be required by pip for the installation of the PyExternals. Openssl currently isn't built automatically in the build (although it is built for other bits of StatAnalysis, just not yet for Python) so you need to ensure openssl 1.1 is installed yourself. I found the following was necessary:

brew install openssl@1.1
cp /usr/local/opt/openssl@1.1/lib/pkgconfig/*.pc /usr/local/lib/pkgconfig/

After build+install (make -j install) you can then setup the release by creating a test area and sourcing the relevant setup script:

mkdir test/build

Developer tips

CMake options let you control if the core dependencies are built in the release or not

cmake -LAH | grep _BUILD_

will list all those build options. The project specific options are in cmake/ProjectOptions.cmake. By default building ROOT is switched on (ATLAS_BUILD_ROOT=TRUE) and controlled by the ATLAS_ROOT_SOURCE and ATLAS_ROOT_VERSION variables in External/ROOT/CMakeLists.txt. Versions of other softwares can be set in the same file.

In order to debug a package within StatAnalysis there is the TRACK_CHANGES option which is set to ON by default. This option set to ON forces the recompilation of packages with changes. For ROOT, there is a specific option TRACK_CHANGES_ROOT, which is OFF by default.

Setting this option to ON will compile only those files that have changed since the previous compilation. In order to force the complete rebuild of one of the packages the build folder relating to that package should be deleted, the build folder can be found in <build folder>/src/<package name>-build.

Known issues

When building natively (on macos) the python build still seems to have references to the build directory, see output of

otool -l `which python`

It may be possible to modify the relevant values with:

install_name_tool -change <oldpath> <newpath> `which python`

using @loader_path/../lib as part of the newpath.

This issue was exposed by trying to import ROOT in ipython - which caused a double-loading of the cppyy libraries. The second copy is coming from the build directory (not the install area). If you mv the build directory that will stop the second load but then you get errors about references to paths in the build directory - these are the errors you can fix with a handful of install_name_tool calls above.

Of course, instead of moving the build dir the copies of the cppyy libraries can be deleted from the build directory too, to stop the double load (just look for the libcppyy files in the Externals/ROOT/CMakeFiles area)