StatAnalysis
StatAnalysis is an assembly of statistical analysis software, leveraging the ATLAS build-configuration system (AtlasCmake) to manage the compilation.
From any machine with cvmfs, such as lxplus, using setupATLAS
StatAnalysis is available anywhere that ATLAS software it available through the asetup
command. This includes lxplus. After you have made the asetup
command available (ATLAS users on lxplus can run setupATLAS
, for example) you can do:
asetup StatAnalysis,<version ref> # For a specific full release version, OR:
asetup StatAnalysis,0.2,latest # For the latest available nightly of the 0.2 branch
There are currently two branches being maintained, with difference use cases:
- 0.2 : The series with ROOT 6.28
- main : The series with ROOT master
Using docker images:
To start an interactive session just do:
docker run --pull always -e DISPLAY=host.docker.internal:0 -it gitlab-registry.cern.ch/atlas/statanalysis:<branch/or/tag>
To get x11 windows working you may need to also run: xhost +${hostname}
Building from source
Fast start
Some parts of StatAnalysis are not located in public repositories. The default access is through KRB5 authentication which may require you to kinit
first. You can control the access method using the CERN_GITLAB_PREFIX
option e.g. add -DCERN_GITLAB_PREFIX="ssh://git@gitlab.cern.ch:7999"
to the cmake command below to use ssh access instead of krb5 auth.
Then just build and install like a regular cmake project - can take up to an hour to fully build:
mkdir build; cd build
cmake -DCMAKE_INSTALL_PREFIX=../install ../StatAnalysis
make -j install
If this succeeds you can then whenever you want to use the project just do:
source /path/to/install/setup.sh
Prerequisites
You must have a modern version of cmake and a relatively modern version of gcc compiler. E.g on cvmfs you can do the following to get cmake 3.21 with gcc 11.2:
asetup none,gcc11,cmakesetup
on a mac you can do:
xcode-select --install
which ensures you have compilers. You will also need to install cmake (google this, there are lots of ways).
You also need git. During the build, this will also be used to checkout code of the tools included in StatAnalysis. If the build appears to freeze it may be that git is asking for your username and/or password/ssh passphrase to perform the checkout. You might then need to cache your username and password:
git config --global credential.helper "cache --timeout=86400"
git config --global credential.https://gitlab.cern.ch.username <yourUserName>
As the build takes a long time, it is beneficial to add the longer timeout for the cache. The default is 15 minutes. After calling that, you may encounter the build asking for your password/ssh passphrase (may be unclear this has happened, just hit enter to see the prompt again then enter password).
If you have ssh set up, you can also compile with the option -DCERN_GITLAB_PREFIX="ssh://git@gitlab.cern.ch:7999"
to use ssh instead.
Checkout the project (ensure this is done with recursive option because project has submodules) and create a build directory.
git clone --recurse-submodules ssh://git@gitlab.cern.ch:7999/atlas/StatAnalysis.git
Decide where you would like the release installed to as well. Then do:
mkdir build; cd build
cmake -DCMAKE_INSTALL_PREFIX=../install ../StatAnalysis
make -j install
Depending on what software you have installed on your machine you may need or otherwise choose to disable the building of some parts of the release if you encounter errors at this stage. Do this with the following options (in the form of -D<OPTION>=OFF
):
List all relevant options with:
cmake -LAH ./ | grep ATLAS_BUILD_
Additionally, the versions of the software built in the release are controlled by further options:
cmake -LAH ./ | grep STATANA_
A note for mac users: Python will require openssl in order to build the ssl module that will be required by pip for the installation of the PyExternals. Openssl currently isn't built automatically in the build (although it is built for other bits of StatAnalysis, just not yet for Python) so you need to ensure openssl 1.1 is installed yourself. I found the following was necessary:
brew install openssl@1.1
cp /usr/local/opt/openssl@1.1/lib/pkgconfig/*.pc /usr/local/lib/pkgconfig/
After build+install (make -j install
) you can then setup the release by creating a test area and sourcing the relevant setup script:
mkdir test/build
Developer tips
CMake options let you control if the core dependencies are built in the release or not
cmake -LAH | grep _BUILD_
will list all those build options. The project specific options are in cmake/ProjectOptions.cmake. By default building ROOT is switched on (ATLAS_BUILD_ROOT=TRUE) and controlled by the ATLAS_ROOT_SOURCE
and ATLAS_ROOT_VERSION
variables in External/ROOT/CMakeLists.txt. Versions of other softwares can be set in the same file.
In order to debug a package within StatAnalysis there is the TRACK_CHANGES option which is set to ON by default. This option set to ON forces the recompilation of packages with changes. For ROOT, there is a specific option TRACK_CHANGES_ROOT, which is OFF by default.
Setting this option to ON will compile only those files that have changed since the previous compilation. In order to force the complete rebuild of one of the packages the build folder relating to that package should be deleted, the build folder can be found in <build folder>/src/<package name>-build
.
Known issues
When building natively (on macos) the python build still seems to have references to the build directory, see output of
otool -l `which python`
It may be possible to modify the relevant values with:
install_name_tool -change <oldpath> <newpath> `which python`
using @loader_path/../lib
as part of the newpath.
This issue was exposed by trying to import ROOT in ipython - which caused a double-loading of the cppyy libraries. The second copy is coming from the build directory (not the install area). If you mv the build directory that will stop the second load but then you get errors about references to paths in the build directory - these are the errors you can fix with a handful of install_name_tool calls above.
Of course, instead of moving the build dir the copies of the cppyy libraries can be deleted from the build directory too, to stop the double load (just look for the libcppyy files in the Externals/ROOT/CMakeFiles area)