Skip to content
Snippets Groups Projects

Preparing migration to CAT

Merged Preparing migration to CAT
migration into master
Merged Patrick Louis S Connor requested to merge migration into master
Compare
and
3 files
+ 107
− 15
Compare changes
  • Side-by-side
  • Inline
Files
3
README.md
+ 75
− 14
# General instruction
# Das Analysis System
Darwin is a general project involving modular tools for physics analysis. This repository provides the necessary tools to install all pieces of software.
Das Analysis System is a general project involving modular tools for physics analysis with high-level objects such as jets, leptons, and photons. The principle is to perform physics directly from the shell in atomic steps, applying one correction at a time and treating all systematic uncertainties simultaneously.
In general, it is recommended that you install the software on a fast disk, but that you process the heavy n-tuples on a dedicated area (e.g. NFS at DESY, EOS at CERN).
The GitLab group is divided in several repositories:
- The `Installer` repo contains the necessary scripts to install the suite.
- The `Core` repo corresponds to a CMSSW module and is where most of the code for the analysis of CMS data is to be found.
- The `Tables` repo contains the calibration of the high-level objects.
- The `Darwin` repo is a mirror of a [general toolkit](https://protodarwin.docs.cern.ch) for physics analysis.
- `PlottingHelper` is a useful [library](https://github.com/zleba/PlottingHelper) originally made by Radek Zlebcík (Charles University) to help make plots with ROOT.
The `Campaigns` subgroup contains actual repos corresponding to different analyses with configs, plotting macros, and possibly CRAB outputs, or anything relevant to the reproducibility of an analysis. A template repo is provided with suggestions and guidelines.
## Installation
First source the minimal environment for CMSSW (not necessary at CERN):
### From scratch
In general, it is recommended that you install the software on a fast disk to ensure fast compilation (e.g. AFS), but that you process the heavy n-tuples on a dedicated area (e.g. NFS at DESY, EOS at CERN).
First clone the `Installer`. Then source the minimal environment for CMSSW, CRAB, and RUCIO:
```
cd DasAnalysisSystem
source ./setup
```
It is currently working at CERN, at DESY, and for GitLab CI. Feel free to make a merge request to include your favourite facility.
It is currently working at CERN (CH), at DESY (DE), at IIHE (BE), and for GitLab CI. Feel free to make a merge request to include your favourite facility.
The installation is then trivial:
Finally, the installation is trivial:
```
make
```
This takes a few minutes, then you're all set.
### Remarks
#### Remarks
1. You may overwrite (at your own risks) the CMSSW release by running `make CMSSW=CMSSW_X_Y_Z` where you tune `X`, `Y`, and `Z` to the values of the release that you want to test. This may be necessary to run over certain data sets.
2. `git-cms-init` is not run, since it is a priori not useful for the framework to run, take 100MB of space, and makes the installation slower. If it may happen to be useful, you have to add it in the Makefile. It must be run right after setting up the release, when it is locally still empty.
1. You may overwrite (at your own risks) the CMSSW release by running `make CMSSW=CMSSW_X_Y_Z` where you tune `X`, `Y`, and `Z` to the values of the release that you want to test. This may be necessary to run over certain data sets; however, we cannot guarantee that the code will compile nor provide the same result as in the default release.
2. `git-cms-init` is not run, since it is a priori not useful for the framework to run, takes 100MB of space, and makes the installation slower. If it may happen to be useful, you have to add it in the Makefile. It must be run right after setting up the release, when it is locally still empty.
3. To use SSH instead of HTTPS, run `make GITLAB=ssh://git@gitlab.cern.ch:7999`.
4. The `PlottingHelper` is not cloned by default as it is only useful for plotting purposes.
## Setting up the environment
### From an existing CMSSW release or from an existing installation.
In the following, we explain the installation of the framework step by step. A few packages external to CMSSW are necessary (e.g. TUnfold), which you will need to tell CMSSW how to find them with `scram setup`.
First source your CMSSW release and close to a directory where you want to install the external packages.
1. Install TUnfold:
```
wget https://www.desy.de/~sschmitt/TUnfold/TUnfold_V17.9.tgz
mkdir -p TUnfold TUnfold/lib
mv TUnfold_V17.9.tgz TUnfold
cd TUnfold
tar xvzf TUnfold_V17.9.tgz
make lib TUNFOLDVERSION='V17' -j
mv -f libunfold.so lib/libtunfold.so
mv -f TUnfoldV17Dict_rdict.pcm lib/
(cd $CMSSW_BASE && scram setup $OLDPWD/../tunfold.xml)
```
2. Tell CMSSW where to find libgit2:
```
(cd $CMSSW_BASE && scram setup $OLDPWD/libgit2.xml)
```
3. Install ProtoDarwin:
```
git clone $(GITLAB)/paconnor/ProtoDarwin.git
cmake3 -B build -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=$PWD/build/install
cmake3 --build build -j`nproc`
cmake3 --build build --target install
(cd $CMSSW_BASE && scram setup $OLDPWD/../protodarwin.xml)
```
4. Clone the `Tables` wherever you like and the `Core` in CMSSW, and compile as usual with `scram b -j`.
First source the minimal environment for CMSSW as above. Then just source the CMSSW release as usual with `cmsenv`.
#### Remark
To run CRAB jobs, you'll also need to set up a valid [grid certificate](https://twiki.cern.ch/twiki/bin/view/CMSPublic/WorkBookStartingGrid#ObtainingCert). If you already have done so, this will enable it:
If you want to rerun jet clustering (e.g. to have AK8 jets at a low transverse momentum), you need to clone the [JetToolbox](https://github.com/cms-jet/JetToolbox.git):
```
git clone https://github.com/cms-jet/JetToolbox.git $CMSSW_BASE/JMEAnalysis/JetToolbox -b jetToolbox_120X
```
That version needs a minor patch:
```
cd $CMSSW_BASE/src/JMEAnalysis/JetToolbox
patch -p1 $OLDPWD/jetToolbox_120X.patch
```
## Setting up the environment
First source the minimal environment for CMSSW with `source ./setup`, then just source the CMSSW release as usual with `cmsenv`.
To run CRAB jobs, you will also need to set up a valid [grid certificate](https://twiki.cern.ch/twiki/bin/view/CMSPublic/WorkBookStartingGrid#ObtainingCert). If you already have done so, this will enable it:
```
voms-proxy-init --rfc --voms cms -valid 192:00
```
To make [RUCIO](https://twiki.cern.ch/twiki/bin/viewauth/CMS/Rucio) requests, the `setup` file tried to guess your RUCIO username from your local username (unless was already set up). This is not guaranteed to work and you may have to define your RUCIO username ahead for sourcing the DAS environment.
### Good practices
Fight against the increase of entropy by
- pulling regularly,
- commiting regularly,
- making regular merge requests.
- committing regularly,
- document your code,
- making frequent merge requests.
Happy analysis!