README.md 12.3 KB
Newer Older
Domenico Giordano's avatar
Domenico Giordano committed
1
2
Collection of HEP workloads for benchmarking purposes.

Domenico Giordano's avatar
Domenico Giordano committed
3
Workloads are packaged in self contained Docker images, including the needed libraries from CVMFS.
Domenico Giordano's avatar
Domenico Giordano committed
4

5
Snapshotting of CVMFS is obtained using the [CVMFS Shrinkwrap utility](https://cvmfs.readthedocs.io/en/stable/cpt-shrinkwrap.html)
root's avatar
root committed
6

7
8
## Summary of currently supported HEP workloads

Domenico Giordano's avatar
Domenico Giordano committed
9
10
| Experiment |  Name  | Description | Experiment license| Latest Container | Readiness | Pipeline status | 
| -------- | -------- | -------- | -------- | -------- | -------- | -------- |
Domenico Giordano's avatar
Domenico Giordano committed
11
12
13
14
15
16
17
18
| Alice    | [gen-sim](alice/gen-sim/)  | [link](https://gitlab.cern.ch/hep-benchmarks/hep-workloads/-/blob/master/alice/gen-sim/alice-gen-sim/DESCRIPTION) | [GNU GPL v3](github.com/AliceO2Group/AliceO2/blob/dev/COPYING) | [docker](docker://gitlab-registry.cern.ch/hep-benchmarks/hep-workloads/alice-gen-sim-bmk:latest) | w.i.p. | ![pipeline status](https://gitlab.cern.ch/hep-benchmarks/hep-workloads/badges/qa-build-alice-gen-sim/pipeline.svg) | 
| Atlas    | [gen](atlas/gen)      | [link](https://gitlab.cern.ch/hep-benchmarks/hep-workloads/-/blob/master/atlas/gen/atlas-gen/DESCRIPTION) | [Apache v2](https://gitlab.cern.ch/atlas/athena/blob/master/LICENSE) | [docker](docker://gitlab-registry.cern.ch/hep-benchmarks/hep-workloads/atlas-gen-bmk:latest) | Y | ![pipeline status](https://gitlab.cern.ch/hep-benchmarks/hep-workloads/badges/qa-build-atlas-gen/pipeline.svg) |
| Atlas    | [sim](atlas/sim)      | [link](https://gitlab.cern.ch/hep-benchmarks/hep-workloads/-/blob/master/atlas/sim/atlas-sim/DESCRIPTION) | [Apache v2](https://gitlab.cern.ch/atlas/athena/blob/master/LICENSE) | [docker](docker://gitlab-registry.cern.ch/hep-benchmarks/hep-workloads/atlas-sim-bmk:latest) | Y | ![pipeline status](https://gitlab.cern.ch/hep-benchmarks/hep-workloads/badges/qa-build-atlas-sim/pipeline.svg) |
| Atlas    | [digi-reco](atlas/digi-reco) | [link](https://gitlab.cern.ch/hep-benchmarks/hep-workloads/-/blob/master/atlas/digi-reco/atlas-digi-reco/DESCRIPTION)  | [Apache v2](https://gitlab.cern.ch/atlas/athena/blob/master/LICENSE) | [docker](docker://gitlab-registry.cern.ch/hep-benchmarks/hep-workloads/atlas-digi-reco-bmk:latest) | w.i.p. | ![pipeline status](https://gitlab.cern.ch/hep-benchmarks/hep-workloads/badges/qa-build-atlas-digi-reco/pipeline.svg) |
| CMS      | [gen-sim](cms/gen-sim)  | [link](https://gitlab.cern.ch/hep-benchmarks/hep-workloads/-/blob/master/cms/gen-sim/cms-gen-sim/DESCRIPTION)  | [Apache v2](https://github.com/cms-sw/cmssw/blob/master/LICENSE) | [docker](docker://gitlab-registry.cern.ch/hep-benchmarks/hep-workloads/cms-gen-sim-bmk:latest) | Y | ![pipeline status](https://gitlab.cern.ch/hep-benchmarks/hep-workloads/badges/qa-build-cms-gen-sim/pipeline.svg) |
| CMS      | [digi](cms/digi)     | [link](https://gitlab.cern.ch/hep-benchmarks/hep-workloads/-/blob/master/cms/digi/cms-digi/DESCRIPTION)   | [Apache v2](https://github.com/cms-sw/cmssw/blob/master/LICENSE) | [docker](docker://gitlab-registry.cern.ch/hep-benchmarks/hep-workloads/cms-digi-bmk:latest) | Y | ![pipeline status](https://gitlab.cern.ch/hep-benchmarks/hep-workloads/badges/qa-build-cms-digi/pipeline.svg) |
| CMS      | [reco](lhcb/gen-sim)     | [link](https://gitlab.cern.ch/hep-benchmarks/hep-workloads/-/blob/master/cms/reco/cms-reco/DESCRIPTION)  | [Apache v2](https://github.com/cms-sw/cmssw/blob/master/LICENSE) | [docker](docker://gitlab-registry.cern.ch/hep-benchmarks/hep-workloads/cms-reco-bmk:latest) | Y | ![pipeline status](https://gitlab.cern.ch/hep-benchmarks/hep-workloads/badges/qa-build-cms-reco/pipeline.svg) |
| LHCb     | [gen-sim](lhcb/gen-sim)  | [link](https://gitlab.cern.ch/hep-benchmarks/hep-workloads/-/blob/master/lhcb/gen-sim/lhcb-gen-sim/DESCRIPTION)  | [GNU GPL v3](indico.cern.ch/event/727095/contributions/2992611/attachments/1646177/2631053/20180509-HSFLicensing-LHCbStatus.pdf) | [docker](docker://gitlab-registry.cern.ch/hep-benchmarks/hep-workloads/lhcb-gen-sim-bmk:latest) | Y | ![pipeline status](https://gitlab.cern.ch/hep-benchmarks/hep-workloads/badges/qa-build-lhcb-gen-sim/pipeline.svg) |
19
| Belle2     | [gen-sim-reco](belle2/gen-sim-reco)  | [link](https://gitlab.cern.ch/hep-benchmarks/hep-workloads/-/blob/master/belle2/gen-sim-reco/belle2-gen-sim-reco/DESCRIPTION)  | [GNU GPL v3](indico.cern.ch/event/727095/contributions/2992611/attachments/1646177/2631053/20180509-HSFLicensing-belle2Status.pdf) | [docker](docker://gitlab-registry.cern.ch/hep-benchmarks/hep-workloads/belle2-gen-sim-reco-bmk:latest) | Y | ![pipeline status](https://gitlab.cern.ch/hep-benchmarks/hep-workloads/badges/qa-build-belle2-gen-sim-reco/pipeline.svg) |
20
21


root's avatar
root committed
22
23
24
25
26
27
28
## HEP application in a standalone container
A standalone container running an HEP workload consists of
   1. A cvmfs area exposing the software libraries
   1. A set of input data (root files and condition data)
   1. An orchestrator script

### The orchestrator script
29
The orchestrator script (example here for [KV](atlas/kv/atlas-kv/atlas-kv-bmk.sh) ) takes care of configuring the application
root's avatar
root committed
30
31
32
environment, run the application, parse the produced output and create the score results.
The script has few utility functions to start a configurable number of parallel (and independent) copies of the same application, so that all the cores of the machine under test receive a workload. Typically the number of copies depends on the number of available cores and the number of threads each copy will spawn.

33
The orchestrator script resides in a directory (example here for [KV](atlas/kv/atlas-kv/) ) that expects all other components (namely /cvmfs and input files) are already available. How to make those components available depends on the approaches followed.
root's avatar
root committed
34
35
36

### Run the HEP application in a Docker container
One of the approaches to run the application is to build a Docker a standalone container starting from the SLC6 base image, 
Domenico Giordano's avatar
Domenico Giordano committed
37
include the directory of the orchestration script, the input files and /cvmfs.
38
This is what achieved by using this [Dockerfile](common/Dockerfile.template), where the local cvmfs area (empty in this repo) can be populated with the snapshot fo CVMFS obtained by using the [CVMFS Shrinkwrap utility](https://cvmfs.readthedocs.io/en/stable/cpt-shrinkwrap.html).
root's avatar
root committed
39
40
41
42
43
44
45

Alternatively cvmfs can be made available on the host, and bind mounted to the docker container using -v /cvmfs:/cvmfs:shared

### Snapshot the CVMFS repository using the CVMFS Shrinkwrap utility
This utilities assumes that 
   * a recent version of CVMFS (still pre-release) is adopted
   * an appropriate configuration of the shrinkwrap utility is present
Domenico Giordano's avatar
Domenico Giordano committed
46
47
   * that the HEP application runs once, in order to open the cvmfs file that will be then extracted to build the snapshot.

Domenico Giordano's avatar
Domenico Giordano committed
48
When this is done, after running the shrinkwrap utility, a local archive of the cvmfs snapshot is build.
root's avatar
root committed
49
50
51
52
53
54
55
56
57
58
The process goes through the creation of a trace file, that traces what has been accesses, followed by the copy of those files to the local archive.

In order to automate this procedure a bash script is available in this repo [main.sh](/build-executor/main.sh)
The script takes care of
   * Installing the appropriate version of cvmfs (if needed)
   * Configuring the cvmfs application for the tracer
   * Trigger the execution of the HEP application
   * Create the cvmfs local archive
   * Create the standalone Docker container with the HEP application and the local copy of the cvmfs archive

59
The gitlab CI of this project is configured to run all those steps (see [.gitlabci.yml](https://gitlab.cern.ch/hep-benchmarks/hep-workloads/blob/master/.gitlab-ci.yml))
Domenico Giordano's avatar
Domenico Giordano committed
60
In order to execute all those steps in an isolate environment, Docker containers are used.
root's avatar
root committed
61

Domenico Giordano's avatar
Domenico Giordano committed
62
#### Example: Run interactively the CVMFS Shrinkwrap procedure to build a standalone LHCb container
Evangelia Santorinaiou's avatar
Evangelia Santorinaiou committed
63

Domenico Giordano's avatar
Domenico Giordano committed
64
 1. Install docker ([doc from official site](https://docs.docker.com/install/linux/docker-ce/centos/))
Domenico Giordano's avatar
Domenico Giordano committed
65
    * `yum remove docker         docker-common         docker-selinux         docker-engine`   
Evangelia Santorinaiou's avatar
Evangelia Santorinaiou committed
66
67
68
69
70
71
        `yum install -y yum-utils device-mapper-persistent-data lvm2`  
        `yum-config-manager  --add-repo   https://download.docker.com/linux/centos/docker-ce.repo`  
        `yum install -y docker-ce`  
        `systemctl  start docker`   
        `systemctl enable docker`   
 
Evangelia Santorinaiou's avatar
Evangelia Santorinaiou committed
72
 2. Clone the repository  
73
    * `git clone https://:@gitlab.cern.ch:8443/hep-benchmarks/hep-workloads.git`
Domenico Giordano's avatar
Domenico Giordano committed
74
    * ``HEPWL=`readlink -f hep-workloads` ``
Evangelia Santorinaiou's avatar
Evangelia Santorinaiou committed
75

76
77
78
 3. Run the automatic procedure   
    * ``$HEPWL/build-executor/run_build.sh -s $HEPWL/lhcb/gen-sim/lhcb-bmk.spec``  
    * run ``$HEPWL/build-executor/run_build.sh -h`` to know all options
79
80
81
82
83

4. In case only cvmfs needs to be exposed, run ``run_build.sh`` with the option ``-e``
    * ``$HEPWL/build-executor/run_build.sh -s $HEPWL/cms/gen-sim/cms-gen-sim.spec -e sleep``
       * This will start a privileged container that will mount the cvmfs repositories listed in `` $HEPWL/cms/gen-sim/cms-gen-sim.spec`` and will stay alive so that a second container can bind mount the exposed cvmfs mount point
          * ``docker run -v /tmp/root/cvmfs_hep/MOUNTPOINT:/cvmfs:shared ...``
Evangelia Santorinaiou's avatar
Evangelia Santorinaiou committed
84
     
85
86
# How to run a built HEP workload container

87
Given a HEP workload docker image ($IMAGE) from the [registry](https://gitlab.cern.ch/hep-benchmarks/hep-workloads/container_registry), get the benchamark to run through
88
  * ``docker run --rm -v /tmp/results:/results $IMAGE``
Domenico Giordano's avatar
Domenico Giordano committed
89
  * In order to access the command line options
90
     * ``docker run --rm -v /tmp/results:/results $IMAGE -h``
Domenico Giordano's avatar
Domenico Giordano committed
91

92
The HEP workload docker images are also compatible with Singularity.  It's possible to run the containers with Singularity by prepending the image with a ``docker://`` URI:
93
  * ``singularity run -C -B /tmp:/tmp -B /tmp/results:/results docker://$IMAGE``
94
      * The ``/tmp/results`` directory must exist.
95
Note that the ``overlay`` or ``underlay`` options must be enabled in your singularity.conf file.  On systems with older Linux kernels that do not support OverlayFS, such as in Scientific Linux 6, underlay must be used.  Also note that full containment ('-C') is recommended when invoking Singularity, in order to avoid potential conflicts with your environment settings and the configuraton files in your home directory, which would otherwise be inherited by the container.
96

Domenico Giordano's avatar
Domenico Giordano committed
97
98
## Example of a fast benchmark: Run Athena KV
  * Run Athena v17.8.0.9 (legacy version of KV)
99
100
      * docker run --rm -v /some_path:/results gitlab-registry.cern.ch/hep-benchmarks/hep-workloads/atlas-kv-bmk:ci1.2
      * singularity run -C -B /tmp:/tmp -B /some_path:/results docker://gitlab-registry.cern.ch/hep-benchmarks/hep-workloads/cms-gen-sim-bmk:latest
Domenico Giordano's avatar
Domenico Giordano committed
101
102
103
104
      * command line options: -h (for help), -n xx (number of copies, default is the number of available cores), -d (debug verbosity)
      * NB: any combination of the docker run options described above for CMS works also here

## Example: Run CMS ttbar GEN-SIM
105
106
107
108
109
110
111

Any combination of the following options
    
   * command line options: -h (for help), -t xx (number of threads, default: 4), -e xx (number of events per thread, default: 100 ), -d (debug verbosity)
      * The script will spawn a number of parallel processes equal to (number of available cores) / (number of defined threads per proces) 
   
   * In order to retrieve json benchmark results and logs from a local directory, mount a host directory (/some_path) as container volume /results 
112
113
      * docker run --rm -v /some_path:/results gitlab-registry.cern.ch/hep-benchmarks/hep-workloads/cms-gen-sim-bmk:latest
      * singularity run -C -B /tmp:/tmp -B /some_path:/results docker://gitlab-registry.cern.ch/hep-benchmarks/hep-workloads/cms-gen-sim-bmk:latest
114
 
115
   * In order to fix number of events per thread (default is 20) and number of threads per cmssw (default is 4)
116
117
      * docker run --rm -v /some_path:/results gitlab-registry.cern.ch/hep-benchmarks/hep-workloads/cms-gen-sim-bmk:latest -t 10 -e 50
      * singularity run -C -B /tmp:/tmp -B /some_path:/results docker://gitlab-registry.cern.ch/hep-benchmarks/hep-workloads/cms-gen-sim-bmk:latest -t 10 -e 50
118
119

   * In order to fix number of running cores 
120
      * docker run --rm --cpuset-cpus=0-7  gitlab-registry.cern.ch/hep-benchmarks/hep-workloads/cms-gen-sim-bmk:latest
121
      * This is not directly possible with Singularity
122
123

   * In order to inspect information about the docker container, including the label with the description of the HEP workload included
124
      * docker inspect gitlab-registry.cern.ch/hep-benchmarks/hep-workloads/cms-gen-sim-bmk:latest
125
      * This is not directly possible with Singularity
126
127


Evangelia Santorinaiou's avatar
Evangelia Santorinaiou committed
128

Evangelia Santorinaiou's avatar
Evangelia Santorinaiou committed
129

130

Evangelia Santorinaiou's avatar
Evangelia Santorinaiou committed
131