Skip to content
Snippets Groups Projects
Select Git revision
  • fixdoc
  • qa default
  • authors
  • ncopies
  • reptestr
  • singci6
  • singci5
  • singci3
  • singci2
  • reptest
  • singci
  • alicesng2
  • BMK-8
  • BMK-7
  • patch-4
  • master
16 results
Compare
Forked from HEP-Benchmarks / hep-workloads
3033 commits behind the upstream repository.
Name Last commit Last update
alice/gen-sim
atlas
build-executor
cms/gen-sim
lhcb/gen-sim
test
.gitignore
.gitlab-ci.sh
.gitlab-ci.yml
README.md
README.md
Branch QA
Pipeline status pipeline status

Collection of HEP workloads for benchmarking purposes.

Workloads are packaged in self contained Docker images, including the needed libraries from CVMFS.

Snapshotting of CVMFS is obtained using the CVMFS Shrinkwrap utility

HEP application in a standalone container

A standalone container running an HEP workload consists of

  1. A cvmfs area exposing the software libraries
  2. A set of input data (root files and condition data)
  3. An orchestrator script

The orchestrator script

The orchestrator script (example here for KV ) takes care of configuring the application environment, run the application, parse the produced output and create the score results. The script has few utility functions to start a configurable number of parallel (and independent) copies of the same application, so that all the cores of the machine under test receive a workload. Typically the number of copies depends on the number of available cores and the number of threads each copy will spawn.

The orchestrator script resides in a directory (example here for KV ) that expects all other components (namely /cvmfs and input files) are already available. How to make those components available depends on the approaches followed.

Run the HEP application in a Docker container

One of the approaches to run the application is to build a Docker a standalone container starting from the SLC6 base image, include the directory of the orchestration script, the input files and /cvmfs. This is what achieved by using this Dockerfile for KV, where the local cvmfs area (empty in this repo) can be populated with the snapshot fo CVMFS obtained by using the CVMFS Shrinkwrap utility.

Alternatively cvmfs can be made available on the host, and bind mounted to the docker container using -v /cvmfs:/cvmfs:shared

Snapshot the CVMFS repository using the CVMFS Shrinkwrap utility

This utilities assumes that

  • a recent version of CVMFS (still pre-release) is adopted
  • an appropriate configuration of the shrinkwrap utility is present
  • that the HEP application runs once, in order to open the cvmfs file that will be then extracted to build the snapshot.

When this is done, after running the shrinkwrap utility, a local archive of the cvmfs snapshot is build. The process goes through the creation of a trace file, that traces what has been accesses, followed by the copy of those files to the local archive.

In order to automate this procedure a bash script is available in this repo main.sh The script takes care of

  • Installing the appropriate version of cvmfs (if needed)
  • Configuring the cvmfs application for the tracer
  • Trigger the execution of the HEP application
  • Create the cvmfs local archive
  • Create the standalone Docker container with the HEP application and the local copy of the cvmfs archive

The gitlab CI of this project is configured to run all those steps (see .gitlabci.yml) In order to execute all those steps in an isolate environment, Docker containers are used.

Example: Run interactively the CVMFS Shrinkwrap procedure to build a standalone LHCb container

  1. Install docker (doc from official site)

    • yum remove docker docker-common docker-selinux docker-engine
      yum install -y yum-utils device-mapper-persistent-data lvm2
      yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
      yum install -y docker-ce
      systemctl start docker
      systemctl enable docker

  2. Clone the repository

    • git clone https://:@gitlab.cern.ch:8443/hep-benchmarks/hep-workloads.git
    • HEPWL=`readlink -f hep-workloads`

  3. Run the automatic procedure

    • $HEPWL/build-executor/run_build.sh -s $HEPWL/lhcb/gen-sim/lhcb-bmk.spec
    • run $HEPWL/build-executor/run_build.sh -h to know all options

The HEP workload docker images are also compatible with Singularity. It's possible to run the containers with Singularity by prepending the image with a docker:// URI:

  • singularity run -B /tmp/results:/results docker:://$IMAGE
    • The /tmp/results directory must exist.

How to run a built HEP workload container

Given a HEP workload docker image ($IMAGE) from the registry, get the benchamark to run through

  • docker run --rm -v /tmp/results:/results $IMAGE
  • In order to access the command line options
    • docker run --rm -v /tmp/results:/results $IMAGE -h

Example of a fast benchmark: Run Athena KV

  • Run Athena v17.8.0.9 (legacy version of KV)
    • docker run --rm -v /some_path:/results gitlab-registry.cern.ch/hep-benchmarks/hep-workloads/atlas-kv-bmk:latest
    • singularity run -B /some_path:/results docker://gitlab-registry.cern.ch/hep-benchmarks/hep-workloads/cms-gen-sim:latest
    • command line options: -h (for help), -n xx (number of copies, default is the number of available cores), -d (debug verbosity)
    • NB: any combination of the docker run options described above for CMS works also here

Example: Run CMS ttbar GEN-SIM

Any combination of the following options

  • command line options: -h (for help), -t xx (number of threads, default: 4), -e xx (number of events per thread, default: 100 ), -d (debug verbosity)

    • The script will spawn a number of parallel processes equal to (number of available cores) / (number of defined threads per proces)

  • In order to retrieve json benchmark results and logs from a local directory, mount a host directory (/some_path) as container volume /results

  • In order to fix number of events per thread (default is 20) and number of threads per cmssw (default is 4)

  • In order to fix number of running cores

    • docker run --rm --cpuset-cpus=0-7 gitlab-registry.cern.ch/hep-benchmarks/hep-workloads/cms-gen-sim:latest
    • This is not directly possible with Singularity

  • In order to inspect information about the docker container, including the label with the description of the HEP workload included

    • docker inspect gitlab-registry.cern.ch/hep-benchmarks/hep-workloads/cms-gen-sim:latest
    • This is not directly possible with Singularity