Skip to content

Enhance CI tests using multiple specialized runners for parallel jobs

David-Leon Pohl requested to merge enhance_ci into development

Example pipeline output here

Enhancements:

  • Unit tests are now part of the bdaq53 package

  • Run 3 jobs in parallel with different tasks using 3 runners:

    imageimage

    1. Specific RD53 runner with RD53 & FPGA co-simulation. This runner is used when for jobs specifying the rd53 tag.
    2. CERN shared runners for testing software. This runner uses a Miniconda docker for faster setup. Software unit tests itself are not ran in parallel, since the job did not terminate when this was tried... This runner is used for jobs specifying the software tag and is unfortunately rather slow, due to insufficient computational time / data IO provided by CERN. The benefit is that many commits can run this runner in parallel.
    3. Specific hardware runner for tests requiring bdaq53 readout hardware + RD53 front-end. This runner is bound to this project, thus cannot be used in forks. This runner is used for jobs specifying the hardware tag.
    4. Specific runner in CERN VM with privileged mode true to allow docker-in-docker for code climate CI. PEP8 is tested now.

Breaking changes:

  • New (global) project configuration: with old CI configuration the jobs cannot be assigned to specific runners, meaning that failures could occur when e.g. the hardware tests are run on the software runner. Thus this MR should be merged first.

For reference:

Setting up a hardware runner is as easy as just typing these commands into the terminal (tested with Ubuntu):

sudo apt install docker.io
curl -L https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh | sudo bash
sudo apt-get install gitlab-runner
sudo gitlab-runner register

The register step needs to be filled with the project specific settings as defined here. The tag should be set to hardware to only use this runner when hardware is needed.

Edited by David-Leon Pohl

Merge request reports