Skip to content

Restructure continuousintegration/ directory

Currently we have the following directory structure:

📂 continuousintegration/
 ┣ 📂 ci_helpers/
 ┃ ┣ 📂 release/
 ┃ ┣ 📂 tape/
 ┣ 📂 ci_runner/
 ┃ ┣ 📂 development/
 ┣ 📂 docker/
 ┃ ┣ 📂 ctafrontend/
 ┣ 📂 orchestration/

There are a few problems with this structure:

  • It uses CTA CI specific terms, making it more difficult to find where things are if you are not already familiar with this
  • The distinction between ci_runner and ci_helpers is rather arbitrary. There are even scripts there that are not necessary ci_runner or ci_helper specific
  • docker has a subdirectory ctafrontend, even though it builts an image that does much more than just the ctafrontend (this is a leftover from the earlier days to my understanding` This has been fixed by #941 (closed)
  • The tests are not properly grouped; there is basically just a bunch of test files in the tests/ directory. Some of them spawn other tests using kubectl commands and some of them are expected to be run within a pod itself (This one is for later)
  • The continuousintegration name is long, tedious to write, clutters the screen; basically needlessly verbose. At some point we can rename this to ci, but I would not do that now, so we leave this.

To fix this, we should restructure things a bit. The first step would be to revisit ci_helpers and ci_runner and collapse docker/ctafrontend/ into docker/:

📂 continuousintegration/
 ┣ 📂 build/
 ┃ ┣ 📂 development/
 ┣ 📂 docker/
 ┣ 📂 orchestration/
 ┣ 📂 release/
 ┣ 📂 utils/
 ┃ ┣ 📂 tape/
 ┣ 📂 validation/
 ┣ 📜 build_deploy.sh
 ┣ 📜 README.md

Using git mv we can easily do this without losing/altering the Git history; git blame still works fine after this. See e.g. here: file was moved but both blame and the history are intact


Explanation from the new README.md:


This directory contains all the files necessary for development and automation workflows, including build scripts, container configurations, orchestration tools, release processes, and utility scripts for the CI pipeline.

  • build/: Files for building the CTA software
  • docker/: Docker files and content to build the docker images
  • orchestration/: Files to set up a local development cluster and all the tests that can run on this.
  • release/: Scripts used by the CI pipeline when doing a new release of the CTA software
  • utils/: Collection of utility scripts
  • validation/: Collection of scripts that perform validation checks within the CI pipeline
  • build_deploy.sh: The main script used for development: builds the project, the corresponding Docker image and deploys a local CTA test instance. See the docs on development commands for more details.
Edited by Niels Alexander Buegel