Create base images for CI: no installs in GitLab pipeline

Currently, we are doing various yum install ... commands in our CI pipeline. The common convention is to use custom Docker images that come pre-installed with the necessary packages: don't do at runtime what you can do at build time. There are a few good reasons for doing this:

• It is easier to run a pipeline from the past, as your Docker image is essentially a time capsule of what the packages were at that time. If you re-run previous commits now, it might install newer versions of various packages and break things. If we have these self-contained, immutable Docker images we don't need to worry about this.
• It saves time on various jobs in the pipeline, as the jobs no longer have to do the same install every time. For context: this would save about 2 minutes on the build jobs alone
• It allows us to always use images that work for our pipeline. Right now, we are experiencing a failure in creating the debug packages due to an upgrade to the alma9 images. As we are using the latest tag, this automatically updated the image in use. In this particular case, the pipelines still run fine(ish) in the sense that they don't fail, but a more serious problem would essentially block all development until resolved.

So how can we go about this without introducing much more complexity and/or work on the development side? The idea is as follows:

• We have a single separate repository that builds all the images we need in the pipeline. The images are all relatively simple, so a single Dockerfile per image should be enough (e.g. cppcheck.Dockerfile, build.Dockerfile, image-base.Dockerfile)
• Once every week (or more/less frequently) we build each of these images, and trigger the CTA pipeline against the newly build images.
• If the triggered pipeline passes, the parent pipeline will create a commit in the CTA repository to update the version of the images.
• If the triggered pipeline fails, a developer goes in and fixes it. This way, a breaking change does not block the rest of the developers.

The alternative would be to tag the latest working versions as e.g. stable and use this in the CTA pipeline (instead of explicitly specifying a version). The downside is that this tag is not immutable, so going back to previous commits means you will have to replace stable with the commit at that time manually.

This does not take a long time to set up and only needs to be set up once. Other than that, it doesn't cost developers any extra time (except maybe having to rebase an additional commit every once in a while when the image is updated). As a matter of fact, developers will gain time from this as the time to feedback in the pipeline is reduced.