Alma 9 CI runners
As we need a streamlined way to instantiate more runners I have started to work on putting together what we need and how to get there.
Indeed several tickets about CI lead to complicated setup that were complicated to maintain and most of the time they completely failed...
Here is a summary of all my current conclusions on what we should target as a main platform:
- a recent linux distribution supported by CERN and WLCG: Alma 9
- a recent kubernetes version that can be easily installed by other sites:
minikube
withpodman
driver andcri-o
container runtime - an easy way to install mhvtl on alma 9 and a recent kernel: here is what I generated with @jcamarer mhvtl-utils packaging and a few tweaks: cta-dependencies#4 (comment 6303912)
- a simple minimal installation environment: bare openstack without puppet is the way to go for runners
- this minimal environment allows us to install on a base alma9 VM provided by any virtualization software on any site
Impact on CI (Already fixed)
We will need to adapt kubectl
calls to a more modern kubernetes version as some of our scripts use deprecated kubectl get pod -a
: changes should be minimal but we need to sort out transition from old runners to new ones.
CI runner configuration
Runs in user mode under cirunner
local account in tape
group.
Puppet won't run on these nodes: we need some mechanisms to populate various configurations:
- GITLAB API TOKEN: to pull from registry, these TOKEN are expiring and need to be updated when they change
- ORACLE DB credentials: same pain: we need to change developer DB access every year (or go to postgres?)
- OBJECTSTORE credentials: runner specific
Could rundeck
distribute all this? How? When (like upon DB password change in teigi
)?