Make CI pipeline run for CTA with PostgreSQL Scheduler DB

With the development of PostgreSQL Scheduler DB we need to test its functionality to catch missing parts and bugs. Creating a runner in CI will make these tests run consistently for every change.

added Building and Packaging Continuous Integration workflowIn Progress labels

assigned to @guenther

changed milestone to %CI update push

WIP in branch: https://gitlab.cern.ch/cta/CTA/-/commits/ci_changes_sched

Running pipeline manually with variables: ORACLE_SUPPORT: OFF SCHED_TYPE: pgsched

Results in wrong CMAKE_OPTIONS (added echo to see the options in the output below).

[...]
$ echo ${CMAKE_OPTIONS}
-DDISABLE_ORACLE_SUPPORT:BOOL=ON
$ cmake3 -DPackageOnly:Bool=true -DVCS_VERSION=${CTA_BUILD_ID} ${CMAKE_OPTIONS} ..
[...]

When fixing this in .gitlab-ci.yml, one can see the CI compiling with the (almost) desired combination:

$ echo ${CMAKE_OPTIONS}
-DSKIP_UNIT_TESTS:STRING=1 -DCTA_USE_PGSCHED:Bool=true -DDISABLE_ORACLE_SUPPORT:BOOL=ON
$ cmake3 -DPackageOnly:Bool=true -DVCS_VERSION=${CTA_BUILD_ID} ${CMAKE_OPTIONS} ..

https://gitlab.cern.ch/cta/CTA/-/jobs/30145636

While the default is unchanged:

$ echo ${CMAKE_OPTIONS}
$ cmake3 -DPackageOnly:Bool=true -DVCS_VERSION=${CTA_BUILD_ID} ${CMAKE_OPTIONS} ..

https://gitlab.cern.ch/cta/CTA/-/jobs/30145526

Why do we disable compiling the unit tests for PostgreSQL in the CI ? https://gitlab.cern.ch/cta/CTA/-/blob/main/.gitlab-ci.yml#L34

The PostgreSQL code does not compile (WIP): https://gitlab.cern.ch/cta/CTA/-/jobs/30145737

Partly due to this commit which changed the signature of few methods in the scheduler/SchedulerDatabase.hpp: c192c8ef

Fixed the PostgreSQL part --> !261 (merged)

changed title from Create CI runner for testing of CTA with PostgreSQL Scheduler DB to Make CI pipeline run for CTA with PostgreSQL Scheduler DB

created branch 383-create-ci-runner-for-testing-of-cta-with-postgresql-scheduler-db to address this issue

created branch 383-make-ci-pipeline-run-for-cta-with-postgresql-scheduler-db to address this issue

mentioned in merge request !247 (merged)

David's code compiles again, merged: !356 (merged)

PGSCHED pipelines:

• default CTA unitTests need to be adjusted to be run with PGSCHED: https://gitlab.cern.ch/cta/CTA/-/pipelines/6257712
• disabled unit tests: https://gitlab.cern.ch/cta/CTA/-/pipelines/6257951 `/opt/run/bin/init.sh: line 58: cta-objectstore-initialize: command not found

TO-DO-LIST:

• creating pgscheddb container in CI
• create pgscheddb initializing executable (to create schema, cleanup, etc.)
• adapt some unit tests to run using pgsched
• write unit tests for existing pgsched methods
• reiterate on the basic schema design
• code has ~120 methods marked as 'not implemented'  --> to be implemented as we go forward

mentioned in merge request !378 (merged)

added 1 deleted label

Instance stress successfully created:
NAME             READY     STATUS      RESTARTS   AGE
client           1/1       Running     0          3m
ctacli           1/1       Running     0          3m
ctaeos           1/1       Running     0          3m
ctafrontend      1/1       Running     0          3m
init             0/1       Completed   0          5m
kdc              1/1       Running     0          3m
postgres         1/1       Running     0          5m
postgres-sched   1/1       Running     0          5m
tpsrv01          2/2       Running     0          3m
tpsrv02          2/2       Running     0          3m

pgsched DB container is running in CI, create schema command debugged and deployed.

changed iteration to EOY 2023 targets Nov 21, 2023 - Dec 22, 2023

Merged: !378 (merged)

TO-DO:

• created a related issue in order to optimise the workload on the development #554 (closed)
• tests of the pgsched container DB and see where the basic file archival fails

created branch 383-make-ci-pipeline-run-for-cta-with-postgresql-scheduler-db-2 to address this issue

mentioned in merge request !411 (merged)