Status Report on Archive and Retrieve workflow implementations with Relational DB scheduler

This ticket is describing the strategy and status of the progress in implementing the Postgres DB backend for CTA.

Starting point:

stateDiagram-v2
    direction LR
    classDef whitesystem fill:white
    classDef needsreview fill:#ff8c1a
    classDef allok fill:#66b3ff
    classDef almostimplemented fill:yellow
    Scheduler\n(cta/scheduler):::allok --> SchedulerDatabase\n(cta/scheduler):::allok
    OStoreDB\n(cta/scheduler/OStoreDB):::needsreview --> objectstore\n(cta/objectstore):::needsreview
    SchedulerDatabase\n(cta/scheduler) --> OStoreDB\n(cta/scheduler/OStoreDB)

• ensuring compatibility with Scheduler:
  • copy of OStoreDB methods put in cta/scheduler/PostgresSchedDB making them throw exceptions
• useful strategy to learn/review all the bits from the very start (i.e. neither taped not ctafrontend start up)
• to start running the system, step by step approach: crash --> implement --> test

Midterm goal (end of 2024):

• functional PostgresSched backend

stateDiagram-v2
    direction LR
    classDef whitesystem fill:white
    classDef needsreview fill:#ff8c1a
    classDef allok fill:#66b3ff
    classDef almostimplemented fill:yellow
    Scheduler\n(cta/scheduler):::allok --> SchedulerDatabase\n(cta/scheduler):::allok
    PostgresSchedDB\n(cta/scheduler/PostgresSchedDB):::allok --> rbms\n(cta/rdbms):::allok
    SchedulerDatabase\n(cta/scheduler) --> PostgresSchedDB\n(cta/scheduler/PostgresSchedDB):::allok

Our longterm target:

• re-think and re-implement part of the Scheduler and SchedulerDatabase (+ refactor the PostgredsSchedDB code) to exploit the advantages of Postgres DB and remove biases in scheduling introduced by objectstore

stateDiagram-v2
    direction LR
    classDef allok fill:#5cd65c
    Scheduler\n(cta/scheduler):::allok --> SchedulerDatabase\n(cta/scheduler):::allok
    PostgresSchedDB\n(cta/scheduler/PostgresSchedDB):::allok --> rbms\n(cta/rdbms):::allok
    SchedulerDatabase\n(cta/scheduler) --> PostgresSchedDB\n(cta/scheduler/PostgresSchedDB):::allok

Current development:

• complies with the skeleton hook
• no changes/improvements to the Scheduler or SchedulerDatabase unless necessary (creating tickets with these for the future)