Skip to content
GitLab
Projects Groups Topics Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
  • Sign in
  • CTA CTA
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributor statistics
    • Graph
    • Compare revisions
    • Locked files
  • Issues 133
    • Issues 133
    • List
    • Boards
    • Service Desk
    • Milestones
    • Iterations
    • Requirements
  • Merge requests 15
    • Merge requests 15
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Artifacts
    • Schedules
    • Test cases
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Package Registry
    • Container Registry
    • Terraform modules
    • Model experiments
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • CI/CD
    • Code review
    • Insights
    • Issue
    • Repository
  • Wiki
    • Wiki
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar

Admin message

GitLab Runners cleaning campaign ongoing (ETA 2nd October 2023). Check https://cern.ch/otg0078219 for further information.

  • ctacta
  • CTACTA
  • Issues
  • #40
Closed
Open
Issue created Feb 24, 2019 by Eric Cano@canoMaintainer

Harmonize retrieve/recall reporting interface for success.

We now have two scenarios for retrieve reporting (repack and user transfer).

The reporting happens in queued report jobs RecallReportPacker::ReportSuccessful::execute(), itself executed by RecallReportPacker::WorkerThread::run()

Things are now finalized in 2 separate ways:

  • For user transfers, the request is asynchronously deleted (by RecallReportPacker::ReportSuccessful::execute() and a chain of direct wrappers). Then when the deletion result is waited for in cta::RetrieveMount::waitAndFinishSettingJobsBatchRetrieved() indirectly through the SchedulerDatabase::RetrieveJob interface, and finally deleted from ownership in SchedulerDatabase::finishSettingJobsBatchSuccessful(),
  • For repack jobs, nothing is actually done at RecallReportPacker::ReportSuccessful::execute() time, and then all is done in OStoreDB::RetrieveMount::batchSucceedRetrieveForRepack(). We first update the status (OStoreDB::RetrieveJob::asyncReportSucceedForRepack()), then wait() for the result, and finally queue the requests in the proper queue for reporting (currently vid of the mount, to be changed to the repack request named queue).

The logic is currently scattered at many level, but the lower level could take up all the value added, and we should move all the user/repack switching to the SchedulerDatabase level. The higher levels just need to report the success (to trigger the async update or delete of the request, which will ensure no unnecessary retry), and then a single flush() call should be done on the SchedulerDatabase::RetrieveMount to wait() for both deletions and updates, queue jobs where necessary (repack) and them remove jobs from ownership.

This will update the requests earlier for repack and simplify the code in general.

Assignee
Assign to
Time tracking