Update scheduler to use file creation time provided by frontend
Current CTA scheduler implementation sorts the request according their submission time. This approach works well when EOS is used as the frontend storage system: the scheduler creation timestamp matches file's creation time. However, when CTA is used with dCache, the scheduler creation time matches only timestamp at which dCache have decided to flush the file. However, when flushing the scheduler still should respect file creation time. Moreover, as single dCache pool might submit multiple requests at once, then multiple cta-taped
processes might decide to flush files from a single host, thus stature the network or disk IO on that host.
To overcome this limitation, the scheduler interface should be update to accept file's creation time provided by front-end and use it when request selection is made.
It looks like that the most of CTA code is already prepared for that, except two small steps:
-
update DiskFileInfo
to include creation timestamp -
update implementations of SchedulerDatabase#queueArchive
to use file creation timestamp when populatingArchiveFile#creationTime
field
If no objections, we are happy contribute this changes.