Simple safe tool for EOS NS injection
Problem to solve
Since the removal of the CASTOR migration tools we need to propose a simple, safe and generic workflow to inject files in the EOS side of a EOSCTA instance to address upcoming migrations (RAL for example as asked in the community) or the following production use case: move files from one eoscta instance to another only migrating metadata.
The start point of this tool: the CTA catalogue part is correctly populated with consistent information for the tape side. We could decide as a convention and for simplicity that DISK_INSTANCE_NAME
is set to eosctabogus
, eosctaincoming
... something that is not a valid production eoscta instance and/or DISK_FILE_ID
is set to a specific string: something like INJECTING
?.
This tool will then take in input a json file: one line per object file that needs to be injected (no JSON array). Each file object contains all the EOS metadata and the ARCHIVE_FILE_ID
.
The tool will then:
- check that that candidate file for injection is valid from the CTA catalogue point of view: size and checksum for the
ARCHIVE_FILE_ID
is the consistent - check that the candidate DISK instance exists
- create a new file in the eos instance and inject consistent information for this file in the CTA catalogue (all the missing/inherited information) notably the
DISK_INSTANCE_NAME
and theDISK_FILE_ID
that was generated by EOS in the previous step
This tool should avoid dangerous constraints that we do not have: there is no point allowing the end user to provide the DISK_FILE_ID
.
It must be idempotent by design.
Some more or less open questions:
Directory tree creation responsibility?
This could easily be done by site CTA operations team ingesting the same JSON to create all the needed directories with the correct ACLs,... To keep this simple we should leave this outside of the scope of this tool as a prerequisite before using it: there are too many constraints on dir attributes not too tie them with this tool.
If the target directory for injection does not exist -> fail with ERROR: dir XXX does not exist as early as possible
.