Improvements on the 'cta-restore-deleted-files' tool: more than one recycle bin file matches the parameters passed
In the Alice restore, when trying to restorethis file: archiveFileId="4477705639" I was hit by this:
cta-restore-deleted-files --id 4477705639
Aug 8 08:35:49.833568 ctaproductionfrontend02.cern.ch cta-restore-deleted-files: LVL="INFO" PID="16992" TID="16992" MSG="Listing deleted file in CTA catalogue" userName="root" archiveFileId="4477705639"
Aug 8 08:35:55.120230 ctaproductionfrontend02.cern.ch cta-restore-deleted-files: LVL="INFO" PID="16992" TID="16992" MSG="Listed deleted file in CTA catalogue" userName="root" archiveFileId="4477705639" nbFiles="2"
Aug 8 08:35:55.159568 ctaproductionfrontend02.cern.ch cta-restore-deleted-files: LVL="INFO" PID="16992" TID="16992" MSG="Restoring file in the EOS namespace" userName="root" diskInstance="eosctaalice" archiveFileId="4477705639" diskFileId="4393021900" diskFilePath="/eos/ctaalice/archive/alice/t0alice/12/18002/872468eb-46ad-11ed-b37f-3cecef03e986"
More than one recycle bin file matches the parameters passed
Indeed, looking in the recycle bin:
[
{
"vid": "L89099",
"fseq": "3752",
"blockId": "29552169",
"copyNb": 1,
"tapeFileCreationTime": "1669171103",
"archiveFileId": "4477705639",
"diskInstance": "eosctaalice",
"diskFileId": "4393021900",
"diskFileIdWhenDeleted": "4393021900",
"diskFileUid": "10367",
"diskFileGid": "1395",
"sizeInBytes": "2123516317",
"checksum": [
{
"type": "ADLER32",
"value": "e565d0ee"
}
],
"storageClass": "aliceraw",
"archiveFileCreationTime": "1669171103",
"reconciliationTime": "1669171103",
"collocationHint": "",
"diskFilePath": "",
"reasonLog": "REPACK",
"recycleLogTime": "1669188012"
},
{
"vid": "L89099",
"fseq": "17208",
"blockId": "32554187",
"copyNb": 1,
"tapeFileCreationTime": "1669188012",
"archiveFileId": "4477705639",
"diskInstance": "eosctaalice",
"diskFileId": "4393021900",
"diskFileIdWhenDeleted": "4393021900",
"diskFileUid": "10367",
"diskFileGid": "1395",
"sizeInBytes": "2123516317",
"checksum": [
{
"type": "ADLER32",
"value": "e565d0ee"
}
],
"storageClass": "aliceraw",
"archiveFileCreationTime": "1669171103",
"reconciliationTime": "1669171103",
"collocationHint": "",
"diskFilePath": "/eos/ctaalice/archive/alice/t0alice/12/18002/872468eb-46ad-11ed-b37f-3cecef03e986",
"reasonLog": "File deleted by aliprod from the eosctaalice instance",
"recycleLogTime": "1690295242"
}
]
There are 2 files with the exact same caracteristics: same archiveId, same tape, diskfileID...
There is no way I can choose one of these for restore: to uniquely identify the tape file, we need to be able to add the vid and fseq.
In addition these files have been restored in the eos namespace but not in CTA catalogue tf. It means that user cannot stage them...
The restore tool should check if tf can be restored before restoring from EOS...
We need a better way to classify first and then inject in EOS all the files correctly restored in CTA.
In addition what is this REPACK file with no diskFilePath?? This is not something we can restore for the end user.
cta-restore-deleted-files
tool:
Requested improvements to the -
Optionally, be able to pass the vid
and/orfseq
, so that it can uniquely identify the tape file. -
Do not restore a file in the eos namespace if it can't be successfully recovered to the CTA catalogue TAPE_FILE
table. -
Guarantee that all files have a diskFilePath
. Otherwise, we cannot recover them.