Collect metadata field from the HTTP TPC command
@smurray has deployed a simple test of the HTTP Archive Metadata feature in gfal2, for us to use in our development and experimentation.
From now on, all HTTP requests sent from fts3-atlas.cern.ch to EOSCTA instances will contain the following Archive Metadata HTTP Header:
- TransferMetadata.
This is documented in the following FTS Jira ticket:
Example from the point of view of an fts3-atlas.cern.ch machine, as explained by Steve:
[root@fts-atlas-001 ~]# gfal-stat -vvv https://cern.ch/test | grep TransferMetadata
> TransferMetadata: eyJrZXkiOiAibXkgbWV0YWRhdGEifQ==
> TransferMetadata: eyJrZXkiOiAibXkgbWV0YWRhdGEifQ==
> TransferMetadata: eyJrZXkiOiAibXkgbWV0YWRhdGEifQ==
gfal-stat error: 2 (No such file or directory) - [gfal2_stat][gfal_plugin_statG][gfal_http_stat] Result HTTP 404 : File not found after 1 attempts
[root@fts-atlas-001 ~]# echo eyJrZXkiOiAibXkgbWV0YWRhdGEifQ== | base64 --decode; echo
{"key": "my metadata"}
Even though the metadata structure is not yet fully defined, we are mostly interested in the DataSet (or similar) field.
We need to discuss how to store this on the CTA side.
Edited by Joao Afonso