Skip to content

Allow cta-admin stream command to print one independent json object per output record

Problem to solve

At the moment, when running a cta-admin --json <cmd> ls command, we always output a single JSON object (one single array with all entries).

This makes it more difficult for operator tools to parse it's output, because they need to way for the full JSON object to be flushed and finalized, instead of being able to parse the entries one-by-one as they are printed to stdout.

For small lists this is not a problem, but some commands (for example tapefile ls) can output millions of lines.

Example:

$ cta-admin --json vo ls
[
  {
    "name": "ATLAS",
    "creationLog": {
      "username": "jleduc",
      "host": "eosctafst0017.cern.ch",
      "time": "1593464175"
    },
    "lastModificationLog": {
      "username": "idlarbi",
      "host": "ctapreproductionfrontend11",
      "time": "1739458822"
    },
    "comment": "ATLAS",
    "readMaxDrives": "2",
    "writeMaxDrives": "2",
    "maxFileSize": "128849018880",
    "diskinstance": "eosctaatlaspps",
    "isRepackVo": false,
    "instanceName": "preproduction"
  },
  {
    "name": "IT",
    "creationLog": {
      "username": "jleduc",
      "host": "eosctafst0017.cern.ch",
      "time": "1593464175"
    },
    "lastModificationLog": {
      "username": "idlarbi",
      "host": "ctapreproductionfrontend11",
      "time": "1752229639"
    },
    "comment": "IT",
    "readMaxDrives": "2",
    "writeMaxDrives": "3",
    "maxFileSize": "20000000000000",
    "diskinstance": "eosctarepack",
    "isRepackVo": false,
    "instanceName": "preproduction"
  }
]

Stakeholders

CTA operator tools.

Proposal

Add a new --jsonl option which, instead of printing a single json file, prints each line as a separate json object.

Example:

cta-admin --jsonl vo ls
{
  "name": "ATLAS",
  "creationLog": {
    "username": "jleduc",
    "host": "eosctafst0017.cern.ch",
    "time": "1593464175"
  },
  "lastModificationLog": {
    "username": "idlarbi",
    "host": "ctapreproductionfrontend11",
    "time": "1739458822"
  },
  "comment": "ATLAS",
  "readMaxDrives": "2",
  "writeMaxDrives": "2",
  "maxFileSize": "128849018880",
  "diskinstance": "eosctaatlaspps",
  "isRepackVo": false,
  "instanceName": "preproduction"
}
{
  "name": "IT",
  "creationLog": {
    "username": "jleduc",
    "host": "eosctafst0017.cern.ch",
    "time": "1593464175"
  },
  "lastModificationLog": {
    "username": "idlarbi",
    "host": "ctapreproductionfrontend11",
    "time": "1752229639"
  },
  "comment": "IT",
  "readMaxDrives": "2",
  "writeMaxDrives": "3",
  "maxFileSize": "20000000000000",
  "diskinstance": "eosctarepack",
  "isRepackVo": false,
  "instanceName": "preproduction"
}
Edited by Joao Afonso