capture the output printed by singularity stream command
This MR allows capturing the error output when the command which has been run in singularity
fails.
In fact, while running singularity's execute
command with stream=True
, if there is an error, it's printed to sys.stderr
, and subprocess.CalledProcessError
is raised, but only with return code and command (output
, stdout
, and stderr
are not set), see: https://github.com/singularityhub/singularity-cli/blob/64358eb33eee6a399461bb0ddeec4951fc01b415/spython/utils/terminal.py#L154-L159.
The effect for us was that if e.g. make configure
fails, there is no information about the error exposed to the frontend (it's only in the logs sent to task logfile): https://lhcb-nightlies.web.cern.ch/nightly/celery-test/42/LHCb/x86_64_v2-centos7-gcc10-opt/build.