Skip to content

capture the output printed by singularity stream command

Maciej Pawel Szymanski requested to merge maszyman-singularity-stderr into master

This MR allows capturing the error output when the command which has been run in singularity fails.

In fact, while running singularity's execute command with stream=True, if there is an error, it's printed to sys.stderr, and subprocess.CalledProcessError is raised, but only with return code and command (output, stdout, and stderr are not set), see: https://github.com/singularityhub/singularity-cli/blob/64358eb33eee6a399461bb0ddeec4951fc01b415/spython/utils/terminal.py#L154-L159.

The effect for us was that if e.g. make configure fails, there is no information about the error exposed to the frontend (it's only in the logs sent to task logfile): https://lhcb-nightlies.web.cern.ch/nightly/celery-test/42/LHCb/x86_64_v2-centos7-gcc10-opt/build.

Merge request reports