Skip to content

cwl: HTCondor compute backend example

Tibor Simko requested to merge github/fork/tiborsimko/cwl-htc into master

The first step is OK, the second not:

$ reana-client status -w roofit-cwl-htc  
NAME             RUN_NUMBER   CREATED               STARTED               ENDED                 STATUS     PROGRESS
roofit-cwl-htc   2            2021-02-18T19:17:39   2021-02-18T19:17:39   2021-02-18T19:21:58   finished   1/2     

$ reana-client logs -w roofit-cwl-htc.2
("Error collecting output for parameter 'data':\nworkflow.json:1:1051: Did not find output file with glob pattern: '['data.root']'", {})
2021-02-18 19:21:58,079 | reana-workflow-engine-cwl | gendata | ERROR | [job gendata] workflow job error:
("Error collecting output for parameter 'data':\nworkflow.json:1:1051: Did not find output file with glob pattern: '['data.root']'", {})
ERROR [step gendata] Output is missing expected field file:///var/reana/users/aaaa-bbbb-cccc-dddd/workflows/aaaa-bbbb-cccc-dddd/workflow.json#main/gendata/data
2021-02-18 19:21:58,079 | cwltool | gendata | ERROR | [step gendata] Output is missing expected field file:///var/reana/users/aaaa-bbbb-cccc-dddd/workflows/aaaa-bbbb-cccc-dddd/workflow.json#main/gendata/data
WARNING [step gendata] completed permanentFail
2021-02-18 19:21:58,081 | cwltool | gendata | WARNING | [step gendata] completed permanentFail
INFO [workflow ] completed permanentFail
2021-02-18 19:21:58,744 | cwltool | MainThread | INFO | [workflow ] completed permanentFail
WARNING Final process status is permanentFail
2021-02-18 19:21:58,756 | cwltool | MainThread | WARNING | Final process status is permanentFail
workflow done

$ reana-client ls -w roofit-cwl-htc.2               
NAME                                     SIZE     LAST-MODIFIED      
.job.ad                                  6723     2021-02-18T19:21:59
inputs.json                              139      2021-02-18T19:21:59
.machine.ad                              7003     2021-02-18T19:21:59
docker_stderror                          1120     2021-02-18T19:21:59
reana_job.5613803.0.err                  0        2021-02-18T19:21:59
job_wrapper.sh                           511      2021-02-18T19:21:59
.update.ad                               7037     2021-02-18T19:21:59
_condor_stderr                           0        2021-02-18T19:21:59
workflow.json                            2132     2021-02-18T19:21:59
.chirp.config                            49       2021-02-18T19:21:59
reana_job.5613803.0.out                  2712     2021-02-18T19:22:00
condor_exec.exe                          511      2021-02-18T19:21:59
_condor_stdout                           2712     2021-02-18T19:21:59
code/gendata.C                           1937     2021-02-18T19:21:59
code/fitdata.C                           1648     2021-02-18T19:21:59
cwl/docker_outdir/.root_hist             67       2021-02-18T19:21:59
cwl/docker_outdir/gendata.C              1937     2021-02-18T19:21:59
cwl/docker_outdir/data.root              154590   2021-02-18T19:21:59
workflow/cwl/fitdata.cwl                 571      2021-02-18T19:21:59
workflow/cwl/workflow-htcondorcern.cwl   980      2021-02-18T19:21:59
workflow/cwl/input.yml                   116      2021-02-18T19:21:59
workflow/cwl/gendata.cwl                 566      2021-02-18T19:21:59
workflow/cwl/workflow.cwl                811      2021-02-18T19:21:59
cwl/outdir/c10xpedt/gendata.C            1937     2021-02-18T19:21:59
cwl/outdir/c10xpedt/data.root            154590   2021-02-18T19:21:59

Step's I/O to be checked.

Merge request reports