make sure a test fails if its reference file is not found

Before this, if a reference file was mentionned in a test but not found during the validation, the test was successful. Now it will fail with reason "Reference file not found" Note that in case no reference file is given, the test will still succeed

• [2018-08-16 16:21] Validation started with lhcb-gaudi-merge#558
• [2018-08-17 00:04] Validation started with lhcb-lcg-dev3#620
• [2018-08-17 00:06] Validation started with lhcb-lcg-dev4#619
• [2018-08-17 00:09] Validation started with lhcb-gaudi-head#1964
• [2018-08-17 00:10] Validation started with lhcb-tdr-test#258
• [2018-08-18 00:04] Validation started with lhcb-lcg-dev3#621
• [2018-08-18 00:06] Validation started with lhcb-lcg-dev4#620
• [2018-08-18 00:09] Validation started with lhcb-tdr-test#259
• [2018-08-18 00:17] Validation started with lhcb-gaudi-head#1965
• [2018-08-19 00:04] Validation started with lhcb-lcg-dev3#622
• [2018-08-19 00:05] Validation started with lhcb-gaudi-head#1966
• [2018-08-19 00:05] Validation started with lhcb-lcg-dev4#621
• [2018-08-19 00:06] Validation started with lhcb-tdr-test#260
• [2018-08-20 00:07] Validation started with lhcb-lcg-dev4#622
• [2018-08-20 00:07] Validation started with lhcb-gaudi-head#1967
• [2018-08-20 00:08] Validation started with lhcb-lcg-dev3#623
• [2018-08-20 00:10] Validation started with lhcb-tdr-test#261
• [2018-08-21 00:05] Validation started with lhcb-lcg-dev4#623
• [2018-08-21 00:05] Validation started with lhcb-lcg-dev3#624
• [2018-08-21 00:07] Validation started with lhcb-tdr-test#262
• [2018-08-21 00:09] Validation started with lhcb-gaudi-head#1968
• [2018-08-21 16:40] Validation started with lhcb-lcg-dev4#624
• [2018-08-22 00:04] Validation started with lhcb-lcg-dev4#625
• [2018-08-22 00:07] Validation started with lhcb-lcg-dev3#625
• [2018-08-22 00:08] Validation started with lhcb-tdr-test#263
• [2018-08-22 00:10] Validation started with lhcb-gaudi-head#1969
• [2018-08-23 00:06] Validation started with lhcb-lcg-dev4#626
• [2018-08-23 00:07] Validation started with lhcb-lcg-dev3#626
• [2018-08-23 00:09] Validation started with lhcb-tdr-test#264
• [2018-08-23 00:10] Validation started with lhcb-gaudi-head#1970
• [2018-08-24 00:04] Validation started with lhcb-lcg-dev3#627
• [2018-08-24 00:05] Validation started with lhcb-lcg-dev4#627
• [2018-08-24 00:10] Validation started with lhcb-gaudi-head#1971
• [2018-08-24 00:15] Validation started with lhcb-tdr-test#265
• [2018-08-25 00:03] Validation started with lhcb-lcg-dev3#628
• [2018-08-25 00:03] Validation started with lhcb-lcg-dev4#628
• [2018-08-25 00:07] Validation started with lhcb-tdr-test#266
• [2018-08-25 00:09] Validation started with lhcb-gaudi-head#1972
• [2018-08-26 00:04] Validation started with lhcb-gaudi-head#1973
• [2018-08-26 00:05] Validation started with lhcb-tdr-test#267
• [2018-08-26 00:05] Validation started with lhcb-lcg-dev4#629
• [2018-08-26 00:06] Validation started with lhcb-lcg-dev3#629
• [2018-08-27 00:03] Validation started with lhcb-lcg-dev3#630
• [2018-08-27 00:05] Validation started with lhcb-lcg-dev4#630
• [2018-08-27 00:06] Validation started with lhcb-gaudi-head#1974
• [2018-08-27 00:07] Validation started with lhcb-tdr-test#268
• [2018-08-27 02:06] Validation started with lhcb-lcg-dev3#630
• [2018-08-27 15:23] Validation started with lhcb-lcg-dev4#631
• [2018-08-28 00:03] Validation started with lhcb-lcg-dev3#631
• [2018-08-28 00:03] Validation started with lhcb-lcg-dev4#632
• [2018-08-28 00:06] Validation started with lhcb-tdr-test#269
• [2018-08-28 00:06] Validation started with lhcb-gaudi-head#1975
• [2018-08-29 00:06] Validation started with lhcb-lcg-dev4#633
• [2018-08-29 00:08] Validation started with lhcb-lcg-dev3#632
• [2018-08-29 00:12] Validation started with lhcb-tdr-test#270
• [2018-08-29 00:12] Validation started with lhcb-gaudi-head#1976
• [2018-08-30 00:21] Validation started with lhcb-tdr-test#271
• [2018-08-30 00:21] Validation started with lhcb-lcg-dev4#634
• [2018-08-30 00:21] Validation started with lhcb-gaudi-head#1977
• [2018-08-30 00:21] Validation started with lhcb-lcg-dev3#633
• [2018-08-31 00:05] Validation started with lhcb-lcg-dev4#635
• [2018-08-31 00:05] Validation started with lhcb-lcg-dev3#634
• [2018-08-31 00:08] Validation started with lhcb-gaudi-head#1978
• [2018-08-31 00:15] Validation started with lhcb-tdr-test#272
• [2018-08-31 10:35] Validation started with lhcb-gaudi-head#1979
• [2018-08-31 15:07] Validation started with lhcb-lcg-dev4#636
• [2018-09-01 00:05] Validation started with lhcb-lcg-dev3#635
• [2018-09-01 00:05] Validation started with lhcb-lcg-dev4#637
• [2018-09-01 00:08] Validation started with lhcb-tdr-test#273
• [2018-09-01 00:11] Validation started with lhcb-gaudi-head#1980
• [2018-09-01 01:53] Validation started with lhcb-lcg-dev4#637
• [2018-09-01 02:12] Validation started with lhcb-gaudi-head#1980
• [2018-09-02 00:06] Validation started with lhcb-lcg-dev3#636
• [2018-09-02 00:06] Validation started with lhcb-gaudi-head#1981
• [2018-09-02 00:06] Validation started with lhcb-lcg-dev4#638
• [2018-09-02 00:08] Validation started with lhcb-tdr-test#274
• [2018-09-02 13:59] Validation started with lhcb-lcg-dev4#639
• [2018-09-03 00:05] Validation started with lhcb-gaudi-head#1982
• [2018-09-03 00:06] Validation started with lhcb-lcg-dev4#640
• [2018-09-03 00:07] Validation started with lhcb-lcg-dev3#637
• [2018-09-03 00:10] Validation started with lhcb-tdr-test#275
• [2018-09-04 00:05] Validation started with lhcb-lcg-dev3#638
• [2018-09-04 00:06] Validation started with lhcb-lcg-dev4#641
• [2018-09-04 00:08] Validation started with lhcb-gaudi-head#1983
• [2018-09-04 00:10] Validation started with lhcb-tdr-test#276
• [2018-09-05 00:05] Validation started with lhcb-lcg-dev4#642
• [2018-09-05 00:05] Validation started with lhcb-lcg-dev3#639
• [2018-09-05 00:08] Validation started with lhcb-tdr-test#277
• [2018-09-05 00:10] Validation started with lhcb-gaudi-head#1984
• [2018-09-06 00:05] Validation started with lhcb-lcg-dev3#640
• [2018-09-06 00:08] Validation started with lhcb-lcg-dev4#643
• [2018-09-06 00:11] Validation started with lhcb-tdr-test#278
• [2018-09-06 00:12] Validation started with lhcb-gaudi-head#1985
• [2018-09-07 00:05] Validation started with lhcb-lcg-dev3#641
• [2018-09-07 00:06] Validation started with lhcb-lcg-dev4#644
• [2018-09-07 00:09] Validation started with lhcb-tdr-test#279
• [2018-09-07 00:19] Validation started with lhcb-gaudi-head#1986
• [2018-09-07 00:52] Validation started with lhcb-lcg-dev4#644
• [2018-09-07 00:56] Validation started with lhcb-tdr-test#279
• [2018-09-07 01:06] Validation started with lhcb-gaudi-head#1986
• [2018-09-07 01:34] Validation started with lhcb-tdr-test#279
• [2018-09-07 09:11] Validation started with lhcb-gaudi-head#1987
• [2018-09-08 00:04] Validation started with lhcb-lcg-dev4#645
• [2018-09-08 00:06] Validation started with lhcb-lcg-dev3#642
• [2018-09-08 00:07] Validation started with lhcb-tdr-test#280
• [2018-09-08 00:10] Validation started with lhcb-gaudi-head#1988
• [2018-09-09 17:03] Validation started with lhcb-gaudi-head#1989
• [2018-09-09 17:03] Validation started with lhcb-lcg-dev3#643
• [2018-09-09 17:05] Validation started with lhcb-lcg-dev4#646
• [2018-09-09 17:05] Validation started with lhcb-tdr-test#281
• [2018-09-10 00:08] Validation started with lhcb-lcg-dev3#644
• [2018-09-10 00:11] Validation started with lhcb-lcg-dev4#647
• [2018-09-10 00:14] Validation started with lhcb-tdr-test#282
• [2018-09-10 00:19] Validation started with lhcb-gaudi-head#1990
• [2018-09-10 09:41] Validation started with lhcb-dd4hep-dev4#1
• [2018-09-10 10:04] Validation started with lhcb-dd4hep-dev4#2
• [2018-09-10 17:08] Validation started with lhcb-dd4hep-dev4#3
• [2018-09-10 22:35] Validation started with lhcb-dd4hep-dev4#4
• [2018-09-11 00:05] Validation started with lhcb-lcg-dev3#645
• [2018-09-11 00:05] Validation started with lhcb-lcg-dev4#648
• [2018-09-11 00:06] Validation started with lhcb-dd4hep-dev4#5
• [2018-09-11 00:09] Validation started with lhcb-tdr-test#283
• [2018-09-11 00:09] Validation started with lhcb-gaudi-head#1991
• [2018-09-11 00:18] Validation started with lhcb-dd4hep-dev4#5
• [2018-09-11 00:27] Validation started with lhcb-dd4hep-dev4#5
• [2018-09-11 09:47] Validation started with lhcb-tdr-test#284
• [2018-09-11 18:18] Validation started with lhcb-dd4hep-dev4#6
• [2018-09-12 00:05] Validation started with lhcb-sanitizers#1
• [2018-09-12 00:05] Validation started with lhcb-dd4hep-dev4#7
• [2018-09-12 00:06] Validation started with lhcb-lcg-dev4#649
• [2018-09-12 00:07] Validation started with lhcb-lcg-dev3#646
• [2018-09-12 00:07] Validation started with lhcb-tdr-test#285
• [2018-09-12 00:10] Validation started with lhcb-gaudi-head#1992
• [2018-09-13 00:07] Validation started with lhcb-dd4hep-dev4#8
• [2018-09-13 00:07] Validation started with lhcb-lcg-dev4#651
• [2018-09-13 00:15] Validation started with lhcb-gaudi-head#1993
• [2018-09-13 00:16] Validation started with lhcb-tdr-test#286
• [2018-09-13 00:23] Validation started with lhcb-lcg-dev3#647
• [2018-09-13 00:38] Validation started with lhcb-dd4hep-dev4#8
• [2018-09-13 00:46] Validation started with lhcb-sanitizers#2
• [2018-09-14 00:05] Validation started with lhcb-lcg-dev3#648
• [2018-09-14 00:05] Validation started with lhcb-dd4hep-dev4#9
• [2018-09-14 00:07] Validation started with lhcb-lcg-dev4#652
• [2018-09-14 00:15] Validation started with lhcb-sanitizers#3
• [2018-09-14 00:23] Validation started with lhcb-tdr-test#287
• [2018-09-14 00:48] Validation started with lhcb-gaudi-head#1994
• [2018-09-15 00:06] Validation started with lhcb-lcg-dev3#649
• [2018-09-15 00:07] Validation started with lhcb-dd4hep-dev4#10
• [2018-09-15 00:08] Validation started with lhcb-tdr-test#288
• [2018-09-15 00:09] Validation started with lhcb-lcg-dev4#653
• [2018-09-15 00:10] Validation started with lhcb-sanitizers#4
• [2018-09-15 00:11] Validation started with lhcb-gaudi-head#1995
• [2018-09-16 00:07] Validation started with lhcb-lcg-dev4#654
• [2018-09-16 00:07] Validation started with lhcb-sanitizers#5
• [2018-09-16 00:08] Validation started with lhcb-lcg-dev3#650
• [2018-09-16 00:08] Validation started with lhcb-gaudi-head#1996
• [2018-09-16 00:08] Validation started with lhcb-tdr-test#289
• [2018-09-16 00:08] Validation started with lhcb-dd4hep-dev4#11
• [2018-09-16 11:08] Validation started with lhcb-dd4hep-dev4#12
• [2018-09-17 00:04] Validation started with lhcb-dd4hep-dev4#13
• [2018-09-17 00:06] Validation started with lhcb-lcg-dev4#655
• [2018-09-17 00:07] Validation started with lhcb-gaudi-head#1997
• [2018-09-17 00:07] Validation started with lhcb-lcg-dev3#651
• [2018-09-17 00:08] Validation started with lhcb-tdr-test#290
• [2018-09-17 00:11] Validation started with lhcb-sanitizers#6
• [2018-09-18 00:03] Validation started with lhcb-dd4hep-dev4#14
• [2018-09-18 00:06] Validation started with lhcb-lcg-dev3#652
• [2018-09-18 00:06] Validation started with lhcb-lcg-dev4#656
• [2018-09-18 00:06] Validation started with lhcb-sanitizers#7
• [2018-09-18 00:07] Validation started with lhcb-gaudi-head#1998
• [2018-09-18 00:19] Validation started with lhcb-tdr-test#291
• [2018-09-19 00:04] Validation started with lhcb-lcg-dev3#653
• [2018-09-19 00:06] Validation started with lhcb-dd4hep-dev4#15
• [2018-09-19 00:06] Validation started with lhcb-sanitizers#8
• [2018-09-19 00:07] Validation started with lhcb-lcg-dev4#657
• [2018-09-19 00:09] Validation started with lhcb-gaudi-head#1999
• [2018-09-19 00:10] Validation started with lhcb-tdr-test#292
• [2018-09-20 00:04] Validation started with lhcb-lcg-dev4#658
• [2018-09-20 00:04] Validation started with lhcb-lcg-dev3#654
• [2018-09-20 00:09] Validation started with lhcb-sanitizers#9
• [2018-09-20 00:11] Validation started with lhcb-tdr-test#293
• [2018-09-20 00:15] Validation started with lhcb-gaudi-head#2000
• [2018-09-20 09:02] Validation started with lhcb-dd4hep#2
• [2018-09-21 00:05] Validation started with lhcb-lcg-dev4#659
• [2018-09-21 00:05] Validation started with lhcb-dd4hep#3
• [2018-09-21 00:06] Validation started with lhcb-lcg-dev3#655
• [2018-09-21 00:10] Validation started with lhcb-tdr-test#294
• [2018-09-21 00:10] Validation started with lhcb-sanitizers#10
• [2018-09-21 00:12] Validation started with lhcb-gaudi-head#2001
• [2018-09-22 00:06] Validation started with lhcb-lcg-dev3#656
• [2018-09-22 00:07] Validation started with lhcb-lcg-dev4#660
• [2018-09-22 00:09] Validation started with lhcb-sanitizers#11
• [2018-09-22 00:09] Validation started with lhcb-dd4hep#4
• [2018-09-22 00:11] Validation started with lhcb-tdr-test#295
• [2018-09-22 00:12] Validation started with lhcb-gaudi-head#2002
• [2018-09-23 00:08] Validation started with lhcb-sanitizers#12
• [2018-09-23 00:08] Validation started with lhcb-lcg-dev3#657
• [2018-09-23 00:09] Validation started with lhcb-dd4hep#5
• [2018-09-23 00:11] Validation started with lhcb-tdr-test#296
• [2018-09-23 00:11] Validation started with lhcb-gaudi-head#2003
• [2018-09-24 00:07] Validation started with lhcb-lcg-dev3#658
• [2018-09-24 00:08] Validation started with lhcb-gaudi-head#2004
• [2018-09-24 00:08] Validation started with lhcb-dd4hep#6
• [2018-09-24 00:10] Validation started with lhcb-lcg-dev4#662
• [2018-09-24 00:12] Validation started with lhcb-tdr-test#297
• [2018-09-24 00:14] Validation started with lhcb-sanitizers#13
• [2018-09-24 09:34] Validation started with lhcb-tdr-test#298
• [2018-09-24 10:58] Validation started with lhcb-gaudi-head#2005
• [2018-09-25 00:15] Validation started with lhcb-lcg-dev4#663
• [2018-09-25 00:19] Validation started with lhcb-tdr-test#299
• [2018-09-25 00:19] Validation started with lhcb-lcg-dev3#659
• [2018-09-25 00:21] Validation started with lhcb-sanitizers#14
• [2018-09-25 00:22] Validation started with lhcb-gaudi-head#2006
• [2018-09-25 00:29] Validation started with lhcb-dd4hep#7
• [2018-09-26 00:07] Validation started with lhcb-lcg-dev3#660
• [2018-09-26 00:08] Validation started with lhcb-dd4hep#8
• [2018-09-26 00:10] Validation started with lhcb-lcg-dev4#664
• [2018-09-26 00:14] Validation started with lhcb-sanitizers#15
• [2018-09-26 00:14] Validation started with lhcb-gaudi-head#2007
• [2018-09-26 00:15] Validation started with lhcb-tdr-test#300
• [2018-09-27 00:05] Validation started with lhcb-lcg-dev4#665
• [2018-09-27 00:06] Validation started with lhcb-lcg-dev3#661
• [2018-09-27 00:08] Validation started with lhcb-dd4hep#9
• [2018-09-27 00:09] Validation started with lhcb-gaudi-head#2008
• [2018-09-27 00:10] Validation started with lhcb-tdr-test#301
• [2018-09-27 00:11] Validation started with lhcb-sanitizers#16
• [2018-09-27 01:07] Validation started with lhcb-tdr-test#301
• [2018-09-28 00:30] Validation started with lhcb-dd4hep#10
• [2018-09-28 00:30] Validation started with lhcb-sanitizers#17
• [2018-09-28 00:31] Validation started with lhcb-lcg-dev4#666
• [2018-09-28 00:33] Validation started with lhcb-lcg-dev3#662
• [2018-09-28 00:33] Validation started with lhcb-tdr-test#302
• [2018-09-28 00:43] Validation started with lhcb-gaudi-head#2009
• [2018-09-29 00:19] Validation started with lhcb-tdr-test#303
• [2018-09-29 00:21] Validation started with lhcb-sanitizers#18
• [2018-09-29 00:24] Validation started with lhcb-gaudi-head#2010
• [2018-09-29 00:33] Validation started with lhcb-lcg-dev4#667
• [2018-09-29 00:33] Validation started with lhcb-lcg-dev3#663
• [2018-09-29 00:35] Validation started with lhcb-dd4hep#11
• [2018-09-29 08:39] Validation started with lhcb-gaudi-head#2011
• [2018-09-29 08:47] Validation started with lhcb-tdr-test#304
• [2018-09-30 00:07] Validation started with lhcb-dd4hep#12
• [2018-09-30 00:07] Validation started with lhcb-gaudi-head#2012
• [2018-09-30 00:07] Validation started with lhcb-lcg-dev3#664
• [2018-09-30 00:08] Validation started with lhcb-tdr-test#305
• [2018-09-30 00:09] Validation started with lhcb-lcg-dev4#668
• [2018-09-30 00:09] Validation started with lhcb-sanitizers#19
• [2018-10-01 00:06] Validation started with lhcb-lcg-dev3#665
• [2018-10-01 00:08] Validation started with lhcb-lcg-dev4#669
• [2018-10-01 00:10] Validation started with lhcb-tdr-test#306
• [2018-10-01 00:11] Validation started with lhcb-dd4hep#13
• [2018-10-01 00:11] Validation started with lhcb-sanitizers#20
• [2018-10-01 00:12] Validation started with lhcb-gaudi-head#2013
• [2018-10-01 10:13] Validation started with lhcb-gaudi-head#2014
• [2018-10-02 00:08] Validation started with lhcb-lcg-dev3#666
• [2018-10-02 00:09] Validation started with lhcb-tdr-test#307
• [2018-10-02 00:09] Validation started with lhcb-dd4hep#14
• [2018-10-02 00:10] Validation started with lhcb-lcg-dev4#670
• [2018-10-02 00:10] Validation started with lhcb-sanitizers#21
• [2018-10-02 00:15] Validation started with lhcb-gaudi-head#2015
• [2018-10-03 00:05] Validation started with lhcb-lcg-dev4#671
• [2018-10-03 00:07] Validation started with lhcb-lcg-dev3#667
• [2018-10-03 00:09] Validation started with lhcb-tdr-test#308
• [2018-10-03 00:10] Validation started with lhcb-dd4hep#15
• [2018-10-03 00:14] Validation started with lhcb-gaudi-head#2016
• [2018-10-03 00:25] Validation started with lhcb-sanitizers#22
• [2018-10-04 00:07] Validation started with lhcb-lcg-dev3#668
• [2018-10-04 00:07] Validation started with lhcb-dd4hep#16
• [2018-10-04 00:10] Validation started with lhcb-lcg-dev4#672
• [2018-10-04 00:10] Validation started with lhcb-tdr-test#309
• [2018-10-04 00:11] Validation started with lhcb-sanitizers#23
• [2018-10-04 00:17] Validation started with lhcb-gaudi-head#2017
• [2018-10-04 08:35] Validation started with lhcb-tdr-test#310

approved this merge request

changed title from make sure a test fails is its reference file is not found to make sure a test fails if its reference file is not found

changed the description

added 1 commit

• cd579fbc - Update BaseTest.py

Compare with previous version

resolved all discussions

assigned to @clemenci

changed milestone to %v30r4

added tests label

approved this merge request

merged

mentioned in commit 4fa3d01f

mentioned in commit clemenci/Gaudi@25864b55

mentioned in commit clemenci/Gaudi@62598c4d