WIP: qmtest Support for remote refs in tests

Working with @rmatev, this is a simple MR to add support for a remote file as the reference for a test in qmtest.

To do this I added some logic to the _expandReferenceFileName method to determine if a file is to be accessed via a remote protocol (for the moment only http(s)).

If the file is remote it's first downloaded to a local temporary location to allow the rest of the test code to run as normal. The few methods which use the _expandReferenceFileName method have then been updated to check for these temporary files and clean up after the test has been completed.

This has been tested by replacing the reference file location in one of the qmtests with the same file hosted by a http server on a remote machine and the test passed as expected when the file was available.

added 1 commit

• 0070dc42 - Fixing formatting

Compare with previous version

added 1 commit

• 9530353e - Applying formatting patch properly

Compare with previous version

added 1 commit

• b3196130 - Fixing backtrace test

Compare with previous version

added 1 commit

• 78feb76c - Applying formatting

Compare with previous version

added 1 commit

• ed5e1720 - Fixing typo

Compare with previous version

changed milestone to %v32r2

assigned to @clemenci

unmarked as a Work In Progress

removed milestone

• [2019-09-18 15:47] Validation started with lhcb-gaudi-merge#817
• [2019-09-18 16:02] Validation started with lhcb-gaudi-merge#817
• [2019-09-18 16:15] Validation started with lhcb-gaudi-merge#817
• [2019-09-19 00:04] Validation started with lhcb-dd4hep#321
• [2019-09-19 00:04] Validation started with lhcb-lcg-dev4#1035
• [2019-09-19 00:04] Validation started with lhcb-lcg-dev3#1025
• [2019-09-19 00:06] Validation started with lhcb-sanitizers#375
• [2019-09-19 00:07] Validation started with lhcb-gaudi-head-py3#178
• [2019-09-19 00:07] Validation started with lhcb-test-throughput2#156
• [2019-09-19 00:09] Validation started with lhcb-run2-patches-dev4#83
• [2019-09-19 00:09] Validation started with lhcb-gaudi-head#2390
• [2019-09-19 00:11] Validation started with lhcb-tdr-test#678

I'm not too convinced it is that useful to get a reference file from a remote location.

In my opinion it's going to make the comparison of stdout even less reliable.

How do you keep references and the code to be tested in sync? At any time there may be different code versions all of which may need their own consistent version of the references. If the references are kept with the code this is trivially consistent but how would that work with remote references?

This breaks a lot of tests in LHCb.

I think it's breaking all the tests...

marked as a Work In Progress

mentioned in merge request lhcb/LHCb!2128 (merged)

I join @clemenci and @graven that I do not really see the point of this other than breaking consistency between code and refs. What is the original intention and need ?

mentioned in merge request !875 (merged)

We are about to drop QMTest support. So closing this as obsolete.

closed