[RTADPA Bandwidth tests]: Test feasibility of copying HLT2 output to read into sprucing test

FYI @lugrazet

Goes with lhcb-datapkg/PRConfig!359 (merged)

This MR adds a small testing block to the BandwidthTestHandler to try to copy the FULL-stream mdf output of the lhcb-master Moore_hlt2_bandwidth test to the UpgradeRateTest area of eos, and time it, to see how feasible this is.

The corresponding PRConfig MR will then try to download this at the end of the sprucing BW test, again to show that it can be done and time how long it takes.

I've tested locally that if I have the correctly-named MDF file locally, then it will try to copy it to eos, but in this local test all uploads fail as I don't have the correct permissions, so can't test fully if the upload will work. Given all the current uploads work in the nightlies, I don't see why this one wouldn't.

All a bit hacky at the mo; if this all works we can refine the implementation soon.

handlers/BandwidthTestHandler.py

@@ -517,22 +517,30 @@ class BandwidthTestHandler(BaseHandler):
         if options == "Moore_hlt2_bandwidth" and slot == "lhcb-master" and status:
             from datetime import datetime
             before_copy = datetime.now()
-            full_stream_mdf = os.path.join(directory, 'tmp', 'MDF', "hlt2_bw_testing__production__full.mdf")
-            log.info(f"Trying to copy {full_stream_mdf} to eos. File size: {os.stat(full_stream_mdf).st_size / 1e6} MB.")
+            full_stream_mdf = os.path.join(
+                directory, 'tmp', 'MDF',
+                "hlt2_bw_testing__production__full.mdf")
+            log.info(
+                f"Trying to copy {full_stream_mdf} to eos. File size: {os.stat(full_stream_mdf).st_size / 1e6} MB."
+            )
             publish.upload_eos_www(
                 full_stream_mdf,
                 os.path.basename(full_stream_mdf),
-                baseurl=os.path.join(os.getenv("LHCBPR_WWW_EOS"), "UpgradeRateTest")
-            )
-            log.info(f"Copied {full_stream_mdf} to eos. This took " + str(datetime.now() - before_copy)) 
+                baseurl=os.path.join(
+                    os.getenv("LHCBPR_WWW_EOS"), "UpgradeRateTest"))
+            log.info(f"Copied {full_stream_mdf} to eos. This took " +
+                     str(datetime.now() - before_copy))
             log.info("Trying to copy to new directory...")
             before_second_copy = datetime.now()
             publish.upload_eos_www(
                 full_stream_mdf,
-                os.path.join("current_hlt2_output", os.path.basename(full_stream_mdf)),
-                baseurl=os.path.join(os.getenv("LHCBPR_WWW_EOS"), "UpgradeRateTest")
-            )
-            log.info(f"Copied {full_stream_mdf} to eos new directory. This took " + str(datetime.now() - before_second_copy)) 
+                os.path.join("current_hlt2_output",
+                             os.path.basename(full_stream_mdf)),
+                baseurl=os.path.join(
+                    os.getenv("LHCBPR_WWW_EOS"), "UpgradeRateTest"))
+            log.info(
+                f"Copied {full_stream_mdf} to eos new directory. This took " +
+                str(datetime.now() - before_second_copy))
 
         # Post GitLab MR feedback; a bit more complex as comparing to a reference build.
         if (slot in [