[RTADPA BW tests] Test feasibility of copying HLT2 output to read into sprucing test

FYI @lugrazet

Goes with lhcb-core/LHCbPR2HD!267 (merged)

This MR adds a dumb new script (modelled on the relevant part in run_bandwidth_test_jobs.py) that downloads the output full-stream MDF of the latest Hlt2 bandwidth job from eos to the local disk. This is then called by the sprucing BW test, and this is all timed. The idea is to test the feasibility of a HLT2->Sprucing chained test, and I'd like to see if there's any speed/plumbing bottlenecks here.

Have tested locally that it can download an MDF from a public dir on eos to my local space at Warwick, and separately that I can xrdcp files from UpgradeRateTest to Warwick, so don't see why it can't pick up something from the UpgradeRateTest area, but we have to test this.

The file in question to be downloaded is being copied to eos via the LHCbPR2HD MR.

python/MooreTests/download_hlt2_output.py

@@ -29,43 +29,42 @@ FALLBACK_CACHE_DIR = os.getenv('XDG_RUNTIME_DIR', tempfile.gettempdir())
 
 FILE_TO_COPY = "mdf:root://eoslhcb.cern.ch//eos/lhcb/storage/lhcbpr/www/UpgradeRateTest/hlt2_bw_testing__production__full.mdf"
 
+
 def default_cache_dirs():
     hostname = socket.getfqdn()
     dirs = DEFAULT_CACHE_DIRS.get(hostname, DEFAULT_CACHE_DIRS['default'])
     return dirs
 
+
 def main():
 
     logging.basicConfig(
-        format='%(levelname)-7s %(message)s',
-        level=logging.INFO)
+        format='%(levelname)-7s %(message)s', level=logging.INFO)
 
     cache_dir = default_cache_dirs()
     if not os.path.isdir(cache_dir):
         fallback_dir = tempfile.mkdtemp(
             prefix='bandwidth-', dir=FALLBACK_CACHE_DIR)
-        logging.warning(
-            'default cache dir {!r} doesnt exist, using {}'.format(
-                cache_dir, fallback_dir))
+        logging.warning('default cache dir {!r} doesnt exist, using {}'.format(
+            cache_dir, fallback_dir))
         cache_dir = fallback_dir
         # if we use the fallback directory, clean up after ourselves
         atexit.register(shutil.rmtree, fallback_dir)
 
     # Now download file
-    logging.info(
-        f'Downloading input file {FILE_TO_COPY}'
-    )
+    logging.info(f'Downloading input file {FILE_TO_COPY}')
     # download_mdf_inputs_locally only downloads if files
     # are not already available locally on the machine
-    logging.info(
-        f'Downloading inputs for bandwidth job to {cache_dir}')
+    logging.info(f'Downloading inputs for bandwidth job to {cache_dir}')
     before_copy = datetime.now()
     kB_to_GB = 1e3
     downloaded_path = download_mdf_inputs_locally(
-        [FILE_TO_COPY],
-        cache_dir,
-        max_size=300 * kB_to_GB * 2e4) # Guesses as to output size and n_events in the FULL stream TODO improve
-    logging.info(f"Downloaded {downloaded_path}. This took: {datetime.now() - before_copy}")
+        [FILE_TO_COPY], cache_dir, max_size=300 * kB_to_GB * 2e4
+    )  # Guesses as to output size and n_events in the FULL stream TODO improve
+    logging.info(
+        f"Downloaded {downloaded_path}. This took: {datetime.now() - before_copy}"
+    )
+
 
 if __name__ == "__main__":
-    main()
\ No newline at end of file
+    main()