TrigValSteering: Memoize path lookup and add more debug messages

This should hopefully fix ATR-21080. The slowest part of the test configuration, and therefore the unit tests, is the path lookup with find_data and get_files. Since they were called multiple times per test, the delays were multiplied. This MR adds the memoize modifier to these calls to avoid multiple lookups of the same path.

Several extra debug messages are also added to help debug such problems in the future. The time format in the logger will now also show seconds.

Trigger/TrigValidation/TrigValTools/python/TrigValSteering/Common.py

@@ -9,6 +9,8 @@ Common variables and functions used in Trigger ART test steering
 import logging
 import sys
 import os
+import subprocess
+from PyUtils.Decorators import memoize
 
 
 # Logging level used across the package
@@ -32,7 +34,7 @@ def get_logger():
     '''Default TrigValSteering logger'''
     logging.basicConfig(stream=sys.stdout,
                         format='%(asctime)s %(name)s %(levelname)-8s %(message)s',
-                        datefmt='%Y-%m-%dT%H%M %Z',
+                        datefmt='%Y-%m-%dT%H%M%S %Z',
                         level=trigvalsteering_logging_level)
     return logging.getLogger('TrigValSteering')
 
@@ -51,3 +53,18 @@ def clear_art_summary():
 
     if os.path.isfile(art_result_summary):
         os.remove(art_result_summary)
+
+
+@memoize
+def check_job_options(jo_path):
+    '''Check if the job options file exists locally or in PATH'''
+    # Check if job options existi locally
+    if os.path.isfile(jo_path):
+        return True
+    # Try to find the file in PATH
+    get_files_output = subprocess.check_output(
+        'get_files -jo -list {}'.format(jo_path), shell=True)
+    if 'nothing found' in get_files_output.decode():
+        return False
+    else:
+        return True