Only configure producers / consumers when required by algorithms in sequence

In general, the goal of this MR is to only trigger the relevant geometry consumers / producers.

• Create new type of Algorithm, ProviderAlgorithm. As with other Allen algorithm types (such as HostAlgorithm, DeviceAlgorithm or SelectionAlgorithm) it doesn't have an impact upon the functionalities the algorithm can have.
• ProviderAlgorithms are expected to have a bank_type property.
• The algorithm type is now propagated to the JSON configurations.
• A new bank string VPRetinaCluster which maps to bank type VP has been created, required to enable the functionality described below.

This MR does various things, depending on the way that Allen is run:

• In Allen STANDALONE and Allen steered from MooreOnline, deduce from the data provider algorithms present in the requested sequence which geometry consumers / producers are needed and only schedule the ones linked to that bank type.
• In Allen STANDALONE and Allen steered from MooreOnline, given that he magnetic field is used by compassUT and the SciFi reconstruction, it is always scheduled.
• In Gaudi-Allen, ProvideRuntimeOptions is now configured with the concrete bank types that are requested. It is not needed anymore to specify the type of banks requested by hand (solves #332 (closed)).

Closes #344 (closed) and #332 (closed).

Should go with MooreOnline!130 (merged)

requested review from @raaij and @dcampora

assigned to @dovombru

mentioned in merge request !887 (closed)

added RTA label

added 1 commit

• 4bd090d1 - Fixed formatting

Compare with previous version

added 1 commit

• faf6cde3 - configure producers and consumers based on which bank providers are scheduled...

Compare with previous version

added 6 commits

• faf6cde3...d288d844 - 3 commits from branch master
• 3178910c - Added new type of Algorithm - ProviderAlgorithm. Added scope to json files.
• 4a0c08be - make compile
• 9d49184e - configure producers and consumers based on which bank providers are scheduled...

Compare with previous version

mentioned in issue Moore#449 (closed)

mentioned in issue #332 (closed)

mentioned in issue Moore#454 (closed)

added 126 commits

• 9d49184e...0d8b0b7b - 123 commits from branch master
• 2d8ba2b3 - Added new type of Algorithm - ProviderAlgorithm. Added scope to json files.
• d95247ab - make compile
• fbb371b0 - configure producers and consumers based on which bank providers are scheduled...

Compare with previous version

Using this with the following options:

def dump_mdf_ft():
    """Dump raw banks and MC info required for Allen standalone running into an MDF file                                                                
                                                                                                                                                        
    Written to MDF:                                                                                                                                     
    - The Velo, UT, SciFi, Muon, ODIN banks                                                                                                             
    - MC info from the TrackerDumper and the PVDumper required for the Allen standalone checker                                                         
    Written to binaries (inside the "geometry" directory):                                                                                              
    - Non-event data such as geometry information                                                                                                       
    """

    data = []

    detector_parts = [
        default_raw_event([bt]) for bt in ["FTCluster"]
    ]
    # get only unique elements while preserving order                                                                                                   
    detector_parts = list(dict.fromkeys(detector_parts).keys())
    combiner = RawEventCombiner(RawEventLocations=detector_parts)

    data.append(combiner)

    data.append(
        mdf_writer(
            options.output_file, location=combiner.RawEvent, compression=0))

    return Reconstruction('write_mdf', data)


with allen_non_event_data_config.bind(
        dump_geometry=True, out_dir=geometry_dir):
    run_allen_reconstruction(options, dump_mdf_ft)

I get

Traceback (most recent call last):
  File "/home/lohenry/stack/Gaudi/InstallArea/x86_64_v2-centos7-gcc11-opt/bin/gaudirun.py", line 584, in <module>
    exec(o, g, l)
  File "<string>", line 1, in <module>
  File "/home/lohenry/stack/Gaudi/InstallArea/x86_64_v2-centos7-gcc11-opt/bin/gaudirun.py", line 543, in __call__
    importOptions(arg)
  File "/home/lohenry/stack/Gaudi/InstallArea/x86_64_v2-centos7-gcc11-opt/python/GaudiKernel/ProcessJobOptions.py", line 552, in importOptions
    _import_function_mapping[ext](optsfile)
  File "/home/lohenry/stack/Gaudi/InstallArea/x86_64_v2-centos7-gcc11-opt/python/GaudiKernel/ProcessJobOptions.py", line 486, in _import_python
    exec(code, {})
  File "/home/lohenry/options/testV7/makeGeometry.py", line 106, in <module>
    run_allen_reconstruction(options, dump_mdf_ft)
  File "/home/lohenry/stack/Moore/Hlt/Moore/python/Moore/config.py", line 831, in run_allen_reconstruction
    non_event_data_node = setup_allen_non_event_data_service()
  File "/home/lohenry/stack/Allen/InstallArea/x86_64_v2-centos7-gcc11-opt/python/Allen/config.py", line 100, in setup_allen_non_event_data_service
    algorithm_converters, algorithm_producers, requested_banks = allen_configure_producers(
  File "/home/lohenry/stack/Allen/InstallArea/x86_64_v2-centos7-gcc11-opt/python/Allen/config.py", line 48, in allen_configure_producers
    if algo[2] == "ProviderAlgorithm":
IndexError: list index out of range

Is the error on my side or something in this MR?

added 1 commit

• 2615560a - Update Provider.cpp and test mep banks to expect the right amount of elements in the configuration.

Compare with previous version

added 1 commit

• 58494a51 - Use configurationreader in Provider.cpp.

Compare with previous version

added 1 commit

• 342116cd - Fixed formatting

Compare with previous version

added 1 commit

• ed0da1a5 - Move configured_bank_types to conf reader. Make standalone work.g

Compare with previous version

added 1 commit

• cecf7bd3 - Fixed formatting

Compare with previous version

changed title from Only configure producers / consumers when required by algorithms in sequence to WIP: Only configure producers / consumers when required by algorithms in sequence

added 1 commit

• 8235f574 - Steps to make Gaudi-Allen work.

Compare with previous version

mentioned in issue Moore#456 (closed)