Type-Erased Allen

This MR type erases all Allen algorithms. It reduces compilation times, and enables running any Allen sequence specified in a python config or json file (see list of changes)

All the changes affect the backend, the frontend (that is, what the average user will see) is not modified, with the exception of requiring specifying INSTANTIATE_ALGORITHM(algorithm_type) in each algorithm source file.

List of changes

• All Allen algorithms are now type-erased. All algorithms require specifying an INSTANTIATE_ALGORITHM(algorithm_type) macro. Selection algorithms do this by their preexisting INSTANTIATE_LINE, which has been extended to also instantiate the algorithm.
• An AlgorithmDB.h containing a string to algorithm correspondence is generated at compile-time.
• Arguments are fully type-erased as well. They were missing the type size and scope, which now allows the memory manager to determine the correct bytesize to use for allocating / freeing and whether to use the host or device memory manager instance.
• The Allen store (ArgumentManager) has now become an UnorderedStore instance. It should be noted that this "store" keeps ArgumentData which contain pointers to the data. The memory where the data lives is managed by the memory manager as before.
• StreamLoader, SchedulerMachinery and ConfiguredSequence.h generation code were not required anymore and have been removed.
• By removing the need to generate a ConfiguredSequence.h, this MR reduces the compilation time and resource requirements significantly.
• AlgorithmTraversalLibClang.py now uses exclusively c++ parsing in contrast to before, where .cuh files were being parsed with -x cuda. The reason for this is that in doing so the parsing is more homogeneous. Other options include "-std=c++17", "-nostdinc++".
• Sequence is now generated by default at runtime (from the python configurations). This means that changes in the python configurations, even those done after compilation, will have an effect at runtime. It is still possible to generate a list of .json sequences using option -DSEQUENCES as before, and run off them specifying --run-from-json 1 at runtime.

./Allen --sequence <my_sequence> # generates config in configuration/sequences/<my_sequence>.py and runs it
./Allen # runs hlt1_pp_default per default

Sequence generation changes

• The compile step for generating sequences now only generates a .json file. Please note that this generation still occurs at compile-time, although it is not a true compile requirement anymore. It can potentially be moved to runtime.
• The .json configuration contains the sequence under the field "sequence" in a predefined format. It expects configured_algorithms, configured_arguments, configured_sequence_arguments and argument_dependencies. As an example:

  "sequence": {
    "configured_algorithms": [
      ["host_init_event_list::host_init_event_list_t", "initialize_event_lists"],
      ["host_data_provider::host_data_provider_t", "host_scifi_banks"],
    ],
    "configured_arguments": [
      ["host", "initialize_event_lists__host_event_list_output_t"],
      ["device", "initialize_event_lists__dev_event_list_output_t"],
      ["host", "host_scifi_banks__host_raw_banks_t"],
      ["host", "host_scifi_banks__host_raw_offsets_t"],
      ["host", "host_scifi_banks__host_raw_bank_version_t"]
    ],
    "configured_sequence_arguments": [
      [["initialize_event_lists__host_event_list_output_t", "initialize_event_lists__dev_event_list_output_t"], []],
      [["host_scifi_banks__host_raw_banks_t",
      "host_scifi_banks__host_raw_offsets_t",
      "host_scifi_banks__host_raw_bank_version_t"], []]
    ],
    "argument_dependencies": {
      "host_scifi_banks__host_raw_banks_t": ["initialize_event_lists__host_event_list_output_t"]
    }
  }

• configured_algorithms - It is of type std::vector<ConfiguredAlgorithm> (due to the nlohmann-json parsing POD-type requirements, it is parsed as a std::vector<std::tuple<std::string, std::string>> which is then converted with std::make_from_tuple to ConfiguredAlgorithms - a similar method is used for the other types). configured_algorithms contains a list of algorithms to be executed with their configured names.
• configured_arguments - Type std::vector<ConfiguredArgument> (std::vector<std::tuple<std::string, std::string>>). It contains a list of argument names with their associated scope (either host or device). These are the arguments that populate the Allen store.
• configured_sequence_arguments - Type std::vector<ConfiguredAlgorithmArguments> (std::vector<std::tuple<std::vector<std::string>, std::vector<std::vector<std::string>>>>). It includes the list of arguments and input aggregates that will be used in each algorithm's invocation.
• argument_dependencies - Type ArgumentDependencies (std::map<std::string, std::vector<std::string>>). It contains the dependencies of each datatype.

Built on top of !431 (closed)