Skip to content

Generation of Gaudi wrapper algorithms for Allen algorithms

Daniel Hugo Campora Perez requested to merge dcampora_nnolte_gaudi_conversion into master

This MR produces automatic Gaudi wrappers to all Allen algorithms when Allen is compiled in non-STANDALONE mode.

  • Automatic algorithm wrappers are generated for all Allen algorithms.
  • Gaudi versions of Allen algorithms invoke the Allen algorithms' set_arguments_size and operator() consecutively.
  • Input and outputs are [const] std::vectors of the underlying Allen type. The exception is bool, which instead uses std::vector<char> as backend to avoid std::vector bool storage shenanigans.
  • Properties are properly populated in the Gaudi wrappers, initialized to the same (parsed) Allen defaults upon initialization.

What this enables in short

All Allen configurations introduced with the MES are now usable from within Gaudi, and all Allen algorithms are now usable as Gaudi algorithms.

A Gaudi configuration like the following is now possible. It works both in Allen and Gaudi:

from AllenConf.hlt1_reconstruction import hlt1_reconstruction
hlt1_reco = hlt1_reconstruction()
velo_tracks = hlt1_reco['velo_tracks']
ut_tracks = hlt1_reco['ut_tracks']
forward_tracks = hlt1_reco['forward_tracks']
kf_tracks = hlt1_reco['kalman_velo_only']

Here is an example of such a configuration.

This works because all Allen algorithms are translated to Gaudi algorithms with automatically compile-time generated wrappers. In addition, the following two python paths are installed as part of Allen:

The above only occurs when Allen is compiled with -DSTANDALONE=OFF, so the STANDALONE build is not affected.

List of changes

  • At configure-time the list of algorithm wrappers that will be built is generated.
  • A new custom CMake target gaudi_wrappers_of_allen_algorithms generates Gaudi wrappers of Allen algorithms (see below for an example).
  • The generation of Gaudi wrappers is only triggered if building with -DSTANDALONE=OFF.
  • The verbosity_t property of all algorithms has now public visibility to allow Gaudi algorithms access.
  • Added a new function set_property_value(const R& value) to Allen Algorithm class.
  • Implemented parsing of default datatypes of Allen algorithms.
  • Refactored ParseAlgorithms.py. Now it accepts three possible generations with option --generate: views (default, generates the Python view of all Allen algorithms), wrapperlist (which generates the list of files used at configure-time) and wrappers (which generates .cpp wrappers).
  • Added two new Gaudi algorithms under Rec/Allen/: ProvideConstants and ProvideRuntimeOptions. These two algorithms are required in any invocation to a wrapper algorithm.
  • Added two new Gaudi algorithms for data conversions: GaudiAllenVeloToV2Tracks and GaudiAllenForwardToV2Tracks. These two algorithms convert Allen data received in the form of std::vectors to v2 tracks.
  • Two types of algorithms can be generated: Consumer and MultiTransformer. All Allen Producer algorithms become MultiTransformer since they now require a constants and runtime_options parameter.
  • Moved configuration/sequences/AllenConf to configuration/python/AllenConf.
  • Created Rec/Allen/python/AllenCore/generator.py which provides make_algorithm in Gaudi.
  • Created Rec/Allen/python/AllenConf/algorithms.py which acts as a proxy for PyConf.Algorithms.
  • Packages configuration and Rec/Allen export now their python directory using gaudi_install_python_modules.

Functionality this enables

  • This MR streamlines the creation of Gaudi tests for Allen algorithms.
  • It allows the creation of subsequences that run as independent algorithms.
  • Concretely, all Allen-to-Gaudi conversions can now become proper Gaudi algorithms.
  • It is a step to removing host_buffers in all Allen algorithms, which was required to persist objects after the Allen sequence was run.

Example - Search by triplet generated conversion

#include "AlgorithmConversionTools.h"
#include </home/nnolte/lb-stack-setup/stack5/Allen/device/velo/search_by_triplet/include/SearchByTriplet.cuh>
#include <GaudiAlg/Transformer.h>
#include <vector>
// output type
using output_t = std::tuple<
  Allen::parameter_vector<velo_search_by_triplet::Parameters::dev_tracks_t>,
  Allen::parameter_vector<velo_search_by_triplet::Parameters::dev_tracklets_t>,
  Allen::parameter_vector<velo_search_by_triplet::Parameters::dev_tracks_to_follow_t>,
  Allen::parameter_vector<velo_search_by_triplet::Parameters::dev_three_hit_tracks_t>,
  Allen::parameter_vector<velo_search_by_triplet::Parameters::dev_hit_used_t>,
  Allen::parameter_vector<velo_search_by_triplet::Parameters::dev_atomics_velo_t>,
  Allen::parameter_vector<velo_search_by_triplet::Parameters::dev_rel_indices_t>,
  Allen::parameter_vector<velo_search_by_triplet::Parameters::dev_number_of_velo_tracks_t>>;
// algorithm definition
struct velo_search_by_triplet_t_gaudi final
  : Gaudi::Functional::MultiTransformer<output_t(
      Allen::parameter_vector<velo_search_by_triplet::Parameters::host_number_of_events_t> const&,
      Allen::parameter_vector<velo_search_by_triplet::Parameters::host_total_number_of_velo_clusters_t> const&,
      Allen::parameter_vector<velo_search_by_triplet::Parameters::dev_event_list_t> const&,
      Allen::parameter_vector<velo_search_by_triplet::Parameters::dev_number_of_events_t> const&,
      Allen::parameter_vector<velo_search_by_triplet::Parameters::dev_sorted_velo_cluster_container_t> const&,
      Allen::parameter_vector<velo_search_by_triplet::Parameters::dev_offsets_estimated_input_size_t> const&,
      Allen::parameter_vector<velo_search_by_triplet::Parameters::dev_module_cluster_num_t> const&,
      Allen::parameter_vector<velo_search_by_triplet::Parameters::dev_hit_phi_t> const&,
      Allen::parameter_vector<velo_search_by_triplet::Parameters::dev_velo_clusters_t> const&,
      const RuntimeOptions&,
      Constants const* const&)> {
  // wrapped algorithm body
  velo_search_by_triplet_t_gaudi(std::string const& name, ISvcLocator* pSvc) :
    MultiTransformer(
      name,
      pSvc,
      {KeyValue("host_number_of_events_t", {""}),
       KeyValue("host_total_number_of_velo_clusters_t", {""}),
       KeyValue("dev_event_list_t", {""}),
       KeyValue("dev_number_of_events_t", {""}),
       KeyValue("dev_sorted_velo_cluster_container_t", {""}),
       KeyValue("dev_offsets_estimated_input_size_t", {""}),
       KeyValue("dev_module_cluster_num_t", {""}),
       KeyValue("dev_hit_phi_t", {""}),
       KeyValue("dev_velo_clusters_t", {""}),
       KeyValue("runtime_options_t", {""}),
       KeyValue("constants_t", {""})},
      {KeyValue("dev_tracks_t", {""}),
       KeyValue("dev_tracklets_t", {""}),
       KeyValue("dev_tracks_to_follow_t", {""}),
       KeyValue("dev_three_hit_tracks_t", {""}),
       KeyValue("dev_hit_used_t", {""}),
       KeyValue("dev_atomics_velo_t", {""}),
       KeyValue("dev_rel_indices_t", {""}),
       KeyValue("dev_number_of_velo_tracks_t", {""})})
  {}
  // operator()
  output_t operator()(
    Allen::parameter_vector<velo_search_by_triplet::Parameters::host_number_of_events_t> const&
      host_number_of_events_t_arg,
    Allen::parameter_vector<velo_search_by_triplet::Parameters::host_total_number_of_velo_clusters_t> const&
      host_total_number_of_velo_clusters_t_arg,
    Allen::parameter_vector<velo_search_by_triplet::Parameters::dev_event_list_t> const& dev_event_list_t_arg,
    Allen::parameter_vector<velo_search_by_triplet::Parameters::dev_number_of_events_t> const&
      dev_number_of_events_t_arg,
    Allen::parameter_vector<velo_search_by_triplet::Parameters::dev_sorted_velo_cluster_container_t> const&
      dev_sorted_velo_cluster_container_t_arg,
    Allen::parameter_vector<velo_search_by_triplet::Parameters::dev_offsets_estimated_input_size_t> const&
      dev_offsets_estimated_input_size_t_arg,
    Allen::parameter_vector<velo_search_by_triplet::Parameters::dev_module_cluster_num_t> const&
      dev_module_cluster_num_t_arg,
    Allen::parameter_vector<velo_search_by_triplet::Parameters::dev_hit_phi_t> const& dev_hit_phi_t_arg,
    Allen::parameter_vector<velo_search_by_triplet::Parameters::dev_velo_clusters_t> const& dev_velo_clusters_t_arg,
    const RuntimeOptions& runtime_options,
    Constants const* const& constants) const override
  {
    output_t output_container {};
    // TES wrappers
    Allen::TESWrapperInput<velo_search_by_triplet::Parameters::host_number_of_events_t>
      host_number_of_events_t_wrapper {host_number_of_events_t_arg, "host_number_of_events_t"};
    Allen::TESWrapperInput<velo_search_by_triplet::Parameters::host_total_number_of_velo_clusters_t>
      host_total_number_of_velo_clusters_t_wrapper {
        host_total_number_of_velo_clusters_t_arg, "host_total_number_of_velo_clusters_t"};
    Allen::TESWrapperInput<velo_search_by_triplet::Parameters::dev_event_list_t> dev_event_list_t_wrapper {
      dev_event_list_t_arg, "dev_event_list_t"};
    Allen::TESWrapperInput<velo_search_by_triplet::Parameters::dev_number_of_events_t> dev_number_of_events_t_wrapper {
      dev_number_of_events_t_arg, "dev_number_of_events_t"};
    Allen::TESWrapperInput<velo_search_by_triplet::Parameters::dev_sorted_velo_cluster_container_t>
      dev_sorted_velo_cluster_container_t_wrapper {
        dev_sorted_velo_cluster_container_t_arg, "dev_sorted_velo_cluster_container_t"};
    Allen::TESWrapperInput<velo_search_by_triplet::Parameters::dev_offsets_estimated_input_size_t>
      dev_offsets_estimated_input_size_t_wrapper {
        dev_offsets_estimated_input_size_t_arg, "dev_offsets_estimated_input_size_t"};
    Allen::TESWrapperInput<velo_search_by_triplet::Parameters::dev_module_cluster_num_t>
      dev_module_cluster_num_t_wrapper {dev_module_cluster_num_t_arg, "dev_module_cluster_num_t"};
    Allen::TESWrapperInput<velo_search_by_triplet::Parameters::dev_hit_phi_t> dev_hit_phi_t_wrapper {
      dev_hit_phi_t_arg, "dev_hit_phi_t"};
    Allen::TESWrapperOutput<velo_search_by_triplet::Parameters::dev_tracks_t> dev_tracks_t_wrapper {
      std::get<0>(output_container), "dev_tracks_t"};
    Allen::TESWrapperOutput<velo_search_by_triplet::Parameters::dev_tracklets_t> dev_tracklets_t_wrapper {
      std::get<1>(output_container), "dev_tracklets_t"};
    Allen::TESWrapperOutput<velo_search_by_triplet::Parameters::dev_tracks_to_follow_t> dev_tracks_to_follow_t_wrapper {
      std::get<2>(output_container), "dev_tracks_to_follow_t"};
    Allen::TESWrapperOutput<velo_search_by_triplet::Parameters::dev_three_hit_tracks_t> dev_three_hit_tracks_t_wrapper {
      std::get<3>(output_container), "dev_three_hit_tracks_t"};
    Allen::TESWrapperOutput<velo_search_by_triplet::Parameters::dev_hit_used_t> dev_hit_used_t_wrapper {
      std::get<4>(output_container), "dev_hit_used_t"};
    Allen::TESWrapperOutput<velo_search_by_triplet::Parameters::dev_atomics_velo_t> dev_atomics_velo_t_wrapper {
      std::get<5>(output_container), "dev_atomics_velo_t"};
    Allen::TESWrapperOutput<velo_search_by_triplet::Parameters::dev_rel_indices_t> dev_rel_indices_t_wrapper {
      std::get<6>(output_container), "dev_rel_indices_t"};
    Allen::TESWrapperOutput<velo_search_by_triplet::Parameters::dev_number_of_velo_tracks_t>
      dev_number_of_velo_tracks_t_wrapper {std::get<7>(output_container), "dev_number_of_velo_tracks_t"};
    Allen::TESWrapperInput<velo_search_by_triplet::Parameters::dev_velo_clusters_t> dev_velo_clusters_t_wrapper {
      dev_velo_clusters_t_arg, "dev_velo_clusters_t"};
    // Inputs to set_arguments_size and operator()
    std::array<std::reference_wrapper<ArgumentData>, 17> tes_wrappers_references {
      host_number_of_events_t_wrapper,
      host_total_number_of_velo_clusters_t_wrapper,
      dev_event_list_t_wrapper,
      dev_number_of_events_t_wrapper,
      dev_sorted_velo_cluster_container_t_wrapper,
      dev_offsets_estimated_input_size_t_wrapper,
      dev_module_cluster_num_t_wrapper,
      dev_hit_phi_t_wrapper,
      dev_tracks_t_wrapper,
      dev_tracklets_t_wrapper,
      dev_tracks_to_follow_t_wrapper,
      dev_three_hit_tracks_t_wrapper,
      dev_hit_used_t_wrapper,
      dev_atomics_velo_t_wrapper,
      dev_rel_indices_t_wrapper,
      dev_number_of_velo_tracks_t_wrapper,
      dev_velo_clusters_t_wrapper};
    HostBuffers host_buffers {};
    Allen::Context context {};
    // set arguments size invocation
    m_algorithm.set_arguments_size(tes_wrappers_references, runtime_options, *constants, host_buffers);
    // algorithm operator() invocation
    m_algorithm(tes_wrappers_references, runtime_options, *constants, host_buffers, context);
    return output_container;
  }

private:
  velo_search_by_triplet::velo_search_by_triplet_t m_algorithm {};
  Gaudi::Property<int> m_verbosity_t {
    this,
    "verbosity",
    3,
    [=](auto&) { m_algorithm.set_property_value<Allen::Algorithm::verbosity_t, int>(m_verbosity_t.value()); },
    Gaudi::Details::Property::ImmediatelyInvokeHandler {true},
    "verbosity of algorithm"};
  Gaudi::Property<float> m_phi_tolerance_t {
    this,
    "phi_tolerance",
    0.045f,
    [=](auto&) {
      m_algorithm.set_property_value<velo_search_by_triplet::Parameters::phi_tolerance_t, float>(
        m_phi_tolerance_t.value());
    },
    Gaudi::Details::Property::ImmediatelyInvokeHandler {true},
    "tolerance in phi"};
  Gaudi::Property<float> m_max_scatter_t {
    this,
    "max_scatter",
    0.08f,
    [=](auto&) {
      m_algorithm.set_property_value<velo_search_by_triplet::Parameters::max_scatter_t, float>(m_max_scatter_t.value());
    },
    Gaudi::Details::Property::ImmediatelyInvokeHandler {true},
    "maximum scatter for seeding and forwarding"};
  Gaudi::Property<unsigned int> m_max_skipped_modules_t {
    this,
    "max_skipped_modules",
    1,
    [=](auto&) {
      m_algorithm.set_property_value<velo_search_by_triplet::Parameters::max_skipped_modules_t, unsigned int>(
        m_max_skipped_modules_t.value());
    },
    Gaudi::Details::Property::ImmediatelyInvokeHandler {true},
    "skipped modules"};
  Gaudi::Property<unsigned int> m_block_dim_x_t {
    this,
    "block_dim_x",
    64,
    [=](auto&) {
      m_algorithm.set_property_value<velo_search_by_triplet::Parameters::block_dim_x_t, unsigned int>(
        m_block_dim_x_t.value());
    },
    Gaudi::Details::Property::ImmediatelyInvokeHandler {true},
    "block dimension x"};
};
DECLARE_COMPONENT(velo_search_by_triplet_t_gaudi)

This MR is built on top of !393 (merged) and !429 (merged).

Should go together with Moore!609 (merged).

Edited by Daniel Hugo Campora Perez

Merge request reports