Implement routing bits

requested review from @raaij

requested review from @mfontana and @dovombru

added 1 commit

7120e7fd - code cleanup

I think this is a good start, but some work will still be needed.

A few general comments

Routing bit definitions start counting from 1.
HLT1 should produce two 32-bit words of routing bits.
Routing bits have been more than a single regex, logical operations (&, |, ~) between regexes have been used. Looking at the Run 2 routing bits for HLT1, only regexes was sufficient, but implementing this later shouldn't be made impossible by the current implementation.
The Run 2 routing bits are defined here: https://edms.cern.ch/document/1146861/1.7 and an example implementation as regexes can be found starting on line 374 of this file: /cvmfs/lhcb.cern.ch/lib/lhcb/HLT/HLT_v28r3p1/Hlt/HltConf/python/HltConf/HltOutput.py.
I don't think anything on the device is needed. I'd suggest to use the host_dec_reports_t as input for a host_algorithm. They are guaranteed to be available on the host after the dec_reporter_t runs.

What I would concretely suggest is to use the configuration in Python to generate a set of functions that are called on the host to create the two routing bit words. The translation of line names to index of the DecReports required by each function can be done on the python side to avoid having to resolve them repeatedly in the C++. Perhaps something like the following (completely untested):

// routing_bits_generated.cpp
namespace Allen {
  namespace RoutingBits {
    inline bool line_passed(unsigned const* reports, unsigned const index) {
      return HltDecReport const report{reports[i]}.decision();
    }
    inline bool routing_bit_33(unsigned const* reports) {
      for (auto i : {4u, 6u, 12u}) { 
        if (line_passed(reports, i)) return true;
      }
      return false;
    }

    inline bool routing_bit_34(unsigned const* reports) {
      for (auto i : {2u, 3u, 7u, 12u}) { 
        if (line_passed(reports, i)) return true;
      }
      return false;
    }
  }

  std::array<bool(unsigned const* dec_reports), 64> routing_bits_functions() {
    std::array<bool(unsigned const* dec_reports), 64> functions{nullptr};
    functions[32] = &Allen::RoutingBits::routing_bit_33;
    functions[33] = &Allen::RoutingBits::routing_bit_34;
    // etc
    return functions;
  }
}

// routing_bits.h
namespace Allen {
  std::array<bool(unsigned const* dec_reports), 64> routing_bits_functions();
}

namespace routing_bits {
  struct Parameters {
    // ...
  };

  struct routing_bits_t : public HostAlgorithm, Parameters {
    void set_arguments_size(
      ArgumentReferences<Parameters> arguments,
      const RuntimeOptions& runtime_options,
      const Constants&,
      const HostBuffers&) const;

    void init();

    void operator()(
      const ArgumentReferences<Parameters>& arguments,
      const RuntimeOptions& runtime_options,
      const Constants&,
      HostBuffers& host_buffers,
      const Allen::Context& context) const;

  private:
    std::array<bool(unsigned const*), 64> m_routing_bits_functions;
  };
} // namespace host_global_event_cut


// routing_bits.cpp
void routing_bits::routing_bits_t::init()
{
  m_routing_bits_functions = Allen::routing_bits_functions();
}

void routing_bits::routing_bits_t::operator()(
  const ArgumentReferences<Parameters>& arguments,
  const RuntimeOptions& runtime_options,
  const Constants&,
  HostBuffers& host_buffers,
  const Allen::Context& context) const
{
  for(unsigned event = 0; event < parameters.host_number_of_events[0]; ++event) {
    unsigned const* words = parameters.routing_bits[2 * event];
    unsigned const* dec_reports = parameters.host_dec_reports + (2 + parameters.host_number_of_active_lines[0]) * event;
    for (unsigned rb = 0; rb < m_routing_bits_functions.size(); ++rb) {
      auto rb_fun = m_routing_bits_functions[rb];
      if (rb_fun == nullptr) continue;
      words[rb / 64] |= (rb_fun(dec_reports) << (rb % 32));
    }
  }
}

Hey @raaij , thanks a lot for the suggestions. I will start working towards the implementation of the regex logic, and I agree that a python configuration seems like the best approach. I am a bit unsure how to go about it so that the configuration is properly generated and passed to Allen, is there an example (for instance a script where this is already being used for another functionality) that you could point me towards?

Concerning your comment on the size of the routing bits output, what is the second word for? I know in Run 2 there was 1 word for ODIN + L0, 1 for HLT1 and 1 for HLT2. Is the second word meant to be a placeholder for HLT2 or for ODIN?

About your last bullet point, I initially thought I would do everything on the host as well, but I was concerned it might be slower so I moved part of the algorithm to the device to have some parallelization if needed. But if you believe there will be no speed-related issues I can migrate the full algorithm to the host.

The first word was indeed filled with L0 and ODIN related routing bits, but HLT1 was always the one setting them. The first 32 routing bits were not used during the last part of Run 2, but since they are free, the possibility to use them should be there.

I think it's fine to do everything on the host. I doubt that there will be any issue with throughput.

Thinking a bit more, code generation is probably not the way to go, sorry for the noise. The main reason being that in general Allen is moving to doing more at runtime. That would mean that the list of active lines is no longer known when generating code, which removes all benefits of code generation.

So, as much at initialization and the rest at runtime then. To keep things moving, that does mean that logical operations are out for the foreseeable future.

The routing bits should become a property of the host algorithm. Unfortunately the Allen property parsers are rather limited in functionality, so a new specialization that takes care of parsing std::map<int, std::string> will have to be added to Property.cuh.

The list of names of configured lines is only available as an input and not when initializing, so I would then suggest to keep things very simple:

assume each routing bit is a single regex.
create an std::regex for each routing bit.
For each event, loop through all routing bits and match them one-by-one to the names of lines that passed, if one matches, set the bit and go to the next routing bit.

Implement routing bits

Merge request reports

Activity