Improvements for MVA inference
A few things came up recently (feel free to add / signal your needs):
-
multiple input nodes (and types) for Tensorflow. With @fbury we made a workaround for the specific case, but for a generic solution we should probably turn the evaluate
method into a variadic template, each argument passed to one node (for the outputs I can't think of a better solution thanstd::any
but so far there hasn't been a need for that) -> done in https://gitlab.cern.ch/cp3-cms/bamboo/-/merge_requests/156 for inputs -
XGBoost (requested by @gsaha, started an implementation, but need a BDT file for testing) -> still pending -
ONNX Runtime seems to be fast (I saw a few cases where it's faster than Tensorflow for the same network), so would be a nice addition (it also dynamically selects instructions depending on the CPU generation, and supports multi-threading). I compiled it, but didn't implement a evaluator bridge class yet. -> done in https://gitlab.cern.ch/cp3-cms/bamboo/-/merge_requests/156, and working quite nicely -
on the latter point, the tensorflow evaluator is not thread-safe because it reuses the tensors, but if the inputs are moved in it should be acceptable to allocate them per call. The rest should be fine (lwtnn and torch(script) should be efficient, TMVA uses locking). -> done in https://gitlab.cern.ch/cp3-cms/bamboo/-/merge_requests/156 -
multiple inputs for PyTorch: should be possible, but I need to have a closer look at the documentation for how to get the shapes from the model then (I don't think anyone is using this yet, or is planning to anytime soon)
Edited by Pieter David