Skip to content

Add a backend with fully compiled code (standalone executable)

Pieter David requested to merge piedavid/bamboo:compiledbackend into master

As suggested in https://gitlab.cern.ch/cp3-cms/bamboo/-/issues/64 and the discussion in https://indico.cern.ch/event/963454/ .

Current status: very early still, but the following simple example works (without on-demand corrections - and too minimal for benchmarking, unfortunately), so most of the basic parts should be there.

bambooRun -v --module=examples/nanozmumu.py:MinimalNanoZMuMu bamboo/examples/test1.yml --output=test_out/nanozmm_test2 --backend=compiled --threads=2

Assuming no other big issues are found, the main remaining challenge is getting C++ extensions included correctly as well. For the jet/met and rochester correction the methods used to configure them already get a reference to the backend, so there I can make sure anything necessary gets passed (also configuration). For the MVA libraries and user-added things, the most straightforward seems to be to ask the user to call a loadXYZ(be) method from prepareTree (which gets a backend reference from the base class method - definePlots does not have this by design, but forceDefine breaks this rule, so it's possible to work around that if really needed); for user extensions, this could be a load method in bamboo.root that takes keyword arguments for an include path, dynamic library path, headers, and shared libraries (if needed also lines of C++ code for initialisation). This is just a first idea, so opinions and feedback are most welcome.

Backwards-incompatible changes:

  • NanoAODModule.prepareTree now takes a backend string argument ("lazy", "compiled" - otherwise the default), instead of a lazyBackend boolean
  • installation of the non-python parts (headers, shared libraries, cmake modules) works differently (using scikit-build, a small standalone package, to call cmake, and putting everything in the bamboo python package directory). This has two consequences: the safest is to recreate the virtualenv when updating, and python setup.py build does not work anymore to get a fully working development install; pip install -e does, though, and it's more efficient and stable than before, so not a huge loss (the absence of scikit-build in the CI is what broke the related test, the easiest is probably to remove that option).
  • as a consequence of the above, pip builds in a isolated environment, so picking up pytorch from the LCG distribution does not work as before. The two options are forcing a non-isolated build, or picking it up as an optional dependency (see the documentation).

Remaining:

  • pass variable and weight types to HistoND
  • improve test coverage for different column and expression output types (or use cling to get/check the result type of expressions)
  • avoid redefining existing columns wherever possible
  • documentation (adding extensions after these changes, compiled backend specifics, loading ML frameworks)
  • IMT with lazy backend (regression) - probably make it a backend constructor parameter
  • add ROOT::PhysicsROOT::GenVector to the default list of linked targets (pulled in by headers)
  • propagate includePath, dynamicPath to CMakeLists
  • keep the tmp directory in case of build errors (it is getting removed automatically by tempfile.TemporaryDirectory, I don't know how to disable it)
  • allow passing extra options to cmake
  • skip dynamic compilation of helper functions with the compiled backend (needs a distinction between declaring instances and functions, currently they are treated the same because both use TInterpreter::Declare)
  • rebase after https://gitlab.cern.ch/cp3-cms/bamboo/-/merge_requests/141 is merged (and adapt a few things there to RangeOp)
  • benchmarking for PR purposes 😎
  • support CutFlowReport
  • support skims - or at least print a clear error to show that they're not supported
  • check if the limit on the number of input files also applies here (otherwise read from the textfile directly)

Afterwards:

  • separate compilation (for all samples, or better: unique graphs) from processing step (running the former locally and the latter distributed, or compiling in slurm jobs with a higher memory limit)
Edited by Pieter David

Merge request reports