feat: Switch to using nanobind from pybind11 (!10)
Now that there is a stable v1.0.0
release of nanobind
, this MR follows the nanobind
docs instructions on getting setup with nanobind
and CMake and porting a pybind11
project to nanobind
.
The main reasons that nanobind
is attractive for the Columnar Analysis project are (in my mind):
More concretely, benchmarks show ~2-3× faster compile time, ~3× smaller binaries, and up to ~8× lower runtime overheads compared to pybind11.
nanobind can exchange n-dimensional arrays (henceforth “ndarrays”) with popular array programming frameworks including NumPy, PyTorch, TensorFlow, and JAX. It supports zero-copy exchange using two protocols:
- The classic buffer protocol.
- DLPack, a GPU-compatible generalization of the buffer protocol.
nanobind knows how to talk to each framework and takes care of all the nitty-gritty details.
Both of those seem like big wins to me!
I also want to thank @gstark and (especially big thanks to) Angus Hollands for their help in debugging some small C++ things that I messed up over in the IRIS-HEP Slack (my C++-foo in my postdoc years had become quite degraded compared to my Ph.D. days
- Use
nanobind
v1.0.0
to provide C++/Python bindings.-
Migrating from pybind11 to nanobind is done by mainly following the v1.0.0 docs on installing as a Git submodule and then providing the necessary information in the CMake
CMakeLists.txt
to properly find and build nanobind, though while the use ofDevelopment.module
is recommendedfind_package(Python 3.8 COMPONENTS Interpreter Development.Module REQUIRED)
Development
is used instead to allow for use ofPython::Python
. -
On the Python side most of what is done is just changing names from 'pybind11' to 'nanobind', and then taking advantage of the
nb::ndarray
API to abstract away all interactions with buffers, which simplifies things quite a bit. -
To properly allow for the Python object that has the array data to live long enough for its data to be referenced by a consumer dynamically allocate the vector and then use
nb::capsule
to wrap the C++ pointer for it and provide a cleanup routine when the capsule is garbage collected.
-