Skip to content

[2025-patches] Optimize hash_object using orjson+xxhash

Part of #397

This optimises hash_object by using a faster JSON encoder and faster hash algorithm. The dependencies currently aren't on CVMFS but are in LCG so @clemenci is going to install them (hence the draft state).

Using three randomly chosen inputs to hash_object of different sizes the timing goes from:

5.28 µs ± 2.81 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
42 µs ± 37.5 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)
599 µs ± 1.12 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

to:

704 ns ± 3.71 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
3.89 µs ± 5.01 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
66.9 µs ± 40.2 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)

cc @sponce @ahennequ @msaur

Edited by Chris Burr

Merge request reports

Loading