Switch to rapidyaml for loading conditions

The goal of this MR is to replace the yaml-cpp library we use when loading conditions by https://github.com/biojppm/rapidyaml

We noticed a big memory consumption comming from yaml-cpp when running HLT2 online, caused by memory fragmentation.

FYI @clemenci

Goes with LHCb!4980 (merged) Rec!4306 (merged) Allen!1928 (merged) Alignment!613 (merged) Boole!689 (merged)

Results:

This MR:

INFO     DependencyHandler         Inserted 3 [3] conditions to pool-iov: run(0):[0-9223372036854775807]   [0.00001 seconds]
INFO     DependencyHandler         Inserted 17 [17] conditions to pool-iov: run(0):[291593-9223372036854775807]   [0.00001 seconds]
INFO     DependencyHandler         Inserted 24 [24] conditions to pool-iov: run(0):[291820-291820]   [0.00001 seconds]
INFO     DependencyHandler         Inserted 77 [77] conditions to pool-iov: run(0):[291820-291821]   [0.00002 seconds]
INFO     DetectorDataService       +  Created/Accessed a total of 6643 conditions (S:  3261,L:  3261,C:   121,M:0)  Load:4.66510s/93896KB Compute:15.46900s/196608KB
...
INFO     DependencyHandler         Inserted 101 [101] conditions to pool-iov: run(0):[292794-292794]   [0.00003 seconds]
INFO     DetectorDataService       +  Created/Accessed a total of 6973 conditions (S:  3611,L:  3261,C:   101,M:0)  Load:0.54199s/0KB Compute:14.11437s/65536KB

Reference:

INFO     DependencyHandler         Inserted 3 [3] conditions to pool-iov: run(0):[0-9223372036854775807]   [0.00001 seconds]
INFO     DependencyHandler         Inserted 17 [17] conditions to pool-iov: run(0):[291593-9223372036854775807]   [0.00001 seconds]
INFO     DependencyHandler         Inserted 24 [24] conditions to pool-iov: run(0):[291820-291820]   [0.00001 seconds]
INFO     DependencyHandler         Inserted 77 [77] conditions to pool-iov: run(0):[291820-291821]   [0.00003 seconds]
INFO     DetectorDataService       +  Created/Accessed a total of 6643 conditions (S:  3261,L:  3261,C:   121,M:0)  Load:17.88175s/1113484KB Compute:17.09771s/14968KB
...
INFO     DependencyHandler         Inserted 101 [101] conditions to pool-iov: run(0):[292794-292794]   [0.00003 seconds]
INFO     DetectorDataService       +  Created/Accessed a total of 6973 conditions (S:  3611,L:  3261,C:   101,M:0)  Load:0.67283s/0KB Compute:14.22873s/0KB

Condition loading time decreased from 17s to 4s and memory consumption (virtual) from 1.1GB to 94MB, indicating less fragmentation.

Also found a bug in derived condition's Compute which slowed down DeUTObject construction, fixing it improve compute time from 17s to 3s:

INFO     DetectorDataService       +  Created/Accessed a total of 6643 conditions (S:  3261,L:  3261,C:   121,M:0)  Load:3.35545s/92148KB Compute:3.10019s/196608KB
...
INFO     DetectorDataService       +  Created/Accessed a total of 6973 conditions (S:  3611,L:  3261,C:   101,M:0)  Load:0.52090s/0KB Compute:2.52502s/65536KB

Validated by

  • Core Software
  • RTA
  • Simulation
Edited by Miroslav Saur

Merge request reports

Loading