Switch to rapidyaml for loading conditions
The goal of this MR is to replace the yaml-cpp library we use when loading conditions by https://github.com/biojppm/rapidyaml
We noticed a big memory consumption comming from yaml-cpp when running HLT2 online, caused by memory fragmentation.
FYI @clemenci
Goes with LHCb!4980 (merged) Rec!4306 (merged) Allen!1928 (merged) Alignment!613 (merged) Boole!689 (merged)
Results:
This MR:
INFO DependencyHandler Inserted 3 [3] conditions to pool-iov: run(0):[0-9223372036854775807] [0.00001 seconds]
INFO DependencyHandler Inserted 17 [17] conditions to pool-iov: run(0):[291593-9223372036854775807] [0.00001 seconds]
INFO DependencyHandler Inserted 24 [24] conditions to pool-iov: run(0):[291820-291820] [0.00001 seconds]
INFO DependencyHandler Inserted 77 [77] conditions to pool-iov: run(0):[291820-291821] [0.00002 seconds]
INFO DetectorDataService + Created/Accessed a total of 6643 conditions (S: 3261,L: 3261,C: 121,M:0) Load:4.66510s/93896KB Compute:15.46900s/196608KB
...
INFO DependencyHandler Inserted 101 [101] conditions to pool-iov: run(0):[292794-292794] [0.00003 seconds]
INFO DetectorDataService + Created/Accessed a total of 6973 conditions (S: 3611,L: 3261,C: 101,M:0) Load:0.54199s/0KB Compute:14.11437s/65536KB
Reference:
INFO DependencyHandler Inserted 3 [3] conditions to pool-iov: run(0):[0-9223372036854775807] [0.00001 seconds]
INFO DependencyHandler Inserted 17 [17] conditions to pool-iov: run(0):[291593-9223372036854775807] [0.00001 seconds]
INFO DependencyHandler Inserted 24 [24] conditions to pool-iov: run(0):[291820-291820] [0.00001 seconds]
INFO DependencyHandler Inserted 77 [77] conditions to pool-iov: run(0):[291820-291821] [0.00003 seconds]
INFO DetectorDataService + Created/Accessed a total of 6643 conditions (S: 3261,L: 3261,C: 121,M:0) Load:17.88175s/1113484KB Compute:17.09771s/14968KB
...
INFO DependencyHandler Inserted 101 [101] conditions to pool-iov: run(0):[292794-292794] [0.00003 seconds]
INFO DetectorDataService + Created/Accessed a total of 6973 conditions (S: 3611,L: 3261,C: 101,M:0) Load:0.67283s/0KB Compute:14.22873s/0KB
Condition loading time decreased from 17s to 4s and memory consumption (virtual) from 1.1GB to 94MB, indicating less fragmentation.
Also found a bug in derived condition's Compute which slowed down DeUTObject construction, fixing it improve compute time from 17s to 3s:
INFO DetectorDataService + Created/Accessed a total of 6643 conditions (S: 3261,L: 3261,C: 121,M:0) Load:3.35545s/92148KB Compute:3.10019s/196608KB
...
INFO DetectorDataService + Created/Accessed a total of 6973 conditions (S: 3611,L: 3261,C: 101,M:0) Load:0.52090s/0KB Compute:2.52502s/65536KB
Validated by
-
Core Software -
RTA -
Simulation
Edited by Miroslav Saur