Skip to content

[RTA/DPA BW Tests] Correct compression level for HLT2 in BW Tests

Luke Grazette requested to merge lugrazet-BWTest-compressionsettings into master

Looking at Rosen's script (/home/rmatev/piquet/stream.py in the Online network), --compress simply does zstd -q -f -o filename, i.e. doesn't specify a compression level.

This would be equivalent to zstd -3, c.f. zstd -5 which we use currently in the BW Tests.
This should make negligible difference, but we want to be as accurate to the data conditions as we can be.

Difference

over some test number of events, they have similar compression factors. 33.8% and 34.8%. Thus we expect this will increase the result of the BWTests by a small margin, but if it's more accurate to data-taking then we want to include it.

[phrhbm@veleta modern_stack]$ zstd -5 tmp/MDF/hlt2_bw_testing__production__full.mdf 
tmp/MDF/hlt2_bw_testing__production__full.mdf : 33.82%   (  12.5 MiB =>   4.22 MiB, tmp/MDF/hlt2_bw_testing__production__full.mdf.zst) 

[phrhbm@veleta modern_stack]$ zstd tmp/MDF/hlt2_bw_testing__production__full.mdf 
tmp/MDF/hlt2_bw_testing__production__full.mdf : 34.76%   (  12.5 MiB =>   4.34 MiB, tmp/MDF/hlt2_bw_testing__production__full.mdf.zst) 

ToDo:

  • Check locally
  • Check in ci-test (Test with lhcb/Moore!3795 + lhcb/LHCb!4661)
  • Check that this is what the data movers do? (Chris Burr / Frederic Hemmer)
Edited by Luke Grazette

Merge request reports