Corrupt HltLumiSummary from HLT1

The encoding key in the HltLumiSummary banks out of HLT1 is corrupted when running a TCK made from hlt1_PbPb_PbSMOG_no_ut on the latest release lhcb-core/lhcbstacks!185 (merged)

Run https://lbrundb.cern.ch/rundb/run/277247/ was taken (without beams) to test the TCK. A problem was observed in the LumiMon task:

Screenshot_2023-09-25_at_11.22.41_PM

An HLT1 output file can be checked with pcie40_mdfreader and it shows that about 1/4 of the events have a wrong encoding key (we expect only 9F DC 6A 9F and B9 E6 7A 41):

[rmatev@pluscc06 ~]$ pcie40_mdfreader /scratch/rmatev/Run_0000277247_20230925-231853-098_ECEB09_0001.mdf | head -n 1000000 | grep -A2 HltLumiSummary | grep -o '0x0000..............' | sort | uniq -c
      6 0x0000 | 9C DC 6A 9F
     29 0x0000 | 9D DC 6A 9F
    185 0x0000 | 9E DC 6A 9F
    632 0x0000 | 9F DC 6A 9F
      3 0x0000 | B8 E6 7A 41
     25 0x0000 | B9 E6 7A 41

Thanks to @gunther @miolocco @oozcelik for uncovering this issue.

Followed up by @dcraik but help from others is appreciated!

/cc @cagapopo @raaij

Edited by Rosen Matev