Extend HltLumiSummary decoding to Run 3 format

Add decoding for the new key-less LumiSummary structure encoded in Allen!743 (merged) and Allen!950 (merged). The list of fields matches that added to Allen. This version is incompatible with upstream run1+2 code (legacy keyed LumiSummaries cannot be read) but is compatible with both V1 and V2 banks from run 3. This version also includes code to decode LumiSummary banks where the structure is encoded in the TCK but this functionality still needs to be tested - this required merging in commits from !3788 (closed).

Another related Moore MR will also be needed to add encoding/decoding tests.

To be tested with Allen!950 (merged) Rec!3108 (merged) Moore!1761 (merged) lhcb-datapkg/PRConfig!266 (merged)

FYI @rmatev

added 1 commit

• 44c9aab6 - Fixed formatting

Compare with previous version

added RTA label

Is this ready for undrafting and testing ?

Allen!968 (merged) has been merged, and we will update the sequence in the pit to start writing raw banks.

We should move forward with this MR ASAP to get it ready for the VdM scan.

A key point that came up in RTA coordination is that two tests are added that:

1. run Allen/HLT1 to write lumi banks in a file
2. read the file, decode the lumi summary and check that it makes sense.

assigned to @dcraik

assigned to @sxian

ping @dcraik, @sxian

I'm working on this now and should have something tonight or tomorrow.

I had one question regarding the counterKey enum - we need to add a further 9 values, which won't fit in the gap before Method = 20, (we're currently up to 16). My plan is to add the extra keys (which are grouped in a set of 4 and a set of 5) starting from 30 but I wanted to check if anyone had a preferred alternative.

Hello @dcraik, @sxian, I "bumped" into this MR. Let me make a little comment: do not hesitate to ping us DPA, especially @pkoppenb, for matters relevant to how lumi information will eventually make it offline. We touched on the topic of lumi at our coords meeting this morning, funny enough, and are keen to know how the "handshake" between lumi info created online and made available offline to users will happen. Advance thanks.

To Dan: there is absolutely no need to be compatible with Run 2 in lumi summary format. You have full freedom to destroy completely backward compatibility.

More specifically, the nano events were used in Run 1,2 in two places ONLY:

• in FSRs which will be new in Run 3 and will keep only the event statistics irrespective of their content,
• and in lumi-analysis. For that the lumi-stream was transformed to ntuples stored in /eos/lhcb/wg/Luminosity/data/counters. The ntuple has a standard format with one column per lumi-counter, independent of the format of lumi-summary. So, all lumi-analysis programs were decoupled from the lumi-summary format (we know it with Rosen because we did the lumi-analysis; note that probably no one else can confirm it).

To summarize, compared to Run 2 the ONLY place where the new lumi summary format will cause changes is in this single job reading the lumi-events and writing the ntuples. There are no other places.

I suppose all code dependent on the old LumiSummary is obsolete. And it is a good criterium to see which parts can be removed to keep the code cleaner. Could you, please, send me a link to the suspects?

added 1 commit

• ca5bc09a - Update for V1 of lumi summary; remove legacy run1+2 version

Compare with previous version

added 1 commit

• 99798f5c - Fixed formatting

Compare with previous version

marked this merge request as ready

changed the description

added 1 commit

• f43fbc90 - Shift enum values so subdetectors start on multiples of 10

Compare with previous version

@dcraik I see you marked this as ready -- please rebase, the branch is far behind master, and ping me if you want to test this :)

added 676 commits

• f43fbc90...76f7cb55 - 670 commits from branch master
• c469ec5d - Extend HltLumiSummary decoding to Run 3 format
• 806fe827 - Fixed formatting
• ddd25d3d - Update for V1 of lumi summary; remove legacy run1+2 version
• 8527c097 - Fixed formatting
• 60d48e76 - Shift enum values so subdetectors start on multiples of 10
• b1929072 - Merge branch 'dcraik_decodeLumiSummary' of ssh://gitlab.cern.ch:7999/lhcb/LHCb...

Compare with previous version

added ci-test-triggered label