v2 HltLumiSumary format and add lumi counters

depends on LHCb!3803 (merged)

implement more proposed lumi counters

• counters to be implement codimd:
  • SciFi in different modules and stations
  • ECal
    • ECal ET
    • ECal E_tot in different position
      • Inner upper
      • Middle upper
      • Outer upper
      • Inner bottom
      • Middle bottom
      • Outer bottom
  • Muon in
    • M2: R1, R2, R3, R4
    • M3: R1, R2, R3, R4
    • M4: R1, R2, R3, R4

change the version of the Allen Lumi bank to 2

clean up make_lumi_summary

• remove the lumi info making parts outside the make_lumi_summary
• standalone lumi counter infos collection algorithm for different sub-detector
  • calo_lumi_counters, muon_lumi_counters, pv_lumi_counters, scifi_lumi_counters, velo_lumi_counters
  • thus the line could run properly with some of the sub-detector banks missing
• add dummy_maker to produce a dummy, to substitute missing inputs

more maintainable, and vanish ambiguity for decoding

• Use the header file LumiSummaryOffsets_V2.h from LHCb
• add an unsigned int encoding key to the start of every bank, which could be used to determine how was the bank encoded
  • since the json format lumi counters map is not ready yet, the encoding key is now decided by hashing a fixed lumi map.
  • this MR should go before the VDM scan, so the full functionality of the encoding key will be left for the next version

added Luminosity label

assigned to @sxian

added RTA label

added 1 commit

• 0b0fa499 - add caloET to lumi summaries

Compare with previous version

added 1 commit

• 27008815 - Fixed formatting

Compare with previous version

changed the description

added 9 commits

• 27008815...27e4c9fb - 6 commits from branch master
• 2494d5d9 - add caloET to lumi summaries
• a5c48e7b - Fixed formatting
• 1ad81b63 - use (0,0,0) as IP; convert adc readout to GeV

Compare with previous version

added 1 commit

• 69a77fee - Fixed formatting

Compare with previous version

added 54 commits

• 69a77fee...19481f96 - 49 commits from branch master
• baa4b48a - add caloET to lumi summaries
• f3d33561 - Fixed formatting
• fef1dafa - use (0,0,0) as IP; convert adc readout to GeV
• 1562929d - Fixed formatting
• 64ed4239 - fix ECalET; add SciFi clusters in different module/station

Compare with previous version

added 1 commit

• 15486f20 - Fixed formatting

Compare with previous version

mentioned in merge request LHCb!3518 (merged)

marked the checklist item SciFi in different modules and stations as completed

added 80 commits

• 15486f20...bc826705 - 73 commits from branch master
• aa7fc31c - add caloET to lumi summaries
• 7f8f4ac4 - Fixed formatting
• 6975cbf7 - use (0,0,0) as IP; convert adc readout to GeV
• 3ab54091 - Fixed formatting
• 5474436c - fix ECalET; add SciFi clusters in different module/station
• c607e341 - Fixed formatting
• 035f8388 - clean up make_lumi_summary, make the counter infos collecting part standalone

Compare with previous version

changed the description

added 1 commit

• 75e67be7 - Fixed formatting

Compare with previous version

added 1 commit

• 8f5988d2 - fix gpu

Compare with previous version

added 1 commit

• 648df67c - fix gpu

Compare with previous version

added 1 commit

• a5717a8f - Fixed formatting

Compare with previous version

added 1 commit

• 37943884 - fix gpu

Compare with previous version

added 2 commits

• 1f065ebd - download Lumi Offsets includes from LHCb
• b196337d - update the ECal and Muon counters to the latest list

Compare with previous version

marked the checklist item Download LumiSummaryOffsets_V1.h from LHCb as completed

added 1 commit

• daa3a458 - Fixed formatting

Compare with previous version

changed the description

marked the checklist item ECal as completed

marked the checklist item counters to be implement codimd: as completed

added 1 commit

• 8888352a - fix version

Compare with previous version

added 36 commits

• 8888352a...ff0d309d - 20 commits from branch master
• abaa2bc3 - add caloET to lumi summaries
• e4c8eeb6 - Fixed formatting
• cd829e87 - use (0,0,0) as IP; convert adc readout to GeV
• 790dc178 - Fixed formatting
• 3e3f8473 - fix ECalET; add SciFi clusters in different module/station
• 083c6a5e - Fixed formatting
• 9be25d13 - clean up make_lumi_summary, make the counter infos collecting part standalone
• 250721c7 - Fixed formatting
• 547c4840 - fix gpu
• cb12a82b - fix gpu
• c41c5dbe - Fixed formatting
• db122df3 - fix gpu
• 9fc9aa26 - download Lumi Offsets includes from LHCb
• 1e1a9483 - update the ECal and Muon counters to the latest list
• 4dd0144e - Fixed formatting
• ade84a3f - fix version, rebase to master

Compare with previous version

added 1 commit

• b4b38dfd - add a 32 bits tck to the start of the lumi summary bank; remove the lumi READ>...

Compare with previous version

added 1 commit

• ed55f849 - Fixed formatting

Compare with previous version

added 1 commit

• e1c66c30 - minor fix

Compare with previous version

marked the checklist item add an unsigned int to the start of every bank, which could be used to determine how was the bank encoded as completed

changed the description

added 60 commits

• e1c66c30...24f5fb55 - 42 commits from branch master
• 34b791ce - add caloET to lumi summaries
• 4ff1a69f - Fixed formatting
• 68416fe3 - use (0,0,0) as IP; convert adc readout to GeV
• 2b1f053d - Fixed formatting
• f53cd9c7 - fix ECalET; add SciFi clusters in different module/station
• d9e26178 - Fixed formatting
• 73cd120c - clean up make_lumi_summary, make the counter infos collecting part standalone
• b244a2c4 - Fixed formatting
• 8730b14e - fix gpu
• 6337f82d - fix gpu
• 9cd78e38 - Fixed formatting
• df6805ce - fix gpu
• 744c53f2 - download Lumi Offsets includes from LHCb
• 94fb4e73 - update the ECal and Muon counters to the latest list
• ce1b8d38 - Fixed formatting
• a8bd5577 - fix version, rebase to master
• e4f7b607 - add a 32 bits tck to the start of the lumi summary bank; remove the lumi READ>...
• 8aa9eddb - Fixed formatting

Compare with previous version

added 1 commit

• 9bc54b24 - a working but not really usable encoding key at the start of the lumi summary

Compare with previous version

added 1 commit

• b907c959 - fix stack building

Compare with previous version

added 1 commit

• 63341afe - fix stack building

Compare with previous version

added 1 commit

• 43ac17b1 - minor fix

Compare with previous version

added 1 commit

• ef350015 - fix calo_lumi_counters

Compare with previous version

Could you please rebase? There were recent changes to the Allen CI and rebasing will reduce the block on CPU EPYC resources we have now.

added 46 commits

• ef350015...09b826d8 - 25 commits from branch master
• e1fa861b - add caloET to lumi summaries
• ab7dc915 - Fixed formatting
• c21a8045 - use (0,0,0) as IP; convert adc readout to GeV
• 7a02aa14 - Fixed formatting
• 010566cc - fix ECalET; add SciFi clusters in different module/station
• 167d4cc9 - Fixed formatting
• aa6c4bcd - clean up make_lumi_summary, make the counter infos collecting part standalone
• d9156092 - Fixed formatting
• c8e5c9ed - fix gpu
• 4ac93afa - fix gpu
• b0d4a34b - Fixed formatting
• 97ee3a39 - fix gpu
• 32f48d69 - download Lumi Offsets includes from LHCb
• 21c0fcab - update the ECal and Muon counters to the latest list
• dec7448d - Fixed formatting
• 70cc0242 - fix version, rebase to master
• e2a16e5d - add a 32 bits tck to the start of the lumi summary bank; remove the lumi READ>...
• c50e1d41 - a working but not really usable encoding key at the start of the lumi summary
• 80b5bdb8 - fix stack building
• 73ee01aa - fix stack building
• 17393745 - fix make_lumi_summary and calo_lumi_counters

Compare with previous version

added 1 commit

• d465e304 - fix make_lumi_summary and calo_lumi_counters, rebase to master

Compare with previous version

How far is this from being ready? It looks like the sequence is building / passing. Shall we launch a Jenkins CI-test?

To @sxian, @edallocc: I would propose avoiding any dependence on the implementation of the decoder. What we want to version here is ONLY the Allen format, and it has nothing to do with the following decoding. Eg. the decoding can be done to the Gaudi bank (like Event/LumiSummaryOffsets_V2.h) or to C++ map<string, unsigned>, but Allen version should not depend on these downstream details.

So, the TCK-like hash (unsigned int field) should depend only on the Allen counter names and offsets.

Of course, it'll be good to implement it. However, if this implementation (hashing of the JSON form or any other) delays (blocks) this MR, I would exceptionally go with just setting it to some random magic number and distinguishing the version in the bank header. This should be sufficient for commissioning. I would prefer to have more counters and ensure we have data in 2022, than to have a clean code but no data if MR will be not ready (we need to ensure at least 2 weeks in advance before the first data taking, for making a release). In the future, we can check separately whether this field is equal to the "magic" number, check the header or even check the ODIN timestamp and determine this was commissioning in 2022 - there should be no problem with that.

Maybe one can even set the magic number to zero. If data is corrupted, most often they have zeros, so for detecting corruption it might be preferable to generate some non-zero random number. This argument is not so important, however, since with so many fields it'll be extremely easy to detect such a corruption (eg. using the timing/orbit/bcid information, values of counters, bank header). In Run 2 no anyone lumi-event was corrupted.

Otherwise, the hash can be calculated by hand: one just needs to invent some algorithm how to do that from the names of the counters and their offsets. It can be automated later.

mentioned in merge request LHCb!3786 (merged)

added 78 commits

• d465e304...3eb7716a - 56 commits from branch master
• 0b51212e - add caloET to lumi summaries
• 7b9800fc - Fixed formatting
• e68f23ec - use (0,0,0) as IP; convert adc readout to GeV
• b4faba21 - Fixed formatting
• 6c26caca - fix ECalET; add SciFi clusters in different module/station
• 651d9c17 - Fixed formatting
• 42697f64 - clean up make_lumi_summary, make the counter infos collecting part standalone
• 46151430 - Fixed formatting
• 8217e040 - fix gpu
• a470daaf - fix gpu
• 7b285bc7 - Fixed formatting
• c2b4342d - fix gpu
• 369296da - download Lumi Offsets includes from LHCb
• e41e750e - update the ECal and Muon counters to the latest list
• 4e8b77de - Fixed formatting
• 66a6bd60 - fix version, rebase to master
• 61275526 - add a 32 bits tck to the start of the lumi summary bank; remove the lumi READ>...
• 9ecdb224 - a working but not really usable encoding key at the start of the lumi summary
• d35afa6f - fix stack building
• 9e998479 - fix stack building
• f8d8bf53 - fix make_lumi_summary and calo_lumi_counters, rebase to master
• 4094239a - minor fix, rebase to master

Compare with previous version

added 1 commit

• dfade4d9 - tempororally switch to downloading the lumi offset file

Compare with previous version

added 26 commits

• dfade4d9...680ec3fe - 2 commits from branch master
• 24b83c4b - add caloET to lumi summaries
• f5d24349 - Fixed formatting
• dbb4445f - use (0,0,0) as IP; convert adc readout to GeV
• e2d09a52 - Fixed formatting
• 49761704 - fix ECalET; add SciFi clusters in different module/station
• b78e344b - Fixed formatting
• 220c78ab - clean up make_lumi_summary, make the counter infos collecting part standalone
• 903ecf9a - Fixed formatting
• 101d2cb0 - fix gpu
• af8d3684 - fix gpu
• 5fa6de30 - Fixed formatting
• b2e27a64 - fix gpu
• 488877ca - download Lumi Offsets includes from LHCb
• 36f6d20b - update the ECal and Muon counters to the latest list
• f8f91db9 - Fixed formatting
• 3e1c6f86 - fix version, rebase to master
• 028d72f6 - add a 32 bits tck to the start of the lumi summary bank; remove the lumi READ>...
• 870046d3 - a working but not really usable encoding key at the start of the lumi summary
• a6106c2c - fix stack building
• f339b20b - fix stack building
• 2d911b9c - fix make_lumi_summary and calo_lumi_counters, rebase to master
• f68734af - minor fix, rebase to master
• 7b926c25 - tempororally switch to downloading the lumi offset file
• 4d80d5dc - tempororally switch to downloading the lumi offset file

Compare with previous version

added 1 commit

• fc97e31f - update lumi CMakeList

Compare with previous version