I don't think that we need opening all the files just for the configuration. Ideally, the options stored in the FSRs of files processed by each AP job should be identical, given that the input files were generated using the same Moore steps.
Therefore, my proposal is to read one file, extract the options needed to configure the AP, and subsequently, during the execution of the job on each input file, DaVinci can verify that the metadata information passed to the AP corresponds with the data stored in its FSR. If this is not the case it can raise an error.
On the other side this could led to a lot of failures, so another idea could be to perform this consistency check on the FSRs content at the end of the production. So it will be performed only once and not every time the AP is run.
Quick update on this topic: I saw that the MR LHCb!4404 (merged) has been merged and so the new FSRs are now available. I discussed with @nskidmor about the possibility to add at the Sprucing level some metadata information in the new FSRs. In particular: [input_type, data_type, simulation, lumi, dddb_tag, conddb_tag], these are the information needed in the AP for allowing an automatic input data configuration. Nicole said that they could be easily added. @eleckste do you think can you add these infos when the work on the encoding keys is finalized?
Hi @gunther, in fact is not easy to implement a test able to recognize if the lack of events is due to a real bug or just to a reduction of the bandwidth. Do you think is it possible to implement some lines just for this kind of test (at least until the situation is not stable)?
Regarding the porting of the test to dd4hep platform, I am working on a MR (see !58) but the Hlt2 step is still failing. I saw that many tests in Moore are still not running on dd4hep platforms. Do you know if it's possible to run a sequence Hlt1-Hlt2-Sprucing on these platforms?
Hi all, I had a look at the test. The GenFSRMerge
function was written for the merging of the generator FSRs along time ago, using the old Lumi codes as starting point.
There is no a legacy code, since the genFSR were not available before Sim10 and as far as I know this is the only function available at the moment for taking care of this task. I can ask to the simulation people if there is any new specific update for Run3.
@egraveri, can you point me the other algorithm for merging FSRs in Run3? Do you know if it works also with the generator FSRs?
Closing since obsolete after !3203
Davide Fazzini (ff5e2b8e) at 21 Mar 14:47
cleanup
yes, I think we still need to use the make_*
functions in order to run the correct reconstruction.
I am investigating with @sesen a dependency circle in the control flow. It's not completely clear at the moment what is causing it.
Adds the needed DaVinci test for the integration test for HLT2 -> Sprucing -> track refitting in DaVinci, see LHCbIntegrationTests!60.
Relies on:
The maximum number of events at the Hlt1 step was set to 250 in order to limit the total time required. However I think it can be increased. Can you test it locally to check how much it should be increased?
Davide Fazzini (ddc218a9) at 19 Mar 12:51
cleanup
Davide Fazzini (77a761e3) at 19 Mar 12:44
pre-commit fixes
Davide Fazzini (4503eaf5) at 19 Mar 12:40
Fixed formatting
Davide Fazzini (16699c67) at 19 Mar 12:39
fix typos
Davide Fazzini (0204540d) at 19 Mar 12:07
fix bug
Davide Fazzini (e7012bc6) at 19 Mar 12:05
cleanup
Davide Fazzini (4416b16d) at 19 Mar 12:03
add corresponding test