Draft: first attempt to ran on a subset of spruce lines provided by the user

requested review from @nskidmor

requested review from @shunan

added RTA label

@nskidmor (cc @shunan) this is the very first prototype (like version -1) of what I am trying to do, a few points to look at:

https://gitlab.cern.ch/lhcb/Moore/-/blob/spruce-dev-abertoli/Hlt/Hlt2Conf/options/sprucing/spruce_all_lines_realtime.py#L95 this is the list of b2cc lines we want, it will have to leave in a different file
https://gitlab.cern.ch/lhcb/Moore/-/blob/spruce-dev-abertoli/Hlt/Hlt2Conf/options/sprucing/spruce_all_lines_realtime.py#L99 this is the list of sl lines we want, it will have to leave in a different file
https://gitlab.cern.ch/lhcb/Moore/-/blob/spruce-dev-abertoli/Hlt/Hlt2Conf/options/sprucing/spruce_all_lines_realtime.py#L115 here I scan the mega-dictionary (I simplified it a bit) of lines and I append only what is in the b2cc or sl list of lines

when I ran that with a grep -i LAZY_AND spruce.log I get:

  LAZY_AND: SpruceB2CC_BsToJpsiPhi_DetachedLineDecisionWithOutput                                              #=10      Sum=0           Eff=|( 0.000000 +- 0.00000 )%|
   LAZY_AND: SpruceB2CC_BsToJpsiPhi_DetachedLine                                                               #=10      Sum=0           Eff=|( 0.000000 +- 0.00000 )%|
  LAZY_AND: SpruceSLB_B0ToDpTauNu_DpToKPiPi_TauToMuNuNu_LineDecisionWithOutput                                 #=10      Sum=0           Eff=|( 0.000000 +- 0.00000 )%|
   LAZY_AND: SpruceSLB_B0ToDpTauNu_DpToKPiPi_TauToMuNuNu_Line                                                  #=10      Sum=0           Eff=|( 0.000000 +- 0.00000 )%|

so just the 2 lines that are in some_b2cc_lines_to_run and in some_sl_lines_to_run

question: at step -1 is it more or less what we want ? if yes, I can add what's missing in the coming days

ps: the fact that the lines are counting 0 is totally irrelevant (for now) just what lines are there is relevant

As a version -1 looks good to me :) What are the next steps?

these are the steps I have in mind right now:

I have to move some_b2cc_lines_to_run and some_sl_lines_to_run to a different file
I have to scream if something unexpected happen, very minimum if the requested line does not exist

then @nskidmor whenever you like and have time we can define additional ones

They are the main ones :)

Hello. Happy to see this work starting. I did not look at details but let me make a few general comments:

We should discuss from the onset with RTA WP3 as the solution is going to be the same as for HLT2 selections. Better add Anton and Mika asap.
Do not forget to add relevant labels and link to relevant tasks.
Maybe we should discuss grand design choices in the related tasks, for sure https://gitlab.cern.ch/lhcb-dpa/project/-/issues/163.
IMO the solution should have in mind https://gitlab.cern.ch/lhcb-dpa/project/-/issues/188 and https://gitlab.cern.ch/lhcb-dpa/project/-/issues/157 to avoid a solution that is too narrow in scope.
Packaging is important but we should also think about versioning so that it is trivial to run the configuration XXX with a compatible newer Moore version months downstream. This is just one of the requirements and there will be far more.

HTH.

@mvesteri @poluekt Eduardo encouraged us to prepare a script that would allow to ran just the spruce lines that appear in a list/dictionary, possibly one dictionary per PWG to be filled by the corresponding MiCo

this is being done in the spruce_all_lines_realtime.py of this MR, as you will see we are still at the stage -1 but at least the very basic functionality is there

as suggested above by Eduardo I add you to the loop

Thanks @abertoli we'll keep an eye on this then

Hey @mvesteri, I really think we need more than an eye, TBH, rather a joint solution since in the end we should only have one way to configure and bookkeep our selections, whether HLT2 or Sprucing.

For the record we had some interesting exchanges back in the days in our task https://gitlab.cern.ch/lhcb-dpa/project/-/issues/81. It is now a closed task since a bit of a duplication, but we ought all to "revise" those and converge on what ticks the boxes (aka fullfills our requirements).

We should consider the mechanics of how to configure the application in production, which may look somewhat different to a python script to run the application interactively.

Indeed. I had written down some requirements in https://gitlab.cern.ch/lhcb-dpa/project/-/issues/81#note_5242458, see also the rest of the thread (we converged already on some bits).