Tupling test running on spruced output

In the discussion on #5 (closed), @sstahl suggested to add a tupling test running on sprucing (cc @nskidmor) output, ideally with not a candidate in each event.

There is such a test in MooreAnalysis (?).

changed due date to September 30, 2021

added DPA-WP3 testing labels

changed the description

I was referring to https://gitlab.cern.ch/lhcb/MooreAnalysis/-/blob/master/HltIntegrationTests/tests/qmtest/hlt2_analysis.qms/read_hlt2_example_output.qmt

Oh boy, that example (https://gitlab.cern.ch/lhcb/MooreAnalysis/-/blob/master/HltIntegrationTests/tests/options/hlt2_analysis/read_hlt2.py#L116) still makes use of appendPostConfigAction? Seems we will never get rid of it :S.

For the tests can we now simply use some of the files from the June FEST, put them in the TestFileDB and run from there?

These are here /eos/lhcb/wg/dpa/wp1/JuneFEST/

Ok, thanks. Do you know which DDDB tags were used in the generation?

OK, found them!

Hi @nskidmor, I trying to run DaVinci with the options file saved here:

• https://gitlab.cern.ch/nskidmor/fest-june2021/-/tree/master/DV.py

on the .dst file stored in this directory:

• /eos/lhcb/wg/dpa/wp1/JuneFEST/spruce_FEST_00135282_00000001_1.dst

and using the two .json related files:

• /eos/lhcb/wg/dpa/wp1/JuneFEST/00135282_00000001_1.annsvc.pol.json
• /eos/lhcb/wg/dpa/wp1/JuneFEST/00135282_00000001_1.annsvc.selid.json

However, I noticed that there are no lines with a '/Event/Spruce/' prefix in the two .json files. Can you confirm that these are the correct files to be used?

Yeh, I was stupid and didnt upload the JSON from the Sprucing job. I need to remake these for you

Ok, thanks!

OK I have added a file sprucert.all_lines.annsvc.pol.json, you need to use this for the packedobjectlocations (pol).

So use /eos/lhcb/wg/dpa/wp1/JuneFEST/spruce_FEST_00135282_00000001_1.dst with 00135282_00000001_1.annsvc.selid.json and sprucert.all_lines.annsvc.pol.json

Hi @sstahl, I had a look at the example you posted however I see the particles are retrieved using the AutomaticData class. Since this is not a PyConf class, PyConf complains when I try to import it into the job. Do you know if a PyConf equivalent class exists?

@dfazzini, that's the data-on-demand service and something we totally banned from our run-3 stack. Is that really the case that code there uses it? Very strange. Another solution must be used. People such as Gerhard or Chris Jones may be able to know the alternative, I mean aside Alex or Rosen.

OK, I will try to post this question to the Upgrade Hlt2 mattermost channel for having a feedback from their side.

@erodrigu, @nskidmor, I have a working test running on spruced ntuples. I have just one question, the unpacking methods used in the hlt2_examples are the ones implemented here: https://gitlab.cern.ch/lhcb/LHCb/-/blob/master/GaudiConf/python/GaudiConf/reading.py However, these methods use classes imported from Configurables which don't work with the PyConf environment. Do we have an alternative version using PyConf algorithms?

I was asking because I had to create a new reading.py module in order to run correctly DaVinci on a spruced sample, so in case I can open a MR and commit the changes I made to the original reading.py script.

Can you @nskidmor comment on this question above from @dfazzini? I confess I am surprised by the issue and would think that importing via the "wrappers" of PyConf would be no problem. Maybe you @dfazzini can provide the example in a WIP MR so that we see and iterate concretely?

You should be able to use them, see https://gitlab.cern.ch/lhcb/Moore/-/snippets/1808#LC6

Do we have a master issue for OktoberFEST? This issue is essentially what DPA:WP3 will do at FEST. @erodrigu

There is https://gitlab.cern.ch/lhcb-dpa/project/-/issues/131 and https://gitlab.cern.ch/lhcb-dpa/project/-/issues/129 (as there is for WP1)

Maybe we can have an OktoberFEST DPA milestone

Hi @pkoppenb, FEST is important enough, with enough activities foreseen, that there's in fact a milestone for it, see https://gitlab.cern.ch/groups/lhcb-dpa/-/milestones/8. There you can see per-WP issues for preparatory work and the actual activities during the week. That worked fine in June for WP1.

Thanks. Added it now.

created merge request !573 (merged) to address this issue

mentioned in merge request !573 (merged)

closed with merge request !573 (merged)

mentioned in commit 8b76e705

reopened

I reopened the issue as I don't think it is solved. See my comment at !573 (comment 4795455) .

I believe this is now sorted by @dfazzini via MR !577 (merged) but can you @dfazzini confirm and act accordingly?

yes, it'sorted.

mentioned in merge request !577 (merged)

closed

mentioned in commit e7031756