tuning of parameters for a qmt test on sprucing lines rate

added 1 commit

f63e1f30 - Fixed formatting

Compare with previous version

requested review from @nskidmor

added DPA-WP1 label

@nskidmor anybody to be added as reviewer ? any label missing ? then as Assignee I never know .. shall I ask Alex ?

added 1 commit

5d5cc169 - update of one parameter in test_spruce_all_lines_realtime.qmt

Compare with previous version

Can you explain briefly (in the description) why you need to run over more events?

added 1 commit

c7beb305 - update comments in test_spruce_all_lines_realtime.qmt

Compare with previous version

resolved all threads

@rmatev I am not able to attach my replay below your comment .. do not get why ... anyway I have added to Hlt/Hlt2Conf/tests/qmtest/test_spruce_all_lines_realtime.qmt this:

# In this test we are running over 1000 HLT1-filtered events that are additionaly 
# filtered with the topo. 2 or 3 body topological trigger requirement (see
# test_hlt2_2or3bodytopo_realtime prerequisite).
# The topo. trigger requirement reduces the sample by about 1/10 so about 100 events 
# are passed to the sprucing.
# We consider a failure if any of the sprucing selections exceeds 10 positive decisions 
# (cut value to be tuned), this corresponds to a retention above 10 %.
# Reducing the input sample to 100 events and keeping the threshould to 10 % would 
# correspond to a failure for selections exceeds 1 positive decision i.e. we will be
# dominated by statistical fluctuations.

so 100 input events are really too few :-(

changed the description

@rmatev are you happy with the above justification? If so we can ask for testing

In fact, I think we should kind of move in the opposite direction. The Hlt/Hlt2Conf/options/hlt2_2or3bodytopo_fromfile.py test already uses 1000 events and is quite slow.

I would say that checking retention is not the point of the qmtests. They should be fast because otherwise 1. the feedback from nightlies/ci-test is delayed and 2. people are discouraged to run all the tests locally (even when having 20 cores at hand).

This kind of slow-to-make checks, especially when running multiple applications one after the other, should be made into LHCbPR tests, but we don't have an example of that just yet (there have been discussion though).

The only thing I can propose today is to run manually HLT2, store the output somewhere in /eos/lhcb/wg/rta/samples/mc and simply use that in the sprucing. That may be painful though as the HLT2 output file might need frequent manual updates (because of updates in selections or in persistency).

Let's try a ci-test and see if this change makes the test prohibitively slow.

/ci-test --merge

added ci-test-triggered label

[2021-07-30 11:17] Validation started with lhcb-master-mr#2635

mentioned in merge request !932 (merged)

Closing this as moved to !932 (merged).

closed

tuning of parameters for a qmt test on sprucing lines rate

Merge request reports

Activity