Duplication/cloning of events in data
See presentation https://indico.cern.ch/event/1179100/#2-report-from-bs-phiphi-emtf-q by @maryr. CC @needham.
The features presented by Mary at the RTA-DPA joint meeting are puzzling and should be investigated asap.
There are apparently two issues. Here data is discussed. In #82 (closed) the MC issue is discussed.
Here I had a first look with what samples DPA could produce so far with the HLT2 outputs available, see the job created last week at !782 (merged). The ntuples were created after Sprucing the sample at /eos/lhcb/wg/dpa/wp1/data/Run251342/.
Judging from the fact that the ntuples created by @nskidmor at the end of the Sprucing job and by us within DaVinci match in the number of entries,
root [15] DecayTree -> Print()
******************************************************************************
*Tree :DecayTree : DecayTree *
*Entries : 13515 : Total = 5171684 bytes File Size = 1962020 *
* : : Tree compression factor = 2.63 *
I wonder if the issue is not upstream rather than in the ntuple making. I would start here to disentangle potential sources of trouble. Nicole, to be effective since you have it all at hand, could you run a GaudiPython job and dump for some 50 events or so the candidates you see straight from looking at the line decision bit? So straight out of HLT2 and also straight out of Sprucing using GaudiPython. Ideal if you can print also the event number so that we compare with the below.
If the bug is in FunTuple, why would we see events with multiple entries and events with single entries?
Here are some examples from inspecting the ntuple produced by the job (see !782 (merged)) in the data test introduced by Patrick in the DaVinci repo (I separate with empty lines the various "kinds" of entries). Let me know what you think of all this.
I only did the below so far. Any other ideas?
Thanks a lot.
- Aside but slightly puzzling - entries are not always in increasing run number when I thought that we are not running in multi-threading mode at the pit. How can the event number not be ordered?
root [13] DecayTree -> Scan("EVENTNUMBER:nPVs:KS_PT:pip_PT:pim_PT")
************************************************************************
* Row * EVENTNUMB * nPVs * KS_PT * pip_PT * pim_PT *
************************************************************************
1) Events with a single entry - all fine:
* 9 * 98772360 * 1 * 1681.0555 * 478.63986 * 1207.8833 *
* 8 * 98772303 * 2 * 1176.7602 * 413.49484 * 798.10601 *
2) Events with multiple but different candidates - all fine:
* 128 * 98774444 * 2 * 1265.8796 * 505.29238 * 760.61444 *
* 129 * 98774444 * 2 * 1099.3618 * 719.09106 * 438.38992 *
* 243 * 98776009 * 1 * 929.07421 * 255.57234 * 750.10491 *
* 244 * 98776009 * 1 * 1001.3006 * 447.48693 * 590.97345 *
(Same pi- used twice with a different pi+. Also fine.)
* 6 * 98772163 * 2 * 1123.2120 * 344.31088 * 788.18908 *
* 7 * 98772163 * 2 * 1120.9774 * 342.64694 * 788.18908 *
3) Events with a single duplicate:
* 25 * 98766112 * 2 * 1585.0288 * 681.44635 * 904.4021 *
* 26 * 98766112 * 2 * 1585.0288 * 681.44635 * 904.4021 *
* 27 * 98766179 * 2 * 737.82562 * 349.00152 * 426.44332 *
* 28 * 98766179 * 2 * 737.82562 * 349.00152 * 426.44332 *
4) Events with duplication of 2-3 different sets of candidates:
* 0 * 98767893 * 5 * 556.11517 * 200.45158 * 444.40976 *
* 1 * 98767893 * 5 * 386.05709 * 248.78660 * 156.83134 *
* 2 * 98767893 * 5 * 556.11517 * 200.45158 * 444.40976 *
* 3 * 98767893 * 5 * 386.05709 * 248.78660 * 156.83134 *
* 16 * 98772725 * 3 * 708.34588 * 494.16180 * 239.18859 *
* 17 * 98772725 * 3 * 868.89819 * 634.54565 * 274.09545 *
* 18 * 98772725 * 3 * 642.05072 * 504.57763 * 274.09545 *
* 19 * 98772725 * 3 * 708.34588 * 494.16180 * 239.18859 *
* 20 * 98772725 * 3 * 868.89819 * 634.54565 * 274.09545 *
* 21 * 98772725 * 3 * 642.05072 * 504.57763 * 274.09545 *
* 56 * 98734193 * 5 * 1188.0792 * 968.69647 * 281.46466 *
* 57 * 98734193 * 5 * 685.58325 * 409.89132 * 281.46466 *
* 58 * 98734193 * 5 * 658.00488 * 515.15161 * 222.01973 *
* 59 * 98734193 * 5 * 1188.0792 * 968.69647 * 281.46466 *
* 60 * 98734193 * 5 * 685.58325 * 409.89132 * 281.46466 *
* 61 * 98734193 * 5 * 658.00488 * 515.15161 * 222.01973 *
* 169 * 98740859 * 1 * 641.35784 * 439.63525 * 271.92327 *
* 170 * 98743961 * 2 * 1146.9110 * 136.32356 * 1058.7207 *
* 171 * 98743961 * 2 * 1222.3657 * 195.26068 * 1058.7207 *
* 172 * 98740859 * 1 * 641.35784 * 439.63525 * 271.92327 *
* 202 * 98775196 * 1 * 1221.8747 * 523.80615 * 715.21972 *
* 203 * 98775208 * 1 * 1052.2995 * 208.17503 * 883.93286 *
* 204 * 98775196 * 1 * 1221.8747 * 523.80615 * 715.21972 *
* 205 * 98775208 * 1 * 1052.2995 * 208.17503 * 883.93286 *
(Variant with the same pi+ used twice with a different pi-.)
* 231 * 98748401 * 2 * 2336.7204 * 541.96386 * 1804.5166 *
* 232 * 98748401 * 2 * 2340.4008 * 541.96386 * 1806.8640 *
* 234 * 98748401 * 2 * 2336.7204 * 541.96386 * 1804.5166 *
* 235 * 98748401 * 2 * 2340.4008 * 541.96386 * 1806.8640 *
5) Even more strange - the same candidate interleaved with a candidate with a different event number:
* 51 * 98741034 * 1 * 3424.5610 * 401.79205 * 3023.2060 *
* 52 * 98734053 * 4 * 675.14556 * 226.65951 * 552.74200 *
* 53 * 98741034 * 1 * 3424.5610 * 401.79205 * 3023.2060 *
* 54 * 98741042 * 1 * 1720.1242 * 1506.6270 * 226.55819 *
* 55 * 98741053 * 1 * 480.13540 * 104.19439 * 468.12710 *
* 83 * 98734975 * 2 * 616.38446 * 524.10888 * 208.72470 *
* 84 * 98734975 * 2 * 616.09045 * 524.10888 * 199.25286 *
* 85 * 98768005 * 1 * 582.66015 * 480.62442 * 254.30824 *
* 86 * 98734892 * 3 * 475.46548 * 338.10369 * 315.56878 *
* 87 * 98734975 * 2 * 616.38446 * 524.10888 * 208.72470 *
* 88 * 98734975 * 2 * 616.09045 * 524.10888 * 199.25286 *
* 292 * 98771062 * 4 * 1010.2778 * 285.43231 * 775.66009 *
* 293 * 98744668 * 3 * 493.99935 * 207.65086 * 449.74417 *
* 294 * 98771062 * 4 * 1010.2778 * 285.43231 * 775.66009 *
* 285 * 98751740 * 1 * 506.01290 * 324.61184 * 255.74403 *
* 286 * 98744474 * 3 * 936.19903 * 198.26982 * 746.64471 *
* 287 * 98751740 * 1 * 506.01290 * 324.61184 * 255.74403 *
* 288 * 98751790 * 2 * 1745.0987 * 1372.5427 * 379.05484 *
* 289 * 98751790 * 2 * 1745.0987 * 1372.5427 * 379.05484 *
* 290 * 98744474 * 3 * 936.19903 * 198.26982 * 746.64471 *
CC'ing several people but please RTA colleagues add anyone you may find relevant. Advance thanks.
@amathad, @nskidmor, @pkoppenb, @gligorov, @rmatev, @mstahl, @mvesteri, @poluekt.