Incorrect Statistics for new Gaudi profile histograms
I have been looking into some more fluctuations I am seeing in the statistics printed for histograms between different builds, and I have realised there is a problem with stats calculations for profiles that is leading to incorrect results being shown.
If I compare these two tests, the first one is when the RICH test in question was using the old style histograms, and the second for after the migration to the new ones.
If you compare the stats, for the regular histograms the stats are similar, not quite the same but perhaps close enough to think the calculations are not too wrong.
However, for profiles the results for the new histograms are completely wrong
Old
RichRecPixelQC SUCCESS 1D profile histograms in directory "RICH/RichRecPixelQC" : 8
| ID | Title | # | Mean | RMS | Skewness | Kurtosis |
| Rich1/hitDensityX | "Rich1 <#hits/mm^2> (X)" | 250000 | 0 | 375.2 | -0.056157 | -2.4172 |
| Rich1/hitDensityY | "Rich1 <#hits/mm^2> (Y)" | 250000 |-9.4739e-19 | 376.81 | -0.0073197 | -2.5269 |
| Rich1/pdOccXLoc | "Rich1 PD Average Occupancy (X)" | 250000 | 0 | 375.2 | -0.056157 | -2.4172 |
| Rich1/pdOccYLoc | "Rich1 PD Average Occupancy (Y)" | 250000 | 0 | 392.52 | -0.0066538 | -2.6078 |
| Rich2/hitDensityX | "Rich2 <#hits/mm^2> (X)" | 250000 | 0 | 461.79 | -0.024105 | -1.5068 |
| Rich2/hitDensityY | "Rich2 <#hits/mm^2> (Y)" | 250000 | 0 | 432.93 | 0.016382 | -2.7402 |
| Rich2/pdOccXLoc | "Rich2 PD Average Occupancy (X)" | 250000 | 0 | 461.79 | -0.022671 | -1.4689 |
| Rich2/pdOccYLoc | "Rich2 PD Average Occupancy (Y)" | 250000 | 0 | 432.93 | 0.034487 | -2.3648 |New
RichRecPixelQC INFO 1D profile histograms in directory "RichRecPixelQC" : 8
| ID | Title | # | Mean | RMS | Skewness | Kurtosis |
| /RICH/RichRecPixelQC/Rich1/hitDensityX | "Rich1 <#hits/mm^2> (X);Local X / mm;hits..." | 250000 | -26 | 375.20 | 0 | -1.201 |
| /RICH/RichRecPixelQC/Rich1/hitDensityY | "Rich1 <#hits/mm^2> (Y);Local Y / mm;hits..." | 240000 | -26 | 378.04 | 0 | -1.198 |
| /RICH/RichRecPixelQC/Rich1/pdOccXLoc | "Rich1 PD Average Occupancy (X);Local X /..." | 250000 | -26 | 375.20 | 0 | -1.201 |
| /RICH/RichRecPixelQC/Rich1/pdOccYLoc | "Rich1 PD Average Occupancy (Y);Local Y /..." | 250000 | -27.2 | 392.52 | 3.08e-16 | -1.201 |
| /RICH/RichRecPixelQC/Rich2/hitDensityX | "Rich2 <#hits/mm^2> (X);Local X / mm;hits..." | 250000 | -32 | 461.79 | 0 | -1.201 |
| /RICH/RichRecPixelQC/Rich2/hitDensityY | "Rich2 <#hits/mm^2> (Y);Local Y / mm;hits..." | 250000 | -32 | 434.84 | 0 | -1.1878 |
| /RICH/RichRecPixelQC/Rich2/pdOccXLoc | "Rich2 PD Average Occupancy (X);Local X /..." | 250000 | -32 | 461.79 | 0 | -1.201 |
| /RICH/RichRecPixelQC/Rich2/pdOccYLoc | "Rich2 PD Average Occupancy (Y);Local Y /..." | 250000 | -30 | 432.93 | 0 | -1.201 |A skewness of zero is certainly not correct here, given how the histograms look.
I suspect the issue is in some way related to how in the implementation for profiles the bin values are used, but I need to investigate more.