Packing of particle covariance matrix is not sufficiently accurate

The packing of the covariance matrix of LHCb::Particle is not sufficiently accurate. As an illustration, I made combinations of 3 charged particles with the standard ParticleCombiner, then packed and unpacked each particle and finally compared the computed error on the mass. (See here for an example of how to compute the covariance of the mass.) The result for a few random events is:

mass-cov-old, masscov-new, ratio
3.56854  -3.93066 -1.10148
6.34917  -8.4234 -1.32669
9.8146  0.456843 0.0465472
6.00942  12.9111 2.14848
3.07106  -7.52585 -2.45057
9.76429  0.596196 0.0610588
3.37543  -4.48747 -1.32945
7.60287  -2.91446 -0.383337
11.2658  18.292 1.62367
6.82377  13.7133 2.00964
7.75931  16.4184 2.11596
6.56639  -0.338362 -0.0515294
3.90194  -1.31369 -0.336676
3.45346  -6.46153 -1.87103
2.43247  -4.66258 -1.91681
6.53259  15.8192 2.42158
6.11018  -1.95685 -0.320261

The covariance after packing is often even negative. It is unclear why the standard packer checks do not catch this.

As a result of this, the covariance matrix for Particles on the DST is compromised. This is not a problem for DecayTreeFitter as it goes back to the tracks (which we should also check carefully). However, if the particles are used as input to another combiner, then the result will not be right. As far as we can see now, it only affects the 'energy row' of the covariance matrix.

It is expected that the main reason for this problem is that in the standard (P,E) representation of four-vectors, PZ and E are highly correlated. Therefore, this representation is not so suitable for packing directly. One pragmatic solution may be to convert the cov matrix to (P,M) representation, which will be much better conditioned. (The packed four-vector itself is already in (P,M) representation.)

This problem is not new. See for instance: https://groups.cern.ch/group/lhcb-davinci/Lists/Archive/Flat.aspx?RootFolder=%2Fgroup%2Flhcb%2Ddavinci%2FLists%2FArchive%2FNegative%20eigenvalues%20in%20LHCbParticlecovMatrix%28%29%20%28matrix%20not%20positive%20definite%29&FolderCTID=0x01200200B795ECDBD8943F4E90E5F02A017196C9

Manuel also suggested different methods to solve this.