Skip to content

Interpolation mode from command-line

Florian Bury requested to merge fbury/inference:exclusion_interpolation into master

Hi,

The problem I faced recently was when running 2D exclusion limits with a very large number of points kl,-25,25,51:kt,-5,5,51 which prompted the following memory error

[...]
  File "/afs/cern.ch/user/f/fbury/work/inference/dhi/plots/exclusion.py", line 477, in plot_exclusion_and_bestfit_2d
    interpolation=interpolation_method,
  File "/afs/cern.ch/user/f/fbury/work/inference/dhi/plots/util.py", line 413, in get_contours
    fill_hist_from_points(h, x_values, y_values, z_values, **kwargs)
  File "/afs/cern.ch/user/f/fbury/work/inference/dhi/plots/util.py", line 296, in fill_hist_from_points
    interp = scipy.interpolate.Rbf(x_values, y_values, z_values, **rbf_args)
  File "/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/py2-scipy/1.2.1-pafccj/lib/python2.7/site-packages/scipy/interpolate/rbf.py", line 241, in __init__
    self.nodes = linalg.solve(self.A, self.di)
  File "/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/py2-scipy/1.2.1-pafccj/lib/python2.7/site-packages/scipy/interpolate/rbf.py", line 247, in A
    r = squareform(pdist(self.xi.T, self.norm))  # Pairwise norm
  File "/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/py2-scipy/1.2.1-pafccj/lib/python2.7/site-packages/scipy/spatial/distance.py", line 2159, in squareform
    M = np.zeros((d, d), dtype=X.dtype)
MemoryError

Turns out the lists x_values, etc, had a length of 50601, squared in the rbf method and my script exceeded 10 GB of RAM then was killed on lxplus.

I do not have a fix for the rbf choice (maybe a smart dropping of some values in the array ?), but the TGraph2D option (=root argument) works well enough and in a few seconds. On the other hand, the scipy linear option takes a much longer time and provides similar result.

Therefore, I think it makes sense that the user can select the interpolation method, and I copied the argument that as in the LikelihoodScan class.

Merge request reports