Self-calibrating the look-elsewhere effect: fast evaluation of the statistical significance using peak heights

Abstract

In experiments where one searches a large parameter space for an anomaly, one often finds many spurious noise-induced peaks in the likelihood. This is known as the look-elsewhere effect, and must be corrected for when performing statistical analysis. Here, this paper introduces a method to calibrate the false alarm probability (FAP), or p-value, for a given dataset by considering the heights of the highest peaks in the likelihood. Specifically, we derive an equation relating the global p-value to the rank and height of local maxima. In the simplest form of self-calibration, the look-elsewhere-corrected $\chi^2$ of a physical peak is approximated by the $\chi^2$ of the peak minus the $\chi^2$ of the highest noise-induced peak, with accuracy improved by considering lower peaks. In contrast to alternative methods, this approach has negligible computational cost as peaks in the likelihood are a byproduct of every peak-search analysis. We apply to examples from astronomy, including planet detection, periodograms, and cosmology.