DOE OSTI2009
The usual procedure for estimating the significance of a peak in a power spectrum is to calculate the probability of obtaining that value or a larger value by chance (known as the 'p-value'), on the assumption that the time series contains only noise-typically that the measurements are derived from random samplings of a Gaussian distribution. However, since the use of p-values in other contexts is known to be misleading, it seems prudent to examine the implications of using p-values for significance estimation of power spectra. We really need to know the probability that the time series is-or is not-compatible with the 'null hypothesis' that the measurements are derived from noise. This probability can be calculated by Bayesian analysis, but this requires one to specify and evaluate a second hypothesis that the time series does contain a contribution other than noise. We show that the requirement that the p-value should be identical to the probability that the null hypothesis is true leads to an unacceptable form for the likelihood function associated with this hypothesis. We claim that, for this reason, the p-value is not an acceptable method for significance estimation of a power spectrum. In order to obtain an acceptable significance estimate, it is necessary to explicitly consider a second hypothesis, and the key challenge is to identify an appropriate likelihood function for this hypothesis. We first propose four simple conditions that it seems reasonable to impose on this function. We then examine a general functional form for the function, and find the simplest form (which has one free parameter) that meets these conditions. We then define two different ways of combining information derived from two independent power estimates. One procedure is to calculate the post-probabilities of the null hypothesis, convert these to odds-values, and sum the log-odds. The second procedure is to combine the p-values using a procedure due to R.A. Fisher, and to calculate the corresponding post-probability and hence the corresponding log-odds. It seems sensible-even if not logically essential-to seek a likelihood function for which the two procedures lead to the same answer. We find that this consistency condition may be satisfied, to good approximation, by a special case of the previously proposed likelihood function. We find that the resulting significance estimates are considerably more conservative than those usually associated with the p-values. As two examples, we apply the new procedure to two recent analyses of solar neutrino data: (1) power spectrum analysis of Super-Kamiokande data and (2) the combined analysis of radiochemical neutrino data and irradiance data.
79 ASTRONOMY AND ASTROPHYSICS↗