Search NASA⌕ Search

SEARCH · Search NASA

Results for “confidence intervals”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Confidence intervals with a priori parameter bounds

We review the methods of constructing confidence intervals that account for a priori information about one-sided constraints on the parameter being estimated. We show that the so-called method of sensitivity limit yields a correct solution of the problem. Derived are the solutions for the cases of a continuous distribution with non-negative estimated parameter and a discrete distribution, specifically a Poisson process with background. For both cases, the best upper limit is constructed that accounts for the a priori information. A table is provided with the confidence intervals for the parameter of Poisson distribution that correctly accounts for the information on the known value of the background along with the software for calculating the confidence intervals for any confidence levels and magnitudes of the background (the software is freely available for download via Internet).

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Calculation of narrower confidence intervals for tree mortality rates when we know nothing but the location of the death/survival events

Abstract Many ecological applications, like the study of mortality rates, require the estimation of proportions and confidence intervals for them. The traditional way of doing this applies the binomial distribution, which describes the outcome of a series of Bernoulli trials. This distribution assumes that observations are independent and the probability of success is the same for all the individual observations. Both assumptions are obviously false in many cases. I show how to apply bootstrap and the Poisson binomial distribution (a generalization of the binomial distribution) to the estimation of proportions. Any information at the individual level would result in better (narrower) confidence intervals around the estimation of proportions. As a case study, I applied this method to the calculation of mortality rates in a forest plot of tropical trees in Lambir Hills National Park, Malaysia. I calculated central estimates and 95% confidence intervals for species‐level mortality rates for 1,007 tree species. I used a very simple model of spatial dependence in survival to estimate individual‐level risk of mortality. The results obtained by accounting for heterogeneity in individual‐level risk of mortality were comparable to those obtained with the binomial distribution in terms of central estimates, but the precision increased in virtually all cases, with an average reduction in the width of the confidence interval of ~20%. Spatial information allows the estimation of individual‐level probabilities of survival, and this increases the precision in the estimates of mortality rates. The general method described here, with modifications, could be applied to reduce uncertainty in the estimation of proportions related to any spatially structured phenomenon with two possible outcomes. More sophisticated approaches can yield better estimates of individual‐level mortality and thus narrower confidence intervals.

59 BASIC BIOLOGICAL SCIENCES↗

Multiplicative scale uncertainties in the unified approach for constructing confidence intervals

We have investigated how uncertainties in the estimation of the detection efficiency affect the 90\% confidence intervals in the unified approach for constructing confidence intervals. The study has been conducted for experiments where the number of detected events is large and can be described by a Gaussian probability density function. We also assume the detection efficiency has a Gaussian probability density and study the range of the relative uncertainties $\sigma_\epsilon$ between 0 and 30\%. We find that the confidence intervals provide proper coverage and increase smoothly and continuously from the intervals that ignore scale uncertainties with a quadratic dependence on $\sigma_\epsilon$.

Smith, Elton↗

Confidence intervals for fitting of atomic models into low-resolution densities

This paper describes procedures for obtaining confidence intervals for coordinate locations resulting from the fitting of atomic models into low-resolution densities. The fitting of high-resolution structures into low-resolution densities obtained from techniques such as electron microscopy or small-angle X-ray scattering can yield powerful new insights. While several algorithms for achieving optimal fits have recently been developed, relatively little effort has been devoted to developing objective measures for judging the quality of the resulting fits, in particular with regard to the danger of overfitting. Here, a general method is presented for obtaining confidence intervals for atomic coordinates resulting from fitting of atomic resolution domain structures into low-resolution densities using well established statistical tools. It is demonstrated that the resulting confidence intervals are sufficiently accurate to allow meaningful statistical tests and to provide tools for detecting potential overfitting.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Correction of confidence intervals in excess relative risk models using Monte Carlo dosimetry systems with shared errors

Introduction: Measurement errors are ubiquitous in epidemiological studies, especially for doses arising from environmental and occupational exposures, which are difficult to assess. Correction methods for independent measurement errors, have been comprehensively described elsewhere, including regression calibration, simulation extrapolation (SIMEX), etc. However, when exposure estimates are constructed using complicated physical and biological models, uncertainties in the dosimetry system can become very complex. In such systems, uncertainties in some parameters may affect a large group of study participants simultaneously, and hence are “shared” as opposed to independent dosimetric uncertainties. Representing the structure of the uncertainty in the dosimetry by providing multiple realizations of possible dose has been described for several studies. The goal of this paper is to implement, and explore the properties of, a generalized estimating equation-based approach to shared uncertainties based on principles described in an earlier paper. Methods: The effects of shared and unshared dosimetry error on parameter estimates and confidence intervals in a generalized excess relative risk model are examined. A post-hoc correction method is implemented that constructs corrected confidence intervals based on an underlying variance structure described in an earlier paper. In this correction method it is assumed that the estimated slope parameter in a linear excess relative risk model is distributed as a mixture of normal and lognormal components. A simulation study of the behavior of this correction was developed based on multiple realizations provided by the Monte Carlo dosimetry system used for the plutonium exposures in the Mayak Worker Cohort and a risk model with close similarity to that used in analyses of lung cancer data in that cohort. Results: In the simulation study we found that shared dose errors severely degraded the coverage probabilities of confidence limits for the slope parameter in the linear excess relative risk model relating uncertain exposure to lung cancer risk. Using our correction method to adjust the confidence intervals provided much improved coverage probabilities for this parameter. A smaller amount of degradation in confidence interval coverage for other parameters (e.g. those in the baseline and dose modification terms) was noted, and the proposed correction method also improved coverage for these parameters as well. Discussion: Compared to other recently described methods our proposed correction method relies on relatively familiar software to fit dose response models. The basic modeling philosophy described here first uses existing Poisson regression methods to estimate parameters in the excess relative risk model; in this first stage mean dose (computed from the multiple realizations) is used as the independent variable in the regression. Once the model is fit we then correct (widen) the standard errors for all parameters using the proposed correction method. The simulation studies indicate that the methods described here are an important contribution to the available tools for dealing with shared measurement errors in complex risk analysis.

61 RADIATION PROTECTION AND DOSIMETRY↗

Statistical variability and confidence intervals for planar dose QA pass rates

Purpose: The most common metric for comparing measured to calculated dose, such as for pretreatment quality assurance of intensity-modulated photon fields, is a pass rate (%) generated using percent difference (%Diff), distance-to-agreement (DTA), or some combination of the two (e.g., gamma evaluation). For many dosimeters, the grid of analyzed points corresponds to an array with a low areal density of point detectors. In these cases, the pass rates for any given comparison criteria are not absolute but exhibit statistical variability that is a function, in part, on the detector sampling geometry. In this work, the authors analyze the statistics of various methods commonly used to calculate pass rates and propose methods for establishing confidence intervals for pass rates obtained with low-density arrays. Methods: Dose planes were acquired for 25 prostate and 79 head and neck intensity-modulated fields via diode array and electronic portal imaging device (EPID), and matching calculated dose planes were created via a commercial treatment planning system. Pass rates for each dose plane pair (both centered to the beam central axis) were calculated with several common comparison methods: %Diff/DTA composite analysis and gamma evaluation, using absolute dose comparison with both local and global normalization. Specialized software was designed to selectively sample the measured EPID response (very high data density) down to discrete points to simulate low-density measurements. The software was used to realign the simulated detector grid at many simulated positions with respect to the beam central axis, thereby altering the low-density sampled grid. Simulations were repeated with 100 positional iterations using a 1 detector/cm{sup 2} uniform grid, a 2 detector/cm{sup 2} uniform grid, and similar random detector grids. For each simulation, %/DTA composite pass rates were calculated with various %Diff/DTA criteria and for both local and global %Diff normalization techniques. Results: For the prostate and head/neck cases studied, the pass rates obtained with gamma analysis of high density dose planes were 2%-5% higher than respective %/DTA composite analysis on average (ranging as high as 11%), depending on tolerances and normalization. Meanwhile, the pass rates obtained via local normalization were 2%-12% lower than with global maximum normalization on average (ranging as high as 27%), depending on tolerances and calculation method. Repositioning of simulated low-density sampled grids leads to a distribution of possible pass rates for each measured/calculated dose plane pair. These distributions can be predicted using a binomial distribution in order to establish confidence intervals that depend largely on the sampling density and the observed pass rate (i.e., the degree of difference between measured and calculated dose). These results can be extended to apply to 3D arrays of detectors, as well. Conclusions: Dose plane QA analysis can be greatly affected by choice of calculation metric and user-defined parameters, and so all pass rates should be reported with a complete description of calculation method. Pass rates for low-density arrays are subject to statistical uncertainty (vs. the high-density pass rate), but these sampling errors can be modeled using statistical confidence intervals derived from the sampled pass rate and detector density. Thus, pass rates for low-density array measurements should be accompanied by a confidence interval indicating the uncertainty of each pass rate.

61 RADIATION PROTECTION AND DOSIMETRY↗

Bootstrap-based confidence interval estimation for thermal security region of bulk power grid

Thermal security region (TSR) is a powerful tool for monitoring and controlling the thermal security of bulk power grid with high penetration of renewable energy. One of the major challenges of TSR in practical application is to obtain the exact analytical expression of the TSR boundary (TSRB) because TSRB is a nonlinear hypersurface described by an implicit function. For this reason, the TSRB is usually approximated by hyperplane instead of getting the exact analytical expression. Traditionally, the coefficients of the hyperplane are estimated with a set of original TSRB points using least square estimation (LSE). However, LSE is a point estimation method, which is incapable of evaluating the quality of the estimation. In addition, the accuracy and computational efficiency of the hyperplane approximation are influenced by the number of TSRB points significantly. Here, a bootstrap based confidence interval estimation is proposed to estimate not only the coefficients of TSRB approximation hyperplane, but also the standard deviations and confidence intervals of the coefficients for evaluating the quality and reliability of the approximation results. First, empirical distribution functions (EDFs) of the coefficients of TSRB approximation hyperplane are approximated from a set of original TSRB points by using residual resampling bootstrap method. Then, the EDFs of the coefficients are employed to estimate the coefficients of the approximation hyperplane. Meanwhile, the standard deviations and confidence intervals of the estimated coefficients of the hyperplanes are also calculated for evaluating the quality and reliability of the approximation. The proposed approach is tested on the China Southern Power Grid (CSG). Results of simulations validate the accuracy and efficiency of the proposed approach in approximating the TSRB.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Modified frequentist determination of confidence intervals for poisson distribution

We propose modified frequentist definition for the determination of confidence intervals for the case of Poisson statistics. Namely, we require that 1−β{sup ′}⩾∑{sub n=0}{sup n=n{sub o}{sub b}{sub s}+k}P(n|λ)⩾α{sup ′}. We show that this definition is equivalent to the Bayesian method with prior π(λ) ∼ λ{sup k}. We also propose modified frequentist definition for the case of nonzero background.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Confidence Intervals from Realizations of Simulated Nuclear Data

Various statistical techniques are discussed that can be used to assign a level of confidence in the prediction of models that depend on input data with known uncertainties and correlations. The particular techniques reviewed in this paper are: 1) random realizations of the input data using Monte-Carlo methods, 2) the construction of confidence intervals to assess the reliability of model predictions, and 3) resampling techniques to impose statistical constraints on the input data based on additional information. These techniques are illustrated with a calculation of the keff value, based on the 235 U(n, f) and 239 Pu (n, f) cross sections.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Monte Carlo method for constructing confidence intervals with unconstrained and constrained nuisance parameters in the NOvA experiment

Measuring observables to constrain models using maximum-likelihood estimation is fundamental to many physics experiments. Wilks' theorem provides a simple way to construct confidence intervals on model parameters, but it only applies under certain conditions. These conditions, such as nested hypotheses and unbounded parameters, are often violated in neutrino oscillation measurements and other experimental scenarios. Monte Carlo methods can address these issues, albeit at increased computational cost. In the presence of nuisance parameters, however, the best way to implement a Monte Carlo method is ambiguous. Furthermore, this paper documents the method selected by the NOvA experiment, the profile construction. It presents the toy studies that informed the choice of method, details of its implementation, and tests performed to validate it. It also includes some practical considerations which may be of use to others choosing to use the profile construction.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

The use of latin hypercube sampling for the efficient estimation of confidence intervals

Latin hypercube sampling (LHS) has long been used as a way of assuring adequate sampling of the tails of distributions in a Monte Carlo analysis and provided the framework for the uncertainty analysis performed in the NUREG-1150 risk assessment. However, this technique has not often been used in the performance of regulatory analyses due to the inability to establish confidence levels on the quantiles of the output distribution. Recent work has demonstrated a method that makes this possible. This method is compared to the procedure of crude Monte Carlo using order statistics, which is currently used to establish confidence levels. The results of several statistical examples demonstrate that the LHS confidence interval method can provide a more accurate and precise solution, but issues remain when applying the technique generally. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

The Profiled Feldman-Cousins Method for Confidence Interval Construction for the Nova 3-Flavor Oscillation Analysis

The small interaction cross-section of neutrinos makes experimental neutrino physics particularly responsive to technological advancements. A significant development leveraged by the NOvA experiment is large-scale parallel processing, enabling novel computational approaches to longstanding experimental challenges. Central to managing the resulting high-throughput data is NOvA’s implementation of the Freight Train model, designed for efficient data production and handling.This dissertation details the methodology and execution of the NOvA 2024 3-Flavor Oscillation Analysis, supported by a comprehensive dataset spanning ten years. It emphasizes frequentist results refined through the Feldman-Cousins (FC) technique, specifically addressing confidence interval corrections in parameter estimation. The computational intensity associated with Feldman-Cousins arises from extensive Monte Carlo simulations, which were substantially mitigated through parallel computing on the Perlmutter supercomputer at the National Energy Research Scientific Computing Center (NERSC), employing the MPI framework.To further enhance computational efficiency, an Importance Sampling method is introduced and evaluated, demonstrating significant potential to reduce complexity, particularly in exploring extreme parameter space regions. This thesis presents both the successful application of advanced computational resources and the development of sophisticated statistical techniques, aiming to enhance the precision and scope of neutrino oscillation analyses.

Dye ajdye11190@gmail.com, Andrew Joseph [Mississip↗

Behavior Detection using Confidence Intervals of Hidden Markov Models

Markov models are commonly used to analyze real-world problems. Their combination of discrete states and stochastic transitions is suited to applications with deterministic and stochastic components. Hidden Markov Models (HMMs) are a class of Markov model commonly used in pattern recognition. Currently, HMMs recognize patterns using a maximum likelihood approach. One major drawback with this approach is that data observations are mapped to HMMs without considering the number of data samples available. Another problem is that this approach is only useful for choosing between HMMs. It does not provide a criteria for determining whether or not a given HMM adequately matches the data stream. In this work, we recognize complex behaviors using HMMs and confidence intervals. The certainty of a data match increases with the number of data samples considered. Receiver Operating Characteristic curves are used to find the optimal threshold for either accepting or rejecting a HMM description. We present one example using a family of HMM's to show the utility of the proposed approach. A second example using models extracted from a database of consumer purchases provides additional evidence that this approach can perform better than existing techniques.

97 MATHEMATICS AND COMPUTING↗

Technical Report: Algorithm and Implementation for Quasispecies Abundance Inference with Confidence Intervals from Metagenomic Sequence Data

This report describes the design and implementation of an algorithm for estimating relative microbial abundances, together with confidence limits, using data from metagenomic DNA sequencing. For the background behind this project and a detailed discussion of our modeling approach for metagenomic data, we refer the reader to our earlier technical report, dated March 4, 2014. Briefly, we described a fully Bayesian generative model for paired-end sequence read data, incorporating the effects of the relative abundances, the distribution of sequence fragment lengths, fragment position bias, sequencing errors and variations between the sampled genomes and the nearest reference genomes. A distinctive feature of our modeling approach is the use of a Chinese restaurant process (CRP) to describe the selection of genomes to be sampled, and thus the relative abundances. The CRP component is desirable for fitting abundances to reads that may map ambiguously to multiple targets, because it naturally leads to sparse solutions that select the best representative from each set of nearly equivalent genomes.

59 BASIC BIOLOGICAL SCIENCES↗

Neutron multiplicity counting: Confidence intervals for reconstruction parameters

From nuclear materials accountability to homeland security, the need for improved nuclear material detection, assay, and authentication has grown over the past decades. Starting in the 1940s, neutron multiplicity counting techniques have enabled quantitative evaluation of masses and multiplications of fissile materials. In this paper, we propose a new method to compute uncertainties on these parameters using a model-based sequential Bayesian processor, resulting in credible regions in the fissile material mass and multiplication space. These uncertainties will enable us to evaluate quantitatively proposed improvements to the theoretical fission chain model. Additionally, because the processor can calculate uncertainties in real time, it is a useful tool in applications such as portal monitoring: monitoring can stop as soon as a preset confidence of non-threat is reached.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Computing confidence intervals on solution costs for stochastic grid generation expansion problems.

A range of core operations and planning problems for the national electrical grid are naturally formulated and solved as stochastic programming problems, which minimize expected costs subject to a range of uncertain outcomes relating to, for example, uncertain demands or generator output. A critical decision issue relating to such stochastic programs is: How many scenarios are required to ensure a specific error bound on the solution cost? Scenarios are the key mechanism used to sample from the uncertainty space, and the number of scenarios drives computational difficultly. We explore this question in the context of a long-term grid generation expansion problem, using a bounding procedure introduced by Mak, Morton, and Wood. We discuss experimental results using problem formulations independently minimizing expected cost and down-side risk. Our results indicate that we can use a surprisingly small number of scenarios to yield tight error bounds in the case of expected cost minimization, which has key practical implications. In contrast, error bounds in the case of risk minimization are significantly larger, suggesting more research is required in this area in order to achieve rigorous solutions for decision makers.

09 BIOMASS FUELS↗