Search NASA⌕ Search

SEARCH · Search NASA

Results for “Model misspecification”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

41 records · Page 3

Effect of likelihood misspecification in Gaussian process-driven autonomous experimentation

In recent years, several groups have designed Autonomous Experiment (AE) models with the aim of using them as an alternative method for neutron scattering scanning. In an AE, Gaussian processes (GPs) are most frequently used due to their interpretability, their non-parametric nature, their universal approximation, and their closed-form predictive distribution. GPs have two key components, namely, the model for the likelihood of a neutron count knowing the underlying dynamic structure factor and the acquisition function. In this paper, we investigate the impact, on the quality of an AE, of the likelihood and acquisition function choices, in energy scans and (Q, ω) ones, with respect to the signal-over-noise ratio. While we hypothesized that the quality of GP predictions would decrease when the normal to Poisson likelihood approximation breaks down at low count rates, we found that the use of the correct Poisson likelihood does not improve the quality of the data collected, as well as yields very poor results in (Q, ω) scans at low count rates. In fact, the best results are obtained with a combination of normal likelihood, including the observation noise, and the change in variance acquisition function. In addition, we find that the performance, or quality of the predictive distribution, is a misleading measure of efficiency, that is, of the quality of the data collected.

Perryman, David Elliott [Inst. Laue-Langevin (ILL)↗

A probabilistic fracture mechanics approach for structural reliability assessment of space flight systems

A probabilistic fracture mechanics approach for predicting the failure life distribution due to subcritical crack growth is presented. A state-of-the-art crack propagation method is used in a Monte Carlo simulation to generate a distribution of failure lives. The crack growth failure model expresses failure life as a function of stochastic parameters including environment, loads, material properties, geometry, and model specification errors. A stochastic crack growth rate model that considers the uncertainties due to scatter in the data and mode misspecification is proposed. The rationale for choosing a particular type of probability distribution for each stochastic input parameter and for specifying the distribution parameters is presented. The approach is demonstrated through a probabilistic crack growth failure analysis of a welded tube in the Space Shuttle Main Engine. A discussion of the results from this application of the methodology is given.

Sutharshana, S.↗

Too Good to Be True? Evaluation of Colonoscopy Sensitivity Assumptions Used in Policy Models

Background: Models can help guide colorectal cancer screening policy. Although models are carefully calibrated and validated, there is less scrutiny of assumptions about test performance. Methods: We examined the validity of the CRC-SPIN model and colonoscopy sensitivity assumptions. Standard sensitivity assumptions, consistent with published decision analyses, assume sensitivity equal to 0.75 for diminutive adenomas (<6 mm), 0.85 for small adenomas (6–10 mm), 0.95 for large adenomas (≥10 mm), and 0.95 for preclinical cancer. We also selected adenoma sensitivity that resulted in more accurate predictions. Targets were drawn from the Wheat Bran Fiber study. In the study, we examined how well the model predicted outcomes measured over a three-year follow-up period, including the number of adenomas detected, the size of the largest adenoma detected, and incident colorectal cancer. Results: Using standard sensitivity assumptions, the model predicted adenoma prevalence that was too low (42.5% versus 48.9% observed, with 95% confidence interval 45.3%–50.7%) and detection of too few large adenomas (5.1% versus 14.% observed, with 95% confidence interval 11.8%–17.4%). Predictions were close to targets when we set sensitivities to 0.20 for diminutive adenomas, 0.60 for small adenomas, 0.80 for 10- to 20-mm adenomas, and 0.98 for adenomas 20 mm and larger. Conclusions: Colonoscopy may be less accurate than currently assumed, especially for diminutive adenomas. Alternatively, the CRC-SPIN model may not accurately simulate onset and progression of adenomas in higher-risk populations. Impact: Misspecification of either colonoscopy sensitivity or disease progression in high-risk populations may affect the predicted effectiveness of colorectal cancer screening. When possible, decision analyses used to inform policy should address these uncertainties.

60 APPLIED LIFE SCIENCES↗

Evaluating Gaussian process metamodels and sequential designs for noisy level set estimation

Abstract We consider the problem of learning the level set for which a noisy black-box function exceeds a given threshold. To efficiently reconstruct the level set, we investigate Gaussian process (GP) metamodels. Our focus is on strongly stochastic simulators, in particular with heavy-tailed simulation noise and low signal-to-noise ratio. To guard against noise misspecification, we assess the performance of three variants: (i) GPs with Student- t observations; (ii) Student- t processes (TPs); and (iii) classification GPs modeling the sign of the response. In conjunction with these metamodels, we analyze several acquisition functions for guiding the sequential experimental designs, extending existing stepwise uncertainty reduction criteria to the stochastic contour-finding context. This also motivates our development of (approximate) updating formulas to efficiently compute such acquisition functions. Our schemes are benchmarked by using a variety of synthetic experiments in 1–6 dimensions. We also consider an application of level set estimation for determining the optimal exercise policy of Bermudan options in finance.

97 MATHEMATICS AND COMPUTING↗

Quickest Detection for Changes in Maximal kNN Coherence of Random Matrices

This paper addresses the problem of quickest detection of a change in the maximal coherence between columns of a n p random matrix based on a sequence of matrix observations having a single unknown change point. The random matrix is assumed to have identically distributed rows and the maximal coherence is defined as the largest of the ( 2 p) correlation coefficients associated with any row. Likewise the κ nearest neighbor (kNN) coherence is defined as the κ-th largest of these correlation coefficients. The forms of the preand postchange distributions of the observed matrices are assumed to belong to the family of elliptically contoured densities with sparse dispersion matrices but are otherwise unknown. A nonparametric stopping rule is proposed that is based on the maximal kNN sample coherence between columns of each observed random matrix. This is a summary statistic that is related to a test of existence of a hub vertex in a sample correlation graph having degree at least κ. Performance bounds on the delay and false alarm performance of the proposed stopping rule are obtained in the purely high-dimensional regime where p → ∞ and n is fixed. When the prechange dispersion matrix is diagonal it is shown that, among all functions of the proposed summary statistic, the proposed stopping rule is asymptotically optimal under a minimax quickest change detection model as the stopping threshold approaches infinity. Furthermore, the theory developed also applies to sequential hypothesis testing and fixed sample size tests.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗