Search NASA⌕ Search

SEARCH · Search NASA

Results for “prediction interval”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Short-Term Load Forecasting Considering EV Charging Loads with Prediction Interval Evaluation

Short-term load forecasting plays a critical role in power system planning and operation. Along with the electrification of various loads, electricity demands are becoming increasingly hard to predict. Notably, the recent rise in electric vehicles (EVs) has further contributed to this unpredictability. To address this issue, this paper proposes a probabilistic load forecasting strategy utilizing Gaussian process regression, structured in a day-ahead manner. While many works focus on deterministic prediction, probabilistic forecasting offers additional insights into variability and uncertainty, enabling more flexible and reliable operation for power systems. To enhance the accuracy of the load forecasting model, the inputs include features related to EV charging habits as well as commonly used weather information. The load forecasting results are evaluated using various metrics, including conventional ones that assess the accuracy of point forecasts, as well as additional metrics that test the reliability of prediction intervals. The proposed load forecasting method is finally tested on real residential power consumption data and EV charging data sampled from real-world sources. The results prove that the new features can greatly improve the performance of the load forecasting method.

electrical vehicle↗
NASA NTRS2012

Bootstrap Prediction Intervals in Non-Parametric Regression with Applications to Anomaly Detection

Prediction intervals provide a measure of the probable interval in which the outputs of a regression model can be expected to occur. Subsequently, these prediction intervals can be used to determine if the observed output is anomalous or not, conditioned on the input. In this paper, a procedure for determining prediction intervals for outputs of nonparametric regression models using bootstrap methods is proposed. Bootstrap methods allow for a non-parametric approach to computing prediction intervals with no specific assumptions about the sampling distribution of the noise or the data. The asymptotic fidelity of the proposed prediction intervals is theoretically proved. Subsequently, the validity of the bootstrap based prediction intervals is illustrated via simulations. Finally, the bootstrap prediction intervals are applied to the problem of anomaly detection on aviation data.

Kumar, Sricharan↗
NASA NTRS1992

Interval prediction in structural dynamic analysis

Methods for assessing the predictive accuracy of structural dynamic models are examined with attention given to the effects of modal mass, stiffness, and damping uncertainties. The methods are based on a nondeterministic analysis called 'interval prediction' in which interval variables are used to describe parameters and responses that are unknown. Statistical databases for generic modeling uncertainties are derived from experimental data and incorporated analytically to evaluate responses. Covariance matrices of modal mass, stiffness, and damping parameters are propagated numerically in models of large space structures by means of three methods. The test data tend to fall within the predicted intervals of uncertainty determined by the statistical databases. The present findings demonstrate the suitability of using data from previously analyzed and tested space structures for assessing the predictive accuracy of an analytical model.

Hasselman, Timothy K.↗
NASA NTRS2014

Prediction Interval Development for Wind-Tunnel Balance Check-Loading

Results from the Facility Analysis Verification and Operational Reliability project revealed a critical gap in capability in ground-based aeronautics research applications. Without a standardized process for check-loading the wind-tunnel balance or the model system, the quality of the aerodynamic force data collected varied significantly between facilities. A prediction interval is required in order to confirm a check-loading. The prediction interval provides an expected upper and lower bound on balance load prediction at a given confidence level. A method has been developed which accounts for sources of variability due to calibration and check-load application. The prediction interval method of calculation and a case study demonstrating its use is provided. Validation of the methods is demonstrated for the case study based on the probability of capture of confirmation points.

Landman, Drew↗
DOE OSTI2022

PI3NN: Out-of-distribution-aware Prediction Intervals from Three Neural Networks

We propose a novel prediction interval (PI) method for uncertainty quantification, which addresses three major issues with the state-of-the-art PI methods. First, existing PI methods require retraining of neural networks (NNs) for every given confidence level and suffer from the crossing issue in calculating multiple PIs. Second, they usually rely on customized loss functions with extra sensitive hyperparameters for which fine tuning is required to achieve a well-calibrated PI. Third, they usually underestimate uncertainties of out-of-distribution (OOD) samples leading to over-confident PIs. Our PI3NN method calculates PIs from linear combinations of three NNs, each of which is independently trained using the standard mean squared error loss. The coefficients of the linear combinations are computed using root-finding algorithms to ensure tight PIs for a given confidence level. We theoretically prove that PI3NN can calculate PIs for a series of confidence levels without retraining NNs and it completely avoids the crossing issue. Additionally, PI3NN does not introduce any unusual hyperparameters resulting in a stable performance. Furthermore, we address OOD identification challenge by introducing an initialization scheme which provides reasonably larger PIs of the OOD samples than those of the in-distribution samples. Benchmark and real-world experiments show that our method outperforms several state-of-the-art approaches with respect to predictive uncertainty quality, robustness, and OOD samples identification.

Liu, Siyan↗
DOE OSTI2018

Nonparametric Prediction Intervals of Wind Power via Linear Programming

This letter proposes a machine learning-based linear programming model that quickly establishes the nonparametric prediction intervals of wind power by integrating extreme learning machine and quantile regression. The proportions of quantiles can be adaptively determined via sensitivity analysis. The proposed method has been proven to be significantly efficient and reliable, with a high application potential in power systems.

extreme learning machine↗
DOE OSTI2021

A prediction interval method for uncertainty quantification of regression models

This paper considers calculation of prediction intervals (PIs) by neural networks (NNs) for quantifying uncertainty in regression tasks, so as to provide fast, accurate and robust emulators to accelerate scientific simulations. We propose a novel method to learn lower and upper bounds of the PI using independent NNs without defining an exclusive loss. Our method requires no distributional assumption, does not introduce extra hyper-parameters, and can effectively identify out-of-distribution samples and quantify their uncertainty. We demonstrate advantages of our method using a benchmark problem and two real-world scientific applications.

Zhang, Pei↗
DOE OSTI2019

On the Effectiveness of Recurrent Neural Networks for Live Modeling of Cyber-Physical Systems

Attention to cyber security of cyber-physical systems (CPS) has led to the development of innovative cyber-resilient methodologies to ensure early detection and mitigation of cyber anomalies and threats. The concept of Digital Twin (DT) has recently emerged as one of the approaches to achieve the objective of resilience. In the approach using DT, a software-based live model of a target CPS is used to continuously monitor, surveil and verify the correctness of the target CPS operation. In this paper, we empirically study the effectiveness of Recurrent Neural Network (RNN)-based models as the basis of DT-based resilience. We uncover the important characteristics of an RNN-based solution with experimentation on a lab-scale Canal Lock CPS emulator with live validations and attack scenarios. For the first time, we demonstrate actual, real-time use of a RNN-based model as a DT for performing live analysis on an operational CPS. Based on the observed results, we highlight the importance of a DT model's training interval, prediction interval and CPS polling interval in the process of anomaly detection. We uncover the limitations in anomaly detection due to real-time synchronization needs of the RNN-based DT. We highlight this uncovered tug of war between synchronization and anomaly detection is inherent in any complex CPS that is monitored and synchronized by relying on the same sensor streams of ground truth for both synchronization as well as anomaly detection.

Yoginath, Srikanth↗
NASA NTRS2017

Predicted Weather Display and Decision Support Interface for Flight Deck

A system and method for providing visual depictions of a predictive weather forecast for in-route vehicle trajectory planning. The method includes displaying weather information on a graphical display, displaying vehicle position information on the graphical display, selecting a predictive interval, displaying predictive weather information for the predictive interval on the graphical display, and displaying predictive vehicle position information for the predictive interval on the graphical display, such that the predictive vehicle position information is displayed relative to the predictive weather information, for in-route trajectory planning.

Johnson, Walter W.↗
DOE OSTI2023

Uncertainty quantification of the virial black hole mass with conformal prediction

Precise measurements of the black hole mass are essential to gain insight on the black hole and host galaxy co-evolution. A direct measure of the black hole mass is often restricted to nearest galaxies and instead, an indirect method using the single-epoch virial black hole mass estimation is used for objects at high redshifts. However, this method is subjected to biases and uncertainties as it is reliant on the scaling relation from a small sample of local active galactic nuclei. In this study, we propose the application of conformalized quantile regression (CQR) to quantify the uncertainties of the black hole predictions in a machine learning setting. We compare CQR with various prediction interval techniques and demonstrated that CQR can provide a more useful prediction interval indicator. In contrast to baseline approaches for prediction interval estimation, we show that the CQR method provides prediction intervals that adjust to the black hole mass and its related properties. That is it yields a tighter constraint on the prediction interval (hence more certain) for a larger black hole mass, and accordingly, bright and broad spectral line width source. Using a combination of neural network model and CQR framework, the recovered virial black hole mass predictions and uncertainties are comparable to those measured from the Sloan Digital Sky Survey. The code is publicly available.

79 ASTRONOMY AND ASTROPHYSICS↗
NASA NTRSDate not supplied

Comparison of Entry Descent and Landing Aerodynamic Databases with Uncertainty Quantification Developed Using Machine Learning Techniques

When developing the aerodynamic databases for use in trajectory simulations, it is important to develop a system of metrics to qualify which aerodynamic models are best to use. Since aerodynamics are just one input into trajectory simulations, the results of these simulations do not reflect on the quality of the aerodynamic database used. This means that aerodynamic database comparisons must be done offline. While traditional metrics that focus on mean/nominal predictions are a good first step, more robust estimates of the prediction interval become important as more focused uncertainty models are developed. We explore the limitations of evaluating aerodynamic models based purely on nominal-centered response surfaces. Before elaborating and evaluating metrics based on distributed models, the value of evaluating prediction interval and confidence interval are discussed to conclude that prediction intervals are more relevant to the use of trajectory analysis. Several metrics to evaluate the prediction interval are introduced with a focus on the standard calibration metric. Finally, we compare candidate models using both mean and distributed metrics. A finalized candidate model developed using state of the art machine learning methods is compared to a baseline model developed using traditional aerodynamic database modeling techniques.

Aerodynamic Database↗
DOE OSTI2015

Online adaptation and verification of VMAT

Purpose: This work presents a method for fast volumetric modulated arc therapy (VMAT) adaptation in response to interfraction anatomical variations. Additionally, plan parameters extracted from the adapted plans are used to verify the quality of these plans. The methods were tested as a prostate class solution and compared to replanning and to their current clinical practice. Methods: The proposed VMAT adaptation is an extension of their previous intensity modulated radiotherapy (IMRT) adaptation. It follows a direct (forward) planning approach: the multileaf collimator (MLC) apertures are corrected in the beam’s eye view (BEV) and the monitor units (MUs) are corrected using point dose calculations. All MLC and MU corrections are driven by the positions of four fiducial points only, without need for a full contour set. Quality assurance (QA) of the adapted plans is performed using plan parameters that can be calculated online and that have a relation to the delivered dose or the plan quality. Five potential parameters are studied for this purpose: the number of MU, the equivalent field size (EqFS), the modulation complexity score (MCS), and the components of the MCS: the aperture area variability (AAV) and the leaf sequence variability (LSV). The full adaptation and its separate steps were evaluated in simulation experiments involving a prostate phantom subjected to various interfraction transformations. The efficacy of the current VMAT adaptation was scored by target mean dose (CTV{sub mean}), conformity (CI{sub 95%}), tumor control probability (TCP), and normal tissue complication probability (NTCP). The impact of the adaptation on the plan parameters (QA) was assessed by comparison with prediction intervals (PI) derived from a statistical model of the typical variation of these parameters in a population of VMAT prostate plans (n = 63). These prediction intervals are the adaptation equivalent of the tolerance tables for couch shifts in the current clinical practice. Results: The proposed adaptation of a two-arc VMAT plan resulted in the intended CTV{sub mean} (Δ ≤ 3%) and TCP (ΔTCP ≤ 0.001). Moreover, the method assures the intended CI{sub 95%} (Δ ≤ 11%) resulting in lowered rectal NTCP for all cases. Compared to replanning, their adaptation is faster (13 s vs 10 min) and more intuitive. Compared to the current clinical practice, it has a better protection of the healthy tissue. Compared to IMRT, VMAT is more robust to anatomical variations, but it is also less sensitive to the different correction steps. The observed variations of the plan parameters in their database included a linear dependence on the date of treatment planning and on the target radius. The MCS is not retained as QA metric due to a contrasting behavior of its components (LSV and AAV). If three out of four plan parameters (MU, EqFS, AAV, and LSV) need to lie inside a 50% prediction interval (3/4—50%PI), all adapted plans will be accepted. In contrast, all replanned plans do not meet this loose criterion, mainly because they have no connection to the initially optimized and verified plan. Conclusions: A direct (forward) VMAT adaptation performs equally well as (inverse) replanning but is faster and can be extended to real-time adaptation. The prediction intervals for the machine parameters are equivalent to the tolerance tables for couch shifts in the current clinical practice. A 3/4—50%PI QA criterion accepts all the adapted plans but rejects all the replanned plans.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2025

Uncertainty Quantification Enabled by Automatic Differentiation for Hydrodynamic Simulation of Shock‐to‐Detonation Transition in High Explosives

Quantifying the effects of uncertainty in a reactive burn model on the run-to-detonation time in high explosives (HEs) provides a robust methodology for assessing the probability of an HE failing the IHE qualification standard. Moreover, uncertainty quantification helps evaluate whether the model calibration accurately represents data outside the calibration set. This study uses a specialized hydrodynamic simulation code for modeling detonation to determine the run-to-detonation time of the HE PBX 9502 for various impact velocities. To quickly approximate uncertainties in the model, a surrogate was constructed using a Taylor series expansion centered at the mean of the input parameters. To obtain the sensitivities required for constructing the Taylor series, HYP-percomplex Automatic Differentiation (HYPAD) was implemented. HYPAD is a methodology for infusing existing codes with automatic differentiation capabilities by augmenting variables with one or more imaginary units to compute step-size independent partial derivatives. These derivatives are accurate to machine precision with respect to the implemented numerical algorithm, meaning their accuracy reflects that of the underlying method (e.g., integration or discretization schemes). Using reduced order modeling techniques, the mean and standard deviation of the run-to-detonation time of a shock within PBX 9502 were computed for a number of initial impact velocities. A weighted least squares regression was then performed to obtain a best fit curve and prediction interval for the computed statistics. Historical data points from explosively driven wedge tests were utilized to validate the prediction interval, ensuring its reliability in predicting future outcomes. With this prediction interval and a known safety constraint curve, the most probable point of failure and the probability of failure for the HE PBX 9502 were determined.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

A comparison of material flow strength models using Bayesian cross-validation

Predicting material flow strength over a range of conditions, such as temperature and strain rate, is necessary for many engineering applications. In this paper, we consider how to compare the predictiveness of several different strength models using a statistical technique called Bayesian cross-validation. Given a dataset of flow strength measurements obtained from mechanictesting experiments, the procedure consists of performing a Bayesian calibration of each strength model on a subset of the data and evaluating how well the trained models predict the remaining data. The predictiveness of a calibrated strength model is quantifiable probabilistically, which provides an interpretable metric for comparing the different models. As an illustrative example, we compare the Johnson-Cook (JC), Zerilli-Armstrong (ZA), Preston-Tonks-Wallace (PTW), and Mechanical Threshold Stress (MTS) flow strength models for the tantalum stress-strain curve data from Chen and Gray (1996). We show that prediction intervals for the four strength models cover the held-out data at most experimental conditions, but also that prediction interval coverage and prediction uncertainty varies by model and experimental condition. Finally, the analysis further allows us to identify experimental regimes for which one of the strength models predicts better than the other three.

36 MATERIALS SCIENCE↗
NASA NTRS1983

Statistical summaries of fatigue data for design purposes

Two methods are discussed for constructing a design curve on the safe side of fatigue data. Both the tolerance interval and equivalent prediction interval (EPI) concepts provide such a curve while accounting for both the distribution of the estimators in small samples and the data scatter. The EPI is also useful as a mechanism for providing necessary statistics on S-N data for a full reliability analysis which includes uncertainty in all fatigue design factors. Examples of statistical analyses of the general strain life relationship are presented. The tolerance limit and EPI techniques for defining a design curve are demonstrated. Examples usng WASPALOY B and RQC-100 data demonstrate that a reliability model could be constructed by considering the fatigue strength and fatigue ductility coefficients as two independent random variables. A technique given for establishing the fatigue strength for high cycle lives relies on an extrapolation technique and also accounts for "runners." A reliability model or design value can be specified.

Wirsching, P. H.↗