Search NASA⌕ Search

SEARCH · Search NASA

Results for “prediction error”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Error field predictability and consequences for ITER

Abstract ITER coil tolerances are re-evaluated using the modern understanding of coupling to least-stable plasma modes and an updated center-line-traced model of ITER’s coil windings. This reassessment finds the tolerances to be conservative through a statistical, linear study of n = 1 error fields (EFs) due to tilted, shifted misplacements and nominal windings of central solenoid and poloidal field coils within tolerance. We also show that a model-based correction scheme remains effective even when metrology quality is sub-optimal, and compare this to projected empirical correction schemes. We begin with an analysis of the necessity of error field correction (EFC) for daily operation in ITER using scalign laws for the EF penetration threshold. We then consider the predictability of EF dominant mode overlap across early planned ITER scenarios and, as measuring EFs in high power scenarios can pose risks to the device, the potential for extrapolation to the ITER Baseline Scenario (IBS). We find that carefully designing a scenario matching currents proportionally to those of the IBS is far more important than plasma shape or profiles in accurately measuring an optimal correction current set.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2024

Evaluation of GlassNet for physics-informed machine learning of glass stability and glass-forming ability

Glassy materials form the basis of many modern applications, including nuclear waste immobilization, touch-screen displays, and optical fibers, and also hold great potential for future medical and environmental applications. However, their structural complexity and large composition space make design and optimization challenging for certain applications. Of particular importance for glass processing and design is an estimate of a given composition's glass-forming ability (GFA). However, there remain many open questions regarding the underlying physical mechanisms of glass formation, especially in oxide glasses. It is apparent that a proxy for GFA would be highly useful in glass processing and design, but identifying such a surrogate property has proven itself to be difficult. While glass stability (GS) parameters have historically been used as a GFA surrogate, recent research has demonstrated that most of these parameters are not accurate predictors of the GFA of oxide glasses. Here, in this work, we explore the application of an open-source pre-trained neural network model, GlassNet, that can predict the characteristic temperatures necessary to compute GS with reasonable performance and assess the feasibility of using these physics-informed machine learning (PIML)-predicted GS parameters to estimate GFA. In doing so, we track the uncertainties at each step of the computation—from the original ML prediction errors to the compounding of errors during GS estimation, and finally to the final estimation of GFA. While GlassNet exhibits reasonable accuracy on all individual properties, we observe a large compounding of error in the combination of these individual predictions for the PIML prediction of GS, finding that random forest models offer similar accuracy to GlassNet. We also break down the performance of GlassNet on different glass families and find that the error in GS prediction is correlated with the error in crystallization peak temperature prediction. Lastly, we utilize this finding to assess the relationship between top-performing GS parameters and GFA for two ternary glass systems: sodium borosilicate and sodium iron phosphate glasses. We conclude that to obtain true ML predictive capability of GFA, significantly more data needs to be collected.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Bosonic field digitization for quantum computers

Quantum simulation of quantum field theory is a flagship application of quantum computers that promises to deliver capabilities beyond classical computing. The realization of quantum advantage will require methods that can accurately predict error scaling as a function of the resolution and parameters of the model and that can be implemented efficiently on quantum hardware. In this paper, we address the representation of lattice bosonic fields in a discretized field amplitude basis, develop methods to predict error scaling, and present efficient qubit implementation strategies. A low-energy subspace of the bosonic Hilbert space, defined by a boson occupation number cutoff, can be represented with exponentially good accuracy by a low-energy subspace of a finite-size Hilbert space. The finite representation construction and the associated errors are directly related to the accuracy of the Nyquist-Shannon sampling and the finite Fourier transforms of the boson number states in the field and the conjugate-field bases. We analyze the relation between the boson mass, the discretization parameters used for wave function sampling, and the finite representation size. Numerical simulations of small size Φ 4 problems demonstrate that the boson mass optimizing the sampling of the ground state wave function is a good approximation to the optimal boson mass yielding the minimum low-energy subspace size. However, we find that accurate sampling of general wave functions does not necessarily result in accurate representation. Finally, we develop methods for validating and adjusting the discretization parameters to achieve more accurate simulations.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2024

A Simplified Method for Predicting Shaker Voltage in IMMATs

Impedance Matched Multi-Axis Tests (IMMATs) can replicate in-service vibration induced stress more accurately than single axis shaker table tests as they can better match a part’s operational boundary conditions and excite it in multiple degrees of freedom simultaneously. Here, the shakers used in IMMATs are less powerful than shaker tables, so shaker force limits can be exceeded during tests if they are not placed adequately for the desired environment. The ability to predict shaker voltage and force before performing a test is, therefore, helpful in selecting shaker locations so that their limits are not exceeded. In this study, electrodynamic shakers were modeled as discrete electromechanical systems, and the shaker parameters were chosen to match experimentally obtained acceleration/voltage frequency response functions (FRFs). These models were coupled to a finite element model of the device under test (DUT) via dynamic substructuring, and the substructured model was demonstrated to accurately predict shaker voltage as well as the error in reproducing the environment at multiple accelerometer locations. A simple method called the FRF Multiplication method, in which the FRF of the substructured system is approximated as the product of two separate FRFs of the shaker and DUT respectively, was proposed and applied to the same system, yielding similar voltage and error predictions to those obtained using substructuring. Simple case studies were presented to explore the applicability of the proposed method, and it was demonstrated to have similar accuracy to the substructuring method in a range of cases. Additionally, we showed that while it was not possible to derive a unique model of the shakers from acceleration/voltage FRFs alone, the models that could be obtained were sufficient to predict test error almost perfectly and shaker voltage with less than 40 percent error.

42 ENGINEERING↗
DOE OSTI2023

Long–short-term memory encoder–decoder with regularized hidden dynamics for fault detection in industrial processes

The ability of recurrent neural networks (RNN) to model nonlinear dynamics of high dimensional process data has enabled data-driven RNN-based fault detection algorithms. Previous studies have focused on detecting faults by identifying the discrepancies in data distribution between the faulty and normal data, as reflected in prediction errors generated by RNN models. However, in industrial processes, variations in data distribution can also result from changes in normal control setpoints and compensatory control adjustments in response to disturbances, making it hard to differentiate between normal and faulty conditions. This paper proposes a fault detection method utilizing a long short-term memory (LSTM) encoder–decoder structure with regularized hidden dynamics and reversible instance normalization (RevIN) to compactly represent high-dimensional measurements for effective monitoring. During training, the hidden states of the model are regularized to form a low-dimensional latent space representation of the original multivariate time series data. As a result, the prediction errors of the latent states can be used to monitor the abnormal dynamic variations, while the reconstruction errors of the measured variables are used to monitor the abnormal static variations. Furthermore, the proposed indices can reflect operating conditions, even when the distribution of test data changes, which helps distinguish faults from normal adjustments and disturbances that controllers can settle. Here, data from numerical simulation and the Tennessee Eastman process are used to illustrate the effectiveness of the proposed fault detection method.

42 ENGINEERING↗
DOE OSTI2022

Imperfect slope measurements drive overestimation in a geometric cone model of lake and reservoir depth

Lake and reservoir (waterbody) depth is a critical characteristic that influences many important ecological processes. Unfortunately, depth measurements are labor-intensive to gather and are only available for a small fraction of waterbodies globally. Therefore, scientists have tried to predict depth from characteristics easily obtained for all waterbodies, such as surface area or the slope of the surrounding land. One approach for predicting waterbody depth simulates basins using a geometric cone model where the nearshore land slope and distance to the center of the waterbody are assumed to be representative proxies for in-lake slope and distance to the deepest point respectively. We tested these assumptions using bathymetry data from ~5000 lakes and reservoirs to examine whether differences in waterbody type or shape influenced depth prediction error. Here, we found that nearshore land slope was not representative of in-lake slope, and using it for prediction increases error substantially relative to models using true in-lake slope for all waterbody types and shapes. Predictions were biased toward overprediction in concave waterbodies (i.e., bowl-shaped; up to 18% of the study population) and reservoir waterbodies (up to 30% of the study population). Despite this systematic overprediction, model errors were fewer (in absolute and relative terms, irrespective of any specific slope covariate) for concave than convex waterbodies, suggesting the geometric cone model is an adequate representation of depth for these waterbodies. But because convex waterbodies are far more common (>72% of our study population), minimizing overall depth prediction error remains a challenge.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

EPIsembleVis: A geo-visual analysis and comparison of the prediction ensembles of multiple COVID-19 models

In this work, we present EPIsembleVis, a web-based comparative visual analysis tool for evaluating the consistency of multiple COVID-19 prediction models. Our approach analyzes a collection of COVID-19 predictions from different epidemiological models as an ensemble and utilizes two metrics to quantify model performance. These metrics include (a) prediction uncertainty (represented as the dispersion of predictions in each ensemble) and (b) prediction error (calculated by comparing individual model predictions with the recorded data). Through an interactive visual interface, our approach provides a data-driven workflow for (a) selecting and constructing the COVID-19 model prediction ensemble based on the spatiotemporal overlap of available predictions of multiple epidemiological models, (b) quantifying the model performance using both the uncertainty of each model prediction ensemble, and the error of each ensemble member that represents individual model predictions, and (c) visualizing the spatiotemporal variability in the projection performance of individual models using a suite of novel ensemble visualization techniques, such as the data availability map, a spatiotemporal textured-tile calendar, multivariate rose chart, and time-series leaflet glyph. We demonstrate the capability of our ensemble visual interface through a case study that investigates the performance of weekly COVID-19 predictions, which are provided through the COVID-19 Forecast Hub UMass-Amherst Influenza Forecasting Center of Excellence [47] for the United States and United States Territories. The EPIsembleVis tool is implemented using open-source web technologies and adaptive system design, rendering it interoperable with Elasticsearch and Kibana for automatically ingesting COVID-19 predictions from online repositories, and it is generalizable for analyzing worldwide projections from more epidemiological models.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2020

Calibration of reactive burn and Jones-Wilkins-Lee parameters for simulations of a detonation-driven flow experiment with uncertainty quantification

Here, uncertainties in the explosive-specific parameters of the Jones-Wilkins-Lee (JWL) equation of state (EOS) are carefully considered in hydrodynamic simulations of an explosive experiment to minimize the error in the flow prediction. Experimental data of the leading shock position in the transverse direction over time serves as the prediction metric for quantifying simulation prediction error. The uncertainty quantification technique, global sensitivity analysis, is utilized to determine the JWL parameters to which the transverse shock propagation is most sensitive. A polynomial response surface (PRS) is constructed in the space of the most influential JWL parameters, and the point of minimum error between the experimental data and the PRS yields calibrated JWL parameters for the experimental flow. The simulation results following the parameter calibration show good agreement with the experimental data. It was found that two significant parameters, the heat release per unit mass of reactant Q and JWL model exponent R 1 are strongly related, which makes it difficult to identify accurate values.

36 MATERIALS SCIENCE↗
DOE OSTI2023

UniKP: a unified framework for the prediction of enzyme kinetic parameters

Prediction of enzyme kinetic parameters is essential for designing and optimizing enzymes for various biotechnological and industrial applications, but the limited performance of current prediction tools on diverse tasks hinders their practical applications. Here, we introduce UniKP, a unified framework based on pretrained language models for the prediction of enzyme kinetic parameters, including enzyme turnover number (k cat ), Michaelis constant (K m ), and catalytic efficiency (k cat / K m ), from protein sequences and substrate structures. A two-layer framework derived from UniKP (EF-UniKP) has also been proposed to allow robust k cat prediction in considering environmental factors, including pH and temperature. In addition, four representative re-weighting methods are systematically explored to successfully reduce the prediction error in high-value prediction tasks. We have demonstrated the application of UniKP and EF-UniKP in several enzyme discovery and directed evolution tasks, leading to the identification of new enzymes and enzyme mutants with higher activity. UniKP is a valuable tool for deciphering the mechanisms of enzyme kinetics and enables novel insights into enzyme engineering and their industrial applications.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

A cross-dimensional analysis of data-driven short-term load forecasting methods with large-scale smart meter data

Electricity load forecasting is essential to utility operation and power grid stability. A wide spectrum of data-driven methods, ranging from linear regression models to more recent deep learning models have been adopted to forecast electric load over the years. However, there still lacks a holistic evaluation of the applicability of conventional statistical and machine learning based algorithms with respect to different temporal and spatial scopes, computational requirements, and sensitivity of model-tuning. Enabled by a large-scale electricity load profile dataset of over 40,000 residential customers in a utility region, we conducted a cross-dimensional analysis of data-driven load forecasting methods. Three regression-based and seven deep learning algorithms with different model configurations were evaluated in terms of their overall and peak load prediction accuracy, and training burdens, across spatial aggregation levels ranging from the transformer, feeder, substation, to neighborhood. We found, first, the load forecasting accuracy is constrained by a predictability boundary, influenced by the forecasting horizon and spatial aggregation level. Specifically, RandomForest, XGBoost, TFT, TSMixer, and TiDE models achieved less than 10 % prediction error for up to 96-h ahead forecasting for district, substation, and feeder levels, while other models struggle at long-horizon predictions; Second, for winter and summer peak load dates, most models were able to predict the peak demand timing within ± 1 h, but the prediction percentage error varied by models, with TFT and TiDE models being the top performers; Third, models with similar prediction accuracy can differ in training burden by an order of magnitude. Therefore, choosing model configurations that balance prediction performance and computational resource is an important practical consideration for large-scale deployment of the machine learning based load forecasting. The outcome of this study can guide researchers and practitioners to choose the proper load forecasting algorithms based on their problem scope, required accuracy, and available resources. The predictability boundary can serve as a benchmark for electricity load forecasting problems with new algorithms and datasets.

Li, Han↗
DOE OSTI2024

Machine learning materials properties with accurate predictions, uncertainty estimates, domain guidance, and persistent online accessibility

One compelling vision of the future of materials discovery and design involves the use of machine learning (ML) models to predict materials properties and then rapidly find materials tailored for specific applications. However, realizing this vision requires both providing detailed uncertainty quantification (model prediction errors and domain of applicability) and making models readily usable. At present, it is common practice in the community to assess ML model performance only in terms of prediction accuracy (e.g. mean absolute error), while neglecting detailed uncertainty quantification and robust model accessibility and usability. Here, we demonstrate a practical method for realizing both uncertainty and accessibility features with a large set of models. We develop random forest ML models for 33 materials properties spanning an array of data sources (computational and experimental) and property types (electrical, mechanical, thermodynamic, etc). All models have calibrated ensemble error bars to quantify prediction uncertainty and domain of applicability guidance enabled by kernel-density-estimate-based feature distance measures. All data and models are publicly hosted on the Garden-AI infrastructure, which provides an easy-to-use, persistent interface for model dissemination that permits models to be invoked with only a few lines of Python code. We demonstrate the power of this approach by using our models to conduct a fully ML-based materials discovery exercise to search for new stable, highly active perovskite oxide catalyst materials.

domain of applicability↗
DOE OSTI2022

Using an Explainable Machine Learning Approach to Characterize Earth System Model Errors: Application of SHAP Analysis to Modeling Lightning Flash Occurrence

Abstract Computational models of the Earth System are critical tools for modern scientific inquiry. Efforts toward evaluating and improving errors in representations of physical and chemical processes in these large computational systems are commonly stymied by highly nonlinear and complex error behavior. Recent work has shown that these errors can be effectively predicted using modern Artificial Intelligence (A.I.) techniques. In this work, we go beyond these previous studies to apply an explainable A.I. technique to not only predict model errors but also move toward understanding the underlying reasons for successful error prediction. We use XGBoost classification trees and SHapley Additive exPlanations analysis to explore the errors in the prediction of lightning occurrence in the NASA Goddard Earth Observing System model, a widely used Earth System Model. This explainable error prediction system can effectively predict the model error and indicates that the errors are strongly related to convective processes and the characteristics of the land surface.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Self-diagnosis of model suitability for continuous measurements of stream-dissolved organic carbon derived from in situ UV–visible spectroscopy

Application of high-frequency monitoring of dissolved organic carbon (DOC) is difficult in instances where training datasets are challenging to develop (e.g., remote locations) and the relationship between optical features and DOC concentration changes due to environmental or landscape shifts (e.g., climate or land-use change). We developed and compared three partial least squares (PLS) models using in situ water level measurements, conductivity, and UV–Vis spectral attenuation to predict DOC. Two site-specific models were developed using data from a hillslope-dominated forest or a low-relief wetland-pond-dominated stream catchment. The third model, using data from both sites, exhibited the best performance (DOC range = 4–15.5 mg C L −1 , mean = 8.38 mg C L −1 , training RMSE = 0.34 mg C L −1 , internal validation RMSE = 0.50 mg C L −1 , external validation RMSE = 2.43 mg C L −1 ). We further demonstrate using PLS model statistics to monitor performance and elucidate when and how models should be updated. These statistics, Hotelling's T 2 and squared prediction errors, are useful consistency checks for the predictions made and detect underlying inconsistencies that, if undetected, can reduce the robustness of DOC prediction. For example, via the T 2 statistic, we identified the summer–autumn transition as a period when DOC composition differed from what was represented in the training dataset. We also determined that elevated SUVA 254 values contributed to the overall bias observed in predictions made during the subsequent year as part of the external validation. This enabled the application of a bias correction that reduced the RMSE from 2.43 to 0.89 mg C L −1 . The method presented here could be applied to future monitoring programs enabling model updates to monitor DOC fluxes accurately from optical datasets (e.g., attenuance or fluorescence) in the face of developing datasets in remote locations or environmental change. In conclusion, implementation of this approach may also identify possible regime shifts or landscape and hydrologic change associated with climate and other environmental changes relevant to terrestrial to aquatic fluxes.

UV-VIS spectroscopy↗
DOE OSTI2026

Parametric reduced order models for graded lattice structures

Graded lattice structures, characterized by smoothly varying mechanical properties, hold significant promise for optimizing material distribution in advanced engineering applications. However, accurately modeling these structures poses substantial computational challenges due to the continuous geometric variations within their unit cells. Here, to address these challenges, this paper introduces a novel Efficient Reduced Order Model (EROM) that integrates the Matrix Discrete Empirical Interpolation Method (MDEIM) and Discrete Empirical Interpolation Method (DEIM) with polynomial regression to efficiently manage geometric parametrization in lattice structures. Unlike traditional reduced order models (ROMs) that require extensive precomputed libraries for each geometric configuration, our approach enables continuous geometric variations through a flexible algebraic formulation, significantly reducing computational costs while preserving high accuracy. The method constructs projection matrices for individual unit cells that can be efficiently assembled into global systems, leveraging the repetitive nature of lattice structures. Numerical studies demonstrate that our EROM achieves displacement errors below 1% and von Mises stress prediction errors below 4%, coupled with computational speedups exceeding two orders of magnitude compared to full-order simulations. The proposed method's modularity and scalability make it particularly suitable for design optimization and real-time simulation of functionally graded lattice structures, with applications spanning aerospace to biomedical engineering.

11 - NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2023

Regional Medium-Term Hourly Electricity Demand Forecasting Based on LSTM

This paper aims to forecast high-resolution (hourly) aggregated load for a certain region in the medium term (a few days to over a year). One region is defined as some places with similar climate characteristics because the climate influences people's daily lifestyles and hence the electric usage. We decom- pose the electric usage records into two parts: base load and seasonal load. Considering both temperature and time factors, different deep learning methods are adopted to characterize them. The first goal of our approach is to predict the peak load which is critical for power system planning. Furthermore, our proposed forecast method can provide the depiction of the hourly load profile to provide customized load curves for high- level real-time applications. The proposed method is tested on real-world historical data collected by CAISO, BPA, and PACW. The experimental results show that trained by three years of data, our method could reduce the prediction error for one-year lead hourly load below 5% MAPE, and predict the occurrence of the peak load for next year in CAISO with an error within three days. Furthermore, as a byproduct, an interesting observation on the impact of COVID-19 on human life was made and discussed based on these case studies.

deep learning↗
DOE OSTI2022

Revenue prediction for integrated renewable energy and energy storage system using machine learning techniques

Revenue estimation for integrated renewable energy and energy storage systems is important to support plant owners or operators’ decisions in battery sizing selection that leads to maximized financial performances. A common approach to optimizing revenues of a hybrid hydro and energy storage system is using mixed-integer linear programming (MILP). Although MILP models can provide accurate production cost estimations, they are typically very computationally expensive. To provide a fast yet accurate first-step information to hydropower plant owners or operators who consider integrating energy storage systems, we propose an innovative approach to predicting optimal revenues of an integrated energy generation and storage system. In this study, we examined the performance of two prediction techniques: Generalized Additive Models (GAMs) and machine learning (ML) models developed based on artificial neural networks (ANN). Predictive equations and models are generated based on optimized solutions from a market participation optimization model, the Conventional Hydropower Energy and Environmental Resource System (CHEERS) model. The two predicting techniques reduce the computational time to evaluate annual revenue for one set of battery configurations from 3 h to 1 to 4 min per run while also being implementable with significantly less data. The model validation prediction errors of developed GAMs and ML models are generally below 5%; for model testing predictions, the ML models consistently outperform the regression equations in terms of root mean square errors. This new approach allows plant owners, operators, or potential investors to quickly access multiple battery configurations under different energy generation and market scenarios. This new revenue prediction method will therefore help reduce the barriers, and thereby promoting the deployment of battery hybridization with existing renewable energy sources.

13 HYDRO ENERGY↗
DOE OSTI2023

Regional Medium-Term Hourly Electricity Demand Forecasting Based on LSTM

This paper aims to forecast high-resolution (hourly) aggregated load for a certain region in the medium term (a few days to over a year). One region is defined as some places with similar climate characteristics because the climate influences people's daily lifestyles and hence the electric usage. We decompose the electric usage records into two parts: base load and seasonal load. Considering both temperature and time factors, different deep-learning methods are adopted to characterize them. The first goal of our approach is to predict the peak load which is critical for power system planning. Furthermore, our proposed forecast method can provide the depiction of the hourly load profile to provide customized load curves for high-level real-time applications. The proposed method is tested on real-world historical data collected by CAISO, BPA, and PACW. The experimental results show that trained by three years of data, our method could reduce the prediction error for a one-year lead hourly load below $5\%$ MAPE, and predict the occurrence of the peak load for next year in CAISO with an error within three days. Furthermore, as a byproduct, an interesting observation on the impact of COVID-19 on human life was made and discussed based on these case studies.

deep learning↗