Search NASA⌕ Search

SEARCH · Search NASA

Results for “model accuracy”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
NASA NTRS1994

The Accuracy of Parameter Estimation in System Identification of Noisy Aircraft Load Measurement

This thesis focuses on the subject of the accuracy of parameter estimation and system identification techniques. Motivated by a complicated load measurement from NASA Dryden Flight Research Center, advanced system identification techniques are needed. The objective of this problem is to accurately predict the load experienced by the aircraft wing structure during flight determined from a set of calibrated load and gage response relationship. We can then model the problem as a black box input-output system identification from which the system parameter has to be estimated. Traditional LS (Least Square) techniques and the issues of noisy data and model accuracy are addressed. A statistical bound reflecting the change in residual is derived in order to understand the effects of the perturbations on the data. Due to the intrinsic nature of the LS problem, LS solution faces the dilemma of the trade off between model accuracy and noise sensitivity. A method of conflicting performance indices is presented, thus allowing us to improve the noise sensitivity while at the same time configuring the degredation of the model accuracy. SVD techniques for data reduction are studied and the equivalence of the Correspondence Analysis (CA) and Total Least Squares Criteria are proved. We also looked at nonlinear LS problems with NASA F-111 data set as an example. Conventional methods are neither easily applicable nor suitable for the specific load problem since the exact model of the system is unknown. Neural Network (NN) does not require prior information on the model of the system. This robustness motivated us to apply the NN techniques on our load problem. Simulation results for the NN methods used in both the single load and the 'warning signal' problems are both useful and encouraging. The performance of the NN (for single load estimate) is better than the LS approach, whereas no conventional approach was tried for the 'warning signals' problems. The NN design methodology is also presented. The use of SVD, CA and Collinearity Index methods are used to reduce the number of neurons in a layer.

Kong, Jeffrey↗
NASA NTRS2007

Thermal Model Correlation for Mars Reconnaissance Orbiter

The Mars Reconnaissance Orbiter (MRO) launched on August 12, 2005 and began aerobraking at Mars in March 2006. In order to save propellant, MRO used aerobraking to modify the initial orbit at Mars. The spacecraft passed through the atmosphere briefly on each orbit; during each pass the spacecraft was slowed by atmospheric drag, thus lowering the orbit apoapsis. The largest area on the spacecraft, most affected by aeroheating, was the solar arrays. A thermal analysis of the solar arrays was conducted at NASA Langley Research Center to simulate their performance throughout the entire roughly 6-month period of aerobraking. A companion paper describes the development of this thermal model. This model has been correlated against many sets of flight data. Several maneuvers were performed during the cruise to Mars, such as thruster calibrations, which involve large abrupt changes in the spacecraft orientation relative to the sun. The data obtained from these maneuvers allowed the model to be well-correlated with regard to thermal mass, conductive connections, and solar response well before arrival at the planet. Correlation against flight data for both in-cruise maneuvers and drag passes was performed. Adjustments made to the model included orientation during the drag pass, solar flux, Martian surface temperature, through-array resistance, aeroheating gradient due to angle of attack, and aeroheating accommodation coefficient. Methods of correlation included comparing the model to flight temperatures, slopes, temperature deltas between sensors, and solar and planet direction vectors. Correlation and model accuracy over 400 aeroheating drag passes were determined, with overall model accuracy better than 5 C.

Amundsen, Ruth M.↗
DOE OSTI2021

Evaluate the impact of sensor accuracy on model performance in data-driven building fault detection and diagnostics using Monte Carlo simulation

The performance of data-driven fault detection and diagnostics (FDD) is heavily dependent on sensors. However, sensor inaccuracy and sensor faults are pervasive in building operation: inaccurate and missing sensor readings deteriorate FDD performance; sensor inaccuracy will also affect the selection of sensor for data-driven FDD in the model training process, which is another key factor of data-driven FDD performance. Sensor accuracy and sensor selection individually are well-studied research topics in this field, but the impact of sensor accuracy on sensor selection and its further impact on FDD performance has not been evaluated and quantified. In this paper, we developed a novel analysis methodology that comprehensively evaluates sensor fault on sensor selection and FDD accuracy. Monte Carlo simulation is applied to deal with multiple stochastic sensor inaccuracy and provide probabilistic analysis results of the impact of sensor inaccuracy on sensor selection and FDD accuracy. This methodology focuses on the net impact of fault states across a full sensor set. The developed methodology can be used for the early-stage sensor design and operation-stage sensor maintenance. Furthermore, a case study is conducted to demonstrate the analysis methodology using a commercial building model crated to Flexible Research Platform located at Oak Ridge National Laboratory, USA.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2023

AI-accelerated protein-ligand docking for SARS-CoV-2 is 100-fold faster with no significant change in detection

Protein-ligand docking is a computational method for identifying drug leads. The method is capable of narrowing a vast library of compounds down to a tractable size for downstream simulation or experimental testing and is widely used in drug discovery. While there has been progress in accelerating scoring of compounds with artificial intelligence, few works have bridged these successes back to the virtual screening community in terms of utility and forward-looking development. We demonstrate the power of high-speed ML models by scoring 1 billion molecules in under a day (50 k predictions per GPU seconds). We showcase a workflow for docking utilizing surrogate AI-based models as a pre-filter to a standard docking workflow. Our workflow is ten times faster at screening a library of compounds than the standard technique, with an error rate less than 0.01% of detecting the underlying best scoring 0.1% of compounds. Our analysis of the speedup explains that another order of magnitude speedup must come from model accuracy rather than computing speed. In order to drive another order of magnitude of acceleration, we share a benchmark dataset consisting of 200 million 3D complex structures and 2D structure scores across a consistent set of 13 million “in-stock” molecules over 15 receptors, or binding sites, across the SARS-CoV-2 proteome. We believe this is strong evidence for the community to begin focusing on improving the accuracy of surrogate models to improve the ability to screen massive compound libraries 100 × or even 1000 × faster than current techniques and reduce missing top hits. The technique outlined aims to be a fast drop-in replacement for docking for screening billion-scale molecular libraries.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Spike-and-Slab Shrinkage Priors for Structurally Sparse Bayesian Neural Networks

Network complexity and computational efficiency have become increasingly significant aspects of deep learning. Sparse deep learning addresses these challenges by recovering a sparse representation of the underlying target function by reducing heavily overparameterized deep neural networks. Specifically, deep neural architectures compressed via structured sparsity (e.g., node sparsity) provide low-latency inference, higher data throughput, and reduced energy consumption. In this article, we explore two well-established shrinkage techniques, Lasso and Horseshoe, for model compression in Bayesian neural networks (BNNs). To this end, we propose structurally sparse BNNs, which systematically prune excessive nodes with the following: 1) spike-and-slab group Lasso (SS-GL) and 2) SS group Horseshoe (SS-GHS) priors, and develop computationally tractable variational inference, including continuous relaxation of Bernoulli variables. We establish the contraction rates of the variational posterior of our proposed models as a function of the network topology, layerwise node cardinalities, and bounds on the network weights. Furthermore, we empirically demonstrate the competitive performance of our models compared with the baseline models in prediction accuracy, model compression, and inference latency.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2018

Numerical and Experimental Study of Local Resin Pressure for the Manufacturing of Composite Structures and Their Effect on Porosity

Porosity can have significant impact on the mechanical performance of composite structures. The primary sources of voids during the cure of composites include air entrapped during lay-up, bag or tool leaks, and the off gassing of volatiles. Capturing the physics of void evolution during composite processing is a challenge due to the number of sources and changing phenomena that give rise to voids as the resin cures. Local changes in resin pressure due to geometric features or cure shrinkage have been experimentally linked to a higher likelihood of void formation. In this work, a preliminary model was developed to predict local resin pressure which can be used to identify regions more susceptible to porosity. Experiments were conducted to validate the model accuracy. Ultimately, this model can be used as a tool to minimize the likelihood of porosity by optimizing the material selection, part design, layup, and cure process.

Bedayat, Houman↗
NASA NTRS1983

Estimation of parameters involved in high angle-of-attack aerodynamic theory using spin flight test data

The difficulty in applying parameter estimation techniques to spinning airplanes is due in part to the unwieldy number of possible combinations of terms in the equations of motion, when the model structure is unknown. The combination of high angle of attack and high rotation rate results in aerodynamic functions which are quite complex. For wing dominated configurations it is advantageous to use aerodynamic theory to generate the model structure. In this way, the number of unknown parameters is reduced and the model accuracy may be increased. Under conditions for which the theory is inadequate, however, model accuracy may be reduced. Strip theory, for example, is incapable of predicting autorotative rolling moments indicated by wind tunnel tests at angles of attack exceeding 40 degrees. An improved aerodynamic theory would be necessary to successfully apply the technique advanced for such regions.

Taylor, L. W., Jr.↗
DOE OSTI2024

Progressive transfer learning for advancing machine learning-based reduced-order modeling

Abstract To maximize knowledge transfer and improve the data requirement for data-driven machine learning (ML) modeling, a progressive transfer learning for reduced-order modeling (p-ROM) framework is proposed. A key concept of p-ROM is to selectively transfer knowledge from previously trained ML models and effectively develop a new ML model(s) for unseen tasks by optimizing information gates in hidden layers. The p-ROM framework is designed to work with any type of data-driven ROMs. For demonstration purposes, we evaluate the p-ROM with specific Barlow Twins ROMs (p-BT-ROMs) to highlight how progress learning can apply to multiple topological and physical problems with an emphasis on a small training set regime. The proposed p-BT-ROM framework has been tested using multiple examples, including transport, flow, and solid mechanics, to illustrate the importance of progressive knowledge transfer and its impact on model accuracy with reduced training samples. In both similar and different topologies, p-BT-ROM achieves improved model accuracy with much less training data. For instance, p-BT-ROM with four-parent (i.e., pre-trained models) outperforms the no-parent counterpart trained on data nine times larger. The p-ROM framework is poised to significantly enhance the capabilities of ML-based ROM approaches for scientific and engineering applications by mitigating data scarcity through progressively transferring knowledge.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

DeepMerge – II. Building robust deep learning algorithms for merging galaxy identification across domains

In astronomy, neural networks are often trained on simulation data with the prospect of being used on telescope observations. Unfortunately, training a model on simulation data and then applying it to instrument data leads to a substantial and potentially even detrimental decrease in model accuracy on the new target dataset. Simulated and instrument data represent different data domains, and for an algorithm to work in both, domain-invariant learning is necessary. Here we employ domain adaptation techniques— Maximum Mean Discrepancy (MMD) as an additional transfer loss and Domain Adversarial Neural Networks (DANNs)— and demonstrate their viability to extract domain-invariant features within the astronomical context of classifying merging and non-merging galaxies. Additionally, we explore the use of Fisher loss and entropy minimization to enforce better in-domain class discriminability. We show that the addition of each domain adaptation technique improves the performance of a classifier when compared to conventional deep learning algorithms. We demonstrate this on two examples: between two Illustris-1 simulated datasets of distant merging galaxies, and between Illustris-1 simulated data of nearby merging galaxies and observed data from the Sloan Digital Sky Survey. The use of domain adaptation techniques in our experiments leads to an increase of target domain classification accuracy of up to ~20%. With further development, these techniques will allow astronomers to successfully implement neural network models trained on simulation data to efficiently detect and study astrophysical objects in current and future large-scale astronomical surveys.

galaxies: interactions↗
DOE OSTI2021

Fast convolutional neural networks on FPGAs with hls4ml

We introduce an automated tool for deploying ultra low-latency, low-power deep neural networks with convolutional layers on field-programmable gate arrays (FPGAs). By extending the hls4ml library, we demonstrate an inference latency of 5 µs using convolutional architectures, targeting microsecond latency applications like those at the CERN Large Hadron Collider. Considering benchmark models trained on the Street View House Numbers Dataset, we demonstrate various methods for model compression in order to fit the computational constraints of a typical FPGA device used in trigger and data acquisition systems of particle detectors. In particular, we discuss pruning and quantization-aware training, and demonstrate how resource utilization can be significantly reduced with little to no loss in model accuracy. We show that the FPGA critical resource consumption can be reduced by 97% with zero loss in model accuracy, and by 99% when tolerating a 6% accuracy degradation.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2025

Dynamical Sketching for Enhanced Communication Efficiency in Federated Learning

Federated learning (FL) has revolutionized distributed machine learning by enabling collaborative model training without sharing local data. However, communication efficiency and privacy guarantees remain significant challenges. This paper introduces a dynamic sketching mechanism in FL, optimizing the trade-off between communication efficiency and model accuracy. By dynamically selecting the sketch matrix size, our approach adapts to the evolving characteristics of the data and the model, ensuring optimal performance across diverse scenarios. We leverage Bayesian optimization to systematically tune the sketch parameters, achieving an effective balance between resource efficiency and model performance. Experimental results on the MNIST dataset using a convolutional neural network (CNN) architecture validate the proposed method's efficiency and scalability. Our dynamic sketching approach significantly outperforms fixed-size sketching techniques, achieving higher compression ratios (up to 62x) and providing better privacy guarantees while maintaining high model accuracy. These findings highlight the robustness and versatility of our approach and make it a valuable solution for privacy-preserving, communication-efficient federated learning.

Afrose, Sharmin [ORNL]↗
DOE OSTI2023

Data-driven modeling to enhance municipal water demand estimates in response to dynamic climate conditions

Altered precipitation and temperature patterns from a changing climate will affect supply, demand, and overall municipal water system operations throughout the arid western U.S. While supply forecasts leverage hydrological models to connect climate influences with surface water availability, demand forecasts typically estimate water use independent of climate and other externalities. Stemming from an increased focus on seasonal water demand management, we use the Salt Lake City, Utah municipal water system as a test bed to assess model accuracy versus complexity trade-offs between simple climate-independent econometric-based models and complex climate-sensitive data-driven models to average to extreme wet and dry climate conditions—representative of a new climate normal. Here, the climate-independent model displayed low performance during extreme dry conditions with predictions exceeding 90% and 40% of the observed monthly and seasonal volumetric demands, respectively, which we attribute to insufficient model complexity. The climate-sensitive models displayed greater accuracy in all conditions, with an ordinary least squares model demonstrating a measurable reduction in prediction bias (3.4% vs. -27.3%) and RMSE (74.0 lpcd vs. 294 lpcd) compared to the climate-independent model. The climate-sensitive workflow increased model accuracy and characterized climate-demand interactions, demonstrating a novel tool to enhance water system management.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Development of a Machine-Learned Cruise Guide Indicator for Rotorcraft

This paper presents a machine-learned virtual cruise guide indicator (vCGI) for Chinook helicopters. Two temporal neural networks were trained and evaluated on measured data from 55 flight tests, one for the fore rotor and another for the aft rotor, to predict a vCGI value, which protects 23 components from fatigue damage during steady-state conditions. Three different classes of machine learning architectures were evaluated for prediction of the vCGI from time sequences: a temporal convolutional neural network with 1D dilated causal convolutions, a long short-term memory recurrent neural network, and an attention-based transformer architecture. The final average model accuracy on unseen flight data is currently greater than 93% for CGI values which could result in fatigue damage and 90% for normal operation CGI values. Model accuracy was improved through a series of advancements in:(1) selection of optimal training data using temporal collective variables and unsupervised learning, (2) dataset augmentation with maximum-entropy temporal collective variables, and (3) implementation of a mixture-of-experts classification- regression approach using an adversarial classification approach to assign maneuver labels. The results are presented for each advancement in model development along with lessons learned in training machine learning models on real- world, time-dependent rotorcraft data.

Boyer, Mathew↗
DOE OSTI2025

FedCSpc: A Cross-Silo Federated Learning System With Error-Bounded Lossy Parameter Compression

Cross-Silo federated learning is widely used for scaling deep neural network (DNN) training over data silos from different locations worldwide while guaranteeing data privacy. Communication has been identified as the main bottleneck when training large-scale models due to large-volume model parameters and gradient transmission across public networks with limited bandwidth. Most previous works focus on gradient compression, while limited work tries to compress parameters that can not be ignored and extremely affect communication performance during the training. Here, to bridge this gap, we propose FedCSpc: an efficient cross-silo federated learning system with an XAI-driven adaptive parameter compression strategy for large-scale model training. Our work substantially differs from existing gradient compression techniques due to the distinct data features of gradient and parameter. The key contributions of this paper are fourfold. (1) Our designed FedCSpc proposes to compress the parameter during the training using the state-of-the-art error-bounded lossy compressor – SZ3. (2) We develop an adaptive compression error bound adjustment algorithm to guarantee the model accuracy effectively. (3) We exploit an efficient approach to utilize the idle CPU resources of clients to compress the parameters. (4) We perform a comprehensive evaluation with a wide range of models and benchmarks on a GPU cluster with 65 GPUs. Results show that FedCSpc can achieve the same model accuracy as FedAvg while reducing the data volume of parameters and gradients in communication by up to 7.39× and 288×, respectively. With 32 clients on a 4 Gb size model, FedCSpc significantly outperforms FedAvg in wall-clock time in the emulated WAN environment (at the bandwidth of 1 Gbps or lower without loss of generality).

SZ3↗
DOE OSTI2021

Predicting nepheline precipitation in waste glasses using ternary submixture model and machine learning

Nepheline precipitation in nuclear waste glasses during vitrification can be detrimental due to its negative effect on chemical durability. Developing models to accurately predict nepheline precipitation from compositions is important to increase waste loading since existing models can be overly conservative. In this study, an expanded dataset containing 955 glasses was compiled from literature data, where 355 glasses are for high-level waste (HLW). Previously developed submixture models were refitted using the new dataset, where a misclassification rate of 7.8% was achieved. Nine machine learning (ML) algorithms (e.g., k-nearest neighbor, Gaussian process regression, artificial neural network, support vector machine, decision tree, etc.) were applied to evaluate their ability of predicting nepheline precipitation from compositions. Model accuracy, precision, recall/sensitivity, and F1 score were systemically compared between different ML algorithms and modeling protocols. Good model prediction with an accuracy ~0.9 (misclassification rate of ~10%) was observed with different algorithms under certain protocol. This study evaluated various ML models to predict nepheline precipitations in waste glasses, highlighting the importance of data preparation, modeling protocol, and their effect on model stability and reproducibility. The results provide insights into applying ML to predict glass properties and suggest areas for future research on modeling nepheline precipitations.

Lu, Xiaonan↗
NASA NTRS2000

Accuracy of Modelled Stratospheric Temperatures in the Winter Arctic Vortex from Infra Red Montgolfier Long Duration Balloon Measurements

The temperature of the stratosphere has been measured in the Arctic vortex every 9-10 minutes along the trajectory of four Infra Red Montgolfier long duration balloons flown for 7 to 22 days during the winters of 1997 and 1999. From a number of comparisons to independent sensors, the accuracy of the measurements is demonstrated to be plus or minus 0.5 K during nighttime and at altitude below 28 km (10 hPa). The performances of the analyses of global meteorological models, European Center for Medium Range Weather Forecasts (ECMWF) 31 and 50 levels, United Kingdom Meteorological Office (UKMO), Data Assimilation Office (DAO), National Climatic Prediction Center (NCEP) and NCEP/NCAR reanalysis, used in photochemical simulations of ozone destruction and interpretation of satellite data, are evaluated by comparison to this large (3500 data points) and homogeneous experimental data set. Most of models, except ECMWF31 in 1999, do show a smal1 average warm bias of between 0 and 1.6 K, with deviations particularly large, up to 20 K at high altitude (5hPa) in stratospheric warming conditions in 1999. Particularly wrong was ECMWF 31 levels near its top level at 10 hPa in 1999 where temperature 25 K colder than the real atmosphere were reported. The average dispersion between models and measurements varies from plus or minus 1.0 to plus or minus 3.0 K depending on the model and the year. It is shown to be the result of three contributions. The largest is a long wave modulation likely caused by the displacement of the temperature field in the analyses compared to real atmosphere. The second is the overestimation of the vertical gradient of temperature particularly in warming conditions, which explains the increase of dispersion from 1997 to 1999. Unexpectedly, the third and smallest (plus or minus 0.6-0.7 K) is the contribution of meso and subgrid scale vertical and horizontal features associated to the vertical propagation of orographic or gravity waves. Compared to other models, the newly available ECMWF 50 levels version assimilating the high vertical resolution radiances of the space borne Advanced Microwave Sounding Unit, performs significantly better (0.03 plus or minus 1.12 K on average between 10 and 140 hPa in 1999) than other models.

Pommereau, J.-P.↗
NASA NTRSDate not supplied

Improving Computational Efficiency of Prognostics Algorithms in Resource-Constrained Settings

In engineering and aerospace applications, it is vital to operational success to have insight into the expected performance and health of physical systems. The field of prognostics and health management provides quantitative methods for monitoring, predicting, and managing system health. Prognostics algorithms can be employed to assess the current state of a system, propagate the system throughout time, and predict potential anomalies or failures that may occur. While they can provide accurate prediction results, effective prognostics algorithms can be challenging to use in resource-constrained settings due to computational limitations and high computational latency, leading to obsolete predictions. Thus, computationally efficient and accurate algorithms are necessary for future remaining useful life predictions. In this work, we implement new algorithmic approaches for prediction, quantitatively compare them via a battery degradation use-case, and provide recommendations of potential improvements to a prognostics framework. One approach to prediction is through sampling, whereby the current state of a physical system is sampled many times and each sample is propagated forward until failure is reached, resulting in a distribution of failure values. To improve the efficiency of this process, we implemented five new algorithmic approaches to prediction, including three distinct sampling methods (standard Monte Carlo, Quasi-Monte Carlo, and Latin Hypercube Sampling), a variable time step algorithm, and a variable sample size algorithm. To compare the algorithms, we employ a variety of metrics designed specifically to analyze both computational efficiency and model accuracy. Our metrics include accuracy to compare the average predicted value to ground truth, mean absolute deviation to illustrate dispersion, specific percentile error to describe accuracy within a user-defined risk tolerance, and code run-time. To quantitatively analyze our results, we employ a use-case of degradation of a Lithium-ion battery. We use an electrochemistry-based model to describe the current health state of the battery, and implement our prediction algorithms to propagate forward in time until end-of-discharge (EOD) is reached. Notably, through this work it was found that none of our sampling approaches had a significant impact on computational efficiency or model accuracy in predicting EOD of the battery. We find that while the sampling methods are unique, the distributions they generate are similar, ultimately producing final predictions that are nearly identical. In exploring the effect of the time step within the prediction algorithm, we found that prediction accuracy was highly dependent on the time step used, and that implementing a variable time step within a particular prediction may provide an increase in computational efficiency while also maintaining prediction accuracy. Finally, implementing a variable sample size also affected prediction, and our results show that tuning both the magnitude and timing of the sample size adjustment can result in improved computation speed and maintained prediction accuracy. Taken together, our findings highlight the challenge of performing prognostics in resource-constrained settings, and illustrate the potential of developing new prediction algorithms to improve computational efficiency.

prognostics↗