Search NASA⌕ Search

SEARCH · Search NASA

Results for “limits of machine learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,621 records · Page 90

Virtual Assistant for First Responders Using Natural Language Understanding and Optical Character Recognition

Commercial deep learning capabilities are available for many applications such as computer vision processing and intelligent chat bots. The Google Cloud Platform product Google Dialogflow provides lifelike conversational artificial intelligence (AI) using machine learning (ML) to generate natural conversations between computers and humans. This ML utilizes natural language understanding (NLU) to recognize a user’s intent and extracts key information into a form of entities. We have developed a user-friendly application through understanding the hazardous material database, first aid safety guidelines and observing the process of first responders who access this information in the field. We created the Trusted and Explainable Artificial Intelligence for Saving Lives (TruePAL) virtual assistant using Dialogflow1 and TensorFlow2 paired with EasyOCR.3 The chatbot supports first responders by providing voice interaction which helps limit additional steps such as browsing through multiple categories when searching for information. Using feedback from our field interviews, the voice interface has been developed to enable the first responder to focus on the immediate emergency. With less distractions, the first responder is able to engage the incident more effectively. The partial hands-free TruePAL chatbot assistant improves the accessibility to the correct guidance by an average of 1.9 seconds compared to the widely used application, NIH WISER, which requires full attention to operate. We combined this intelligent chatbot with a separate visual processing capability to produce hazardous signage analysis and generate the proper guidance for first responders. With the evolving functionality of AI tools, the use of virtual assistants in first responder technology will be an advancement, benefiting the safety of both first responders and civilians.

Chow, Edward↗

Two-Step Hyperparameter Optimization Method: Accelerating Hyperparameter Search by Using a Fraction of a Training Dataset

Abstract Hyperparameter optimization (HPO) is an important step in machine learning (ML) model development, but common practices are archaic—primarily relying on manual or grid searches. This is partly because adopting advanced HPO algorithms introduces added complexity to the workflow, leading to longer computation times. This poses a notable challenge to ML applications, as suboptimal hyperparameter selections curtail the potential of ML model performance, ultimately obstructing the full exploitation of ML techniques. In this article, we present a two-step HPO method as a strategic solution to curbing computational demands and wait times, gleaned from practical experiences in applied ML parameterization work. The initial phase involves a preliminary evaluation of hyperparameters on a small subset of the training dataset, followed by a reevaluation of the top-performing candidate models postretraining with the entire training dataset. This two-step HPO method is universally applicable across HPO search algorithms, and we argue it has attractive efficiency gains. As a case study, we present our recent application of the two-step HPO method to the development of neural network emulators for aerosol activation. Although our primary use case is a data-rich limit with many millions of samples, we also find that using up to 0.0025% of the data—a few thousand samples—in the initial step is sufficient to find optimal hyperparameter configurations from much more extensive sampling, achieving up to 135× speedup. The benefits of this method materialize through an assessment of hyperparameters and model performance, revealing the minimal model complexity required to achieve the best performance. The assortment of top-performing models harvested from the HPO process allows us to choose a high-performing model with a low inference cost for efficient use in global climate models (GCMs).

97 MATHEMATICS AND COMPUTING↗

Gaming the beamlines—employing reinforcement learning to maximize scientific outcomes at large-scale user facilities

Abstract Beamline experiments at central facilities are increasingly demanding of remote, high-throughput, and adaptive operation conditions. To accommodate such needs, new approaches must be developed that enable on-the-fly decision making for data intensive challenges. Reinforcement learning (RL) is a domain of AI that holds the potential to enable autonomous operations in a feedback loop between beamline experiments and trained agents. Here, we outline the advanced data acquisition and control software of the Bluesky suite, and demonstrate its functionality with a canonical RL problem: cartpole. We then extend these methods to efficient use of beamline resources by using RL to develop an optimal measurement strategy for samples with different scattering characteristics. The RL agents converge on the empirically optimal policy when under-constrained with time. When resource limited, the agents outperform a naive or sequential measurement strategy, often by a factor of 100%. We interface these methods directly with the data storage and provenance technologies at the National Synchrotron Light Source II, thus demonstrating the potential for RL to increase the scientific output of beamlines, and layout the framework for how to achieve this impact.

36 MATERIALS SCIENCE↗

Atomistic modeling of meso -timescale processes with $\mathrm{SEAKMC}$: A perspective and recent developments

On-the-fly kinetic Monte Carlo (kMC) methods have recently garnered significant attentions after successful applications to various atomic-scale problems using a timescale outside the reach of classical molecular dynamics. These methods play a critical role in modeling atomistic meso-timescale processes, and it is therefore essential to further improve their capabilities. In this report we review one of the on-the-fly kMC methods, Self-Evolving Atomistic kinetic Monte Carlo (SEAKMC) and propose two schemes that considerably enhance the efficiency of saddle point searches (SPSs) during the simulations. The performance of these schemes is tested using the diffusion of point defects in bcc Fe. In addition, we discuss approaches to significantly mitigate limitations of these schemes, which further improves their efficiencies. Importantly, these schemes improve the SPS efficiency not only for SEAKMC but also for other on-the-fly kMC methods, broadening the applications of on-the-fly kMC simulations to complex meso-timescale problems.

36 MATERIALS SCIENCE↗

Extreme Temperature Cryptography Based On Nitrogen-Incorporated Ultrananocrystalline Diamond

Physical entropy sources that remain stable under extreme temperatures are essential for cryptography in emerging technological frontiers in deep space exploration, geothermal energy harvesting, and nuclear energy. However, conventional semiconductor platforms fail to generate stable and reliable cryptographic keys above 200 degrees C due to performance degradation. Here, we report a diamond-based cryptographic primitive that exploits the defect-rich sp 2 -bonded grain boundary network in nitrogen-incorporated ultrananocrystalline diamond (n-UNCD) film as a robust entropy source to generate cryptographic keys that remain operationally stable even after enduring extreme temperatures of 700 degrees C for 54 h while also surviving thermal cycling between room temperature and 700 degrees C for 48 h. The strength of the generated keys is assessed through several cryptographic metrics such as bit uniformity, entropy, hamming distances, and correlation coefficients, all of which are found to be near their respective ideal values. Moreover, the generated keys pass the NIST SP 800 and SP 800-90B tests and are also resilient to supply bias variations and a regression-based machine learning attack model based on the Fourier series. The robustness of the keys is attributed to the better thermal stability and chemical inertness of the n-UNCD film. This is supported by high-resolution energy-dispersive X-ray spectroscopy (EDS), which shows no significant lateral diffusion of metal atoms into the n-UNCD layer, and by Raman spectroscopy, which reveals no significant changes in the bonding configuration of the n-UNCD structure. Our findings highlight the remarkable potential of n-UNCD film for extreme environment cryptography by expanding the operational limits of conventional hardware security platforms.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Parameter uncertainties for imperfect surrogate models in the low-noise regime

Abstract Bayesian regression determines model parameters by minimizing the expected loss, an upper bound to the true generalization error. However, this loss ignores model form error, or misspecification, meaning parameter uncertainties are significantly underestimated and vanish in the large data limit. As misspecification is the main source of uncertainty for surrogate models of low-noise calculations, such as those arising in atomistic simulation, predictive uncertainties are systematically underestimated. We analyze the true generalization error of misspecified, near-deterministic surrogate models, a regime of broad relevance in science and engineering. We show that posterior parameter distributions must cover every training point to avoid a divergence in the generalization error and design a compatible ansatz which incurs minimal overhead for linear models. The approach is demonstrated on model problems before application to thousand-dimensional datasets in atomistic machine learning. Our efficient misspecification-aware scheme gives accurate prediction and bounding of test errors in terms of parameter uncertainties, allowing this important source of uncertainty to be incorporated in multi-scale computational workflows.

Swinburne, Thomas D. (ORCID:0000000232554257)↗

Automated Labeling of Electron Microscopy Images Using Deep Learning

© 2018 IEEE. Searching for scientific data requires metadata providing a relevant context. Today, generating metadata is a time and labor intensive manual process that is often neglected, and important datasets are not accessible through search. We investigate the use of machine learning to generalize metadata from a subset of labeled data, thus increasing the availability of meaningful metadata for search. Specifically, we consider electron microscopy images collected at the National Center for Electron Microscopy (NCEM) at the Lawrence Berkeley National Laboratory (LBNL) and use deep learning to discern characteristics from a small subset of labeled images and transfer labels to the entire image corpus. Relatively small training set sizes and a minimum resolution of 512\times 512 pixels required by the application domain pose unique challenges. We overcome these challenges by using a simple yet powerful convolutional network architecture that limits the number of free parameters to lower the required amount of computational power and reduce the risk of overfitting. We achieve a classification accuracy of approximately 80% in discerning between images recorded in two operating modes of the electron microscope-transmission electron microscopy (TEM) and scanning transmission electron microscopy (STEM). We use transfer learning-i.e., re-using the pre-trained convolution layers from the TEM vs. STEM classification problem-to generalize labels and achieve an accuracy of approximately 70 % despite current experiments being limited to small training data sets. We present these predictions as suggestions to domain scientists through a web-based tool to accelerate the labeling process with the goal of further validating our approach and improving the accuracy of automatically created labels.

Weber, Gunther H↗

Study of MHD Stability, Active Mode Control, and Disruption Avoidance in NSTX-U

Learning how to achieve sustained operation at high plasma pressure is an important goal of the National Spherical Torus Upgrade (NSTX-U) program and is critically important to any future fusion power plant. This research addresses high priority stability and control issues outlined by the NSTX-U Program. Reduced models of the stabilization physics learned continue to be created and tested against actual plasma operation in NSTX, NSTX-U, and other tokamak devices. This research is forming physics-based models for disruption forecasting that aims to unify results across different fusion machines for more confident extrapolation of the techniques studied to future devices including ITER. The Disruption Event Characterization and Forecasting (DECAF) code was expanded to analyze data from the TCV tokamak, including real-time MHD analysis signals. The presence of a mode and likelihood of its m/n numbers can be tracked, as well as the mode frequency, although a limitation at low frequency does not yet allow tracking modes to the locking point in TCV. Initial analyses of NSTX-U operation were conducted using the DECAF code. The timing of sawtooth oscillations was investigated taking advantage of the more extensive L-mode operation of the device in its first run campaign. An initial investigation of thermal collapses in NSTX-U was also conducted with specific comparison to the Greenwald density limit. A local island power balance model to explain the density limit in tokamaks has been tested with NSTX data and is being evaluated as a potential disruption forecaster in the DECAF code, presently showing similarities to the global Greenwald density evolution and limit. A module was written for the DECAF code that generalizes magnetic spectrogram analysis of MHD modes. This new portable code, written in Python, discriminates the toroidal mode number of instabilities and can track each mode as a DECAF event, monitoring the evolution of the mode through bifurcation and locking points. This is a critical new capability to allow disruption event characterization for general tokamak databases.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

WE-A-BRE-01: Debate: To Measure or Not to Measure

Recent studies have highlighted some of the limitations of patient-specific pre-treatment IMRT QA measurements with respect to assessing plan deliverability. Pre-treatment QA measurements are frequently performed with detectors in phantoms that do not involve any patient heterogeneities or with an EPID without a phantom. Other techniques have been developed where measurement results are used to recalculate the patient-specific dose volume histograms. Measurements continue to play a fundamental role in understanding the initial and continued performance of treatment planning and delivery systems. Less attention has been focused on the role of computational techniques in a QA program such as calculation with independent dose calculation algorithms or recalculation of the delivery with machine log files or EPID measurements. This session will explore the role of pre-treatment measurements compared to other methods such as computational and transit dosimetry techniques. Efficiency and practicality of the two approaches will also be presented and debated. The speakers will present a history of IMRT quality assurance and debate each other regarding which types of techniques are needed today and for future quality assurance. Examples will be shared of situations where overall quality needed to be assessed with calculation techniques in addition to measurements. Elements where measurements continue to be crucial such as for a thorough end-to-end test involving measurement will be discussed. Operational details that can reduce the gamma tool effectiveness and accuracy for patient-specific pre-treatment IMRT/VMAT QA will be described. Finally, a vision for the future of IMRT and VMAT plan QA will be discussed from a safety perspective. Learning Objectives: Understand the advantages and limitations of measurement and calculation approaches for pre-treatment measurements for IMRT and VMAT planning Learn about the elements of a balanced quality assurance program involving modulated techniques Learn how to use tools and techniques such as an end-to-end test to enhance your IMRT and VMAT QA program.

60 APPLIED LIFE SCIENCES↗

A Convolutional Neural Network for Enhancement of Multi-Scale Localization in Granular Metallic Representative Unit Cells

A convolutional neural network was used to enhance the localization of strain and stress for a generalized method of cells model of a metallic microstructure. Enhanced shear strains, measured in terms of the linear regression coefficients as a function of ground truth strains, were improved from inaccurate and uncorrelated (slope=0.003, Rsq=0.000) to accurate and well correlated (slope=0.890, Rsq=0.882) relative to ground truth (slope=1.0, Rsq=1.0). In applying the convolutional neural network, a convolutional stride of 1.0 (padding=’same’) was only modestly effective while strides of 2 or 3 were more effective yet at higher cost. Additional convolutional layers were generally more expensive than additional dense layers, often with limited benefit. The accuracy of enhanced localized shear strains and stress is expected to yield benefits for damage progression models, especially in the context of hierarchical multi-scale methods where the generalized method of cells is applied at the intermediate scale.

Machine Learning↗

Algorithmic Detection of Elemental Biosignatures

Machine learning models that classify a sample as indicative or non-indicative of life could play an important role in life-detection missions. Their predictions result from agnostic algorithms and thereby add redundancy to judgements resulting from human expertise. Additionally, their important features can reveal the most informative measurements within the operational constraints of a life-detection mission. The Ladder of Life Detection (Neveu 2018) identifies the need for an understanding of how combinations of multiple biosignatures affect overall confidence. The present work provides a starting point to answer this need, and future work will expand the data types to obtain even more predictive combinations of features. Elemental abundance was chosen as a starting set of features due to its availability in diverse sample types, which are needed to train a generalizable model. A standardized dataset was collected, including 35 non-indicative, e.g., lunar rock, basalt; 19 indicative mixed, e.g., seawater, agricultural soil; 46 indicative non-alive, e.g., coal, chalk; and 10 indicative alive, e.g., biofilm, bacteria. This dataset could be valuable for complementary biosignature research. The samples were standardized to the same limit of detection of a simulated mission scenario. Four classification models were used: k-nearest neighbors (KNN), logistic regression (LR), linear support vector machines (SVM), and Gaussian naïve Bayes (GNB). To obtain feature importances, KNN was run on three principal components of the training data and LR and SVM were run with L1 and L2 regularization. The performances and feature importances of the six model variants on 40:60 train to validation ratios were assessed with Monte Carlo simulations. ROC AUC and mean accuracy scores ranged between 82% - 94%, with sensitivity greater than specificity. For indicative of life predictors, all models had C and Ca as strong and Cl as medium; a majority of models had N, K, and P as medium. For non-indicative of life predictors, all models had Si as strong, and a majority of models had Mg, Al, and Ti as medium. Varied elements were Fe (slightly non-indicative), H (slightly indicative), O (widely varied), Na, Mn, and S. These results serve as a proof of concept and suggest important elemental signals beyond merely the CHNOPS of Earth-based life.

Algorithmic↗

Uncertainty Quantification for Data-Driven Machine Learning Models in Nuclear Engineering Applications: Where We Are and What Do We Need?

Machine learning (ML) has been leveraged to tackle a diverse range of tasks in almost all branches of nuclear engineering. Many of the successes in ML applications can be attributed to the recent performance breakthroughs in deep learning, the growing availability of computational power, data, and easy-to-use ML libraries. However, these empirical successes have often outpaced our formal understanding of the ML algorithms. An important but under-rated area is uncertainty quantification (UQ) of ML. ML-based models are subject to approximation uncertainty when they are used to make predictions, due to sources including but not limited to, data noise, data coverage, extrapolation, imperfect model architecture and the stochastic training process. The goal of this paper is to clearly explain and illustrate the importance of UQ of ML. We will elucidate the differences in the basic concepts of UQ of physics-based models and data-driven ML models. Various sources of uncertainties in physical modeling and data-driven modeling will be discussed, demonstrated, and compared. We will also present and demonstrate a few techniques to quantify the ML prediction uncertainties, including Monte Carlo dropout, deep ensemble, Bayesian neural networks, Gaussian Processes and conformal prediction. Lastly, we will discuss the need for building a verification, validation and UQ framework to establish ML credibility.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Automated Design-for-Reliability of a Power Electronics Module

The design of a power electronics module is a multiphysics problem and involves electrical, thermal, and reliability targets and constraints. Although the electrical and thermal design consider multiple aspects such as materials, geometry, and energy losses, reliability design is often limited to the selection of the attachment materials and electrical interconnect types. In this paper, we incorporate reliability metrics in the design phase by investigating the impact of package geometry on the thermomechanical behavior of the die-attachment material. To this end, we conducted thermal and power cycling simulations of a commercial six-pack power module with silicon carbide devices to compute junction temperature and strain energy density per cycle, respectively. We performed multiple simulations with different geometric dimensions and established a correlation between input features and output variables using subspace-based dimension reduction. The machine learning-based dimension reduction method serves as a surrogate model, which can be employed to identify the optimal module design from a thermal and reliability standpoint.

design-for-reliability↗

New Era Towards Autonomous Additive Manufacturing: A Review of Recent Trends and Future Perspectives

Abstract The Additive Manufacturing (AM) landscape has significantly transformed in alignment with Industry 4.0 principles, primarily driven by the integration of Artificial Intelligence (AI) and Digital Twin (DT). However, current Intelligent Additive Manufacturing (IAM) systems face limitations such as fragmented AI tool usage and suboptimal human-machine interaction (HMI). This paper reviews existing IAM solutions, emphasizing control, monitoring, process autonomy, and end-to-end integration, and identifies key limitations, such as the absence of a high-level controller for global decision-making. To address these gaps, we propose a transition from IAM to Autonomous Additive Manufacturing (AAM), featuring a hierarchical framework with four integrated layers: knowledge, generative solution, operational, and cognitive. In the cognitive layer, AI agents notably enable machines to independently observe, analyze, plan, and execute operations that traditionally require human intervention. These capabilities streamline production processes and expand the possibilities for innovation, particularly in sectors like in-space manufacturing (ISM). Additionally, this paper discusses the role of AI in self-optimization and lifelong learning, positing that the future of AM will be characterized by a symbiotic relationship between human expertise and advanced autonomy, fostering a more adaptive, resilient manufacturing ecosystem.

Fan, Haolin↗

Correlating Time-Resolved Pressure Measurements With Rim Sealing Effectiveness for Real-Time Turbine Health Monitoring

Purge flow is bled from the upstream compressor and supplied to the under-platform region to prevent hot main gas path ingress that damages vulnerable under-platform hardware components. A majority of turbine rim seal research has sought to identify methods of improving sealing technologies and understanding the physical mechanisms that drive ingress. While these studies directly support the design and analysis of advanced rim seal geometries and purge flow systems, the studies are limited in their applicability to real-time monitoring required for condition-based operation and maintenance. As operational hours increase for in-service engines, this lack of rim seal performance feedback results in progressive degradation of sealing effectiveness, thereby leading to reduced hardware life. To address this need for rim seal performance monitoring, this study utilizes measurements from a one-stage turbine research facility operating with true-scale engine hardware at engine-relevant conditions. Time-resolved pressure measurements collected from the rim seal region are regressed with sealing effectiveness through the use of common machine learning techniques to provide real-time feedback of sealing effectiveness. Two modeling approaches are presented that use a single sensor to predict sealing effectiveness accurately over a range of two turbine operating conditions. Here, the results show that an initial purely data-driven model can be further improved using domain knowledge of relevant turbine operations, which yields sealing effectiveness predictions within 3% of measured values.

42 ENGINEERING↗

Neural Networks to Find the Optimal Forcing for Offsetting the Anthropogenic Climate Change Effects

Abstract Of great relevance to climate engineering is the systematic relationship between the radiative forcing to the climate system and the response of the system, a relationship often represented by the linear response function (LRF) of the system. However, estimating the LRF often becomes an ill-posed inverse problem due to high-dimensionality and nonunique relationships between the forcing and response. Recent advances in machine learning make it possible to address the ill-posed inverse problem through regularization and sparse system fitting. Here, we develop a convolutional neural network (CNN) for regularized inversion. The CNN is trained using the surface temperature responses from a set of Green’s function perturbation experiments as imagery input data together with data sample densification. The resulting CNN model can infer the forcing pattern responsible for the temperature response from out-of-sample forcing scenarios. This promising proof of concept suggests a possible strategy for estimating the optimal forcing to negate certain undesirable effects of climate change. The limited success of this effort underscores the challenges of solving an inverse problem for a climate system with inherent nonlinearity. Significance Statement Predicting the climate response for a given climate forcing is a direct problem, while inferring the forcing for a given desired climate response is often an inverse, ill-posed, problem, posing a new challenge to the climate community. This study makes the first attempt to infer the radiative forcing for a given target pattern of global surface temperature response using a deep learning approach. The resulting deeply trained convolutional neural network inversion model shows promise in capturing the forcing pattern corresponding to a given surface temperature response, with a significant implication on the design of an optimal solar radiation management strategy for curbing global warming. This study also highlights the technical challenges that future research should prioritize in seeking feasible solutions to the inverse climate problem.

Ren, Huiying↗

CrossCheck: Rapid, Reproducible, and Interpretable Model Evaluation

Evaluation beyond aggregate performance metrics, e.g. F1-score, is crucial to both establish an appropriate level of trust in machine learning models and identify future model improvements. In this paper we demonstrate CrossCheck, an interactive visualization tool for rapid crossmodel comparison and reproducible error analysis. We describe the tool and discuss design and implementation details. We then present three use cases (named entity recognition, reading comprehension, and clickbait detection) that show the benefits of using the tool for model evaluation. CrossCheck allows data scientists to make informed decisions to choose between multiple models, identify when the models are correct and for which examples, investigate whether the models are making the same mistakes as humans, evaluate models’ generalizability and highlight models’ limitations, strengths and weaknesses. Furthermore, CrossCheck is implemented as a Jupyter widget, which allows rapid and convenient integration into data scientists’ model development workflows.

Arendt, Dustin L.↗

Spatiotemporal Downscaling Model for Solar Irradiance Forecast Using Nearest-Neighbor Random Forest and Gaussian Process

Accurate solar photovoltaic (PV) capacity estimation requires high-resolution, site-specific solar irradiance data to account for localized variability. However, global datasets, such as the National Solar Radiation Database (NSRDB), provide regional averages that fail to capture the fine-scale fluctuations critical for large-scale grid integration. This limitation is particularly relevant in the context of increasing distributed energy resources (DERs) penetration, such as rooftop PV. Additionally, it is critical to the implementation of the U.S. Federal Energy Regulatory Commission (FERC) Order 2222, which facilitates DER participation in U.S. bulk power markets. To address this challenge, this study evaluates Nearest-Neighbor Random Forest (NNRF) and Nearest-Neighbor Gaussian Process (NNGP) models for spatiotemporal downscaling of global solar irradiance data. By leveraging historical irradiance and meteorological data, these models incorporate spatial, temporal, and feature-based correlations to enhance local irradiance predictions. The NNRF model, a machine-learning approach, prioritizes computational efficiency and predictive accuracy, while the NNGP model offers a level of interpretability and prediction uncertainty by numerically quantifying correlations and dependencies in the data. Model validation was conducted using day-ahead predictions. The results showed that the average Goodness of Fit (GoF) of the NNRF model of 90.61% across all eight sites outperformed the GoF of the NNGP of 85.88%. Additionally, the computational speed of NNRF was 2.5 times faster than the NNGP. Finally, the NNGP displayed polynomial scaling while the NNRF scaled linearly with increasing number of nearest neighbors. Additional validation of the model on five sites in Puerto Rico further confirmed the superiority of the NNRF model over the NNGP model. These findings highlight the robustness and computational efficiency of NNRF for large-scale solar irradiance downscaling, making it a strong candidate for improving PV capacity estimation and real-time electricity market integration for DERs.

Asiedu, Shadrack (ORCID:0009000646004826)↗