Search NASA⌕ Search

SEARCH · Search NASA

Results for “machine learning models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 793 records · Page 44
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Structured information extraction from scientific text with large language models

Extracting structured knowledge from scientific text remains a challenging task for machine learning models. Here, we present a simple approach to joint named entity recognition and relation extraction and demonstrate how pretrained large language models (GPT-3, Llama-2) can be fine-tuned to extract useful records of complex scientific knowledge. We test three representative tasks in materials chemistry: linking dopants and host materials, cataloging metal-organic frameworks, and general composition/phase/morphology/application information extraction. Records are extracted from single sentences or entire paragraphs, and the output can be returned as simple English sentences or a more structured format such as a list of JSON objects. This approach represents a simple, accessible, and highly flexible route to obtaining large databases of structured specialized scientific knowledge extracted from research papers.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2021

Predictive Models of Genetic Redundancy in Arabidopsis thaliana

Abstract Genetic redundancy refers to a situation where an individual with a loss-of-function mutation in one gene (single mutant) does not show an apparent phenotype until one or more paralogs are also knocked out (double/higher-order mutant). Previous studies have identified some characteristics common among redundant gene pairs, but a predictive model of genetic redundancy incorporating a wide variety of features derived from accumulating omics and mutant phenotype data is yet to be established. In addition, the relative importance of these features for genetic redundancy remains largely unclear. Here, we establish machine learning models for predicting whether a gene pair is likely redundant or not in the model plant Arabidopsis thaliana based on six feature categories: functional annotations, evolutionary conservation including duplication patterns and mechanisms, epigenetic marks, protein properties including posttranslational modifications, gene expression, and gene network properties. The definition of redundancy, data transformations, feature subsets, and machine learning algorithms used significantly affected model performance based on holdout, testing phenotype data. Among the most important features in predicting gene pairs as redundant were having a paralog(s) from recent duplication events, annotation as a transcription factor, downregulation during stress conditions, and having similar expression patterns under stress conditions. We also explored the potential reasons underlying mispredictions and limitations of our studies. This genetic redundancy model sheds light on characteristics that may contribute to long-term maintenance of paralogs, and will ultimately allow for more targeted generation of functionally informative double mutants, advancing functional genomic studies.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Estimating Submicron Aerosol Mixing State at the Global Scale With Machine Learning and Earth System Modeling

Abstract This study integrates machine learning and particle‐resolved aerosol simulations to develop emulators that predict submicron aerosol mixing state indices from the Earth system model (ESM) simulations. The emulators predict aerosol mixing state using only quantities that are predicted by the ESM, including bulk aerosol species concentrations, which do not by themselves carry mixing state information. We used PartMC‐MOSAIC as the particle‐resolved model and NCAR's CESM as the ESM. We trained emulators for three different mixing state indices for submicron aerosol in terms of chemical species abundance ( χ a ), the mixing of optically absorbing and nonabsorbing species ( χ o ), and the mixing of hygroscopic and nonhygroscopic species ( χ h ). Our global mixing state maps show considerable spatial and seasonal variability unique to each mixing state index. Seasonal averages varied spatially between 13% and 94% for χ a , between 38% and 94% for χ o , and between 20% and 87% for χ h with global annual averages of 67%, 68%, and 56%, respectively. High values in one index can be consistent with low values in another index depending on the grouping of species and their relative abundance, meaning that each mixing state index captures different aspects of the population mixing state. Although a direct validation with observational data has not been possible yet, our results are consistent with mixing state index values derived from ambient observations. This work is a prototypical example of using machine learning emulators to add information to ESM simulations.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Accelerating phase-field-based microstructure evolution predictions via surrogate models trained by machine learning methods

Abstract The phase-field method is a powerful and versatile computational approach for modeling the evolution of microstructures and associated properties for a wide variety of physical, chemical, and biological systems. However, existing high-fidelity phase-field models are inherently computationally expensive, requiring high-performance computing resources and sophisticated numerical integration schemes to achieve a useful degree of accuracy. In this paper, we present a computationally inexpensive, accurate, data-driven surrogate model that directly learns the microstructural evolution of targeted systems by combining phase-field and history-dependent machine-learning techniques. We integrate a statistically representative, low-dimensional description of the microstructure, obtained directly from phase-field simulations, with either a time-series multivariate adaptive regression splines autoregressive algorithm or a long short-term memory neural network. The neural-network-trained surrogate model shows the best performance and accurately predicts the nonlinear microstructure evolution of a two-phase mixture during spinodal decomposition in seconds, without the need for “on-the-fly” solutions of the phase-field equations of motion. We also show that the predictions from our machine-learned surrogate model can be fed directly as an input into a classical high-fidelity phase-field model in order to accelerate the high-fidelity phase-field simulations by leaping in time. Such machine-learned phase-field framework opens a promising path forward to use accelerated phase-field simulations for discovering, understanding, and predicting processing–microstructure–performance relationships.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Predicting Failure Using Deep Learning SAND Report

Accurate prediction of ductile failure is critical to Sandia’s NW mission, but the models are computationally heavy. The costs of including high-fidelity physics and mechanics that are germane to the failure mechanisms are often too burdensome for analysts either because of the person-hours it requires to input them or because of the additional computational time, or both. In an effort to deliver analysts a tool for representing these phenomena with minimal impact to their existing workflow, our project sought to develop modern data-driven methods that would add microstructural information to business-as-usual calculations and expedite failure predictions. The goal is a tool that receives as input a structural model with stress and strain fields, as well as a machine-learned model, and output predictions of structural response in time, including failure. As such, our project spent substantial time performing high-fidelity, three-dimensional experiments to elucidate materials mechanisms of void nucleation and evolution. We developed crystal-plasticity finite-element models from the experimental observations to enrich the findings with fields not readily measured. We developed engineering length-scale simulations of replicated test specimens to understand how the engineering fields evolve in the presence of fine-scale defects. Finally, we developed deep learning convolutional neural networks, and graph-based neural networks to encode the findings of the experiments and simulations and make forward predictions in time for structural performance. This project demonstrated the power of data-driven methods for model development, which have the potential to vastly increase both the accuracy and speed of failure predictions. These benefits and the methods necessary to develop them are highlighted in this report. However, many challenges remain to implementing these in real applications, and these are discussed along with potential methods for overcoming them.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Scalable statistical inference of photometric redshift via data subsampling

Handling big data has largely been a major bottleneck in traditional statistical models. Consequently, when accurate point prediction is the primary target, machine learning models are often preferred over their statistical counterparts for bigger problems. But full probabilistic statistical models often outperform other models in quantifying uncertainties associated with model predictions. We develop a data-driven statistical modeling framework that combines the uncertainties from an ensemble of statistical models learned on smaller subsets of data carefully chosen to account for imbalances in the input space. We demonstrate this method on a photometric redshift estimation problem in cosmology, which seeks to infer a distribution of the redshift—the stretching effect in observing the light of far-away galaxies—given multivariate color information observed for an object in the sky. Our proposed method performs balanced partitioning, graph-based data subsampling across the partitions, and training of an ensemble of Gaussian process models.

data subsampling↗
DOE OSTI2023

A Physics-Based Data-Driven Approach for Modeling of Environmental Degradation in Elastomers

Abstract Elastomers are now commonly used in a number of industries, including aerospace, structure, transportation, shipbuilding, and automotive, due to their excellent workability, formability, and flexibility. During their activity, elastomers are subjected to harsh environmental conditions, which decreases their resilience. False predictions made early in their lives can have major financial and environmental implications. Elastomers’ performance and properties, such as strength, durability, and density, are influenced by chemical changes in these materials, known as degradation, which occurs over time. This process can alter the morphology of a polymer matrix as well as cause chain scission and cross-linking, resulting in different behaviors than that of the unaged material. To demonstrate the effect of thermaloxidative aging on the mechanical behavior of elastomers, several experimental and theoretical models have been proposed. In view of the large volume of experimental data available on micro-structural evolution in the course of aging, we propose a physics-based data-driven approach to overcome the shortcomings of both phenomenological and micro-mechanical models. This work presents a novel thermodynamically consistent, multiagent machine-learned model for predicting the constitutive behavior of cross-linked elastomers during environmental aging, such as thermo-oxidative and hydrolytic aging for various states of deformation. Single mechanism degradation changes the polymer matrix over time where it is causing chain scission, reduction of cross-links, and morphology change. To capture the idealized Mullins effect and permanent set due to the effect of single aging mechanisms on nonlinear mechanical responses of elastomers, we propose a data-driven model for simulating inelastic elements in a polymer matrix. By using a sequential order reduction, we were able to reduce the 3D stress-strain tensor mapping problem to a small number of super-constrained 1D mapping problems. To systematically classify such mapping problems into a few categories, an assembly of multiple replicated conditional neural network learning agents (L-agents) is used based on our recent work. Each category is represented by a different type of agent. The effect of deformation history, aging time, and aging temperature is captured by this model. The model is validated using a broad collection of data, ranging from our experimental results to data from the literature. In addition, thermodynamic consistency and frame independence are investigated. The most significant achievements of this model are its precision, simplicity, and prediction of inelasticity under various states of deformation. The model’s accuracy and simplicity make it a good option for commercial and industrial applications. Conveniently, due to the model modular nature, it can be expanded in the future to include viscoelasticity and non-isotropic formation for better precision.

Ghaderi, Aref↗
DOE OSTI2020

A Causal Inference Model Based on Random Forests to Identify the Effect of Soil Moisture on Precipitation

Soil moisture influences precipitation mainly through its impact on land–atmosphere interactions. Understanding and correctly modeling soil moisture–precipitation (SM–P) coupling is crucial for improving weather forecasting and subseasonal to seasonal climate predictions, especially when predicting the persistence and magnitude of drought. However, the sign and spatial structure of SM–P feedback are still being debated in the climate research community, mainly due to the difficulty in establishing causal relationships and the high degree of nonlinearity in land–atmosphere processes. To this end, we developed a causal inference model based on the Granger causality analysis and a nonlinear machine learning model. This model includes three steps: nonlinear anomaly decomposition, nonlinear Granger causality analysis, and evaluation of the quality of SM–P feedback, which eliminates the nonlinear response of interannual and seasonal variability and the memory effects of climatic factors and isolates the causal relationship of local SM–P feedback. We applied this model by using National Climate Assessment–Land Data Assimilation System (NCA-LDAS) datasets over the United States. Here, the results highlight the importance of nonlinear atmosphere responses in land–atmosphere interactions. In addition, the strong feedback over the southwestern United States and the Great Plains both highlight the impacts of topographic factors rather than only the sensitivity of evapotranspiration to soil moisture. Furthermore, the SM–P index defined by our framework is used to benchmark Earth system models (ESMs), which provides a new metric for efficiently identifying potential model biases in modeling local land–atmosphere interactions and may help the development of ESMs in improving simulations of water cycle variability and extremes.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRSDate not supplied

Uncertainty in Servicing and Assembly Tasks for Space Robotic Manipulators

This presentation will discuss a subset of the sources of uncertainty that impact autonomous in-space servicing, assembly, and manufacturing missions. These include robotic manipulator modeling uncertainties in both kinematics and dynamics, perception error associated machine learning models for pose estimation, and sensor noise. Mitigation strategies will be discussed including the incorporation of capture envelopes in the design of robotic tools and selection of robot goal poses to minimize end-effector sensitivity in manipulators with redundant degrees of freedom.

robotics↗
DOE OSTI2019

Learning nonlinear level sets for dimensionality reduction in function approximation

We developed a Nonlinear Level-set Learning (NLL) method for dimensionality reduction in high-dimensional function approximation with small data. This work is motivated by a variety of design tasks in real-world engineering applications, where practitioners would replace their computationally intensive physical models (e.g., high-resolution fluid simulators) with fast-to-evaluate predictive machine learning models, so as to accelerate the engineering design processes. There are two major challenges in constructing such predictive models: (a) high-dimensional inputs (e.g., many independent design parameters) and (b) small training data, generated by running extremely time-consuming simulations. Thus, reducing the input dimension is critical to alleviate the over-fitting issue caused by data insufficiency. Existing methods, including sliced inverse regression and active subspace approaches, reduce the input dimension by learning a linear coordinate transformation; our main contribution is to extend the transformation approach to a nonlinear regime. Specifically, we exploit reversible networks (RevNets) to learn nonlinear level sets of a high-dimensional function and parameterize its level sets in low-dimensional spaces. A new loss function was designed to utilize samples of the target functions' gradient to encourage the transformed function to be sensitive to only a few transformed coordinates. The NLL approach is demonstrated by applying it to three 2D functions and two 20D functions for showing the improved approximation accuracy with the use of nonlinear transformation, as well as to an 8D composite material design problem for optimizing the buckling-resistance performance of composite shells of rocket inter-stages.

Zhang, Guannan↗
DOE OSTI2025

Resimulation-based self-supervised learning for pretraining physics foundation models

Self-supervised learning (SSL) is at the core of training modern large machine learning models, providing a scheme for learning powerful representations that can be used in a variety of downstream tasks. However, SSL strategies must be adapted to the type of training data and downstream tasks required. We propose resimulation-based self-supervised representation learning (RS3L), a novel simulation-based SSL strategy that employs a method of resimulation to drive data augmentation for contrastive learning in the physical sciences, particularly, in fields that rely on stochastic simulators. By intervening in the middle of the simulation process and rerunning simulation components downstream of the intervention, we generate multiple realizations of an event, thus producing a set of augmentations covering all physics-driven variations available in the simulator. Using experiments from high-energy physics, we explore how this strategy may enable the development of a foundation model; we show how RS3L pretraining enables powerful performance in downstream tasks such as discrimination of a variety of objects and uncertainty mitigation. In addition to our results, we make the RS3L dataset publicly available for further studies on how to improve SSL strategies.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

The Effects of Compounded Model Size Reductions on Adversarial Robustness

Recent advances in Edge AI and Tiny Machine Learning (TinyML) have enabled the deployment of machine learning models on resource-constrained environments. However, deploying these models on edge devices, such as micro-controllers, requires significant model footprint reduction through a variety of techniques such as quantization, pruning, and clustering. While these optimization methods offer considerable advantages, they potentially introduce AI-related security vulnerabilities, particularly concerning model robustness with respect to adversarial AI attacks. Prior research has extensively examined the impact of quantization on adversarial robustness; however, the effects of alternative reduction techniques and their combinations remain understudied. This paper investigates the impact of model size reduction techniques on adversarial robustness, when applied individually and combined. We utilized Fast Gradient Sign Method (FGSM) and Projected Gradient Descent (PGD) attacks to generate adversarial perturbations for both training and testing data, and then evaluated the models' accuracy under adversarial training conditions. Our findings revealed that reduction techniques generally diminished robustness; although, combining techniques was not found to make robustness any worse than when applied individually. Moreover, specific techniques can potentially enhance resistance to small size perturbations. This research provides insights into the trade-offs between model size reduction and security, establishing a foundation for future investigations into improving adversarial training techniques and methodologies for maintaining robustness while preserving memory footprint benefits.

Austria, Phillipe [ORNL] (ORCID:0000000236223973)↗
DOE OSTI2020

Advanced data science toolkit for non-data scientists – A user guide

Emerging modern data analytics attracts much attention in materials research and shows great potential for enabling data-driven design. Data populated from the high-throughput CALPHAD approach enables researchers to better understand underlying mechanisms and to facilitate novel hypotheses generation, but the increasing volume of data makes the analysis extremely challenging. Here in this paper, we introduce an easy-to-use, versatile, and open-source data analytics frontend, ASCENDS (Advanced data SCiENce toolkit for Non-Data Scientists), designed with the intent of accelerating data-driven materials research and development. The toolkit is also of value beyond materials science as it can analyze the correlation between input features and target values, train machine learning models, and make predictions from the trained surrogate models of any scientific dataset. Various algorithms implemented in ASCENDS allow users performing quantified correlation analyses and supervised machine learning to explore any datasets of interest without extensive computing and data science background. The detailed usage of ASCENDS is introduced with an example of experimental high-temperature alloy data.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2022

Physics-informed machine learning with differentiable programming for heterogeneous underground reservoir pressure management

Abstract Avoiding over-pressurization in subsurface reservoirs is critical for applications like CO $$_2$$ 2 sequestration and wastewater injection. Managing the pressures by controlling injection/extraction are challenging because of complex heterogeneity in the subsurface. The heterogeneity typically requires high-fidelity physics-based models to make predictions on CO $$_2$$ 2 fate. Furthermore, characterizing the heterogeneity accurately is fraught with parametric uncertainty. Accounting for both, heterogeneity and uncertainty, makes this a computationally-intensive problem challenging for current reservoir simulators. To tackle this, we use differentiable programming with a full-physics model and machine learning to determine the fluid extraction rates that prevent over-pressurization at critical reservoir locations. We use DPFEHM framework, which has trustworthy physics based on the standard two-point flux finite volume discretization and is also automatically differentiable like machine learning models. Our physics-informed machine learning framework uses convolutional neural networks to learn an appropriate extraction rate based on the permeability field. We also perform a hyperparameter search to improve the model’s accuracy. Training and testing scenarios are executed to evaluate the feasibility of using physics-informed machine learning to manage reservoir pressures. We constructed and tested a sufficiently accurate simulator that is 400 000 times faster than the underlying physics-based simulator, allowing for near real-time analysis and robust uncertainty quantification.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Evaluating large scale aqueous organic redox flow battery performance with a hybrid numerical and machine learning framework

Aqueous organic redox flow battery (AORFB) is a promising cost-competitive technology for large-scale energy storage. Among existing work, the dihydroxyphenazine (DHP)-based AORFB has demonstrated high energy density and low-capacity degradation in 10 cm$^2$ cells during lab tests. However, its commercial-scale performance in more complex environments remains unknown, posing a barrier to commercialization. To address this gap, this work presents a comprehensive performance evaluation of a 780 cm$^2$ DHP-based AORFB by combining a physics-based numerical model, machine learning (ML)-based surrogate models, and ML-derived sensitivity quantification. Specifically, we first select 12 key battery parameters that include 10 physicochemical and 2 operation quantities, then select 6 performance metrics that include energy efficiency (EE), discharging capacity, charging energy, and power losses due to concentration, activation, and ohmic over-potentials. With such selection, 12800 combinations of the 12 parameters are subsequently generated using the Latin Hypercube Sampling method. Such combinations, together with 38 pre-defined State of Charge, are then integrated to a validated AORFB model developed in COMSOL to compute the performance metrics. With both input parameters and performance metrics, 60 deep neural network (DNN) surrogate models are then trained to approximate the relationship between the 10 physicochemical quantities and 6 performance metrics at each flow rate and current density. Sensitivity scores are then calculated based on the DNN models. Two additional sensitivity analysis tools, i.e., MARS, and SHAP, are also used to cross-validate the sensitivity scores from the DNN. The results demonstrate that 1) the standard potential ranks first in controlling EE and charging energy, 2) the membrane conductivity is most critical for power loss and EE, and 3) specific area and reaction rate control activation power loss.

25 ENERGY STORAGE↗
DOE OSTI2022

Data augmentation for disruption prediction via robust surrogate models

The goal of this work is to generate large statistically representative datasets to train machine learning models for disruption prediction provided by data from few existing discharges. Such a comprehensive training database is important to achieve satisfying and reliable prediction results in artificial neural network classifiers. Here, we aim for a robust augmentation of the training database for multivariate time series data using Student-t process regression. We apply Student-t process regression in a state space formulation via Bayesian filtering to tackle challenges imposed by outliers and noise in the training data set and to reduce the computational complexity. Thus, the method can also be used if the time resolution is high. We use an uncorrelated model for each dimension and impose correlations afterwards via coloring transformations. We demonstrate the efficacy of our approach on plasma diagnostics data of three different disruption classes from the DIII-D tokamak. To evaluate if the distribution of the generated data is similar to the training data, we additionally perform statistical analyses using methods from time series analysis, descriptive statistics, and classic machine learning clustering algorithms.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Turbulence Modeling with Nek5000/RS, SOD2D and Alya

We present Validation and Verification (V & V) study of two high-order spectral element method (SEM) based Computational Fluid Dyanimcs (CFD) codes that we will utilize for work on turbulence modeling: Nek5000 and SOD2D. While the former solves the incompressible form of Navier-Stokes equation, the latter works with compressible set of equations and uses an entropy-viscosity formulation to account for the discontinuities for high Mach number flows. We demonstrate the accuracy of these codes for two benchmark sub-sonic turbulent flows: periodic channel and pipe flow, by carrying out first and second order statistical analysis, grid convergence and turbulent structure analysis using wall-resolved large eddy simulations (WRLES). Later, we report on the implementation and testing of various wall-modeling strategies for large eddy simulation of turbulent flows in Nek5000. These include both classical log-law based and decision-tree based machine-learning models. Accuracy of these closure strategies are analyzed and necessary future work is outlined.

97 MATHEMATICS AND COMPUTING↗