Search NASA⌕ Search

SEARCH · Search NASA

Results for “Neural Machine Learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,315 records · Page 73
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Neural-network based collision operators for the Boltzmann equation

Kinetic gas dynamics in rarefied and moderate-density regimes have complex behavior associated with collisional processes. These processes are generally defined by convolution integrals over a high-dimensional space (as in the Boltzmann operator), or require evaluating complex auxiliary variables (as in Rosenbluth potentials in Fokker-Planck operators) that are challenging to implement and computationally expensive to evaluate. In this work, we develop a data-driven neural network model that augments a simple and inexpensive BGK collision operator with a machine-learned correction term, which improves the fidelity of the simple operator with a small overhead to overall runtime. The composite collision operator has a tunable fidelity and, in this work, is trained using and tested against a direct-simulation Monte-Carlo (DSMC) collision operator.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2024

Comparison of Machine Learning Approaches for Prediction of the Equivalent Alkane Carbon Number for Microemulsions Based on Molecular Properties

The chemical properties of oils are vital in the design of microemulsion systems. The hydrophilic–lipophilic difference equation used to predict microemulsions’ phase behavior expresses the oils’ physiochemical properties as the equivalent alkane carbon number (EACN). The experimental determination of EACN requires knowledge of the temperature dependence of the microemulsion system and the effects of different surfactant concentrations. Thus, the experimental determination is time-intensive and tedious, requiring days to months for proper separations. Furthermore, the experiments require high purity of chemicals because microemulsions are sensitive to impurities. Our work focuses on the quick and reliable predictions of the EACN with machine learning (ML) models. Due to the immaturity of ML chemical predictions, we compare three graph neural networks (GNNs) and a gradient-boosted tree algorithm, known as XGBoost. The GNNs use the molecular structures represented as simplified molecular-input line-entry system (SMILES) codes for the initial input, which allows us to assess whether geometry optimization is necessary for reliable results. The XGBoost model also begins with the SMILES representations of the molecules but uses molecular descriptors instead of geometry optimizations. As a result, the best model tested (crystal graph convolutional neural network with Merck molecular force field-94) has an error of 1.15 EACN units of the true EACN for unknown data with the errors skewed toward zero and an R² score of 0.9

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Advancing energy storage through solubility prediction: leveraging the potential of deep learning

Solubility prediction plays a crucial role in energy storage applications, such as redox flow batteries, because it directly affects the efficiency and reliability. Researchers have developed various methods that utilize quantum calculations and descriptors to predict the aqueous solubilities of organic molecules. Notably, machine learning models based on descriptors have shown promise for solubility prediction. As deep learning tools, graph neural networks (GNNs) have emerged to capture complex structure–property relationships for material property prediction. Specifically, MolGAT, a type of GNN model, was designed to incorporate n-dimensional edge attributes, enabling the modeling of intricacies in molecular graphs and enhancing the prediction capabilities. In a previous study, MolGAT successfully screened 23 467 promising redox-active molecules from a database of over 500 000 compounds, based on redox potential predictions. This study focused on applying the MolGAT model to predict the aqueous solubility (log S) of a broad range of organic compounds, including those previously screened for redox activity. The model was trained on a diverse sample of 8494 organic molecules from AqSolDB and benchmarked against literature data, demonstrating superior accuracy compared with other state of the art graph-based and descriptor-based models. Subsequently, the trained MolGAT model was employed to screen redox-active organic compounds identified in the first phase of high-throughput virtual screening, targeting favorable solubility in energy storage applications. The second round of screening, which considered solubility, yielded 12 332 promising redox-active and soluble organic molecules suitable for use in aqueous redox flow batteries. Thus, the two-phase high-throughput virtual screening approach utilizing MolGAT, specifically trained for redox potential and solubility, is an effective strategy for selecting suitable intrinsically soluble redox-active molecules from extensive databases, potentially advancing energy storage through reliable material development. This indicates that the model is reliable for predicting the solubility of various molecules and provides valuable insights for energy storage, pharmaceutical, environmental, and chemical applications.

25 ENERGY STORAGE↗
DOE OSTI2021

Modeling laser-driven ion acceleration with deep learning

Developments in machine learning promise to ameliorate some of the challenges of modeling complex physical systems through neural-network-based surrogate models. High-intensity, short-pulse lasers can be used to accelerate ions to mega-electronvolt energies, but to model such interactions requires computationally expensive techniques such as particle-in-cell simulations. Multilayer neural networks allow one to take a relatively sparse ensemble of simulations and generate a surrogate model that can be used to rapidly search the parameter space of interest. In this study, we created an ensemble of over 1,000 simulations modeling laser-driven ion acceleration and developed a surrogate to study the resulting parameter space. A neural-network-based approach allows for rapid feature discovery not possible for traditional parameter scans given the computational cost. A notable observation made during this study was the dependence of ion energy on the pre-plasma gradient length scale. While this methodology harbors great promise for ion acceleration, it has ready application to all topics in which large-scale parameter scans are restricted by significant computational cost or relatively large, but sparse, domains.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2025

Reconstruction of beam parameters and betatron radiation spectra measured with a Compton spectrometer

The photon flux resulting from high-energy electron beam interactions with high-field systems, such as those found in the upcoming FACET-II experiments at the SLAC National Accelerator Laboratory, yields deep insight into the electron beam’s underlying dynamics during the interaction. However, extracting this information is an intricate process. To demonstrate how to approach this challenge using modern methods, this paper utilizes simulated data that models plasma wakefield acceleration-derived betatron radiation in experiments to determine reliable methods of reconstructing key beam and beam-plasma interaction properties. For betatron radiation measurements, translating the observed 200⁢ keV to 30⁢ MeV photon double-differential energy-angle spectra obtained from an advanced Compton spectrometer requires testing multiple methods to optimize the pipeline from its response to incident electron beam information. The paper compares maximum likelihood estimation and machine learning to refine the translation of photon spectra into precise electron beam metrics, such as spot size, energy, and emittance, enhancing the understanding of beam behavior within these dense, high-field environments. We also introduce machine learning and the expected maximization algorithm to reconstruct the primary photon spectrum, employing a multilayer neural network for regression analysis of the energy and angle spectra. With appropriate modifications, the advanced methods reproduce relevant incident beam parameters with high accuracy, even for beam sizes in the <10 μ⁢m range. This capacity is critical to understanding intense beam propagation and its optimization in plasma.

Beam code development & simulation techniques↗
DOE OSTI2022

H-GCN: A Graph Convolutional Network Accelerator on Versal ACAP Architecture

Recently Graph Neural Networks (GNNs) have drawn tremendous attentions due to their unique capability to extend the Machine Learning (ML) approaches to broadly defined applications with unstructured data, especially graphs. Comparing with other ML modalities, the acceleration of GNNs is as critical but even more challenging due to the irregularity and heterogeneity from graph typologies that together limit the performance. Existing efforts mainly focus on handling graphs’ irregularity, however, have not studied the heterogeneity. To this end, in this work, we propose H-GCN, a PL-AIE-based hybrid accelerator that leverages the emerging heterogeneity of Xilinx Versal ACAPs to achieve high-performance GNN inference. In particular, H-GCN partitions each graph into three subgraphs based on its inherent heterogeneity and processes them using PL and the newly emerged AIE respectively. To further improve the performance, we explore the sparsity support of AIE and develop an efficient density-aware method to map tiles of SpMM onto the systolic tensor array automatically. Compared with the current state-of-the-art GCN accelerator, HGCN achieves on average 1.5× speedups.

Zhang, Chengming↗
DOE OSTI2021

Codon2Vec v1.0

Background: Codon2Vec is an embedding neural network that predicts 'high' or 'low' gene expression directly from the protein-coding sequences. Embedding neural networks are commonly used for natural language processing (NLP) applications. Analogous to how an English sentence is a string of words, a gene can be thought of as a string of codons. Similar to how NLP neural networks model English sentences as a non-random sequence of words, we considered a coding sequence as a non-random non-overlapping array of codons (k-mers of length = 3). Value Proposition: - Codon2Vec achieved a high median AUC-ROC score of 83.8% when trained and applied to transcriptomic data from 300 fungal species - Unlike Codo2Vec, conventional methods predicting for expression based on codon usage rely on a priori knowledge of optimal codons or a set of reference genes. - Unlike Codon2vec, these methods do not account for the effect of codon order on gene expression. - Codon2Vec neural network bypasses the need for artisanal feature selection step that is necessary for traditional machine learning models.

Wint, Rhondene↗
DOE OSTI2024

PhyML

Python scripts for physics informed machine learning for time-dependent problems. The code implements hard constrained sequential physics-informed neural networks (HCSPINNs) using JAX library. The details of the method and implementation can be found in the following paper: Roy, P., & Castonguay, S. (2024). Exact Enforcement of Temporal Continuity in Sequential Physics- Informed Neural Networks. arXiv preprint arXiv:2403.03223. (https://arxiv.org/abs/2403.03223)

Roy, Pratanu↗
DOE OSTI2021

Predictive Data-driven Platform for Subsurface Energy Production

Subsurface energy activities such as unconventional resource recovery, enhanced geothermal energy systems, and geologic carbon storage require fast and reliable methods to account for complex, multiphysical processes in heterogeneous fractured and porous media. Although reservoir simulation is considered the industry standard for simulating these subsurface systems with injection and/or extraction operations, reservoir simulation requires spatio-temporal “Big Data” into the simulation model, which is typically a major challenge during model development and computational phase. In this work, we developed and applied various deep neural network-based approaches to (1) process multiscale image segmentation, (2) generate ensemble members of drainage networks, flow channels, and porous media using deep convolutional generative adversarial network, (3) construct multiple hybrid neural networks such as convolutional LSTM and convolutional neural network-LSTM to develop fast and accurate reduced order models for shale gas extraction, and (4) physics-informed neural network and deep Q-learning for flow and energy production. We hypothesized that physicsbased machine learning/deep learning can overcome the shortcomings of traditional machine learning methods where data-driven models have faltered beyond the data and physical conditions used for training and validation. We improved and developed novel approaches to demonstrate that physics-based ML can allow us to incorporate physical constraints (e.g., scientific domain knowledge) into ML framework. Outcomes of this project will be readily applicable for many energy and national security problems that are particularly defined by multiscale features and network systems.

58 GEOSCIENCES↗
DOE OSTI2026

Overview of IMPACT Data Acquisition System and Data Reduction Process

This report documents the development of the data acquisition system (DAS) and data reduction methodologies for the Irradiated Material Property Accelerated Characterization Test (IMPACT) experiment at the Advanced Test Reactor (ATR). The IMPACT experiment is designed to enable in-pile measurement of thermal conductivity in metallic nuclear fuels, specifically U-10Zr, using an instrumented thermal conductivity probe. The DAS supports both passive temperature monitoring and active thermal interrogation of the probe through controlled AC and DC excitation. Significant modifications to laboratory-scale systems were required to accommodate the higher resistance paths associated with the in-pile application. Custom electronics and relay-controlled measurement sequencing were developed to enable the measurement and sufficient power delivery to the sensing region. A reduced-order, axisymmetric thermal model based on the thermal quadrupoles method is presented to support data interpretation. This model enables efficient evaluation of transient heat transfer behavior and facilitates solution of the inverse problem required to extract thermal properties from measured signals. Multiple boundary condition formulations are discussed to address varying experimental time scales and geometries. Additionally, machine learning techniques are introduced to support data reduction and improve confidence in inverse solutions. Convolutional neural networks are applied to identify the presence of gas gaps and other evolving geometric features that significantly impact thermal response during irradiation. These efforts contribute to the broader integration of digital twin frameworks and real-time modeling capabilities within the Advanced Fuels Campaign.

46 - INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AN↗
DOE OSTI2020

Measure Utility, Gain Trust: Practical Advice for XAI Researchers

Research into explanation of machine learning models, i.e. explainable AI (XAI), has seen a sympathetic exponential growth alongside deep artificial neural networks throughout the past decade. For historical reasons explanation and trust have been intertwined. However this focus on trust is too narrow, and has led the research community astray from tried and true empirical methods that lead to more defensible scientific knowledge about people and explanations. To address this, we contribute a practical path forward for researchers in the XAI field. We recommend researchers focus on the utility and impact of their explanations instead of trust. We outline five broad use cases where explanations are useful and, for each, we describe pseudo-experiments that rely on objective empirical measurements and falsifiable hypotheses. We believe that this experimental rigor is necessary to contribute to scientific knowledge in the field of XAI.

Davis, Brittany F.↗
DOE OSTI2024

Intelligent Pixel Detectors: Towards a Radiation Hard ASIC with On-Chip Machine Learning in 28 nm CMOS

Detectors at future high energy colliders will face enormous technical challenges. Disentangling the unprecedented numbers of particles expected in each event will require highly granular silicon pixel detectors with billions of readout channels. With event rates as high as 40 MHz, these detectors will generate petabytes of data per second. To enable discovery within strict bandwidth and latency constraints, future trackers must be capable of fast, power efficient, and radiation hard data-reduction at the source. We are developing a radiation hard readout integrated circuit (ROIC) in 28nm CMOS with on-chip machine learning (ML) for future intelligent pixel detectors. We will show track parameter predictions using a neural network within a single layer of silicon and hardware tests on the first tape-outs produced with TSMC. Preliminary results indicate that reading out featurized clusters from particles above a modest momentum threshold could enable using pixel information at 40 MHz.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2024

Enhancing Lattice Kinetic Schemes for Fluid Dynamics with Lattice-Equivariant Neural Networks

A new class of equivariant neural networks is presented, hereby dubbed lattice-equivariant neural networks (LENNs), designed to satisfy local symmetries of a lattice structure. The approach develops within a recently introduced framework aimed at learning neural network-based surrogate models’ lattice Boltzmann collision operators. Whenever neural networks are employed to model physical systems, respecting symmetries and equivariance properties has been shown to be key for accuracy, numerical stability, and performance. Here, hinging on ideas from group representation theory, trainable layers are defined whose algebraic structure is equivariant with respect to the symmetries of the lattice cell. In this work, the presented method naturally allows for efficient implementations, in terms of both memory usage and computational costs, supporting scalable training/testing for lattices in two spatial dimensions and higher (in which the size of symmetry group grows). The approach is validated and tested considering 2D and 3D flowing dynamics, both in laminar and turbulent regimes. It is compared with group-averaged-based symmetric networks and with plain, nonsymmetric, networks, showing how the presented approach unlocks the (a posteriori) accuracy and training stability of the former models and the train/inference speed of the latter networks. (LENNs are about one order of magnitude faster than group-averaged networks in 3D.) The work in this paper opens toward practical use of machine learning-augmented lattice Boltzmann CFD in real-world simulations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

A Machine Learning Framework to Predict Images of Edge-on Protoplanetary Disks

The physical structure and properties of protoplanetary disks are typically derived from spatially resolved disk images. Edge-on disks in particular provide an important view point on the vertical structure and degree of settling of disks. Such analyses rely on radiative transfer (RT) calculations that are generally computationally intensive due to the high optical depth of disks. Here we present a machine learning framework that has the potential to dramatically speed up the forward modeling process by approximating the results of RT calculations. This framework, trained on an initial set of RT calculations, utilizes an autoencoder neural network to enable the generation of synthetic scattered light images of edge-on disks directly from a set of physical parameters. We demonstrate that this framework generates synthetic images 2–3 orders of magnitude faster than using RT calculations. These machine learning-generated images appear to approximate the RT images well, in particular preserving their size and shape. We also find a strong correlation between the latent space representations of the generated disk images and several of their associated physical parameters. Finally, we discuss potential changes to the framework, such as methods to further improve the image quality, extending the framework to multiple wavelengths, and inverting the process to infer physical parameters from observed images. Overall, these new tools have the potential to enable a more efficient and uniform analysis of edge-on disk properties and the initial conditions of planet formation.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Universal and interpretable classification of atomistic structural transitions via unsupervised graph learning

Materials processing often occurs under extreme dynamic conditions leading to a multitude of unique structural environments. These structural environments generally occur at high temperatures and/or high pressures, often under non-equilibrium conditions, which results in drastic changes in the material's structure over time. Computational techniques, such as molecular dynamics simulations, can probe the atomic regime under these extreme conditions. However, characterizing the resulting diverse atomistic structures as a material undergoes extreme changes in its structure has proved challenging due to the inherently non-linear relationship between structures as large-scale changes occur. Here, we introduce SODAS++, a universal graph neural network framework, that can accurately and intuitively quantify the atomistic structural evolution corresponding to the transition between any two arbitrary phases. We showcase SODAS++ for both solid–solid and solid–liquid transitions for systems of increasing geometric and chemical complexity, such as colloidal systems, elemental Al, rutile and amorphous TiO 2 , and the non-stoichiometric ternary alloy Ag 26 Au 5 Cu 19 . Finally, we show that SODAS++ can accurately quantify all transitions in a physically interpretable manner, showcasing the power of unsupervised graph neural network encodings for capturing the complex and non-linear pathway, a material's structure takes as it evolves.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Scalable training of trustworthy and energy-efficient predictive graph foundation models for atomistic materials modeling: a case study with HydraGNN

We present our work on developing and training scalable, trustworthy, and energy-efficient predictive graph foundation models (GFMs) using HydraGNN, a multi-headed graph convolutional neural network architecture. HydraGNN expands the boundaries of graph neural network (GNN) computations in both training scale and data diversity. It abstracts over message passing algorithms, allowing both reproduction of and comparison across algorithmic innovations that define nearest-neighbor convolution in GNNs. This work discusses a series of optimizations that have allowed scaling up the GFMs training to tens of thousands of GPUs on datasets consisting of hundreds of millions of graphs. Our GFMs use multitask learning (MTL) to simultaneously learn graph-level and node-level properties of atomistic structures, such as energy and atomic forces. Using over 154 million atomistic structures for training, we illustrate the performance of our approach along with the lessons learned on two state-of-the-art US Department of Energy (US-DOE) supercomputers, namely the Perlmutter petascale system at the National Energy Research Scientific Computing Center and the Frontier exascale system at Oak Ridge Leadership Computing Facility. The HydraGNN architecture enables the GFM to achieve near-linear strong scaling performance using more than 2000 GPUs on Perlmutter and 16,000 GPUs on Frontier.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Elevated temperature contact creep and friction of nickel-based superalloys using machine learning assisted finite element analysis

Nickel-based superalloys with superior thermochemical, mechanical, and tribological properties are highly utilized for critical components in several high temperature applications such as gas turbines and nuclear reactors. Inconel 617, in particular, is considered as one of the main candidate superalloys for tribo-components in very-high-temperature gas-cooled nuclear reactors. Recent findings indicate that this alloy grows unique surface oxide especially in a high-temperature helium environment with distinctive wear, friction, and contact properties. Here, this study investigates the high temperature contact area evolution and frictional behavior of Inconel 617 using finite element simulation and provides predictive models for the contact and friction performance at different normal loads, dwell times, and temperatures. High temperature helium-aged Inconel 617 top surface properties (up to 600 °C) are utilized along with a single asperity-based deformable elastic-plastic contact model under combined normal and tangential loading. Machine learning is used to assist the finite element results and to predict friction coefficient as well as contact area evolution. While a small difference is observed in the instantaneous friction coefficient (no dwell time) for all temperatures, friction coefficient increases considerably with dwell time. This shows that the effect of contact creep for longer dwell times significantly dominants the effect of high temperature variation in basic mechanical parameters such as modulus and yield strength. It is found that increasing temperature and dwell times lead to the friction coefficient increase, yet the dominance of dwell time effects decreases at higher temperatures and loads. While the analysis is presented for Inconel 617, the methodology is easy to be generalized and can be applied to other HT alloys.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Train Like a (Var)Pro: Efficient Training of Neural Networks with Variable Projection

Deep neural networks (DNNs) have achieved state-of-the-art performance across a variety of traditional machine learning tasks, e.g., speech recognition, image classification, and segmentation. The ability of DNNs to efficiently approximate high-dimensional functions has also motivated their use in scientific applications, e.g., to solve partial differential equations and to generate surrogate models. In this paper, we consider the supervised training of DNNs, which arises in many of the above applications. We focus on the central problem of optimizing the weights of the given DNN such that it accurately approximates the relation between observed input and target data. Devising effective solvers for this optimization problem is notoriously challenging due to the large number of weights, nonconvexity, data sparsity, and nontrivial choice of hyperparameters. To solve the optimization problem more efficiently, we propose the use of variable projection (VarPro), a method originally designed for separable nonlinear least-squares problems. Our main contribution is the Gauss--Newton VarPro method (GNvpro) that extends the reach of the VarPro idea to nonquadratic objective functions, most notably cross-entropy loss functions arising in classification. These extensions make GNvpro applicable to all training problems that involve a DNN whose last layer is an affine mapping, which is common in many state-of-the-art architectures. In our four numerical experiments from surrogate modeling, segmentation, and classification, GNvpro solves the optimization problem more efficiently than commonly used stochastic gradient descent (SGD) schemes. Finally, GNvpro finds solutions that generalize well, and in all but one example better than well-tuned SGD methods, to unseen data points.

97 MATHEMATICS AND COMPUTING↗