Search NASA⌕ Search

SEARCH · Search NASA

Results for “Neural Machine Learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 991 records · Page 55
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Deep Learning for Multigroup Cross-Section Representation in Two-Step Core Calculations

Here we investigate using deep learning, a type of machine-learning algorithm employing multiple layers of artificial neurons, for the mathematical representation of multigroup cross sections for use in the Griffin reactor multiphysics code for two-step deterministic neutronics calculations. A three-dimensional fuel element typical of a high-temperature gas reactor as well as a two-dimensional sodium-cooled fast reactor lattice are modeled using the Serpent Monte Carlo code, and multigroup macroscopic cross sections are generated for various state parameters to produce a training data set and a separate validation data set. A fully connected, feedforward neural network is trained using the open-source PyTorch machine-learning framework, and its accuracy is compared against the standard piecewise linear interpolation model. Additionally, we provide in this work a generic technique for propagating the cross-section model errors up to the k eff using sensitivity coefficients with the first-order uncertainty propagation rule. Quantifying the eigenvalue error due to the cross-section regression errors is especially practical for appropriately selecting the mathematical representation of the cross sections. We demonstrate that the artificial neural network model produces lower errors and therefore enables better accuracy relative to the piecewise linear model when the cross sections exhibit nonlinear dependencies; especially when a coarse grid is employed, where the errors can be halved by the artificial neural network. However, for linearly dependent multigroup cross sections as found for the sodium-cooled fast reactor case, a simpler linear regression outperforms deeper networks.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
NASA NTRS2019

High-Performance Computing Optimization for Aladyn – Adaptive Neural Network Molecular Dynamics Mini-Application

This report provides a description and performance evaluation of the optimization techniques for high performance computing (HPC) implementation of the open source Computational Materials mini-application Aladyn (https://github.com/nasa/aladyn). Aladyn is a basic molecular dynamics code written in FORTRAN 2003, which is designed to demonstrate the use of adaptive neural networks (ANNs) in atomistic simulations. The role of ANNs is to efficiently reproduce the very complex energy landscape resulting from the atomic interactions in materials with the accuracy of the more expensive quantum mechanics-based calculations. The ANN is trained on a large set of atomic structures calculated using the density functional theory (DFT) method. While achieving orders of magnitude faster computational performance than DFT, the ANN-based approach was still very computationally demanding compared to the conventional approach of using empirically fitted energy functions. After its initial development, Aladyn was evaluated and optimized by experts at the NASA Advanced Supercomputing (NAS) division to exploit modern supercomputer architectures. The code has been optimized for execution on multicore central processing units (CPUs), including Intel® Skylake microarchitecture, and on graphic accelerators, such as Nvidia® V100 graphic processing units (GPUs), using Open Multi-Processing (OpenMP) and Open Accelerators (OpenACC) programming interfaces. The optimization achieved a speedup of 4.7 times the baseline version on CPU performance and an additional 2.4 times on CPU+GPU performance. Atomistic computer simulations are a fundamental tool in materials research to model material properties form physics-based first principles. Atomic interaction, governed by Quantum Mechanics (QM) require sophisticated and highly computationally demanding mathematical models to calculate [1]. Classical methods use approximate functional forms, empirically fitted through a set of variable parameters to emulate atomic energies as direct functions of atomic coordinates [2]. While empirical potentials are computationally much simpler, allowing simulations of large-scale systems of up to a trillion (1012) atoms [3], they are substantially less accurate compared to quantum calculations and applicable only to very specific atomic configurations or predefined crystallographic phases. A recently suggested approach is to use heuristic machine learning methods [4], such as those based on Adaptive Neural Networks (ANNs) to predict atomic energies, after being trained on a sufficiently large database of QM-calculated structures [5,6]. This approach reduces significantly the computational complexity, allowing for simulations of orders of magnitude larger systems compared to QM-based methods without compromising accuracy. Still, compared to classical methods using empirical energy functions, ANN methods remain two- to three orders of magnitude more computationally demanding. Hence, the computational cost of simulations, together with the need for extensive training of ANNs, still makes the practical implementation of ANN-based methods quite challenging. The purpose of the Aladyn mini-application software [7], available as open source at https://github.com/nasa/aladyn, is to be a testbed for exploring possible optimization strategies to develop highly scalable parallel algorithms for ANN-based atomistic simulations. Aladyn is aimed at utilizing the architecture of the high-end modern highperformance computing (HPC) hardware based on multicore central processing units (CPUs) equipped with graphic processing unit (GPU) accelerators. Specifically, the goal is to optimize the performance on a single HPC compute node, before implementing scaling to multi-node parallelization using message passing interface (MPI). At the same time, the open source code of Aladyn can serve as a training model for students and professors in academia.

Yamakov, Vesselin I.↗
DOE OSTI2025

Jensen–Shannon divergence based novel loss functions for Bayesian neural networks

Bayesian neural networks (BNNs) are state-of-the-art machine learning methods that can naturally regularize and systematically quantify uncertainties using their stochastic parameters. Kullback–Leibler (KL) divergence-based variational inference used in BNNs suffer from unstable optimization and challenges in approximating light-tailed posteriors due to the unbounded nature of the KL divergence. To resolve these issues, we formulate a novel loss function for BNNs based on a new modification to the generalized Jensen–Shannon (JS) divergence, which is bounded. In addition, we propose a Geometric JS divergence-based loss, which is computationally efficient since it can be evaluated analytically. We found that the JS divergence-based variational inference is intractable, and hence employed a constrained optimization framework to formulate these losses. Our theoretical analysis and empirical experiments on multiple regression and classification data sets suggest that the proposed losses perform better than the KL divergence-based loss, especially when the data sets are noisy or biased. Specifically, there are approximately 5% and 8% improvements in accuracy for a noise-added CIFAR-10 dataset and a regression dataset, respectively. There is about 13% reduction in false negative predictions of a biased histopathology dataset. Additionally, we quantify and compare the uncertainty metrics for the regression and classification tasks.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2021

Automated Data Accountability for Missions in Mars Rover Data

As the Mars Curiosity Rover transmits data to the JPL Ground Data System (GDS), it frequently observes data loss and corruption, requiring re-transmits from the rover and Ground Data System Analysts (GDSA) to monitor the downlink process. As new missions are launched, the GDSA team redistributes analysts to these new missions, causing shortages in previous missions. The GDSA team can significantly benefit from the automation and optimization of the downlink process of telemetry data. In fact, there is a need for a better understanding of why the data is corrupted, so that the GDSA team can best determine the root cause of the issues in the GDS. This paper presents machine learning and deep learning based approaches to automate and optimize the detection of data loss. We first created a pipeline to automatically accumulate data from the telemetry databases (MAROS, Telemetry Data Storage, and GDS Elastic Search Database) in the downlink process. With our newly created datasets, we perform feature selection to supplement the GDSA understanding of the downlink process and provide supplemental analysis on the importance of different features. We implement various machine learning and deep learning based models, including support vector machines, ensemble methods, and deep neural networks and evaluate their accuracies in identifying whether a downlink process is complete or incomplete. We utilize fast hyperparameter optimization methods that allow our models to quickly be re-trained, allowing them to quickly be tuned and optimized on daily incoming data in real time. This hyperparameter optimization also allows our methods to be quickly integrated into other JPL missions. Our results show that our best-performing machine learning and deep learning based models outperform the existing GDSA detection software by 6 accuracy points and can aid analysts by providing insights into the data accountability problem. Since these various machine learning and deep learning approaches vary significantly in interpretability, we provide a discussion on the tradeoffs between their performance and trustworthiness in helping detect issues in data transmission.

Divsalar, Dariush↗
DOE OSTI2024

Acceleration of Graph Neural Network-Based Prediction Models in Chemistry via Co-Design Optimization on Intelligence Processing Units

Atomic structure prediction and associated property calculations are the bedrock of chemical physics. Since high-fidelity ab initio modeling techniques for computing the structure and properties can be prohibitively expensive, this motivates the development of machine-learning (ML) models that make these predictions more efficiently. Training graph neural networks over large atomistic databases introduces unique computational challenges such as the need to process millions of small graphs with variable size and support communication patterns that are distinct from learning over large graphs such as social networks. We demonstrate a novel hardware-software co-design approach to scale up the training of atomistic graph neural networks (GNN) for structure and property prediction. First, to eliminate redundant computation and memory associated with alternative padding techniques and to improve throughput via minimizing communication, we formulate the effective coalescing of the batches of variable-size atomistic graphs as the bin packing problem and introduce a hardware-agnostic algorithm to pack these batches. In addition, we propose hardware-specific optimizations including a planner and vectorization for the gather-scatter operations targeted for Graphcore’s Intelligence Processing Unit (IPU), as well as model-specific optimizations such as merged communication collectives and optimized softplus. Putting these all together, we demonstrate the effectiveness of the proposed co-design approach by providing an implementation of a well-established atomistic GNN on the Graphcore IPUs. We evaluate the training performance on multiple atomistic graph databases with varying degrees of graph counts, sizes and sparsity. Here, we demonstrate that such a co-design approach can reduce the training time of atomistic GNNs and can improve the performance by up to 1.5× compared to the baseline implementation of the model on the IPUs. Additionally, we compare our IPU implementation with a Nvidia GPU-based implementation and show that our atomistic GNN implementation on the IPUs can run 1.8× faster on average compared to the execution time on the GPUs.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Preliminary report on applications of machine learning techniques to the Nevada play fairway analysis

We are applying machine learning (ML) techniques, including training set augmentation and artificial neural networks, to mitigate key challenges in the Nevada play fairway project. The study area includes ~85 active geothermal systems as potential training sites and >12 geologic, geophysical, and geochemical features. The main goal is to develop an algorithmic approach to identify new geothermal systems in the Great Basin region. Major objectives include: 1) integrate ML techniques into the geothermal community; 2) develop open community datasets, whereby all play fairway and ML datasets and algorithms are publicly released and available for modification by various user groups; 3) identify data acquisition targets with high value for future work; 4) identify new signatures to detect blind geothermal systems; and 5) foster new capabilities for characterizing subsurface temperature and permeability. Initially, ML techniques are being applied to the same play fairway datasets and workflow. ML will then be applied to both enhanced and additional datasets, with modification of the PFA workflow to incorporate the new datasets. Finally, ML will be applied to define new workflows using the enhanced and additional datasets. An algorithmic approach that empirically learns to estimate weights of influence for diverse parameters can potentially scale and perform better than the play fairway analysis. Initial work on this project has involved 1) evaluating potential positive and negative training sites, 2) transformation of datasets into formats suitable for ML, and 3) initial development and testing of ML techniques.

58 GEOSCIENCES↗
DOE OSTI2024

A Machine Learning Approach to Quantitative Analysis of Enamel Microstructure from Scanning Electron Microscopy Images

Dental enamel, the outermost tissue of mammalian teeth, must withstand a lifetime of wear and cyclic contact. To meet this demand, enamel possesses a combination of high hardness and resistance to fracture, properties that are typically mutually exclusive. The impressive damage tolerance has been attributed largely to decussation of the enamel rods, the principal unit of its microstructure. As such, enamel is inspiring the design of next‐generation structural materials. However, quantitative descriptions of the decussated enamel rod microstructure remain limited due to challenges encountered in applying computed tomography and in acquiring quality images appropriate for traditional digital processing methods. Here, a machine learning segmentation method is applied to images of the enamel obtained using scanning electron microscopy to support quantitative analysis of the microstructure. A pretrained convolutional neural network is used to expand the input training image dataset to allow the training of a random forest classifier, which ultimately segments the image with a very small training set ( n = 3 images). A validation of this segmentation method is presented, in addition to its application to calculate relevant microstructural parameters for images of tooth enamel from selected mammalian species. The methodology applied here is equally applicable to other hard tissues.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Benchmarking Operators in Deep Neural Networks for Improving Performance Portability of SYCL

SYCL is a portable programming model for heterogeneous computing, so it is important to obtain reasonable performance portability of SYCL. Towards the goal of better understanding and improving performance portability of SYCL for machine learning workloads, we have been developing benchmarks for basic operators in deep neural networks (DNNs). These operators could be offloaded to heterogeneous computing devices such as graphics processing units (GPUs) to speed up computation. In this paper, we introduce the benchmarks, evaluate the performance of the operators on GPU-based systems, and describe the causes of the performance gap between the SYCL and Compute Unified Device Architecture (CUDA) kernels. We find that the causes are related to the utilization of the texture cache for read-only data, optimization of the memory accesses with strength reduction, use of local memory, and register usage per thread. We hope that the efforts of developing benchmarks for studying performance portability will stimulate discussion and interactions within the community.

Jin, Zheming [ORNL] (ORCID:000000027197780X)↗
DOE OSTI2024

Evaluating Operators in Deep Neural Networks for Improving Performance Portability of SYCL

SYCL is a portable programming model for heterogeneous computing, so it is important to obtain reasonable performance portability of SYCL. Towards the goal of better understanding and improving performance portability of SYCL for machine learning workloads, we have been developing benchmarks for basic operators in deep neural networks (DNNs). These operators could be offloaded to heterogeneous computing devices such as graphics processing units (GPUs) to speed up computation. In this work, we introduce the benchmarks, evaluate the performance of the operators on GPU-based systems, and describe the causes of the performance gap between the SYCL and Compute Unified Device Architecture (CUDA) kernels. We find that the causes are related to the utilization of the texture cache for read-only data, optimization of the memory accesses with strength reduction, shared local memory accesses, and register usage per thread. We hope that the efforts of developing benchmarks for studying performance portability will stimulate discussion and interactions within the community.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Deep learning for NLTE spectral opacities

Computer simulations of high energy density science experiments are computationally challenging, consisting of multiple physics calculations including radiation transport, hydrodynamics, atomic physics, nuclear reactions, laser–plasma interactions, and more. To simulate inertial confinement fusion (ICF) experiments at high fidelity, each of these physics calculations should be as detailed as possible. However, this quickly becomes too computationally expensive even for modern supercomputers, and thus many simplifying assumptions are made to reduce the required computational time. Much of the research has focused on acceleration techniques for the various packages in multiphysics codes. In this work, we explore a novel method for accelerating physics packages via machine learning. The non-local thermodynamic equilibrium (NLTE) package is one of the most expensive calculations in the simulations of indirect drive inertial confinement fusion, taking several tens of percent of the total wall clock time. We explore the use of machine learning to accelerate this package, by essentially replacing the physics calculation with a deep neural network that has been trained to emulate the physics code. Overall, we demonstrate the feasibility of this approach on a simple problem and perform a side-by-side comparison of the physics calculation and the neural network inline in an ICF Hohlraum simulation. We show that the neural network achieves a 10× speed up in NLTE computational time while achieving good agreement with the physics code for several quantities of interest.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2022

Automated Stellar Spectra Classification with Ensemble Convolutional Neural Network

Large sky survey telescopes have produced a tremendous amount of astronomical data, including spectra. Machine learning methods must be employed to automatically process the spectral data obtained by these telescopes. Classification of stellar spectra by applying deep learning is an important research direction for the automatic classification of high-dimensional celestial spectra. In this paper, a robust ensemble convolutional neural network (ECNN) was designed and applied to improve the classification accuracy of massive stellar spectra from the Sloan digital sky survey. We designed six classifiers which consist six different convolutional neural networks (CNN), respectively, to recognize the spectra in DR16. Then, according the cross-entropy testing error of the spectra at different signal-to-noise ratios, we integrate the results of different classifiers in an ensemble learning way to improve the effect of classification. The experimental result proved that our one-dimensional ECNN strategy could achieve 95.0% accuracy in the classification task of the stellar spectra, a level of accuracy that exceeds that of the classical principal component analysis and support vector machine model.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

Evolutionary reinforcement learning of dynamical large deviations

In this work, we show how to bound and calculate the likelihood of dynamical large deviations using evolutionary reinforcement learning. An agent, a stochastic model, propagates a continuous-time Monte Carlo trajectory and receives a reward conditioned upon the values of certain path-extensive quantities. Evolution produces progressively fitter agents, potentially allowing the calculation of a piece of a large-deviation rate function for a particular model and path-extensive quantity. For models with small state spaces, the evolutionary process acts directly on rates, and for models with large state spaces, the process acts on the weights of a neural network that parameterizes the model's rates. This approach shows how path-extensive physics problems can be considered within a framework widely used in machine learning.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Stochastic machine learning via sigma profiles to build a digital chemical space

This work establishes a different paradigm on digital molecular spaces and their efficient navigation by exploiting sigma profiles. To do so, the remarkable capability of Gaussian processes (GPs), a type of stochastic machine learning model, to correlate and predict physicochemical properties from sigma profiles is demonstrated, outperforming state-of-the-art neural networks previously published. The amount of chemical information encoded in sigma profiles eases the learning burden of machine learning models, permitting the training of GPs on small datasets which, due to their negligible computational cost and ease of implementation, are ideal models to be combined with optimization tools such as gradient search or Bayesian optimization (BO). Gradient search is used to efficiently navigate the sigma profile digital space, quickly converging to local extrema of target physicochemical properties. While this requires the availability of pretrained GP models on existing datasets, such limitations are eliminated with the implementation of BO, which can find global extrema with a limited number of iterations. A remarkable example of this is that of BO toward boiling temperature optimization. Holding no knowledge of chemistry except for the sigma profile and boiling temperature of carbon monoxide (the worst possible initial guess), BO finds the global maximum of the available boiling temperature dataset (over 1,000 molecules encompassing more than 40 families of organic and inorganic compounds) in just 15 iterations (i.e., 15 property measurements), cementing sigma profiles as a powerful digital chemical space for molecular optimization and discovery, particularly when little to no experimental data is initially available.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Virtual node graph neural network for full phonon prediction

Understanding the structure-property relationship is crucial for designing materials with desired properties. The past few years have witnessed remarkable progress in machine-learning methods for this connection. However, substantial challenges remain, including the generalizability of models and prediction of properties with materials-dependent output dimensions. Here we present the virtual node graph neural network to address the challenges. By developing three virtual node approaches, we achieve Γ-phonon spectra and full phonon dispersion prediction from atomic coordinates. We show that, compared with the machine-learning interatomic potentials, our approach achieves orders-of-magnitude-higher efficiency with comparable to better accuracy. This allows us to generate databases for Γ-phonon containing over 146,000 materials and phonon band structures of zeolites. Additionally, our work provides an avenue for rapid and high-quality prediction of phonon band structures enabling materials design with desired phonon properties. The virtual node method also provides a generic method for machine-learning design with a high level of flexibility. In this study, the authors present a virtual node graph neural network to enable the prediction of material properties with variable output dimensions. This method offers fast and accurate predictions of phonon band structures in complex solids.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Co-optimizing water-alternating-carbon dioxide injection projects using a machine learning assisted computational framework

In this report, a robust machine-learning-based computational framework that couples multi-layer neural network (MLNN) proxies and a multi-objective particle swarm optimizer (MOPSO) to design water-alternating-carbon dioxide injection (CO 2 -WAG) projects is presented. The proposed optimization protocol considers various objectives, including oil recovery and CO 2 storage volume. Expert MLNN systems are trained and employed as surrogate models of the high-fidelity compositional simulator in the optimization workflow. When multiple objective functions are considered, two approaches are employed to treat the objectives: the weighted sum method and the Pareto-front-based scheme. A field-scale implementation focusing on tertiary recovery in the Morrow B formation at Farnsworth Unit (FWU) is presented. The developed Pareto-optimal solutions indicate the maximal available oil production can be 1.64 × 10 7 barrels and maximal carbon storage can achieve 2.35 × 10 7 tons. Trade-offs factor is defined to divide the constructed Pareto front into 4 sections with the trade-off factors’ value ranges from 0.35 to 49.9. This work also compares the optimum solution found by the aggregative objective function and the solution repository covered by the Pareto front that considers the physical and operational constraints and reduces uncertainties involved by the multi-objective optimization process. Our comparison indicates multiple solutions exist to satisfy the objective criteria of the WAG design, and these results cannot be found using the traditional weighted sum method. The Pareto front solution can provide more options for project designers, but decisions regarding necessary trade-offs must be made using the solution repository to balance the project economics and CO 2 storage amount.

10 SYNTHETIC FUELS↗
DOE OSTI2022

Data-driven model for divertor plasma detachment prediction

We present a fast and accurate data-driven surrogate model for divertor plasma detachment prediction leveraging the latent feature space concept in machine learning research. Our approach involves constructing and training two neural networks: an autoencoder that finds a proper latent space representation (LSR) of plasma state by compressing the multi-modal diagnostic measurements and a forward model using multi-layer perception (MLP) that projects a set of plasma control parameters to its corresponding LSR. By combining the forward model and the decoder network from autoencoder, this new data-driven surrogate model is able to predict a consistent set of diagnostic measurements based on a few plasma control parameters. In order to ensure that the crucial detachment physics is correctly captured, highly efficient 1D UEDGE model is used to generate training and validation data in this study. The benchmark between the data-driven surrogate model and UEDGE simulations shows that our surrogate model is capable of providing accurate detachment prediction (usually within a few per cent relative error margin) but with at least four orders of magnitude speed-up, indicating that performance-wise, it has the potential to facilitate integrated tokamak design and plasma control. Comparing with the widely used two-point model and/or two-point model formatting, the new data-driven model features additional detachment front prediction and can be easily extended to incorporate richer physics. This study demonstrates that the complicated divertor and scrape-off-layer plasma state has a low-dimensional representation in latent space. Understanding plasma dynamics in latent space and utilising this knowledge could open a new path for plasma control in magnetic fusion energy research.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2026

A variational framework for residual-based adaptivity in neural PDE solvers and operator learning

Residual-based adaptive strategies are widely used in scientific machine learning yet remain largely heuristic. We introduce a variational framework that formalizes these methods through convex transformations of the residual, where different transformations correspond to distinct objective functionals. For instance, exponential weights target uniform error minimization, while linear weights recover quadratic error minimization. This perspective reveals adaptive weighting as a means of selecting sampling distributions that optimize a primal objective, directly linking discretization choices to error metrics. This principled approach yields three key benefits: it enables systematic design of adaptive schemes, reduces discretization error by lowering estimator variance, and enhances learning dynamics by improving gradient signal-to-noise ratio. Extending the framework to operator learning, we demonstrate substantial performance gains across diverse optimizers and architectures. Our results provide a theoretical perspective for residual-based adaptivity and establish a foundation for principled discretization and training.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Selecting durable building envelope systems with machine learning assisted hygrothermal simulations database

Hygrothermal simulations provide insight into the energy performance and moisture durability of building envelope components under dynamic conditions. The inputs required for hygrothermal simulations are extensive, and carrying out simulations and analyses requires expert knowledge. An expert system, the Building Science Advisor (BSA), has been developed to predict the performance and select the energy-efficient and durable building envelope systems for different climates. The BSA consists of decision rules based on expert opinions and thousands of parametric simulation results for selected wall systems. The number of potential wall systems results in millions, too many to simulate all of them. We present how machine learning can help predict durability data, such as mold growth, while minimizing the number of simulations needed to run. The simulation results are used for training and validation of machine learning tools for predicting wall durability. We tested Artificial Neural Network (ANN) and Gradient Boosted Decision Trees (GBDT) for their applicability and model accuracy. Models developed with both methods showed adequate prediction performance (root mean square error of 0.195 and 0.209, respectively). Finally, we introduce how the information supports guidance for envelope design via an easy-to-use web-based tool that does not require the end-user to run hygrothermal simulations.

Salonvaara, Mikael↗