Search NASA⌕ Search

SEARCH · Search NASA

Results for “Neural Machine Learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 595 records · Page 33
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Implementing a neural network interatomic model with performance portability for emerging exascale architectures

The two main thrusts of computational science are increasingly accurate predictions and faster calculations; to this end, the zeitgeist in molecular dynamics (MD) simulations is pursuing machine learned and data driven interatomic models, e.g. neural network potentials, and novel hardware architectures, e.g. GPUs. Current implementations of neural network potentials are orders of magnitude slower than traditional interatomic models and while looming exascale computing offers the ability to run large, accurate simulations with these models, achieving portable performance for MD with new and varied exascale hardware requires rethinking traditional algorithms, using novel data structures, and library solutions. We re-implement a neural network interatomic model in CabanaMD, an MD proxy application, built on libraries developed for performance portability. Our implementation shows significantly improved thread scaling in this complex kernel as compared to a current LAMMPS implementation, across both strong and weak scaling. Our single-source solution enables simulations up to 20 million atoms on a single CPU node and 4 million atoms with improved performance on a single GPU. Furthermore, we also explore parallelism and data layout choices (using flexible data structures called AoSoAs) and their effect on performance, seeing up to ~50% and ~5% improvements in performance on a GPU by choosing the right level of parallelism and data layout respectively.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Neural Architecture and Feature Search for Predicting the Ridership of Public Transportation Routes

Accurately predicting the ridership of public-transit routes provides substantial benefits to both transit agencies, who can dispatch additional vehicles proactively before the vehicles that serve a route become crowded, and to passengers, who can avoid crowded vehicles based on publicly available predictions. The spread of the coronavirus disease has further elevated the importance of ridership prediction as crowded vehicles now present not only an inconvenience but also a public-health risk. At the same time, accurately predicting ridership has become more challenging due to evolving ridership patterns, which may make all data except for the most recent records stale. One promising approach for improving prediction accuracy is to fine-tune the hyper-parameters of machine-learning models for each transit route based on the characteristics of the particular route, such as the number of records. However, manually designing a machine-learning model for each route is a labor-intensive process, which may require experts to spend a significant amount of their valuable time. To help experts with designing machine-learning models, we propose a neural-architecture and feature search approach, which optimizes the architecture and features of a deep neural network for predicting the ridership of a public-transit route. Our approach is based on a randomized local hyper-parameter search, which minimizes both prediction error as well as the complexity of the model. We evaluate our approach on real-world ridership data provided by the public transit agency of Chattanooga, TN, and we demonstrate that training neural networks whose architectures and features are optimized for each route provides significantly better performance than training neural networks whose architectures and features are generic.

Ayman, Afiya↗
DOE OSTI2023

CCSI Toolset 3.19 Release

CCSI Toolset 3.19 Release Highlights A gradient generation tool was developed to support GENN models in FOQUS. Certain machine learning tools train gradient-enhanced neural network (GENN) models which can be more accurate for complex datasets given a priori knowledge of model derivatives. However, the derivatives must be known beforehand and are not often available for process data. This tool automatically predicts the gradients for a training dataset in a form usable by common GENN trainers, such as Surrogate Modeling Toolbox. Support was added for Surrogate Modeling Toolbox GENN models in FOQUS, including updates to the run methods, node properties, test framework, documentation and optional dependencies list. Users can train/save Surrogate Modeling Toolbox gradient-enhanced neural network (GENN) models with custom objects and produce .pkl files compatible with the Machine Learning/Artificial Intelligence Plugin in FOQUS. A simpler implementation of the ordering algorithm in the Sequential Design of Experiments (SDOE) module was included. The SDOE examples documentation was updated. The Optimality-Based Design of Experiments was updated to improve the error handling when the results are None.

AS↗
DOE OSTI2021

Deep learning classification of lipid droplets in quantitative phase images

We report the application of supervised machine learning to the automated classification of lipid droplets in label-free, quantitative-phase images. By comparing various machine learning methods commonly used in biomedical imaging and remote sensing, we found convolutional neural networks to outperform others, both quantitatively and qualitatively. We describe our imaging approach, all implemented machine learning methods, and their performance with respect to computational efficiency, required training resources, and relative method performance measured across multiple metrics. Overall, our results indicate that quantitative-phase imaging coupled to machine learning enables accurate lipid droplet classification in single living cells. As such, the present paradigm presents an excellent alternative of the more common fluorescent and Raman imaging modalities by enabling label-free, ultra-low phototoxicity, and deeper insight into the thermodynamics of metabolism of single cells.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Machine learning–based extreme event attribution

The observed increase in extreme weather has prompted recent methodological advances in extreme event attribution. We propose a machine learning–based approach that uses convolutional neural networks to create dynamically consistent counterfactual versions of historical extreme events under different levels of global mean temperature (GMT). We apply this technique to one recent extreme heat event (southcentral North America 2023) and several historical events that have been previously analyzed using established attribution methods. We estimate that temperatures during the southcentral North America event were 1.18° to 1.42°C warmer because of global warming and that similar events will occur 0.14 to 0.60 times per year at 2.0°C above preindustrial levels of GMT. Additionally, we find that the learned relationships between daily temperature and GMT are influenced by the seasonality of the forced temperature response and the daily meteorological conditions. Our results broadly agree with other attribution techniques, suggesting that machine learning can be used to perform rapid, low-cost attribution of extreme events.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2026

Machine learning for seismic low-frequency extrapolation

The cycle-skipping problem that plagues full waveform inversion (FWI) can be at least partially mitigated if low frequencies (which encode the kinematics of wave propagation in seismic data) are recorded. However, seismic sources and receivers are band-limited, so seismic data does not generally include signals down to 0 Hz. To improve our ability to solve the seismic inverse problem, one can synthesize this missing low-frequency (LF) content from the recorded high-frequency (HF) data using machine learning (ML) models. Deep learning models such as convolutional neural networks (CNNs) demonstrate impressive ability to perform low frequency extrapolation. However, such models require powerful hardware (GPU machines) and careful training. We assess the extrapolation capabilities of three different ML models that do not require GPU machines, namely, random forest, Gaussian process regression and gradient boosting, on both synthetic and real data. Experimental results on two synthetic data sets (generated from a low velocity lens embedded in a homogeneous medium, and the Marmousi model) demonstrate that FWI applied to the extrapolated data consistently improves inversion accuracy relative to FWI applied to the original data sets that do not contain low frequencies. Application of low-frequency extrapolation to real data from the Northwest Shelf of Australia demonstrates that tree-based ML models such as gradient boosting can outperform CNNs in terms of both accuracy and computational cost on non-GPU architectures.

58 GEOSCIENCES↗
DOE OSTI2025

CuXASNet: Rapid and accurate prediction of copper L-edge x-ray absorption spectra using machine learning

In this work, we have developed CuXASNet, a dense neural network that predicts simulated Cu -edge x-ray absorption spectra (XAS) from atomic structures. Featurization of the Cu local environment is performed using a component of M3GNet, a graph neural network developed for predicting the potential energy surface. CuXASNet is trained on simulated spectra from FEFF9 at the multiple scattering level of theory, and can predict the and edges for Cu sites to quantitative accuracy. To validate our approach, we compare 14 experimental spectra extracted from the literature with the predictions of CuXASNet. The agreement of CuXASNet with experiments is shown by an average mean absolute error of 0.125 and an average Spearman's correlation coefficient of 0.891, which is comparable to FEFF9's values of 0.131 and 0.898 for the same metrics. As such, CuXASNet can rapidly predict a large number of -edge XAS spectra at the same accuracy as FEFF9 simulations. This can be used as a drop-in replacement for multiple scattering codes for fast screening of candidate atomic structure models of a measured system. This model establishes a general framework for Cu XAS prediction, and can be extended to more computationally expensive levels of theory and to other transition metal edges.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Uncertainty quantification in multivariable regression for material property prediction with Bayesian neural networks

With the increased use of data-driven approaches and machine learning-based methods in material science, the importance of reliable uncertainty quantification (UQ) of the predicted variables for informed decision-making cannot be overstated. UQ in material property prediction poses unique challenges, including multi-scale and multi-physics nature of materials, intricate interactions between numerous factors, limited availability of large curated datasets, etc. In this work, we introduce a physics-informed Bayesian Neural Networks (BNNs) approach for UQ, which integrates knowledge from governing laws in materials to guide the models toward physically consistent predictions. To evaluate the approach, we present case studies for predicting the creep rupture life of steel alloys. Experimental validation with three datasets of creep tests demonstrates that this method produces point predictions and uncertainty estimations that are competitive or exceed the performance of conventional UQ methods such as Gaussian Process Regression. Additionally, we evaluate the suitability of employing UQ in an active learning scenario and report competitive performance. The most promising framework for creep life prediction is BNNs based on Markov Chain Monte Carlo approximation of the posterior distribution of network parameters, as it provided more reliable results in comparison to BNNs based on variational inference approximation or related NNs with probabilistic outputs.

36 MATERIALS SCIENCE↗
DOE OSTI2019

Machine-learning error models for approximate solutions to parameterized systems of nonlinear equations

Here, this work proposes a machine-learning framework for constructing statistical models of errors incurred by approximate solutions to parameterized systems of nonlinear equations. These approximate solutions may arise from early termination of an iterative method, a lower-fidelity model, or a projection-based reduced-order model, for example. The proposed statistical model comprises the sum of a deterministic regression-function model and a stochastic noise model. The method constructs the regression-function model by applying regression techniques from machine learning (e.g., support vector regression, artificial neural networks) to map features (i.e., error indicators such as sampled elements of the residual) to a prediction of the approximate-solution error. The method constructs the noise model as a mean-zero Gaussian random variable whose variance is computed as the sample variance of the approximate-solution error on a test set; this variance can be interpreted as the epistemic uncertainty introduced by the approximate solution. This work considers a wide range of feature-engineering methods, data-set-construction techniques, and regression techniques that aim to ensure that (1) the features are cheaply computable, (2) the noise model exhibits low variance (i.e., low epistemic uncertainty introduced), and (3) the regression model generalizes to independent test data. Finally, numerical experiments performed on several computational-mechanics problems and types of approximate solutions demonstrate the ability of the method to generate statistical models of the error that satisfy these criteria and significantly outperform more commonly adopted approaches for error modeling.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

A neural network for beam background decomposition in Belle II at SuperKEKB

Here, we describe a neural network for predicting the background hit rate in the Belle II detector produced by the SuperKEKB electron-positron collider. The neural network, BGNet, learns to predict the individual contributions of different physical background sources, such as beam-gas scattering or continuous top-up injections into the collider, to Belle II sub-detector rates. The samples for learning are archived 1 Hz time series of diagnostic variables from the SuperKEKB collider subsystems and measured hit rates of Belle II used as regression targets. We test the learned model by predicting detector hit rates on archived data from different run periods not used during training. We show that a feature attribution method can help interpret the source of changes in the background level over time.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2018

The TENNLab Exploratory Neuromorphic Computing Framework

Spiking, neuromorphic computing systems are in a period of active exploration by the computing community. While they feature computational expressiveness beyond both von Neumann computing models and feed-forward neural networks, they are also challenging to design and program. The TENNLab exploratory neuromorphic computing framework is a software infrastructure, soon to be open-source, whose goal is to enable potential users of spiking, neuromorphic computing systems to develop applications and evaluate computing architectures, and for architecture researchers to develop and evaluate their architectures with a variety of applications. In this letter, we present the software architecture of the TENNLab framework.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

A workflow for segmenting soil and plant X-ray computed tomography images with deep learning in Google’s Colaboratory

X-ray micro-computed tomography (X-ray μCT) has enabled the characterization of the properties and processes that take place in plants and soils at the micron scale. Despite the widespread use of this advanced technique, major limitations in both hardware and software limit the speed and accuracy of image processing and data analysis. Recent advances in machine learning, specifically the application of convolutional neural networks to image analysis, have enabled rapid and accurate segmentation of image data. Yet, challenges remain in applying convolutional neural networks to the analysis of environmentally and agriculturally relevant images. Specifically, there is a disconnect between the computer scientists and engineers, who build these AI/ML tools, and the potential end users in agricultural research, who may be unsure of how to apply these tools in their work. Additionally, the computing resources required for training and applying deep learning models are unique, more common to computer gaming systems or graphics design work, than to traditional computational systems. To navigate these challenges, we developed a modular workflow for applying convolutional neural networks to X-ray μCT images, using low-cost resources in Google’s Colaboratory web application. Here we present the results of the workflow, illustrating how parameters can be optimized to achieve best results using example scans from walnut leaves, almond flower buds, and a soil aggregate. We expect that this framework will accelerate the adoption and use of emerging deep learning techniques within the plant and soil sciences.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Backpropagation-based learning with local derivative approximation and memory replay in biologically plausible neural systems

When learning, the brain modifies individual synaptic connections to reach a desired behavior. Animal and human brains have been shown to be incredibly capable of learning complex and varied functions across a wide variety of tasks. In recent years, artificial neural networks, inspired by human and animal brains, have shown great capabilities in learning a wide variety of difficult tasks. However, artificial neural networks primarily teach themselves through the use of backpropagation, a learning method which has no clear analogue within the brain. Additionally, Artificial Neural Networks primarily use continuous activation functions, which differ significantly from the spiking neuronal behavior present in the brain. In this paper, we discuss and demonstrate a biologically plausible learning method that approximates backpropagation through two techniques on Spiking Neural Networks. First, we show that the local temporal derivatives that are necessary for backpropagation can be approximately recovered through reconstruction using spike timings. Second, we show that through learning during a sleep phase, inspired by neuroscience research into memory replay, the localized parallel feedback path can learn to approximate the derivative through the forward path weight matrix, thus solving the weight transport problem. Lastly, we demonstrate that the combination of these two methods can approach or exceed the accuracy of backpropagation-based methods for a variety of neuromorphic vision tasks while maintaining biological plausibility.

42 ENGINEERING↗
DOE OSTI2021

Hierarchical Convolutional Neural Networks for Event Classification on PMU Measurements

Event classification is one of the central components of automated disturbance analysis based on PMU measurements. Obtaining high-quality event labels remains a challenge for supervised learning-based classification of local and system-wide events in power grids due to its labor-intensive requirement. We present a sensitivity study considering rapidly refined, partially and fully inspected event labels that leads to evidence that hierarchical convolutional neural networks (HCNNs) outperform traditional classification models regardless of the quality of the available event labels. Furthermore, it is demonstrated that performance similar to the one obtained using entirely domain-driven labeling can be achieved as long as the involved expert does not mislabel more than ~5% of the event data captured by PMU measurements.

47 OTHER INSTRUMENTATION↗
DOE OSTI2024

A Graph Neural Network Surrogate Model for hls4ml

Recent advancements in use of machine learning (ML) techniques on field-programmable gate arrays (FPGAs) have allowed for the implementation of embedded neural networks with extremely low latency. This is invaluable for particle detectors at the Large Hadron Collider, where latency and used area are strictly bounded. The hls4ml framework is a procedure that converts trained ML model software to a synthesis result to can be used on an FPGA. However, running the pipeline is a time-consuming procedure, and there is a strong risk of failure. In particular, it may not be possible to successfully convert a model into a synthesis result, or the resource consumption of the model may exceed the resources of the target FPGA. To aid with this development, we introduce wa-hls4ml, a surrogate model using a graph neural network to emulate the structure of the source models. The goal is to estimate the chance of success and resource consumption of a given model when passed through the hls4ml pipeline, without needing to run the pipeline.

Plotnikov, Dennis↗
DOE OSTI2022

Z-Contrast Enhancement in Au–Pt Nanocatalysts by Correlative X-ray Absorption Spectroscopy and Electron Microscopy: Implications for Composition Determination

The properties of bimetallic nanoparticles (BNPs) vary widely as a function of their composition and size distributions. X-ray absorption fine structure analysis is commonly used to characterize their structure, but its application to elements that are close to each other in the periodic table is hampered by poor Z-contrast. For this study, we trained an artificial neural network to recognize the partial coordination numbers in AuPt NPs synthesized via peptide templating using their X-ray absorption near-edge structure spectra. This approach, combined with scanning transmission electron microscopy analysis, revealed unique details of this prototype catalytic system that has different forms of heterogeneities at both the intra- and inter-particle levels. Our method based on the enhancement of Z-contrast of metal species will have implications for compositional studies of BNPs.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Real-time capable modeling of ICRF heating on NSTX and WEST via machine learning approaches

Abstract A real-time capable core Ion Cyclotron Range of Frequencies (ICRF) heating model on NSTX and WEST is developed. The model is based on two nonlinear regression algorithms, the random forest ensemble of decision trees and the multilayer perceptron neural network. The algorithms are trained on TORIC ICRF spectrum solver simulations of the expected flat-top operation scenarios in NSTX and WEST assuming Maxwellian plasmas. The surrogate models are shown to successfully capture the multi-species core ICRF power absorption predicted by the original model for the high harmonic fast wave and the ion cyclotron minority heating schemes while reducing the computational time by six orders of magnitude. Although these models can be expanded, the achieved regression scoring, computational efficiency and increased model robustness suggest these strategies can be implemented into integrated modeling frameworks for real-time control applications.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗