Search NASA⌕ Search

SEARCH · Search NASA

Results for “Neural Networks (Computer)”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 379 records · Page 21
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Resolving the Solvation Structure and Transport Properties of Aqueous Zinc Electrolytes from Salt-in-Water to Water-in-Salt Using Neural Network Potential

Zn Cl 2 solutions are promising electrolytes for aqueous zinc-ion batteries. Here, we report a joint computational and experimental study of the structural and dynamic properties of aqueous Zn Cl 2 electrolytes with concentrations ranging from salt-in-water to water-in-salt (WIS). By developing a neural network potential (NNP) model, we perform molecular dynamics (MD) simulations with accuracy but at much larger lengths and longer timescales. The NNP predicted structures are validated by the structure factors measured by X-ray total scattering experiments. The MD trajectories provide a comprehensive and quantitative picture of the Zn 2 + solvation shell structures. Additionally, we find that the O − H covalent bonds in water are strengthened with increasing salt concentration, thus expanding the electrochemical stability window of aqueous electrolytes. In terms of dynamic properties, the calculated and experimentally measured conductivities are in good agreement. Through the analysis of the calculated cation transference number, we propose a three-stage charge carrier transport mechanism with increasing concentration: independent ion transport, strongly correlated ion transport, and small positive charge carrier diffusion through negatively charged polymeric clusters. Our study provides fundamental atomic scale insights into the structure and transport properties of the Zn Cl 2 electrolyte that can aid the optimization and development of WIS electrolytes. Published by the American Physical Society 2025

25 ENERGY STORAGE↗
NASA NTRS2004

Neural network uncertainty assessment using Bayesian statistics: a remote sensing application

Neural network (NN) techniques have proved successful for many regression problems, in particular for remote sensing; however, uncertainty estimates are rarely provided. In this article, a Bayesian technique to evaluate uncertainties of the NN parameters (i.e., synaptic weights) is first presented. In contrast to more traditional approaches based on point estimation of the NN weights, we assess uncertainties on such estimates to monitor the robustness of the NN model. These theoretical developments are illustrated by applying them to the problem of retrieving surface skin temperature, microwave surface emissivities, and integrated water vapor content from a combined analysis of satellite microwave and infrared observations over land. The weight uncertainty estimates are then used to compute analytically the uncertainties in the network outputs (i.e., error bars and correlation structure of these errors). Such quantities are very important for evaluating any application of an NN model. The uncertainties on the NN Jacobians are then considered in the third part of this article. Used for regression fitting, NN models can be used effectively to represent highly nonlinear, multivariate functions. In this situation, most emphasis is put on estimating the output errors, but almost no attention has been given to errors associated with the internal structure of the regression model. The complex structure of dependency inside the NN is the essence of the model, and assessing its quality, coherency, and physical character makes all the difference between a blackbox model with small output errors and a reliable, robust, and physically coherent model. Such dependency structures are described to the first order by the NN Jacobians: they indicate the sensitivity of one output with respect to the inputs of the model for given input data. We use a Monte Carlo integration procedure to estimate the robustness of the NN Jacobians. A regularization strategy based on principal component analysis is proposed to suppress the multicollinearities in order to make these Jacobians robust and physically meaningful.

Neural Networks (Computer)↗
NASA NTRS2019

A Neural Network Correction to the Scalar Approximation in Radiative Transfer

The next generation of advanced high-resolution sensors in geostationary orbit will gather detailed information for studying the Earth system. There is an increasing desire to perform observing system simulation experiments (OSSEs) for new sensors during the development phase of the mission in order to better leverage information content from the new and existing sensors. Forward radiative transfer calculations that simulate the observing characteristics of a new instrument are the first step to an OSSE, and they are computationally intensive. The scalar approximation to the radiative transfer equation, a simplification of the vector representation, can save considerable computational cost, but produces errors in top of the atmosphere (TOA) radiance as large as 10% due to neglecting polarization effects. This article presents an artificial neural network technique to correct scalar TOA radiance over both land and ocean surfaces to within 1% of vector-calculated radiance. A neural network was trained on a database of scalar-vector TOA radiance differences at a large range of solar and viewing angles for several thousand realistic atmospheric vertical profiles that were sampled from a high resolution (7 km) global atmospheric transport model. The profiles include Rayleigh scattering and aerosol scattering and absorption. Training and validation of the neural network was demonstrated for two wavelengths in the ultraviolet-visible (US-Vis) spectral range (354 nm and 670 nm). The significant computational savings accrued from using a scalar approximation plus neural network correction approach to simulating TOA radiance will make feasible hyperspectral forward simulations of high-resolution sensors on geostationary satellites, such as TEMPO, GOES-R, GEMS, and SENTINEL-4.

TOA↗
DOE OSTI2024

Deep learning with mixup augmentation for improved pore detection during additive manufacturing

In additive manufacturing (AM), process defects such as keyhole pores are difficult to anticipate, affecting the quality and integrity of the AM-produced materials. Hence, considerable efforts have aimed to predict these process defects by training machine learning (ML) models using passive measurements such as acoustic emissions. This work considered a dataset in which keyhole pores of a laser powder bed fusion (LPBF) experiment were identified using X-ray radiography and then registered both in space and time to acoustic measurements recorded during the LPBF experiment. Due to AM’s intrinsic process controls, where a pore-forming event is relatively rare, the acoustic datasets collected during monitoring include more non-pores than pores. In other words, the dataset for ML model development is imbalanced. Moreover, this imbalanced and sparse data phenomenon remains ubiquitous across many AM monitoring schemes since training data is nontrivial to collect. Hence, we propose a machine learning approach to improve this dataset imbalance and enhance the prediction accuracy of pore-labeled data. Specifically, we investigate how data augmentation helps predict pores and non-pores better. This imbalance is improved using recent advances in data augmentation called Mixup, a weak-supervised learning method. Convolutional neural networks (CNNs) are trained on original and augmented datasets, and an appreciable increase in performance is reported when testing on five different experimental trials. When ML models are trained on original and augmented datasets, they achieve an accuracy of 95% and 99% on test datasets, respectively. We also provide information on how dataset size affects model performance. Lastly, we investigate the optimal Mixup parameters for augmentation in the context of CNN performance.

42 ENGINEERING↗
DOE OSTI2024

Neural Network Prediction of Strong Lensing Systems with Domain Adaptation and Uncertainty Quantification

Modeling strong gravitational lenses is computationally expensive for the complex data from modern and next-generation cosmic surveys. Deep learning has emerged as a promising approach for finding lenses and predicting lensing parameters, such as the Einstein radius. Mean-variance Estimators (MVEs) are a common approach for obtaining aleatoric (data) uncertainties from a neural network prediction. However, neural networks have not been demonstrated to perform well on out-of-domain target data successfully - e.g., when trained on simulated data and applied to real, observational data. In this work, we perform the first study of the efficacy of MVEs in combination with unsupervised domain adaptation (UDA) on strong lensing data. The source domain data is noiseless, and the target domain data has noise mimicking modern cosmology surveys. We find that adding UDA to MVE increases the accuracy on the target data by a factor of about two over an MVE model without UDA. Including UDA also permits much more well-calibrated aleatoric uncertainty predictions. Advancements in this approach may enable future applications of MVE models to real observational data.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
NASA NTRS2000

Intelligence Applied to Air Vehicles

The exponential growth in information technology has provided the potential for air vehicle capabilities that were previously unavailable to mission and vehicle designers. The increasing capabilities of computer hardware and software, including new developments such as neural networks, provide a new balance of work between humans and machines. This paper will describe several NASA projects, and review results and conclusions from ground and flight investigations where vehicle intelligence was developed and applied to aeronautical and space systems. In the first example, flight results from a neural network flight control demonstration will be reviewed. Using, a highly-modified F-15 aircraft, a NASA/Dryden experimental flight test program has demonstrated how the neural network software can correctly identify and respond to changes in aircraft stability and control characteristics. Using its on-line learning capability, the neural net software would identify that something in the vehicle has changed, then reconfigure the flight control computer system to adapt to those changes. The results of the Remote Agent software project will be presented. This capability will reduce the cost of future spacecraft operations as computers become "thinking" partners along with humans. In addition, the paper will describe the objectives and plans for the autonomous airplane program and the autonomous rotorcraft project. Technologies will also be developed.

Rosen, Robert↗
NASA NTRS2022

A Deep Learning Approach to Fast Radiative Transfer

Due to the sheer volume of data, leveraging satellite instrument observations effectively in a data assimilation context for numerical weather prediction or for remote sensing requires a radiative transfer model as an observation operator that is both fast and accurate at the same time. Physics-based line-by-line radiative transfer (RT) models fulfil the requirement for accuracy, but are too slow and too costly in computational terms for operational applications. Therefore, fast methods were developed to be able to perform fast RT calculations using techniques such as spectral sampling or pre-computed look-up tables. The operational fast models currently calculate the absorption and scattering coefficients from the pre-computed regression coefficients and atmospheric state and cloud profiles. As a novel solution to this problem, this work investigates a deep learning approach to replace the regression coefficients in the fast RT models. A selection of hidden-layer neural network configurations is trained against atmospheric transmittance profile data computed by an accurate line-by-line model and their performance is evaluated and their advantages and disadvantages are discussed.

Machine learning↗
DOE OSTI2026

Measuring the Energy Consumption and Efficiency of Deep Neural Networks: An Empirical Analysis and Design Recommendations

Addressing the "Red-AI" trend of rising energy consumption by large-scale neural networks, this study investigates the measured energy consumption of training various fully connected neural network architectures. We introduce the BUTTER-E dataset, an augmentation to the BUTTER Empirical Deep Learning dataset, containing energy consumption and performance data from 41,129 individual experimental runs spanning 30,582 distinct configurations: 13 datasets, 20 sizes (trainable parameters), 8 "shapes", and 14 depths on both CPUs and GPUs using node-level watt-meters. This dataset reveals the complex relationship between dataset size, network structure, and energy use. Our analysis uncovers a surprising, hardware-mediated non-linear relationship between energy efficiency and network design, challenging the assumption that reducing the number of parameters or FLOPs is the best way to achieve greater energy efficiency. We propose a straightforward and effective energy model that accounts for network size, computing, and memory hierarchy. Highlighting the need for cache-considerate algorithm development, we suggest a codesign approach to energy efficient network, algorithm, and hardware design. This work contributes to the fields of sustainable computing and Green AI, offering practical guidance for creating more energy-efficient neural networks and promoting sustainable AI.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Anticipating Technical Expertise and Capability Evolution in Research Communities Using Dynamic Graph Transformers

The ability to anticipate global technical expertise and capability evolution trends is essential for national and global security, especially in safety-critical domains such as nuclear nonproliferation (NN) and rapidly emerging fields like artificial intelligence (AI). Here, in this work, we extend traditional statistical relational learning approaches (e.g., link prediction in collaboration networks) and formulate a problem of anticipating technical expertise and capability evolution using dynamic heterogeneous graph representations. We develop novel capabilities to forecast collaboration patterns, authorship behavior, and technical capability evolution at different granularities (e.g., scientist and institution levels) in two distinct research fields. We implement a dynamic graph transformer (DGT) neural architecture, which pushes the state-of-the-art graph neural network models by: 1) forecasting heterogeneous (rather than homogeneous) nodes and edges; and 2) relying on both discrete- and continuous-time inputs. We demonstrate that our DGT models predict collaboration, partnership, and expertise patterns with 0.26, 0.73, and 0.53 mean reciprocal rank values for AI and 0.48, 0.93, and 0.22 for NN domains. DGT model performance exceeds the best-performing static graph baseline models by 30%–80% across AI and NN domains. Our findings demonstrate that DGT models boost inductive task performance when previously unseen nodes appear in the test data for the domains with emerging collaboration patterns (e.g., AI). Specifically, models accurately predict which established scientists will collaborate with early career scientists and vice versa in the AI domain.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1992

Recognition of partially occluded threat objects using the annealed Hopefield network

Recognition of partially occluded objects has been an important issue to airport security because occlusion causes significant problems in identifying and locating objects during baggage inspection. The neural network approach is suitable for the problems in the sense that the inherent parallelism of neural networks pursues many hypotheses in parallel resulting in high computation rates. Moreover, they provide a greater degree of robustness or fault tolerance than conventional computers. The annealed Hopfield network which is derived from the mean field annealing (MFA) has been developed to find global solutions of a nonlinear system. In the study, it has been proven that the system temperature of MFA is equivalent to the gain of the sigmoid function of a Hopfield network. In our early work, we developed the hybrid Hopfield network (HHN) for fast and reliable matching. However, HHN doesn't guarantee global solutions and yields false matching under heavily occluded conditions because HHN is dependent on initial states by its nature. In this paper, we present the annealed Hopfield network (AHN) for occluded object matching problems. In AHN, the mean field theory is applied to the hybird Hopfield network in order to improve computational complexity of the annealed Hopfield network and provide reliable matching under heavily occluded conditions. AHN is slower than HHN. However, AHN provides near global solutions without initial restrictions and provides less false matching than HHN. In conclusion, a new algorithm based upon a neural network approach was developed to demonstrate the feasibility of the automated inspection of threat objects from x-ray images. The robustness of the algorithm is proved by identifying occluded target objects with large tolerance of their features.

Kim, Jung H.↗
DOE OSTI2025

Spin-informed universal graph neural networks for simulating magnetic ordering

The screening and discovery of magnetic materials are hindered by the computational cost of first-principles density-functional theory (DFT) calculations required to find the ground state magnetic ordering. Although universal machine-learning interatomic potentials (uMLIPs), also known as atomistic foundation models, offer high-fidelity models of many atomistic systems with significant speedup, they currently lack the inputs required for predicting magnetic ordering. In this work, we present a data-efficient, spin-informed graph neural network framework that incorporates spin degrees of freedom as inputs and preserves physical symmetries, extending the functionality of uMLIPs to simulate magnetic orderings. This framework speeds up DFT calculations through better initial guesses for magnetic moments, determines the ground-state ordering of bulk materials and even generalizes to magnetic ordering in surfaces. Furthermore, we implement a closed-loop anomaly detection approach that effectively addresses the classic "chicken-and-egg" problem of creating a high-quality dataset while developing a uMLIP, unearthing anomalies in large benchmark datasets and boosting model accuracy.

Xu, Wenbin↗
DOE OSTI2024

Scalable Risk Assessment of Rare Events in Power Systems With Uncertain Wind Generation and Loads

Risk assessment of rare events has become increasingly important in power system planning and operation with the increasing integration of renewable energy and the presence of system uncertainties. However, quantifying the risk posed by rare events via the traditional method, i.e., Monte Carlo sampling (MCS), incurs substantial computational expense stemming from the vast ensemble of power flow simulations. To accelerate the assessment, this paper proposes a Deep Neural Network (DNN)-kernelized vector-valued Gaussian Process (VVGP) approach with excellent computational efficiency while maintaining high accuracy. Consequently, serving as a surrogate model for the power flow solver, the DNN-kernelized VVGP enables significantly faster but accurate risk assessment compared to the power flow solver. The developed surrogate model evaluates low-order N - k events that contain more than 90% instances by adeptly capturing the topological features while the high-order N - k events are assessed via a power flow solver, thereby striking a balance between computational efficiency and uncertainty quantification accuracy. Moreover, the model incorporates a Support Vector Machine (SVM) classifier to resample concerning low-probability tail events to counteract the biases potentially introduced during the DNN-kernelized VVGP evaluations. Simulations conducted on the modified IEEE 24-bus, 118-bus, and European 1354-bus systems demonstrate that the proposed method maintains the accuracy benchmark set by MCS while significantly reducing computational demands in large-scale power systems as compared to other state-of-the-art methods.

17 WIND ENERGY↗
DOE OSTI2024

Effects of input gradient regularization on neural networks time-series forecasting of thermal power systems

This study proposes using neural networks, specifically gated recurrent unit (GRU), long-short-term memory (LSTM), and transformer networks, to improve control strategies in a 450 MW coal-fired power plant. However, neural networks face issues of becoming overly dependent on just a few variables to make predictions, which negatively impacts control decisions that rely on the model to determine the value of all manipulated variables. The paper introduces regularization techniques, including noise injection and input gradient regularization, during the training phase. Here, the work presents novel contributions in adapting neural networks to control industrial systems and applying regularization techniques from computer vision to industrial process control. Results demonstrate the effectiveness of input gradient regularization in reducing model dependence on subsets of variables, emphasizing the balance between fidelity and controllability. Further exploration is recommended, including the development of recurrent transformers, closed-loop control testing, and a sensitivity analysis on computer models to provide further insight.

20 FOSSIL-FUELED POWER PLANTS↗
NASA NTRS1988

Introduction to a system for implementing neural net connections on SIMD architectures

Neural networks have attracted much interest recently, and using parallel architectures to simulate neural networks is a natural and necessary application. The SIMD model of parallel computation is chosen, because systems of this type can be built with large numbers of processing elements. However, such systems are not naturally suited to generalized communication. A method is proposed that allows an implementation of neural network connections on massively parallel SIMD architectures. The key to this system is an algorithm permitting the formation of arbitrary connections between the neurons. A feature is the ability to add new connections quickly. It also has error recovery ability and is robust over a variety of network topologies. Simulations of the general connection system, and its implementation on the Connection Machine, indicate that the time and space requirements are proportional to the product of the average number of connections per neuron and the diameter of the interconnection network.

Tomboulian, Sherryl↗
NASA NTRS1988

Introduction to a system for implementing neural net connections on SIMD architectures

Neural networks have attracted much interest recently, and using parallel architectures to simulate neural networks is a natural and necessary application. The SIMD model of parallel computation is chosen, because systems of this type can be built with large numbers of processing elements. However, such systems are not naturally suited to generalized elements. A method is proposed that allows an implementation of neural network connections on massively parallel SIMD architectures. The key to this system is an algorithm permitting the formation of arbitrary connections between the neurons. A feature is the ability to add new connections quickly. It also has error recovery ability and is robust over a variety of network topologies. Simulations of the general connection system, and its implementation on the Connection Machine, indicate that the time and space requirements are proportional to the product of the average number of connections per neuron and the diameter of the interconnection network.

Tomboulian, Sherryl↗
DOE OSTI2025

Resistive Switching of Spinel Li 4 Ti 5 O 12 Lithium-Ion Battery Material for Neuromorphic Computing

The rapid rise of AI has exposed significant limitations in conventional Von Neumann computing architecture, particularly in regard to speed and energy efficiency. To address these challenges, researchers are exploring a brain-inspired neuromorphic architecture that mimics biological neural networks, enabling massive parallel processing with reduced power consumption for complex AI computational demands. Recent interest has focused on utilizing battery electrodes and solid electrolyte materials for their resistive switching properties in developing a neuromorphic architecture. These properties are precisely tuned through local- and bulk-level chemical composition modifications via voltage bias stimuli. In this study, we demonstrate fabricating a three-terminal lithium-ion electrochemical transistor based on lithium titanium oxide (Li 4 Ti 5 O 12 ), a popular lithium-ion battery anode material. We deposited and characterized LTO thin films using RF sputtering, demonstrating a 6 orders of magnitude increase in electronic conductivity upon lithiation, with conductivity plateauing after 20% lithiation. Density functional theory calculations revealed transformation from the insulating to conducting state, supported by experimental characterization through X-Ray Photoelectron Spectroscopy (XPS) and Direct Current (DC) polarization analyses. The fabricated transistor consisted of LTO as the channel layer, gold as source/drain terminals, lithium phosphorus oxynitride (LiPON) as the lithium-ion conductor, and copper as the gate terminal. The device exhibited clear hysteresis in transfer characteristics due to lithium insertion/extraction processes. Long-term potentiation (LTP) and long-term depression (LTD) measurements showed an asymmetric ratio of 1.425 and maximum/minimum conductance ratio of 7.83. When implemented in a deep neural network (DNN) for MNIST handwritten digit recognition, the device achieved 92.03% accuracy over 20 training epochs. Detailed transport mechanism analysis revealed the crucial role of oxygen vacancies and interface effects in device operation. Our preliminary findings establish LTO-based lithium-ion electrochemical transistors as promising candidates for energy-efficient neuromorphic computing applications, offering potential solutions to traditional Von Neumann architecture limitations.

25 ENERGY STORAGE↗
DOE OSTI2025

Advancements on Multi-Fidelity Random Fourier Neural Networks: Application to Hurricane Modeling for Wind Energy

Multi-fidelity approaches are emerging as effective strategies in computational science to handle otherwise intractable tasks like Uncertainty Quantification (UQ), training of Machine Learning (ML) models, and optimization, for expensive high-fidelity applications in which the amount of available simulations or data is limited. The main idea is simple: large datasets generated for low-fidelity approximations of the problem at hand are fused with a much sparser dataset for the target (high-fidelity) system. In this paper, we build on our recent success in designing random Fourier Neural Networks (rFNNs) [1] to target problems arising in wind energy applications and in particular problems of interest for hurricane modeling. In this context, data for the high-fidelity models are limited and lower fidelity alternatives are needed. In this work, we introduce a novel multi-fidelity training approach for our rFNNs and demonstrate its use on a simple verification problem and on a hurricane modeling problem in which high-fidelity data are generated via Large-Eddy Simulations (LES), while low-fidelity data are given by a mesoscale model. Initial results demonstrate how the multi-fidelity training approach can improve the quality of the resulting surrogate.

Fourier Neural Networks↗
DOE OSTI2025

TorchBraid: High-Performance Layer-Parallel Training of Deep Neural Networks with MPI and GPU Acceleration

TorchBraid is a high-performance implementation of layer-parallel training for deep neural networks (DNNs) supporting MPI-based parallelism and GPU acceleration. Layer-parallel training has been developed to overcome the serialization inherent in forward and backward propagation of DNNs that limits utilization of computational resources in the strong scaling limit. To achieve this, TorchBraid integrates the PyTorch neural network framework with the state-of-the-art XBraid time-parallel library. Furthermore, this article presents the use and performance of TorchBraid, in addition to solutions for overcoming the algorithmic challenges inherent in combining automatic differentiation with layer-parallel. Results are presented with and without GPU acceleration for the Tiny ImageNet and MNIST image classification data sets, as well as recurrent neural networks. Overall, TorchBraid enables fast training of DNNs, both in a strong and weak scaling context. In addition to the TorchBraid software, several new advances in applying layer-parallel algorithms are detailed. Integration of layer-parallel with data-parallel algorithms is presented for the first time, showing the computational advantages of the combination. Standard deep learning techniques, like batch-normalization, are developed for layer-parallel training. Finally, a new approach combining layer-parallel with spatial coarsening in order to accelerate training for 3D image classification shows roughly a 10× speedup over serial execution.

Layer-parallel↗