Search NASA⌕ Search

SEARCH · Search NASA

Results for “NEURAL NETWORKS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,441 records · Page 80

Adaptive coupling of a deep neural network potential to a classical force field

An adaptive modeling method (AMM) that couples a deep neural network potential and a classical force field is introduced to address the accuracy-efficiency dilemma faced by the molecular simulation community. The AMM simulated system is decomposed into three types of regions. The first type captures the important phenomena in the system and requires high accuracy, for which we use the Deep Potential Molecular Dynamics (DeePMD) model in this work. The DeePMD model is trained to accurately reproduce the statistical properties of the ab initio molecular dynamics. The second type does not require high accuracy, and a classical force field is used to describe it in an efficient way. The third type is used for a smooth transition between the first and the second types of regions. By using a force interpolation scheme and imposing a thermodynamics force in the transition region, we make the DeePMD region embedded in the AMM simulated system as if it were embedded in a system that is fully described by the accurate potential. A representative example of the liquid water system is used to show the feasibility and promise of this method.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Graph Neural Network for Object Reconstruction in Liquid Argon Time Projection Chambers

The Exa.TrkX project presents a graph neural network (GNN) technique for low-level reconstruction of neutrino interactions in a Liquid Argon Time Projection Chamber (LArTPC). GNNs are still a relatively novel technique, and have shown great promise for similar reconstruction tasks in the Large Hadron Collider (LHC). Graphs describing particle interactions are formed by treating each detector hit as a node, with edges describing the relationships between hits. We utilise a multi-head attention message passing network which performs graph convolutions in order to label each node with a particle type.We present an updated variant of our GNN architecture, with several improvements. After testing the model on more realistic simulation with regions of unresponsive wires, the target was modified from edge classification to node classification in order to increase robustness. Removing edges as a classification target opens up a broader possibility space for edge-forming techniques; we explore the model’s performance across a variety of approaches, such as Delaunay triangulation, kNN, and radius-based methods. We also extend this model to the 3D context, sharing information between detector views. By using reconstructed 3D spacepoints to map detector hits from each wire plane, the model naively constructs 2D representations that are independent yet fully consistent.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Structures of neural network effective theories

We develop a diagrammatic approach to effective field theories (EFTs) corresponding to deep neural networks at initialization, which dramatically simplifies computations of finite-width corrections to neuron statistics. The structures of EFT calculations make it transparent that a single condition governs criticality of all connected correlators of neuron preactivations. Understanding of such EFTs may facilitate progress in both deep learning and field theory simulations. Published by the American Physical Society 2024

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Neural network based emulation of galaxy power spectrum covariances: A reanalysis of BOSS DR12 data

We train neural networks to quickly generate redshift-space galaxy power spectrum covariances from a given parameter set (cosmology and galaxy bias). This covariance emulator utilizes a combination of traditional fully connected network layers and transformer architecture to accurately predict covariance matrices for the high redshift, north galactic cap sample of the BOSS DR12 galaxy catalog. We run simulated likelihood analyses with emulated and brute-force computed covariances, and we quantify the network’s performance via two different metrics: (1) difference in Χ 2 and (2) likelihood contours for simulated BOSS DR 12 analyses. We find that the emulator returns excellent results over a large parameter range. We then use our emulator to perform a reanalysis of the BOSS HighZ NGC galaxy power spectrum, and find that varying covariance with cosmology along with the model vector produces Ω m = $0.27⁢6$$^{+0.013}_{–0.015}$, H 0 = 70.2 ± 1.9 km/s/Mpc, and σ 8 = $0.67⁢4$$^{+0.058}_{–0.077}$. These constraints represent an average 0.46⁢σ shift in best-fit values and a 5% increase in constraining power compared to fixing the covariance matrix (Ω m = 0.293 ± 0.017, H 0 = 70.3 ± 2.0 km/s/Mpc, σ 8 = $0.70⁢2$$^{+0.063}_{–0.075}$). As a result, this work demonstrates that emulators for more complex cosmological quantities than second-order statistics can be trained over a wide parameter range at sufficiently high accuracy to be implemented in realistic likelihood analyses.

79 ASTRONOMY AND ASTROPHYSICS↗

Efficient I/O for Neural Network Training with Compressed Data

FanStore is a shared object store that enables efficient and scalable neural network training on supercomputers. By providing a global cache layer on node-local burst buffers using a compressed representation, it significantly enhances the processing capability of deep learning (DL) applications on existing hardware. In addition, FanStore allows POSIX-compliant file access to the compressed data in user space. We investigate the tradeoff between runtime overhead and data compression ratio using real-world datasets and applications, and propose a compressor selection algorithm to maximize storage capacity given performance constraints. We consider both asynchronous (i.e., with prefetching) and synchronous I/O strategies, and propose mechanisms for selecting compressors for both approaches. Using FanStore, the same storage hardware can host 2-13x more data for example applications without significant runtime overhead. Empirically, our experiments show that FanStore scales to 512 compute nodes with near linear performance scalability.

Zhang, Zhao↗

Physics Informed Neural Networks as Computational Physics Emulators

This report is a brief overview and evaluation of Physics Informed Neural Networks (PINNs). Karniadakis and co-workers, e.g., Karniadakis et al. (2021) assert that the PINNs approach integrates seamlessly both data and mathematical physics models, even in partially understood, uncertain and high-dimensional contexts. They further claim that PINNs are effective and efficient for ill-posed and inverse problems, and when combined with domain decomposition, are scalable to large problems and a tool to discover hidden physics. While they demonstrate the capabilities in specific academic instances, their overarching claims about PINNs seem to be an overstatement, at least at the current time. We have briefly considered a few of the limitations of PINNs in this investigation. It is not clear to us if the PINNs approach can ever be competitive with approaches that use specialized algorithms to achieve high-accuracy solutions of governing equations and other techniques that can combine observational data with such solutions. As an example of the latter, consistent with the principles of Bayesian inference, data assimilation, or more generally data-model fusion, is a process that fuses observational data typically with a computational model that respects certain constraints such as conservation laws. For example, improvements in observational network combined with data assimilation have been key in improving weather predictions over the past four decades Kalnay (2003).

97 MATHEMATICS AND COMPUTING↗

Neural network based feed-forward high density associative memory

A novel thin film approach to neural-network-based high-density associative memory is described. The information is stored locally in a memory matrix of passive, nonvolatile, binary connection elements with a potential to achieve a storage density of 10 to the 9th bits/sq cm. Microswitches based on memory switching in thin film hydrogenated amorphous silicon, and alternatively in manganese oxide, have been used as programmable read-only memory elements. Low-energy switching has been ascertained in both these materials. Fabrication and testing of memory matrix is described. High-speed associative recall approaching 10 to the 7th bits/sec and high storage capacity in such a connection matrix memory system is also described.

Daud, T.↗

Connectivity strategies for higher-order neural networks applied to pattern recognition

Different strategies for non-fully connected HONNs (higher-order neural networks) are discussed, showing that by using such strategies an input field of 128 x 128 pixels can be attained while still achieving in-plane rotation and translation-invariant recognition. These techniques allow HONNs to be used with the larger input scenes required for practical pattern-recognition applications. The number of interconnections that must be stored has been reduced by a factor of approximately 200,000 in a T/C case and about 2000 in a Space Shuttle/F-18 case by using regional connectivity. Third-order networks have been simulated using several connection strategies. The method found to work best is regional connectivity. The main advantages of this strategy are the following: (1) it considers features of various scales within the image and thus gets a better sample of what the image looks like; (2) it is invariant to shape-preserving geometric transformations, such as translation and rotation; (3) the connections are predetermined so that no extra computations are necessary during run time; and (4) it does not require any extra storage for recording which connections were formed.

Spirkovska, Lilly↗

Learning and optimization with cascaded VLSI neural network building-block chips

To demonstrate the versatility of the building-block approach, two neural network applications were implemented on cascaded analog VLSI chips. Weights were implemented using 7-b multiplying digital-to-analog converter (MDAC) synapse circuits, with 31 x 32 and 32 x 32 synapses per chip. A novel learning algorithm compatible with analog VLSI was applied to the two-input parity problem. The algorithm combines dynamically evolving architecture with limited gradient-descent backpropagation for efficient and versatile supervised learning. To implement the learning algorithm in hardware, synapse circuits were paralleled for additional quantization levels. The hardware-in-the-loop learning system allocated 2-5 hidden neurons for parity problems. Also, a 7 x 7 assignment problem was mapped onto a cascaded 64-neuron fully connected feedback network. In 100 randomly selected problems, the network found optimal or good solutions in most cases, with settling times in the range of 7-100 microseconds.

Duong, T.↗

Artificial Neural Networks and AI in high Assurance Applications: Gaps and Techniques

In recent years, capabilties of Deep Neural Networks (DNN) and Artificial Intelligence (AI) systems have grown tremendously. They are now applied in many areas ranging from game playing, social media, science, to robotics, automotive, and aerospace applications.Based upon requirements for safety of DNN and AI in high assurance automotive and aerospace applications, I will discuss the necessity to ensure that AI technqiues for the analysis of Earth observation data and reasoning are working correctly and reliably.In this talk I will present modern techniques for the verification and validation (V&V) of DNN and other AI components as well as approaches for interpretable AI. I will discuss how these techniques can help to ensure quality of the AI results, improve confidence in their application, and facilitate human-AI interaction and collaboration.

Johann Schumann↗

Multimodel ensemble predictions of precipitation using bayesian neural networks

Multimodel ensembling improves predictions and considers model uncertainties. In this study, we present a Bayesian Neural Network (BNN) ensemble approach for large-scale precipitation predictions based on a set of climate models. BNN infers spatiotemporally varying model weights and biases through the calibration against observations. This ensemble scheme of BNN sufficiently leverages individual model skill for accurate predictions as well as provides interpretability about which models contribute more to the ensemble prediction at which locations and times to inform model development. Additionally, BNN accurately quantifies epistemic uncertainty to avoid overconfident projections. We demonstrate BNN’s superior prediction performance to three state-of-the-art ensemble approaches and discuss its interpretability and uncertainty quantification.

Fan, Patrick↗

Multimodel Ensemble Predictions of Precipitation using Bayesian Neural Networks

Multimodel ensembling improves predictions and considers model uncertainties. In this study, we present a Bayesian Neural Network (BNN) ensemble approach for large-scale precipitation predictions based on a set of climate models. BNN infers spatiotemporally varying model weights and biases through the calibration against observations. This ensemble scheme of BNN sufficiently leverages individual model skill for accurate predictions as well as provides interpretability about which models contribute more to the ensemble prediction at which locations and times to inform model development. Additionally, BNN accurately quantifies epistemic uncertainty to avoid overconfident projections. We demonstrate BNN’s superior prediction performance to three state-of-the-art ensemble approaches and discuss its interpretability and uncertainty quantification.

Fan, Patrick↗

Instantaneous Photosynthetically Available Radiation (IPAR) prediction models based on Neural Network for ocean waters.

Instantaneous photosynthetically available radiation (IPAR) at the ocean surface and its vertical profile below the surface play a critical role in models to calculate net primary productivity of marine phytoplankton. In this work, we report two IPAR prediction models based on neural network (NN) approach, one for open ocean and the other for coastal waters. These models are trained, validated, and tested using a large volume of synthetic datasets for open ocean and coastal waters simulated by a radiative transfer model. Our NN models are designed to predict IPAR under a large range of atmospheric and oceanic conditions. The NN models can compute subsurface IPAR profile very accurately up to euphotic zone depth. The root mean square errors associated with the diffuse attenuation coefficient of IPAR are less than 0.011 𝑚−1 and 0.036 𝑚−1 for open ocean and coastal waters respectively. The performance of the NN models is better than presently available semi analytical models, with significant superiority in coastal waters.

PACE↗

Permutation invariant polynomial neural network approach to fitting potential energy surfaces. IV. Coupled diabatic potential energy matrices

A machine learning method is proposed for representing the elements of diabatic potential energy matrices (PEMs) with high fidelity. This is an extension of the so-called permutation invariant polynomial-neural network (PIP-NN) method for representing adiabatic potential energy surfaces. While for one-dimensional irreducible representations the diagonal elements of a diabatic PEM are invariant under exchange of identical nuclei in a molecular system, the off-diagonal elements require special symmetry consideration, particularly in the presence of a conical intersection. A multiplicative factor is introduced to take into consideration the particular symmetry properties while maintaining the PIP-NN framework. Here, we demonstrate here that the extended PIP-NN approach is accurate in representing diabatic PEMs, as evidenced by small fitting errors and by the reproduction of absorption spectra and product branching ratios in both H 2 O$(\tilde{X} / \tilde{B})$ and NH 3 $(\tilde{X} $/Ã) non-adiabatic photodissociation.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Physics-Informed Neural Networks for PDE-Constrained Optimization and Control

The goal of optimal control is to determine a sequence of inputs for maximizing or minimizing a given performance criterion subject to the dynamics and constraints of the system under observation. This work introduces Control Physics-Informed Neural Networks (PINNs), which simultaneously learn both the system states and the optimal control signal in a single-stage framework that leverages the system’s underlying physical laws. While prior approaches often follow a two-stage process-modeling, the system first and then devising its control—the presented novel framework embeds the necessary optimality conditions directly into the network architecture and loss function. We demonstrate the effectiveness of the novel methodology by solving various open-loop optimal control problems governed by analytical, one-dimensional, and two-dimensional partial differential equations (PDEs).

97 MATHEMATICS AND COMPUTING↗

Applications of neural networks to the studies of phase transitions of two-dimensional Potts models

We study the phase transitions of two-dimensional (2D) Q-states Potts models on the square lattice, using the first principles Monte Carlo (MC) simulations as well as the techniques of neural networks (NN). We demonstrate that the ideas from NN can be adopted to study these considered phase transitions efficiently. In particular, even with a simple NN constructed in this investigation, we are able to obtain the relevant information of the nature of these phase transitions, namely whether they are first order or second order. Our results strengthen the potential applicability of machine learning in studying various states of matters. Subtlety of applying NN techniques to investigate many-body systems is briefly discussed as well.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Tackling the curse of dimensionality in fractional and tempered fractional PDEs with physics-informed neural networks

Fractional and tempered fractional partial differential equations (PDEs) are effective models of long-range interactions, anomalous diffusion, and non-local effects. Traditional numerical methods for these problems are mesh-based, thus struggling with the curse of dimensionality (CoD). Physics-informed neural networks (PINNs) offer a promising solution due to their universal approximation, generalization ability, and mesh-free training. In principle, Monte Carlo fractional PINN (MC-fPINN) estimates fractional derivatives using Monte Carlo methods and thus could lift CoD. However, this may cause significant variance and errors, hence affecting convergence; in addition, MC-fPINN is sensitive to hyperparameters. In general, numerical methods and specifically PINNs for tempered fractional PDEs are under-developed. Herein, we extend MC-fPINN to tempered fractional PDEs to address these issues, resulting in the Monte Carlo tempered fractional PINN (MC-tfPINN). To reduce possible high variance and errors from Monte Carlo sampling, we replace the one-dimensional (1D) Monte Carlo with 1D Gaussian quadrature, applicable to both MC-fPINN and MC-tfPINN. We validate our methods on various forward and inverse problems of fractional and tempered fractional PDEs, scaling up to 100,000 dimensions. Our improved MC-fPINN/MC-tfPINN using quadrature consistently outperforms the original versions in accuracy and convergence speed in very high dimensions.

42 ENGINEERING↗