Search NASA⌕ Search

SEARCH · Search NASA

Results for “Neural Network Models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 883 records · Page 49

A deep learning-accelerated data assimilation and forecasting workflow for commercial-scale geologic carbon storage

Fast assimilation of monitoring data to update forecasts of pressure buildup and carbon dioxide (CO 2 ) plume migration under geologic uncertainties is a challenging problem in geologic carbon storage. The high computational cost of data assimilation with a high-dimensional parameter space impedes fast decision-making for commercial-scale reservoir management. We propose to leverage physical understandings of porous medium flow behavior with deep learning techniques to develop a fast data assimilation-reservoir response forecasting workflow. Applying an Ensemble Smoother Multiple Data Assimilation (ES-MDA) framework, the workflow updates geologic properties and predicts reservoir performance with quantified uncertainty from pressure history and CO 2 plumes interpreted through seismic inversion. As the most computationally expensive component in such a workflow is reservoir simulation, we developed surrogate models to predict dynamic pressure and CO 2 plume extents under multi-well injection. The surrogate models employ deep convolutional neural networks, specifically, a wide residual network and a residual U-Net. The workflow is validated against a flat threedimensional reservoir model representative of a clastic shelf depositional environment. Intelligent treatments are applied to bridge between quantities in a true-3D reservoir model and those in a single-layer reservoir model. The workflow can complete history matching and reservoir forecasting with uncertainty quantification in less than one hour on a mainstream personal workstation.

25 ENERGY STORAGE↗

Interpreting and Stabilizing Machine-Learning Parametrizations of Convection

Neural networks are a promising technique for parameterizing subgrid-scale physics (e.g., moist atmospheric convection) in coarse-resolution climate models, but their lack of interpretability and reliability prevents widespread adoption. For instance, it is not fully understood why neural network parameterizations often cause dramatic instability when coupled to atmospheric fluid dynamics. This paper introduces tools for interpreting their behavior that are customized to the parameterization task. First, we assess the nonlinear sensitivity of a neural network to lower-tropospheric stability and the midtropospheric moisture, two widely studied controls of moist convection. Second, we couple the linearized response functions of these neural networks to simplified gravity wave dynamics, and analytically diagnose the corresponding phase speeds, growth rates, wavelengths, and spatial structures. To demonstrate their versatility, these techniques are tested on two sets of neural networks, one trained with a super-parameterized version of the Community Atmosphere Model (SPCAM) and the second with a near-global cloud-resolving model (GCRM). Additionally, even though the SPCAM simulation has a warmer climate than the cloud-resolving model, both neural networks predict stronger heating/drying in moist and unstable environments, which is consistent with observations. Moreover, the spectral analysis can predict that instability occurs when GCMs are coupled to networks that support gravity waves that are unstable and have phase speeds larger than 5 m s -1 . In contrast, standing unstable modes do not cause catastrophic instability. Using these tools, differences between the SPCAM-trained versus GCRM-trained neural networks are analyzed, and strategies to incrementally improve both of their coupled online performance unveiled.

54 ENVIRONMENTAL SCIENCES↗

Neural network acceleration of large-scale structure theory calculations

Here, we make use of neural networks to accelerate the calculation of power spectra required for the analysis of galaxy clustering and weak gravitational lensing data. For modern perturbation theory codes, evaluation time for a single cosmology and redshift can take on the order of two seconds. In combination with the comparable time required to compute linear predictions using a Boltzmann solver, these calculations are the bottleneck for many contemporary large-scale structure analyses. Here, we construct neural network-based surrogate models for Lagrangian perturbation theory (LPT) predictions of matter power spectra, real and redshift space galaxy power spectra, and galaxy-matter cross power spectra that attain ~ 0.1% (at one sigma) accuracy over a broad range of scales in a ωCDM parameter space. The neural network surrogates can be evaluated in approximately one millisecond, a factor of 1000 times faster than the full Boltzmann code and LPT computations. In a simulated full-shape redshift space galaxy power spectrum analysis, we demonstrate that the posteriors obtained using our surrogates are accurate compared to those obtained using the full LPT model. We make our surrogate models public at https://github.com/sfschen/EmulateLSS, so that others may take advantage of the speed gains they provide to enable rapid iteration on analysis settings, something that is essential in complex contemporary large-scale structure analyses.

79 ASTRONOMY AND ASTROPHYSICS↗

A Fusion of Geothermal and InSAR Data with Machine Learning for Enhanced Deformation Forecasting at the Geysers

The Geysers geothermal field in California is experiencing land subsidence due to the seismic and geothermal activities taking place. This poses a risk not only to the underlying infrastructure but also to the groundwater level which would reduce the water availability for the local community. Because of this, it is crucial to monitor and assess the surface deformation occurring and adjust geothermal operations accordingly. In this study, we examine the correlation between the geothermal injection and production rates as well as the seismic activity in the area, and we show the high correlation between the injection rate and the number of earthquakes. This motivates the use of this data in a machine learning model that would predict future deformation maps. First, we build a model that uses interferometric synthetic aperture radar (InSAR) images that have been processed and turned into a deformation time series using LiCSBAS, an open-source InSAR time series package, and evaluate the performance against a linear baseline model. The model includes both convolutional neural network (CNN) layers as well as long short-term memory (LSTM) layers and is able to improve upon the baseline model based on a mean squared error metric. Then, after getting preprocessed, we incorporate the geothermal data by adding them as additional inputs to the model. This new model was able to outperform both the baseline and the previous version of the model that uses only InSAR data, motivating the use of machine learning models as well as geothermal data in assessing and predicting future deformation at The Geysers as part of hazard mitigation models which would then be used as fundamental tools for informed decision making when it comes to adjusting geothermal operations.

Yazbeck, Joe (ORCID:0000000302235260)↗

A hybrid deep neural operator/finite element method for ice-sheet modeling

One of the most challenging and consequential problems in climate modeling is to provide probabilistic projections of sea level rise. A large part of the uncertainty of sea level projections is due to uncertainty in ice sheet dynamics. At the moment, accurate quantification of the uncertainty is hindered by the cost of ice sheet computational models. In this work we develop a hybrid approach to approximate existing ice sheet models at a fraction of their cost. Our approach consists of replacing the finite element model for the momentum equations for the ice velocity, the most expensive part of an ice sheet model, with a Deep Operator Network, while we retain a classic finite element discretization for the evolution of the ice thickness. We show that the resulting hybrid model is very accurate and it is an order of magnitude faster than the traditional finite element model. Further, a distinctive feature of the proposed model, compared to other neural network approaches, is that it can handle high-dimensional parameter spaces (parameter fields) such as the basal friction at the bed of the glacier and can therefore be used for generating samples for uncertainty quantification. Further, we study the impact of hyper-parameters, number of unknowns and correlation length of the parameter distribution on the training and accuracy of the Deep Operator Network on a synthetic ice sheet model. We then target the evolution of the Humboldt glacier in Greenland and show that our hybrid model can provide accurate statistics of the glacier mass loss and can be effectively used to accelerate the quantification of uncertainty.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Learning constitutive relations using symmetric positive definite neural networks

In this work, we present a new neural-network architecture, called the Cholesky-factored symmetric positive definite neural network (SPD-NN), for modeling constitutive relations in computational mechanics. Instead of directly predicting the stress of the material, the SPD-NN trains a neural network to predict the Cholesky factor of the tangent stiffness matrix, based on which the stress is calculated in incremental form. As a result of this special structure, SPD-NN weakly imposes convexity on the strain energy function, satisfies the second order work criterion (Hill's criterion) and time consistency for path-dependent materials, and therefore improves numerical stability, especially when the SPD-NN is used in finite element simulations. Depending on the types of available data, we propose two training methods, namely direct training for strain and stress pairs and indirect training for loads and displacement pairs. We demonstrate the effectiveness of SPD-NN on hyperelastic, elasto-plastic, and multiscale fiber-reinforced plate problems from solid mechanics. The generality and robustness of SPD-NN make it a promising tool for a wide range of constitutive modeling applications.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Accelerating Multiscale Materials Modeling with Machine Learning

The focus of this project is to accelerate and transform the workflow of multiscale materials modeling by developing an integrated toolchain seamlessly combining DFT, SNAP, LAMMPS, (shown in Figure 1-1) and a machine-learning (ML) model that will more efficiently extract information from a smaller set of first-principles calculations. Our ML model enables us to accelerate first-principles data generation by interpolating existing high fidelity data, and extend the simulation scale by extrapolating high fidelity data (10 2 atoms) to the mesoscale (10 4 atoms). It encodes the underlying physics of atomic interactions on the microscopic scale by adapting a variety of ML techniques such as deep neural networks (DNNs), and graph neural networks (GNNs). We developed a new surrogate model for density functional theory using deep neural networks. The developed ML surrogate is demonstrated in a workflow to generate accurate band energies, total energies, and density of the 298K and 933K Aluminum systems. Furthermore, the models can be used to predict the quantities of interest for systems with more number of atoms than the training data set. We have demonstrated that the ML model can be used to compute the quantities of interest for systems with 100,000 Al atoms. When compared with 2000 Al system the new surrogate model is as accurate as DFT, but three orders of magnitude faster. We also explored optimal experimental design techniques to choose the training data and novel Graph Neural Networks to train on smaller data sets. These are promising methods that need to be explored in the future.

36 MATERIALS SCIENCE↗

Neural Network Enhanced RKPM for Electrochemical-Mechanical Coupled Damage Modeling of Energy Storage Materials

Energy storage materials undergo significant charge cycling, which makes understanding their reliability and durability fundamental in predicting performance and service life. Strong electrochemical-mechanical coupling and highly anisotropic material properties contribute to the formation and propagation of micro-cracking, largely along material interfaces and grain boundaries. For Li-ion batteries, for example, lithium moving between electrodes during charging and discharging process causes expansion and contraction of grains, and the strongly anisotropic and nonlinearly [Li]-dependent grain material properties can cause grains to expand into and contract away from each other, leading to chemo-mechanical cracking. In the first part of this work, a RKPM based computational framework for solving the coupled solid-phase lithium conservation with Fickian diffusion and the lithium concentration dependent anisotropic mechanical problem subjected to a highly nonlinear Butler-Volmer boundary condition is introduced. The choice of RKPM completeness conditions for lithium concentration and mechanical deformation fields, and the variational consistency condition for the domain integration of the coupled problem is first determined. In the second part of this work, a neural network-enhanced reproducing kernel particle method (NN-RKPM) [1] is leveraged to accurately capture damage and crack propagation throughout the material, by learning the location, orientation, and sharpness of discontinuity while allowing for a coarser nodal distribution than that is necessary for capturing sharp solution transitions using traditional mesh-based methods. NN-RKPM is used to inform how crack opening and closure in turn affect the coupled chemical equations and material microstructure.

damage modeling↗

Towards Compact Neural Networks via End-to-End Training: A Bayesian Tensor Approach with Automatic Rank Determination

Post-training model compression can reduce the inference costs of deep neural networks, but uncompressed training still consumes enormous hardware resources and energy. To enable low-energy training on edge devices, it is highly desirable to directly train a compact neural network from scratch with a low memory cost. Low-rank tensor decomposition is an effective approach to reduce the memory and computing costs of large neural networks. However, directly training low-rank tensorized neural networks is a very challenging task because it is hard to determine a proper tensor rank a priori, and the tensor rank controls both model complexity and accuracy. Here, this paper presents a novel end-to-end framework for low-rank tensorized training. We first develop a Bayesian model that supports various low-rank tensor formats (e.g., CANDECOMP/PARAFAC, Tucker, tensor-train, and tensor-train matrix) and reduces neural network parameters with automatic rank determination during training. Then we develop a customized Bayesian solver to train large-scale tensorized neural networks. Our training methods shows orders-of-magnitude parameter reduction and little accuracy loss (or even better accuracy) in the experiments. On a very large deep learning recommendation system with over 4.2 ×10 9 model parameters, our method can reduce the parameter number to 1.6 ×10 5 automatically in the training process (i.e., by 2.6 ×10 4 times) while achieving almost the same accuracy. Code is available at https://github.com/colehawkins/bayesian-tensor-rank-determination.

compact neural networks↗

Neural Network-Enhanced Reproducing Kernel Particle Method for Image-Based Multiphysics Damage Modeling of Energy Storage Materials

Energy storage materials undergo significant stresses during charge/discharge cycling, which makes understanding their reliability and durability fundamental in predicting performance and service life. Strong electrochemical-mechanical coupling and highly anisotropic material properties contribute to the formation and propagation of micro-cracking, largely along material interfaces and grain boundaries. With microstructural images supplied by the National Renewable Energy Laboratory (NREL), image-based modeling techniques are used to represent the complex material microstructures that dictate the coupled physics of these systems. Traditional electrochemical-mechanical models rely on mesh-based finite element methods, which can lead to difficulties in capturing crack propagation due to mesh dependency. Additionally, commonly used damage models, such as the continuous damage model and the cohesive zone model, often have steep tradeoffs between discontinuous field accuracy and computational expense. In this work, a neural network-enhanced reproducing kernel particle method (NN-RKPM) [1] is leveraged to accurately capture damage and crack propagation throughout the material by learning the location, orientation, and sharpness of discontinuity while allowing for a coarser nodal distribution than that necessary for capturing sharp solution transitions using traditional mesh-based methods. NN-RKPM is used to inform how crack opening and closure in turn affect the coupled chemical equations and material microstructure. Reference: [1] Baek, J., Chen, J. S., Susuki, K., "Neural Network enhanced Reproducing Kernel Particle Method for Modeling Localizations," International Journal for Numerical Methods in Engineering, Vol. 123, pp 4422-4454, https://doi.org/10.1002/nme.7040, 2022.

damage modeling↗

Federated Quantum Machine Learning

Distributed training across several quantum computers could significantly improve the training time and if we could share the learned model, not the data, it could potentially improve the data privacy as the training would happen where the data is located. One of the potential schemes to achieve this property is the federated learning (FL), which consists of several clients or local nodes learning on their own data and a central node to aggregate the models collected from those local nodes. However, to the best of our knowledge, no work has been done in quantum machine learning (QML) in federation setting yet. In this work, we present the federated training on hybrid quantum-classical machine learning models although our framework could be generalized to pure quantum machine learning model. Specifically, we consider the quantum neural network (QNN) coupled with classical pre-trained convolutional model. Our distributed federated learning scheme demonstrated almost the same level of trained model accuracies and yet significantly faster distributed training. It demonstrates a promising future research direction for scaling and privacy aspects.

97 MATHEMATICS AND COMPUTING↗

Fault Diagnosis of Double Pitch Time-Sharing Meshing Toothed Conveyor Chain Transmission System Based on Neural Network

With the rapid development of industry, the production and demand of automobiles have increased significantly. The generation of cars often requires tens of thousands of processes, consisting of hundreds of assembly lines to complete. Chain conveyors are widely used in the transportation of automobile assembly lines. The existing conveyor chains are roller chains. With the improvement of the requirements for conveyor reliability, synchronization, and environmental friendliness, the characteristics of roller conveyor chains restrict the further development of chain conveyors. It is urgent to study a new conveyor chain system to improve the synchronization, reliability, and environmental friendliness of chain conveyors on hundreds of assembly lines. In this paper, the mathematical modeling, meshing analysis, reliability, and environmental friendliness of the parameters of the components of the double pitch time-sharing meshing toothed conveyor chain system are studied. In addition, a novel fault diagnosis method of double pitch time-sharing meshing toothed conveyor chain transmission system based on neural network model is proposed in this paper.

Ding, Song↗

Deciphering the Solvation Structure of Aqueous ZnCl 2 Solutions from X-ray Absorption Spectra Using the Interpretable Graph Neural Network

Machine learning (ML) provides powerful pathways for predicting spectroscopic observables from atomic structures, but its broader impact depends on making model predictions interpretable in terms of physical and chemical principles. Here, we introduce a physics-guided graph neural network (GNN) model that predicts Zn K-edge X-ray spectroscopy (XAS) spectra of aqueous ZnCl 2 solutions. Training data are generated from ab initio XAS calculations on molecular dynamics snapshots obtained using a machine learning interatomic potential. The GNN reproduces experimental spectra across concentrations from dilute (<0.1 m) to highly concentrated (30 m, “water-in-salt”) regimes and scales efficiently to large, disordered liquid systems beyond the reach of conventional ab initio approaches. Gradient-based attribution analysis reveals that the model learns physically meaningful structure-spectrum relationships. Ligand-specific attributions reflect orbital hybridization patterns and the origin of the excitations derived from the density functional theory. Bond-length attributions recover spectral shifts consistent with multiple-scattering theory. Finally, this work bridges data-driven prediction with electronic-structure theory, establishing a general paradigm for interpretable ML that links atomic structure, electronic structure, and spectroscopic observables.

25 ENERGY STORAGE↗

DEEP CELLULAR RECURRENT NEURAL ARCHITECTURE FOR EFFICIENT MULTIDIMENSIONAL TIME-SERIES DATA PROCESSING

Efficient processing of time series data is a fundamental yet challenging problem in pattern recognition. Though recent developments in machine learning and deep learning have enabled remarkable improvements in processing large scale datasets in many application domains, most are designed and regulated to handle inputs that are static in time. Many real-world data, such as in biomedical, surveillance and security, financial, manufacturing and engineering applications, are rarely static in time, and demand models able to recognize patterns in both space and time. Current machine learning (ML) and deep learning (DL) models adapted for time series processing tend to grow in complexity and size to accommodate the additional dimensionality of time. Specifically, the biologically inspired learning based models known as artificial neural networks that have shown extraordinary success in pattern recognition, tend to grow prohibitively large and cumbersome in the presence of large scale multi-dimensional time series biomedical data such as EEG. Consequently, this work aims to develop representative ML and DL models for robust and efficient large scale time series processing. First, we design a novel ML pipeline with efficient feature engineering to process a large scale multi-channel scalp EEG dataset for automated detection of epileptic seizures. With the use of a sophisticated yet computationally efficient time-frequency analysis technique known as harmonic wavelet packet transform and an efficient self-similarity computation based on fractal dimension, we achieve state-of-the-art performance for automated seizure detection in EEG data. Subsequently, we investigate the development of a novel efficient deep recurrent learning model for large scale time series processing. For this, we first study the functionality and training of a biologically inspired neural network architecture known as cellular simultaneous recurrent neural network (CSRN). We obtain a generalization of this network for multiple topological image processing tasks and investigate the learning efficacy of the complex cellular architecture using several state-of-the?art training methods. Finally, we develop a novel deep cellular recurrent neural network (CDRNN) architecture based on the biologically inspired distributed processing used in CSRN for processing time series data. The proposed DCRNN leverages the cellular recurrent architecture to promote extensive weight sharing and efficient, individualized, synchronous processing of multi-source time series data. Experiments on a large scale multi-channel scalp EEG, and a machine fault detection dataset show that the proposed DCRNN offers state-of-the-art recognition performance while using substantially fewer trainable recurrent units.

Vidyaratne, Lasitha S.↗

Efficient mapping between void shapes and stress fields using Deep Convolutional Neural Networks with sparse data

Establishing fast and accurate structure-to-property relationships is an important component in the design and discovery of advanced materials. Physics-based simulation models like the finite element method (FEM) are often used to predict deformation, stress, and strain fields as a function of material microstructure in material and structural systems. Such models may be computationally expensive and time intensive if the underlying physics of the system is complex. This limits their application to solve inverse design problems and identify structures that maximize performance. In such scenarios, surrogate models are employed to make the forward mapping computationally efficient to evaluate. However, the high dimensionality of the input microstructure and the output field of interest often renders such surrogate models inefficient, especially when dealing with sparse data. Deep convolutional neural network (CNN) based surrogate models have shown great promise in handling such high-dimensional problems. In this paper, a single ellipsoidal void structure under a uniaxial tensile load represented by a linear elastic, high-dimensional and expensive-to-query, FEM model. We consider two deep CNN architectures, a modified convolutional autoencoder framework with a fully connected bottleneck and a UNet CNN, and compare their accuracy in predicting the von Mises stress field for any given input void shape in the FEM model. Additionally, a sensitivity analysis study is performed using the two approaches, where the variation in the prediction accuracy on unseen test data is studied through numerical experiments by varying the number of training samples from 20 to 100.

surrogate modeling; convolutional neural networks;↗

Machine Learning Moment Closure Models for the Radiative Transfer Equation III: Enforcing Hyperbolicity and Physical Characteristic Speeds

This is the third paper in a series in which we develop machine learning (ML) moment closure models for the radiative transfer equation. In our previous work (Huang et al. in J Comput Phys 453:110941, 2022), we proposed an approach to learn the gradient of the unclosed high order moment, which performs much better than learning the moment itself and the conventional $P_N$ closure. However, while the ML moment closure has better accuracy, it is not able to guarantee hyperbolicity and has issues with long time stability. In our second paper (Huang et al., in: Machine learning moment closure models for the radiative transfer equation II: enforcing global hyperbolicity in gradient based closures, 2021. arXiv:2105.14410), we identified a symmetrizer which leads to conditions that enforce that the gradient based ML closure is symmetrizable hyperbolic and stable over long time. The limitation of this approach is that in practice the highest moment can only be related to four, or fewer, lower moments. In this paper, we propose a new method to enforce the hyperbolicity of the ML closure model. Motivated by the observation that the coefficient matrix of the closure system is a lower Hessenberg matrix, we relate its eigenvalues to the roots of an associated polynomial. Here, we design two new neural network architectures based on this relation. The ML closure model resulting from the first neural network is weakly hyperbolic and guarantees the physical characteristic speeds, i.e., the eigenvalues are bounded by the speed of light. The second model is strictly hyperbolic and does not guarantee the boundedness of the eigenvalues. Several benchmark tests including the Gaussian source problem and the two-material problem show the good accuracy, stability and generalizability of our hyperbolic ML closure model.

97 MATHEMATICS AND COMPUTING↗

A Framework for Neural Network Inference on FPGA-Centric SmartNICs

FPGA-based SmartNICs offer great potential to significantly improve the performance of high-performance computing and warehouse data processing by tightly coupling support for reconfigurable data-intensive computation with cross-node communication, thereby mitigating the von Neumann bottleneck. Existing work, however, has been generally been limited in that it assumes an accelerator model where kernels are offloaded to SmartNICs, but most control tasks are left to the CPUs. This leads to frequent waiting, inferior performance, and scaling challenges. In this work, we propose a new distributive data-centric computing framework, named FCsN, for reconfigurable SmartNIC-based systems. Through a lightweight task circulation execution model and its implementation architecture, FCsN allows the complete detaching of kernel execution, control logic, system scheduling, and network communication to the SmarNICs. This boosts performance by: (i) avoiding the control dependency with CPUs and (ii) supporting streaming kernel execution and network communication at line rate and in a very fine-grained manner. We demonstrate the efficiency and flexibility of FCsN using various types of neural network applications including graph neural networks; as these last are both irregular and data intensive they offer an especially robust demonstration. Evaluations using commonly-used neural network models and graph datasets show that a system with the support of FCsN can achieve, on average, 144 speedups over the MPI-based standard CPU baselines.

Guo, Anqi↗

Karhunen–Loève deep learning method for surrogate modeling and approximate Bayesian parameter estimation

We evaluate the performance of the Karhunen-Loève Deep Neural Network (KL-DNN) framework for surrogate modeling and approximate Bayesian parameter estimation in partial differential equation models. In the surrogate model, the Karhunen-Loève (KL) expansions are used for the dimensionality reduction of the number of unknown parameters and variables, and a deep neural network is employed to relate the reduced space of parameters to that of the state variables. The KL-DNN surrogate model is used to formulate a maximum-a-posteriori-like least-squares problem, which is randomized to draw samples of the posterior distribution of the parameters. We test the proposed framework for a hypothetical unconfined aquifer via comparison with the forward MODFLOW and inverse PEST++ iterative ensemble smoother (IES) solutions as well as the state-of-the-art Fourier neural operator (FNO) and deep operator networks (DeepONets) operator learning surrogate models. Our results show that the KL-DNN surrogate model outperforms FNO and DeepONet for forward predictions. For solving inverse problems, the randomized algorithm provides the same or more accurate Bayesian predictions of the parameters than IES as evidenced by the higher log-predictive probability of both the estimated parameter field and the forecast hydraulic head. The posterior mean obtained from the randomized algorithm is closer to the reference parameter field than that obtained with FNO as the maximum a posteriori estimate.

Approximate Bayesian inference↗