Search NASA⌕ Search

SEARCH · Search NASA

Results for “network model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 343 records · Page 19

Explaining machine-learning models for gamma-ray detection and identification

As more complex predictive models are used for gamma-ray spectral analysis, methods are needed to probe and understand their predictions and behavior. Recent work has begun to bring the latest techniques from the field of Explainable Artificial Intelligence (XAI) into the applications of gamma-ray spectroscopy, including the introduction of gradient-based methods like saliency mapping and Gradient-weighted Class Activation Mapping (Grad-CAM), and black box methods like Local Interpretable Model-agnostic Explanations (LIME) and SHapley Additive exPlanations (SHAP). In addition, new sources of synthetic radiological data are becoming available, and these new data sets present opportunities to train models using more data than ever before. In this work, we use a neural network model trained on synthetic NaI(Tl) urban search data to compare some of these explanation methods and identify modifications that need to be applied to adapt the methods to gamma-ray spectral data. We find that the black box methods LIME and SHAP are especially accurate in their results, and recommend SHAP since it requires little hyperparameter tuning. We also propose and demonstrate a technique for generating counterfactual explanations using orthogonal projections of LIME and SHAP explanations.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

On the Training and Generalization of Deep Operator Networks

Here, we present a novel training method for deep operator networks (DeepONets), one of the most popular neural network models for operators. DeepONets are constructed by two subnetworks, namely the branch and trunk networks. Typically, the two subnetworks are trained simultaneously, which amounts to solving a complex optimization problem in a high dimensional space. In addition, the nonconvex and nonlinear nature makes training very challenging. To tackle such a challenge, we propose a two-step training method that trains the trunk network first and then sequentially trains the branch network. The core mechanism is motivated by the divide-and-conquer paradigm and is the decomposition of the entire complex training task into two subtasks with reduced complexity. Therein the Gram–Schmidt orthonormalization process is introduced which significantly improves stability and generalization ability. On the theoretical side, we establish a generalization error estimate in terms of the number of training data, the width of DeepONets, and the number of input and output sensors. Numerical examples are presented to demonstrate the effectiveness of the two-step training method, including Darcy flow in heterogeneous porous media.

deep operator networks↗

Improving multiwell petrophysical interpretation from well logs via machine learning and statistical models

Well-log interpretation estimates in situ rock properties along well trajectory, such as porosity, water saturation, and permeability, to support reserve-volume estimation, production forecasts, and decision making in reservoir development. However, due to measurement errors, variability of well logs caused by multiple measurement vendors, different borehole tools, and nonuniform drilling/borehole conditions, estimations of rock properties with original well logs without proper preprocessing may not be accurate, especially in the context of multiwell estimation. Well-log normalization techniques such as two-point scaling and mean-variance normalization are commonly used to improve the robustness of multiwell rock-property estimation. However, these techniques do not consider the correlation between well logs and require subjective knowledge for their effective implementation. To reduce uncertainties and processing time associated with multiwell rock-property estimation from well logs, we develop discriminative adversarial (DA) and linear constraint models for well-log normalization and rock-property estimation. The DA neural network model developed for well-log normalization and interpretation can perform linear and nonlinear well-log normalization while considering the joint distribution of each well log and rock properties. However, the linear constraint model uses an ensemble of predictions from linear models to constrain well-log normalization and rock-property estimation. We also develop a divergence-based type well identification method to select type (training) wells for a test well based on the statistical similarity of associated well-log distributions instead of the interwell distance. We apply the DA model to perform well-log normalization and prediction of permeability for the Seminole San Andres Unit carbonate reservoir. Compared with the permeability predicted with the classical machine learning model without well-log normalization and models with two-point scaling normalization, the DA model yields the most accurate permeability prediction by decreasing the mean-squared error of permeability prediction by 20%–50%.

Geochemistry & Geophysics↗

LCA-PyTorch

LCA-PyTorch is a code repository which contains PyTorch implementations of the Locally Competitive Algorithm (LCA), which is a biologically-plausible sparse coding model. LCA-PyTorch allows for the training, testing, and analysis of single layer LCA networks, multi-layer LCA networks, and hybrid LCA-based deep neural network models on a wide variety of applications and data types. LCA-PyTorch was developed in Python, a high-level programming language that takes advantage of the Python ecosystem of high-quality open-source packages for machine learning. LCA-PyTorch interfaces heavily with the open-source PyTorch Python package.

Teti, Michael↗

Machine learning prediction on the fractional free volume of polymer membranes

Fractional free volume (FFV) characterizes the microstructural level features of polymers and affects their properties including thermal, mechanical, and separation performance. Experimental measurements and theoretical analyses have been used to quantify the FFV of polymers, but challenges remain because of their limitations. Experimental measurements are laborious and based on semi empirical equations, while Bondi’s group contribution theory involves ambiguities like the determination of van der Waals volume and the choice of factor values in the theoretical equation. To efficiently evaluate the FFV of polymers, this study utilizes high-throughput molecular dynamics (MD) simulations to build a large dataset regarding polymer’s FFV. Based on this large dataset, we further build machine learning (ML) models to establish the composition-structure relation. Inspired by group contribution theory which correlates polymer’s functional groups to FFV, our ML models correlate polymer’s substructures or physico-chemical indexes to FFV. Here, our study first benchmarks the MD simulation protocol to obtain reliable FFV of polymers and then carries out high-throughput MD simulations for more than 6,500 homopolymers and 1,400 polyamides. Such a large and diverse dataset makes the well-trained ML models more generalizable, compared with the group contribution theory. The efficiency of a feed forward neural network model is further demonstrated by applying it to a hypothetical polyimide dataset of more than 8 million chemical structures. The predicted FFVs of hypothetical polyimides are further validated by MD simulations. The obtained FFVs of the 8 million polymers, plus their previously reported gas separation performances, demonstrate the promising capability of ML virtual screening for the discovery of polymer membranes with exceptional permeability/selectivity.

36 MATERIALS SCIENCE↗

Prevalence and scalable control of localized networks

The ability to control network dynamics is essential for ensuring desirable functionality of many technological, biological, and social systems. Such systems often consist of a large number of network elements, and controlling large-scale networks remains challenging because the computation and communication requirements increase prohibitively fast with network size. Here, we introduce a notion of network locality that can be exploited to make the control of networks scalable, even when the dynamics are nonlinear. We show that network locality is captured by an information metric and is almost universally observed across real and model networks. In localized networks, the optimal control actions and system responses are both shown to be necessarily concentrated in small neighborhoods induced by the information metric. This allows us to develop localized algorithms for determining network controllability and optimizing the placement of driver nodes. This also allows us to develop a localized algorithm for designing local feedback controllers that approach the performance of the corresponding best global controllers, while incurring a computational cost orders-of-magnitude lower. Here, we validate the locality, performance, and efficiency of the algorithms in Kuramoto oscillator networks, as well as three large empirical networks: synchronization dynamics in the Eastern US power grid, epidemic spreading mediated by the global air-transportation network, and Alzheimer’s disease dynamics in a human brain network. Taken together, our results establish that large networks can be controlled with computation and communication costs comparable to those for small networks.

42 ENGINEERING↗

Predictive Integrated Modeling of Heated Tokamak Plasmas (Final Technical Report)

The research carried out by the Lehigh University fusion group during this reporting period: (1) Self-consistent predictive core-pedestal ITER scenario modeling; (2) Microtearing instabilities and electron thermal transport in standard and spherical tokamak discharges; (3) Neural network model of the Multi-Mode anomalous transport module; (4) Anomalous transport in KSTAR, JET, EAST, Alcator C-Mod and DIII-D discharges; (5) Theory for transport in magnetized plasmas; (6) Control-oriented modeling of the q-profile dynamic response in preparation for advanced scenario control in KSTAR and in EAST; (7) Modeling of edge harmonic oscillation in DIII-D QH mode discharges; (8) Elucidating plasma dynamics in Hasegawa-Wakatani turbulence by information geometry; (9) Non-linear Langevin and Fractional Fokker–Planck equations for anomalous diffusion by Levy stable processes; and (10) Investigation of the plasma shaping effects on the H-mode pedestal structure.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

A generalizable machine learning-assisted fast Fourier transform algorithm to simulate the large strain phenomena in polycrystalline materials

Machine learning methods have shown initial promise in constitutive modeling for single crystals or homogenized polycrystals, delivering notable computational efficiency. However, existing machine learning-based constitutive models often lack generalizability, limiting their application across diverse boundary value problems. This study introduces a thermodynamics-informed artificial neural network model to accelerate rate-tangent crystal plasticity fast Fourier transform simulations for cross-scale deformation behaviors of polycrystals under complex loading. Our model integrates microstructural variability and local interactions effectively. To address local effects in each grain, we employ K-means clustering to group Gauss points within the microstructure into clusters assumed to be in similar mechanical states. This approach, based on self-clustering analysis, extends model scope from macroscopic stress response to the granular level, capturing mechanical responses and orientation evolution across grains. This reduces the number of nonlinear problems to solve, with cluster responses propagated throughout each group. The thermodynamics-based artificial neural network-extracted features are further processed using local material state clusters to account for history-dependent deformation and evolving microstructures. Additionally, representative volume element simulations with rate-tangent crystal plasticity fast Fourier transform provide reliable datasets for model training. The proposed model demonstrates high efficiency, accuracy, self-consistency, and enhanced generalizability in predicting strain–stress responses and orientation evolution at both individual grain and aggregate scales under complex loading conditions, such as biaxial tension and arbitrary loading scenarios.

36 MATERIALS SCIENCE↗

Reactive Flows in Porous Media: Challenges in Theoretical and Numerical Methods

We review theoretical and computational research, primarily from the past 10 years, addressing the flow of reactive fluids in porous media. The focus is on systems where chemical reactions at the solid–fluid interface cause dissolution of the surrounding porous matrix, creating nonlinear feedback mechanisms that can often lead to greatly enhanced permeability. Here, we discuss insights into the evolution of geological forms that can be inferred from these feedback mechanisms, as well as some geotechnical applications such as enhanced oil recovery, hydraulic fracturing, and carbon sequestration. Until recently, most practical applications of reactive transport have been based on Darcy-scale modeling, where averaged equations for the flow and reactant transport are solved. We summarize the successes and limitations of volume averaging, which leads to Darcy-scale equations, as an introduction to pore-scale modeling. Pore-scale modeling is computationally intensive but offers new insights as well as tests of averaging theories and pore-network models. We include recent research devoted to validation of pore-scale simulations, particularly the use of visual observations from microfluidic experiments.

wormhole formation↗

Power system restoration incorporating diverse distributed energy resources

An example system includes an aggregator configured to receive a service collaboration request and iteratively determine, based on minimum and maximum power values for DERs under its management, an optimized operation schedule. The aggregator may also be configured to iteratively determine, based on the optimized operation schedule, an estimated flexibility range for devices under its management and output an indication thereof. The system may also include a power management unit (PMU) configured to iteratively receive the indication and determine, based on a network model that includes the estimated flexibility range, a reconfiguration plan and an overall optimized operation schedule for the network. The PMU may also be configured to iteratively cause reconfiguration of the network based on the plan. The PMU and aggregator may also be configured to iteratively, at a fast timescale, cause energy resources under their management to modify operation based on the overall optimized operation schedule.

Ding, Fei↗

Model validation and selection in metabolic flux analysis and flux balance analysis

13C-Metabolic Flux Analysis (13C-MFA) and Flux Balance Analysis (FBA) are widely used to investigate the operation of biochemical networks in both biological and biotechnological research. Both methods use metabolic reaction network models of metabolism operating at steady state so that reaction rates (fluxes) and the levels of metabolic intermediates are constrained to be invariant. They provide estimated (MFA) or predicted (FBA) values of the fluxes through the network in vivo, which cannot be measured directly. These fluxes can shed light on basic biology and have been successfully used to inform metabolic engineering strategies. Several approaches have been taken to test the reliability of estimates and predictions from constraint-based methods and to compare alternative model architectures. Despite advances in other areas of the statistical evaluation of metabolic models, such as the quantification of flux estimate uncertainty, validation and model selection methods have been underappreciated and underexplored. We review the history and state-of-the-art in constraint-based metabolic model validation and model selection. Applications and limitations of the χ 2 -test of goodness-of-fit, the most widely used quantitative validation and selection approach in 13C-MFA, are discussed, and complementary and alternative forms of validation and selection are proposed. A combined model validation and selection framework for 13C-MFA incorporating metabolite pool size information that leverages new developments in the field is presented and advocated for. Finally, we discuss how adopting robust validation and selection procedures can enhance confidence in constraint-based modeling as a whole and ultimately facilitate more widespread use of FBA in biotechnology.

59 BASIC BIOLOGICAL SCIENCES↗

An introduction to neuromorphic computing and its potential impact for unattended ground sensors

Neuromorphic computers are hardware systems that mimic the brain’s computational process phenomenology. This is in contrast to neural network accelerators, such as the Google TPU or the Intel Neural Compute Stick, which seek to accelerate the fundamental computation and data flows of neural network models used in the field of machine learning. Neuromorphic computers emulate the integrate and fire neuron dynamics of the brain to achieve a spiking communication architecture for computation. While neural networks are brain-inspired, they drastically oversimplify the brain’s computation model. Neuromorphic architectures are closer to the true computation model of the brain (albeit, still simplified). Neuromorphic computing models herald a 1000x power improvement over conventional CPU architectures. Sandia National Labs is a major contributor to the research community on neuromorphic systems by performing design analysis, evaluation, and algorithm development for neuromorphic computers. Space-based remote sensing development has been a focused target of funding for exploratory research into neuromorphic systems for their potential advantage in that program area; SNL has led some of these efforts. Recently, neuromorphic application evaluation has reached the NA-22 program area. This same exploratory research and algorithm development should penetrate the unattended ground sensor space for SNL’s mission partners and program areas. Neuromorphic computing paradigms offer a distinct advantage for the SWaP-constrained embedded systems of our diverse sponsor-driven program areas.

97 MATHEMATICS AND COMPUTING↗

Determining the Reaction Kinetics and Thermodynamics of a Diels–Alder Network Using Dynamic Gel Criteria

We undertook a detailed rheological investigation to evaluate the kinetic parameters of the forward and reverse Diels–Alder (DA) reactions of a model network cross-linked using a furan prepolymer and a common aromatic bismaleimide. At high temperature where the Winter–Chambon’s criterion of frequency-independence was more applicable, a multiwave technique permitted van’t Hoff analysis and calculation of the reaction thermodynamic parameters, specifically the enthalpy and entropy of the reaction: ΔH° = –38.3 ± 5.2 kJ mol –1 and ΔS° = –94.3 ± 13.4 J mol –1 . At mild temperatures where the G'–G" crossover point is experimentally convenient to measure gelation, isothermal tests were used to obtain reasonable fDA kinetic parameters from Eyring analysis such as the apparent activation enthalpy and entropy of ΔH$^{‡}_{fDA}$ = 76.8 ± 6.9 kJ mol –1 and ΔS$^{‡}_{fDA}$= –82.8 ± 22.2 J mol –1 K –1 . Comparable rheokinetic methods include cross-linking density measurements and stress relaxation tests to calculate effective kinetics, whereas the critical gel conversion was consistently applied here. As a result, rate data are fitted with the Arrhenius equation for comparison purposes and the Eyring equation to demonstrate its broader utility.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Graph-Based Approaches for Predicting Solvation Energy in Multiple Solvents: Open Datasets and Machine Learning Models

The solvation properties of molecules, often estimated using quantum chemical simulations, are important in the synthesis of energy storage materials, drugs, and industrial chemicals. Here, we develop machine learning models of solvation energies to replace expensive quantum chemistry calculations with inexpensive-to-compute message-passing neural network models that require only the molecular graph as inputs. Our models are trained on a new database of solvation energies for 130,258 molecules taken from the QM9 dataset computed in five solvents (acetone, ethanol, acetonitrile, dimethyl sulfoxide, and water) via an implicit solvent model. Our best model achieves a mean absolute error of 0.5 kcal/mol for molecules with nine or fewer non-hydrogen atoms and 1 kcal/mol for molecules with between 10 and 14 non-hydrogen atoms. We make the entire dataset of 651,290 computed entries openly available and provide simple web and programmatic interfaces to enable others to run our solvation energy model on new molecules. This model calculates the solvation energies for molecules using only the SMILES string and also provides an estimate of whether each molecule is within the domain of applicability of our model. We envision that the dataset and models will provide the functionality needed for the rapid screening of large chemical spaces to discover improved molecules for many applications.

25 ENERGY STORAGE↗

Phase behavior of continuous-space systems: A supervised machine learning approach

The phase behavior of complex fluids is a challenging problem for molecular simulations. Supervised machine learning (ML) methods have shown potential for identifying the phase boundaries of lattice models. In this work, we extend these ML methods to continuous-space systems. We propose a convolutional neural network model that utilizes grid-interpolated coordinates of molecules as input data of ML and optimizes the search for phase transitions with different filter sizes. We test the method for the phase diagram of two off-lattice models, namely, the Widom–Rowlinson model and a symmetric freely jointed polymer blend, for which results are available from standard molecular simulations techniques. The ML results show good agreement with results of previous simulation studies with the added advantage that there is no critical slowing down. We find that understanding intermediate structures near a phase transition and including them in the training set is important to obtain the phase boundary near the critical point. The method is quite general and easy to implement and could find wide application to study the phase behavior of complex fluids.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Modeling the 4D discharge of lithium-ion batteries with a multiscale time-dependent deep learning framework

The lithium-ion battery (LIB) field is moving towards the direction of investigating spatially resolved physical phenomena in the 3D porous microstructure of electrodes. These pore-scale simulations give new insights into the local dynamics of lithiation/de-lithiation and charge transport, Nevertheless, the computational time of these simulations limits the integration of these models in optimization workflows of cycling conditions or electrode manufacturing processes. Machine learning models present a way of assessing in real-time the performance of materials. While several successful techniques for replicating simulations with machine learning have been proposed, this case study presents a more demanding problem, due to the necessity of understanding the behavior of heterogeneous 3D local data, as it evolves in time: this poses both a scientific and a technical challenge. To this end, we propose an autoregressive multiscale convolutional neural network model to predict relevant quantities at the pore-scale in the solid phase: the lithium concentration (in the active material) and potential (in the active material and carbon binder). Here, these are ultimately used to reconstruct the battery discharge curve. 3D images of the electrode microstructures are the input to the network, trained with a dataset of finite element method simulations to predict the discharge behavior of the cathode side in lithium ion batteries. We propose this machine learning model as a proof-of-concept of the applicability of multiscale networks for time-dependent physics problems. The trained model exhibits very high accuracy (with errors lower than 2 %) in forecasting the discharge behavior of new unseen cathodes.

25 ENERGY STORAGE↗

Lithium-Ion Battery Diagnostics Using Electrochemical Impedance via Machine-Learning

Diagnosing battery states such as health, state-of-charge, or temperature is crucial for ensuring the safety and reliability of electrochemical energy storage systems. While some states, such as temperature, may be measured using cheap sensors, accurate diagnosis of battery health metrics usually requires time-consuming performance measurements, making them infeasible for use in real-world operation. These health metrics can be measured during lab-testing and then estimated on-line using predictive life models or via state observer algorithms such as Kalman filters, but these predictive methods should be supplemented by actual measurement of battery health whenever possible to ensure reliability. Rapid measurement of battery health may be done by various types of fast diagnostic techniques such as electrochemical impedance spectroscopy (EIS), which can be performed in only a few minutes and require only a fraction of the energy and power needed for a full charge and discharge measurement. But there is a substantial challenge for estimating battery health using EIS data, as EIS is sensitive to cell temperature, state-of-charge, current, and resting time in addition to health. Thus, utilizing EIS data to predict battery capacity requires correcting for all these additional variables, a task that is extremely difficult to handle analytically. This talk utilizes machine-learning methods to estimate the effectiveness of battery capacity prediction from EIS data, leveraging a data set of hundreds of EIS measurements recorded at varying temperature and state-of-charge throughout a 500-day aging study of 32 commercial, large-format NMC-Graphite lithium-ion batteries. Using EIS as input to machine-learning models is complicated by the nonlinear response of impedance to battery health, temperature, and state-of-charge, as well as the collinearity between the impedance response at neighboring frequencies, which can easily lead to overfit models. To train robust models, features from EIS data need to be extracted from the data or some subset of critical frequencies selected. Many approaches for extracting and selecting features from EIS data from electrochemical analysis and machine-learning fields were identified for analysis: using the entire raw spectra; selection of one, two, or many frequencies from the entire spectra; selecting interesting points from the EIS measurement using domain knowledge; fitting EIS with an equivalent-circuit model; calculating statistics on the raw impedance values; and reducing the dimensionality of the data using unsupervised linear (principal component analysis) and non-linear (uniform manifold approximation and projection) methods. These approaches were rigorously compared using a machine-learning pipeline approach, training linear, Gaussian process, and random forest regression models and quantifying performance using cross-validation as well as a held-out test set. An artificial neural network model trained on the raw spectra was also tested. Promising pipelines were fine-tuned via Bayesian hyperparameter optimization using cross-validation loss and training with class-specific weights to counter data set imbalance. The most reliable method for utilizing impedance in this work was the selection of two optimal frequencies through an exhaustive search, resulting in about 2% mean absolute error on test data for both Gaussian process and random forest model architectures. Interrogation of a variety of models reveals critical frequencies of 100 Hz and 103 Hz for this data set, though the optimal set of frequencies is not necessarily intuitive, i.e., the best performing models are not simply those that use impedance at frequencies that have the highest correlation to the relative discharge capacity. The best performing model is an ensemble model, which is able to predict battery capacity with 1.9% mean absolute error for unseen cells using impedance recorded at a variety of temperatures and states-of-charge.

battery↗

Solving sparse finite element problems on neuromorphic hardware

The finite element method (FEM) is one of the most important and ubiquitous numerical methods for solving partial differential equations (PDEs) on computers for scientific and engineering discovery. Applying the FEM to larger and more detailed scientific models has driven advances in high-performance computing for decades. Here we demonstrate that scalable spiking neuromorphic hardware can directly implement the FEM by constructing a spiking neural network that solves the large, sparse, linear systems of equations at the core of the FEM. We show that for the Poisson equation, a fundamental PDE in science and engineering, our neural circuit achieves meaningful levels of numerical accuracy and close to ideal scaling on modern, inherently parallel and energy-efficient neuromorphic hardware, specifically Intel’s Loihi 2 neuromorphic platform. We illustrate extensions to irregular mesh geometries in both two and three dimensions as well as other PDEs such as linear elasticity. Our spiking neural network is constructed from a recurrent network model of the brain’s motor cortex and, in contrast to black-box deep artificial neural network-based methods for PDEs, directly translates the well-understood and trusted mathematics of the FEM to a natively spiking neuromorphic algorithm.

Applied mathematics↗