Search NASA⌕ Search

SEARCH · Search NASA

Results for “differentiable machine learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 271 records · Page 15

Denial of Service Attack Detection via Differential Analysis of Generalized Entropy Progressions

Denial-of-Service (DoS) attacks are one the most common and consequential cyber attacks in computer networks. While existing research offers a plethora of detection methods, the issue of achieving scalability, a low false positive rate, and high detection accuracy remains open. In this work, we address this problem by developing a differential method based on generalized entropy progression. In this method, named as DoDGE, we continuously fit the line of best fit to the entropy progression of destination addresses and check if the derivative, that is, the slope of this line is less than the negative of the dynamically computed standard deviation of the derivatives. Furthermore, to distinguish from flash events, we leverage the symmetry that when a flash event occurs, the derivative of the entropy progression of source addresses is positive. With this design, we omit the usage of the thresholds and the results with five real-world network traffic datasets confirm that DoDGE outperforms threshold-based DoS attack detection by two orders of magnitude in terms of false positives on average. When compared to ten machine learning (ML) models, DoDGE achieves a balanced accuracy of 99%, while the average balanced accuracy for the ML models is 52%. Moreover, the results show that DoDGE successfully differentiates between a flash event and a DoS attack. Furthermore, since the main computation cost of DoDGE is the entropy computation, which is linear in the volume of the unit-time network flow, uses integer only operations, and works on a small fraction of the total flow, it is lightweight and scalable.

Cybersecurity, wireless communication↗

On the Investigation of Phase Fault Classification in Power Grid Signals: A Case Study for Support Vector Machines, Decision Tree and Random Forest

In monitoring the power grid, an ability to differentiate between fault types is essential to ensuring electrical safety. Accordingly, this study introduces a fault detection and classification method by considering different machine learning (ML) and feature extraction (FE) methods combinations. Specifically, the proposed method is established in two classification layers; the first layer determines the fault, and the second layer distinguishes the type of fault. Based on the proposed system model, this study seeks to determine the influential data attributes in a power grid signal using FE methods, including fast Fourier transform, power spectral density (PSD), auto-correlation, and wavelet transform (WT). A cross-comparison of the effectiveness of the Support Vector Machine (SVM), Decision Tree (DT), and Random Forest (RF) is also performed to accomplish the classification layers of the proposed method. The designed algorithm is analyzed under the various combinations of FE and ML methods, and outcomes are presented by considering the trade-off between computational complexity and prediction accuracy. The results reveal that the RF-based ML algorithm shows the most accurate classification performance with PSD, and the most time-saving of the models is the DT WT. Also, SVM emerges superior on a subsequent test of the simulated models on real-world signals.

Galbraith, Kelli↗

Domain Aware Deep-learning Algorithms Integrated with Scientific-computing Technologies (DADAIST)

This technical report summarized the contribution of the DADAIST project funded by the Data Model Convergence Initiative via the Laboratory Directed Research and Development (LDRD) investments at Pacific Northwest National Laboratory (PNNL). Specifically, we report the development of the NeuroMANCER (Neural Modules with Adaptive Nonlinear Constraints and Efficient Regularizations), a new open-source Scientific Machine Learning library for formulating and solving parametric constrained optimization problems, physics-informed system identification, and parametric optimal control problems. NeuroMANCER is using differentiable programming to combine modern data-driven models and optimization modeling language into a coherent algorithmic and software framework. NeuroMANCER is a Pytorch-based framework and adopts much of its philosophy focused on research and development, rapid prototyping, and streamlined deployment. Strong emphasis is given to extensibility, interoperability with the PyTorch ecosystem, and quick adaptability to custom domain problems. Neuromancer repository contains a comprehensive library of differentiable modules, including custom activation functions, matrix factorizations, deep learning architectures, neural differential equations, differential equation solvers, implicit layers such as iterative solvers, high-level API for symbolic expressions, API for modeling and control of dynamical systems, and extensive set of tutorial code examples in the form of python scripts and jupyter notebooks.

97 MATHEMATICS AND COMPUTING↗

NEML2: A High Performance Library for Constitutive Modeling

NEML2, the New Engineering Material model Library, version 2, is an offshoot of NEML, an earlier material modeling code developed at Argonne National Laboratory. NEML2 extends the key philosophy of its predecessor, i.e., material models are flexible, modular, and can be built from smaller blocks. It also provides modern features that do not exist in the framework of its predecessor such as material model vectorization, automatic differentiation, device-portable just-in-time compilation, operator fusion, lazy tensor evaluation, etc. Moreover, NEML2 can seamlessly integrate with the popular machine learning package PyTorch to take advantage of modern and fast-growing machine learning techniques. In this fiscal year, the development of core library features and capabilities are complete. The purpose of this report is not to serve as a verbatim copy of the software API reference (which is available online at https://reverendbedford.github.io/neml2/). Instead, this report documents the motivation, implementation, design choices, and usage of each core capability as well as their applications in solving practical engineering problems. This report is compiled based on the NEML2 major release 2.0.0.

36 MATERIALS SCIENCE↗

Aircraft Icing Severity Evaluation

Aircraft icing refers to the ice buildup on the surface of an aircraft flying in icing conditions. The ice accretion on the aircraft alters the original aerodynamic configuration and degrades the aerodynamic performances and may lead to unsafe flight conditions. Evaluating the flow structure, icing mechanism and consequences is of great importance to the development of an anti/deicing technique. Studies have shown computational fluid dynamics (CFD) and machine learning (ML) to be effective in predicting the ice shape and icing severity under different flight conditions. CFD solves a set of partial differential equations to obtain the air flow fields, water droplets trajectories and ice shape. ML is a branch of artificial intelligence and, based on the data, the self-improved computer algorithms can be effective in finding the nonlinear mapping relationship between the input flight conditions and the output aircraft icing severity features.

Li, Sibo (ORCID:000000021705844X)↗

A Perspective on Traditional and Data Driven Electrochemical Modeling and Analysis

To understand the behavior of electrochemical systems, we need to reduce the dimensionality of the measured current-voltage-time (I-V-t) data by fitting models, thus enabling us to analyze and compare the governing physics. Traditionally, the process for this is an 'expert first' approach: defining the model and its explicit assumptions based on inductive reasoning or empirical observation, fitting small portions of the I-V-t data where assumptions are most valid or carefully designing experiments to enforce key assumptions, and then interpreting the model parameters. However, modern data-driven methods enable a new paradigm: a 'data first' approach, where the latent behaviors governing the system's measured response are identified directly using machine-learning models that optimize both model structure and parameters from the I-V-t data, guaranteeing that the learned model explains as much of the observed system response as possible. After model identification, the model can then be interrogated by an expert to connect observed behaviors with underlying physics. This talk will review several different types of electrochemical analysis (electrochemical impedance, differential voltage-capacity, electrochemical kinetics) and compare the traditional and data-driven methods for analyzing the data.

42 ENGINEERING↗

Using Co-Optimized Machine Learned Manifolds for Modeling Chemically Reacting Flows

Chemically reacting flows play a key role in a wide range of engineered systems, from chemical and polymer processing to combustion-based energy conversion technologies. Simulations of these flows involve solving a coupled set of partial differential equations for mass, momentum, energy, and all relevant chemical species in the system. Chemical reaction pathways may be extremely complex and involve hundreds or more intermediate species, with reactions that occur over timescales varying by several orders of magnitude - presenting a significant numerical stiffness challenge. The combination of these factors makes simulation of chemically reacting flows vastly more expensive than nonreactive simulations, and often makes direct solution of the governing equations intractable. It is necessary to apply lower-fidelity models in place of the detailed governing equations in order to reduce computational cost to enable reacting flow simulation tools to be used in the engineering design process. Many of the models employed for this purpose are based on reducing the dimension of the thermochemical state, motivated by the observation that the observed thermochemical states in a system lie on a low-dimensional manifold in thermochemical state space. This behavior occurs due to the fast equilibration of certain reactive and transport processes, and physics-based manifold models rely on idealized assumptions about the balance of timescales and the way in which chemistry and transport are coupled. In this work, we apply a novel method for data-driven manifold-based modeling that can leverage data from high-fidelity reacting flow simulations to improve model accuracy in cases where the physics-based modeling assumptions break down. The approach is designed to be broadly applicable across chemically reacting flow systems but is applied here to turbulent combustion modeling.

machine learning↗

Cartesian equivariant representations for learning and understanding molecular orbitals

Qualitative and quantitative orbital properties such as bonding/antibonding character, localization, and orbital energies are critical to how chemists understand reactivity, catalysis, and excited-state behavior. Despite this, representations of orbitals in deep learning models have been very underdeveloped relative to representations of molecular geometries and Hamiltonians. Here, we apply state-of-the-art equivariant deep learning architectures to the task of assigning global labels to orbitals, namely energies characterizations, given the molecular coefficients from Hartree–Fock or density functional theory. The architecture we have developed, the Cartesian Equivariant Orbital Network (CEONET), shows how molecular orbital coefficients are readily featurized as equivariant node features common to all graph-based machine-learned potentials. We find that CEONET performs well at predicting difficult quantitative labels such as the orbital energy and orbital entropy. Furthermore, we find that the CEONET representation provides an intuitive latent space for differentiating orbital character for the qualitative assignment of e.g. bonding or antibonding character. In addition to providing a useful representation for further integrating deep learning with electronic structure theory, we expect CEONET to be useful for automatizing and interpreting the results of advanced electronic structure methods such as complete active space self-consistent field theory. In particular, the ability of CEONET to infer multireference character via the orbital entropy paves the way toward the machine-learned selection of active spaces.

chemical reactions↗

Endophyte‐induced systemic spatial reprogramming of metabolism in Populus trichocarpa roots under drought

Beneficial, facultative endophytes help plants thrive in challenging environments by altering their host's metabolism, but how these cellular scale metabolic changes propagate to the systems biology scale is unknown. In this work, we employed a high-resolution chemical imaging approach to map metabolic changes at the Populus trichocarpa root-zone and cell-type levels combined with machine learning (ML) models to identify root metabolites and exudates that have predictive power over treatment class. We found that a nine-strain consortium of beneficial endophytes differentially altered the metabolome of droughted root tissues in a manner specific to cell type and root zone, with endophyte abundance showing a clear correlation to individual metabolites. Our study demonstrates that integrating spatial metabolomics with ML can reveal localized metabolic patterns linked to root–microbe interactions and generate novel hypotheses about underlying biological mechanisms.

Drought↗

Adaptive Activation Functions Accelerate Convergence in Deep and Physics-informed Neural Networks

We employ adaptive activation functions for regression in deep and physics-informed neural networks (PINNs) to approximate smooth and discontinuous functions as well as solutions of linear and nonlinear partial differential equations. In particular, we solve the nonlinear Klein-Gordon equation, which has smooth solutions, the nonlinear Burgers equation, which can admit high gradient solutions, and the Helmholtz equation. We introduce a scalable hyper-parameter in the activation function, which can be optimized to achieve best performance of the network as it changes dynamically the topology of the loss function involved in the optimization process. The adaptive activation function has better learning capabilities than the traditional one (fixed activation) as it improves greatly the convergence rate, especially at early training, as well as the solution accuracy. To better understand the learning process, we plot the neural network solution in the frequency domain to examine how the network captures successively different frequency bands present in the solution. We consider both forward problems, where the approximate solutions are obtained, as well as inverse problems, where parameters involved in the governing equation are identified. Our simulation results show that the proposed method is a very simple and effective approach to increase the efficiency, robustness and accuracy of the neural network approximation of nonlinear functions as well as solutions of partial differential equations, especially for forward problems. We theoretically prove that in the proposed method, gradient descent algorithms are not attracted to suboptimal critical points or local minima.

machine leaning, Bad minima, Inverse problems, Phy↗

Learning emergent partial differential equations in a learned emergent space

We propose an approach to learn effective evolution equations for large systems of interacting agents. This is demonstrated on two examples, a well-studied system of coupled normal form oscillators and a biologically motivated example of coupled Hodgkin-Huxley-like neurons. For such types of systems there is no obvious space coordinate in which to learn effective evolution laws in the form of partial differential equations. In our approach, we accomplish this by learning embedding coordinates from the time series data of the system using manifold learning as a first step. In these emergent coordinates, we then show how one can learn effective partial differential equations, using neural networks, that do not only reproduce the dynamics of the oscillator ensemble, but also capture the collective bifurcations when system parameters vary. The proposed approach thus integrates the automatic, data-driven extraction of emergent space coordinates parametrizing the agent dynamics, with machine-learning assisted identification of an emergent PDE description of the dynamics in this parametrization.

97 MATHEMATICS AND COMPUTING↗

Exploring the $A$-V-Sb landscape beyond $A$$\mathrm{V_3}$ $\mathrm{Sb_5}$: A case study on the $\mathrm{KV_6}$ $\mathrm{Sb_6}$ Kagome compound

Kagome compounds have garnered attention in the past few years for their intriguing magnetic properties arising from spin frustration dictated by the geometry of the Kagome sublattice. In this paper, we highlight the success of the unconventional hydride route for the fast and easy synthesis of the Kagome compound $\mathrm{KV_6}$ $\mathrm{Sb_6}$. Here, high-temperature in situ powder x-ray diffraction (PXRD) studies proved to be useful in hinting at the existence of $\mathrm{KV_6}$ $\mathrm{Sb_6}$, identifying its synthesis conditions, and understanding the reaction mechanism. The crystal structure for $\mathrm{KV_6}$ $\mathrm{Sb_6}$ was determined from high-resolution PXRD data. The compound has a layered structure [$R\overline{3}m, a = 5.5318$ (9) Å, $c = 34.23$ (3)Å 3 $Z = 3$] at room temperature] and features a Kagome bilayer of V atoms. $\mathrm{KV_6}$ $\mathrm{Sb_6}$ is isostructural to the previously reported $\mathrm{RbV_6}$ $\mathrm{Sb_6}$ and $\mathrm{CsV_6}$ $\mathrm{Sb_6}$ compounds. $\mathrm{KV_6}$ $\mathrm{Sb_6}$ is thermally stable in vacuum up to 1173 K, as evident from the high-temperature in situ PXRD and differential scanning calorimetric analysis. Investigation of magnetic properties for $\mathrm{KV_6}$ $\mathrm{Sb_6}$ between 2 and 300 K reveals temperature-independent paramagnetism and an absence of superconductivity, like the Rb and Cs analogs. Furthermore, we compare the magnetic properties of K V 3 Sb 5 , another ternary Kagome compound, synthesized via two different methods: the hydride route and the traditional route from elements. Low-temperature transport property measurements of $\mathrm{KV_6}$ $\mathrm{Sb_6}$ indicate metallic behavior and an intrinsically low thermal conductivity of $\mathrm{1.0 W K^-1}$ $\mathrm{m^-1}$ at 300 K. The layered structure of $\mathrm{KV_6}$ $\mathrm{Sb_6}$ makes it an attractive candidate for deintercalation and doping studies to tune both magnetic and transport properties, laying a foundation for further studies.

36 MATERIALS SCIENCE↗

Joint Modeling of Quasar Variability and Accretion Disk Reprocessing Using Latent Stochastic Differential Equations

Quasars are bright active galactic nuclei powered by the accretion of matter around supermassive black holes at the center of galaxies. Their stochastic brightness variability depends on the physical properties of the accretion disk and black hole. The upcoming Rubin Observatory Legacy Survey of Space and Time (LSST) is expected to observe tens of millions of quasars, so there is a need for efficient techniques like machine learning that can handle the large volume of data. Quasar variability is believed to be driven by an X-ray corona, which is reprocessed by the accretion disk and emitted as UV/optical variability. We are the first to introduce an auto-differentiable simulation of the accretion disk and reprocessing. We use the simulation as a direct component of our neural network to jointly model the driving variability and reprocessing, trained with supervised learning on simulated LSST-like 10 yr quasar light curves. We encode the light curves using a transformer encoder, and the driving variability is reconstructed using latent stochastic differential equations, a physically motivated generative deep learning method that can model continuous-time stochastic dynamics. By embedding the physical processes of the driving signal and reprocessing into our network, we achieve a model that is more robust and interpretable. We demonstrate that our model outperforms a Gaussian process regression baseline and can infer accretion disk parameters and time delays between wave bands, even for out-of-distribution driving signals. Our approach provides a powerful framework that can be adapted to solve other inverse problems in multivariate time series.

Fagin, Joshua [City Univ. of New York (CUNY), NY (↗

Residential Demand Flexibility: Modeling Occupant Behavior using Sociodemographic Predictors

Demand flexibility (DF) has the potential to increase the saturation of renewables in the grid and reduce operating costs for both utilities and customers. However, less than 8% of U.S. residential electric customers are enrolled in DF programs. A major research gap on this topic is an uneven understanding of behavioral drivers of electricity use and DF program participation at the household level. In this study, we employ machine learning models to predict residential occupant behavior in activities relevant to DF. We model occupants' extensive decisions (i.e., choice of action) and intensive behaviors (i.e., amount of time spent) during peak and off-peak time periods using the publicly available American Time Use Survey, which includes activities data for approximately 200,000 respondents. In our machine learning models, predictions for both extensive and intensive behavior fell within a +/-20% error margin at the aggregate level. We identify 13 key sociodemographic predictors of DF-related intensive behavior using LASSO inference and beta coefficient ranking. However, these top predictors differ by activity, suggesting potential scope for differential user targeting for DF events and technologies during program design. This work also contributes to understanding when and who might adopt these DF technologies based on their daily routine activities.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

cymyc: $\underline{C}$alabi-$\underline{Y}$au $\underline{M}$etrics, $\underline{Y}$ukawas, and $\underline{C}$urvature

We introduce cymyc, a high-performance Python library for numerical investigation of the geometry of a large class of string compactification manifolds and their associated moduli spaces. We develop a well-defined geometric ansatz to numerically model tensor fields of arbitrary degree on a large class of Calabi-Yau manifolds. cymyc includes a machine learning component which incorporates this ansatz to model tensor fields of interest on these spaces by finding an approximate solution to the system of partial differential equations they should satisfy.

differential and algebraic geometry↗

Gaussian processes meet NeuralODEs: a Bayesian framework for learning the dynamics of partially observed systems from scarce and noisy data

We present a machine learning framework (GP-NODE) for Bayesian model discovery from partial, noisy and irregular observations of nonlinear dynamical systems. The proposed method takes advantage of differentiable programming to propagate gradient information through ordinary differential equation solvers and perform Bayesian inference with respect to unknown model parameters using Hamiltonian Monte Carlo sampling and Gaussian Process priors over the observed system states. This allows us to exploit temporal correlations in the observed data, and efficiently infer posterior distributions over plausible models with quantified uncertainty. The use of the Finnish Horseshoe as a sparsity-promoting prior for free model parameters also enables the discovery of parsimonious representations for the latent dynamics. A series of numerical studies is presented to demonstrate the effectiveness of the proposed GP-NODE method including predator–prey systems, systems biology and a 50-dimensional human motion dynamical system. This article is part of the theme issue ‘Data-driven prediction in dynamical systems’.

Science & Technology - Other Topics↗

Data-driven Whitney forms for structure-preserving control volume analysis

Control volume analysis models physics via the exchange of generalized fluxes between subdomains. Here, we introduce a scientific machine learning framework adopting a partition of unity architecture to identify physically-relevant control volumes, with generalized fluxes between subdomains encoded via Whitney forms. The approach provides a differentiable parameterization of geometry which may be trained in an end-to-end fashion to extract reduced models from full field data while exactly preserving physics. The architecture admits a data-driven finite element exterior calculus allowing discovery of mixed finite element spaces with closed form quadrature rules. An equivalence between Whitney forms and graph networks reveals that the geometric problem of control volume learning is equivalent to an unsupervised graph discovery problem. The framework is developed for manifolds in arbitrary dimension, with examples provided for H(div) problems in $\mathbb{R}$ establishing convergence and structure preservation properties. Finally, we consider a lithium-ion battery problem where we discover a reduced finite element space encoding transport pathways from high-fidelity microstructure resolved simulations. The approach reduces the 5.89M finite element simulation to 136 elements while reproducing pressure to under 0.1% error and preserving conservation.

97 MATHEMATICS AND COMPUTING↗

DONUT: physics-aware machine learning for real-time X-ray nanodiffraction analysis

Coherent X-ray scattering techniques are critical for investigating the fundamental structural properties of materials at the nanoscale. While advancements have made these experiments more accessible, real-time analysis remains a significant bottleneck, often hindered by artifacts and computational demands. In scanning X-ray nanodiffraction microscopy, which is widely used to spatially resolve structural heterogeneities, this challenge is compounded by the convolution of the divergent beam with the sample’s local structure. To address this, we introduce DONUT (Diffraction with Optics for Nanobeam by Unsupervised Training), a physics-aware neural network designed for the rapid and automated analysis of nanobeam diffraction data. By incorporating a differentiable geometric diffraction model directly into its architecture, DONUT learns to predict crystal lattice strain and orientation in real-time. Crucially, this is achieved without reliance on labeled datasets or pre-training, overcoming a fundamental limitation for supervised machine learning in X-ray science. We demonstrate experimentally that DONUT accurately extracts all features within the data over 200 times more efficiently than conventional fitting methods.

Materials science↗