Search NASA⌕ Search

SEARCH · Search NASA

Results for “latent space”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Unsupervised machine learning discovery of structural units and transformation pathways from imaging data

We show that unsupervised machine learning can be used to learn chemical transformation pathways from observational Scanning Transmission Electron Microscopy (STEM) data. To enable this analysis, we assumed the existence of atoms, a discreteness of atomic classes, and the presence of an explicit relationship between the observed STEM contrast and the presence of atomic units. With only these postulates, we developed a machine learning method leveraging a rotationally invariant variational autoencoder (VAE) that can identify the existing molecular fragments observed within a material. The approach encodes the information contained in STEM image sequences using a small number of latent variables, allowing the exploration of chemical transformation pathways by tracing the evolution of atoms in the latent space of the system. The results suggest that atomically resolved STEM data can be used to derive fundamental physical and chemical mechanisms involved, by providing encodings of the observed structures that act as bottom-up equivalents of structural order parameters. The approach also demonstrates the potential of variational (i.e., Bayesian) methods in the physical sciences and will stimulate the development of more sophisticated ways to encode physical constraints in the encoder–decoder architectures and generative physical laws and causal relationships in the latent space of VAEs.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Evaluating quenching in cosmological simulations of galaxy formation with spectral covariance in the optical window

ABSTRACT Cosmological hydrodynamical simulations provide valuable insights on galaxy evolution when coupled with observational data. Comparisons with real galaxies are typically performed via scaling relations of the observables. Here, we follow an alternative approach based on the spectral covariance in a model-independent way. We build upon previous work by Sharbaf et al. that studied the covariance of high-quality SDSS (Sloan Digital Sky Survey) continuum-subtracted spectra in a relatively narrow range of velocity dispersion ($\sigma \in [100,150]$ km s$^{-1}$). Here, the same analysis is applied to synthetic data from the eagle and IllustrisTNG100 simulations, to assess the ability of these runs to mimic real galaxies. The real and simulated spectra are consistent regarding spectral covariance, although with subtle differences that can inform the implementation of subgrid physics. Spectral fitting done a posteriori on stacks segregated with respect to latent space reveals that the first principal component (PC1) is predominantly influenced by the stellar age distribution, with an underlying age–metallicity degeneracy. Good agreement is found regarding star formation prescriptions but there is disagreement with active galactic nucleus (AGN) feedback, that also affects the subset of quiescent galaxies. We show a substantial difference in the implementation of the AGN subgrid prescriptions, regarding central black hole seeding, that could lead to the mismatch. Differences are manifest between these two simulations in the star formation histories stacked with respect to latent space. We emphasize that this methodology only relies on the spectral covariance to assess whether simulations provide a true representation of galaxy formation.

Sharbaf, Z. (ORCID:0009000450545946)↗
DOE OSTI2021

Nonlinear proper orthogonal decomposition for convection-dominated flows

Autoencoder techniques find increasingly common use in reduced order modeling as a means to create a latent space. This reduced order representation offers a modular data-driven modeling approach for nonlinear dynamical systems when integrated with a time series predictive model. In this Letter, we put forth a nonlinear proper orthogonal decomposition (POD) framework, which is an end-to-end Galerkin-free model combining autoencoders with long short-term memory networks for dynamics. By eliminating the projection error due to the truncation of Galerkin models, a key enabler of the proposed nonintrusive approach is the kinematic construction of a nonlinear mapping between the full-rank expansion of the POD coefficients and the latent space where the dynamics evolve. We test our framework for model reduction of a convection-dominated system, which is generally challenging for reduced order models. Our approach not only improves the accuracy, but also significantly reduces the computational cost of training and testing.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2026

Scaling kinetic Monte-Carlo simulations of grain growth with combined convolutional and graph neural networks

Graph neural networks (GNN) have emerged as a promising machine learning method for microstructure simulations such as grain growth. However, accurate modeling of realistic grain boundary networks requires large simulation cells, which GNN has difficulty scaling up to. To alleviate the computational costs and memory footprint of GNN, we suggest a hybrid architecture combining a convolutional neural network (CNN) based bijective autoencoder to compress the spatial dimensions, and a GNN that evolves the microstructure in the latent space of reduced spatial sizes. Our results demonstrate that the new design significantly reduces computational costs with using fewer message passing layer (from 12 down to 3) compared with GNN alone. The reduction in computational cost becomes more pronounced as the spatial size increases, indicating strong computational scalability. For the largest mesh evaluated (160 3 ), our method reduces memory usage and runtime in inference by 117× and 115×, respectively, compared with GNN-only baseline. More importantly, it shows higher accuracy and stronger spatiotemporal capability than the GNN-only baseline, especially in long-term testing. Such combination of scalability and accuracy is essential for simulating realistic material microstructures over extended time scales. The improvements can be attributed to the bijective autoencoder’s ability to compress information losslessly from spatial domain into a high dimensional feature space, thereby producing more expressive latent features for the GNN to learn from, while also contributing its own spatiotemporal modeling capability. Training data are generated from stochastic grain growth simulations, providing realistic variability for learning robust microstructure evolution. Comprehensive system validation confirms that the model is accurate, robust, and scalable.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Learning nuclear cross sections across the chart of nuclides with graph neural networks

We explore the use of deep learning techniques to learn how nuclear cross sections change as we add or remove protons and neutrons. As a proof of principle, we focus on the neutron-induced reactions in the fast energy regime. Our approach follows a two-stage learning framework. First, we apply representation learning to encode cross section data into a latent space using either variational autoencoders (VAEs) or implicit neural representations (INRs). Then, we train graph neural networks (GNNs) on the resulting embeddings to predict missing values across the nuclear chart by leveraging the topological structure of neighboring isotopes. We demonstrate accurate cross section predictions within a 9 × 9 block of missing nuclei. We also find that the optimal GNN training strategy depends on the type of latent representation used, with VAE embeddings performing best under end-to-end optimization in the original space, while INR embeddings achieve better results when the GNN is trained only in the latent space. Furthermore, using clustering algorithms, we map groups of latent vectors into regions of the nuclear chart and show that VAEs and INRs can discover some of the neutron magic numbers. These findings suggest that deep-learning models based on the representation encoding of cross sections combined with graph neural networks hold significant potential in augmenting nuclear theory models, e.g., by providing reliable estimates of covariances of cross sections, including cross-material covariances.

Machine learning↗
DOE OSTI2021

Disentangling Rotational Dynamics and Ordering Transitions in a System of Self-Organizing Protein Nanorods via Rotationally Invariant Latent Representations

The dynamics of complex ordering systems with active rotational degrees of freedom exemplified by protein self-assembly is explored using a machine learning workflow that combines deep learning-based semantic segmentation and rotationally invariant variational autoencoder-based analysis of orientation and shape evolution. The latter allows for disentanglement of the particle orientation from other degrees of freedom and compensates for lateral shifts. The disentangled representations in the latent space encode the rich spectrum of local transitions that can now be visualized and explored via continuous variables. The time dependence of ensemble averages allows insight into the time dynamics of the system and, in particular, illustrates the presence of the potential ordering transition. Finally, analysis of the latent variables along the single-particle trajectory allows tracing these parameters on a single-particle level. The proposed approach is expected to be universally applicable for the description of the imaging data in optical, scanning probe, and electron microscopy seeking to understand the dynamics of complex systems where rotations are a significant part of the process.

representation learning↗
DOE OSTI2023

Visualization and efficient generation of constrained high-dimensional theoretical parameter spaces

We describe a set of novel methods for efficiently sampling high-dimensional parameter spaces of physical theories defined at high energies, but constrained by experimental measurements made at lower energies. Often, theoretical models such as supersymmetry are defined by many parameters, $\mathcal{O}$(10 – 100), expressed at high energies, while relevant experimental constraints are often defined at much lower energies, preventing them from directly ruling out portions of the space. Instead, the low-energy constraints define a complex, potentially non-contiguous subspace of the theory parameters. Naive scanning of the theory space for points which satisfy the low-energy constraints is hopelessly inefficient due to the high dimensionality, and the inverse problem is considered intractable. As a result, many theoretical spaces remain under-explored. We introduce a class of modified generative autoencoders, which attack this problem by mapping the high-dimensional parameter space to a structured low-dimensional latent space, allowing for easy visualization and efficient generation of theory points which satisfy experimental constraints. An extension without dimensional compression, which focuses on limiting potential information loss, is also introduced.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2023

A variational encoder–decoder approach to precise spectroscopic age estimation for large Galactic surveys

Constraints on the formation and evolution of the Milky Way Galaxy require multidimensional measurements of kinematics, abundances, and ages for a large population of stars. Ages for luminous giants, which can be seen to large distances, are an essential component of studies of the Milky Way, but they are traditionally very difficult to estimate precisely for a large data set and often require careful analysis on a star-by-star basis in asteroseismology. Because spectra are easier to obtain for large samples, being able to determine precise ages from spectra allows for large age samples to be constructed, but spectroscopic ages are often imprecise and contaminated by abundance correlations. Here we present an application of a variational encoder–decoder on cross-domain astronomical data to solve these issues. The model is trained on pairs of observations from APOGEE and Kepler of the same star in order to reduce the dimensionality of the APOGEE spectra in a latent space while removing abundance information. The low dimensional latent representation of these spectra can then be trained to predict age with just ∼1000 precise seismic ages. We demonstrate that this model produces more precise spectroscopic ages (∼ 22 per cent overall, ∼ 11 per cent for red-clump stars) than previous data-driven spectroscopic ages while being less contaminated by abundance information (in particular, our ages do not depend on [α/M]). We create a public age catalogue for the APOGEE DR17 data set and use it to map the age distribution and the age-[Fe/H]-[α/M] distribution across the radial range of the Galactic disc.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Manifold Learning-Based Polynomial Chaos Expansions for High-Dimensional Surrogate Models

In this work we introduce a manifold learning-based method for uncertainty quantification (UQ) in systems describing complex spatiotemporal processes. Our first objective is to identify the embedding of a set of high-dimensional data representing quantities of interest of the computational or analytical model. For this purpose, we employ Grassmannian diffusion maps, a two-step nonlinear dimension reduction technique which allows us to reduce the dimensionality of the data and identify meaningful geometric descriptions in a parsimonious and inexpensive manner. Polynomial chaos expansion is then used to construct a mapping between the stochastic input parameters and the diffusion coordinates of the reduced space. An adaptive clustering technique is proposed to identify an optimal number of clusters of points in the latent space. The similarity of points allows us to construct a number of geometric harmonic emulators which are finally utilized as a set of inexpensive pretrained models to perform an inverse map of realizations of latent features to the ambient space and thus perform accurate out-of-sample predictions. Thus, the proposed method acts as an encoder-decoder system which is able to automatically handle very high-dimensional data while simultaneously operating successfully in the small-data regime. The method is demonstrated on two benchmark problems and on a system of advection-diffusion-reaction equations which model a first-order chemical reaction between two species. In all test cases, the proposed method is able to achieve highly accurate approximations which ultimately lead to the significant acceleration of UQ tasks.

42 ENGINEERING↗
DOE OSTI2023

Multiscale graph neural network autoencoders for interpretable scientific machine learning

The goal of this work is to address two limitations in autoencoder-based models: latent space interpretability and compatibility with unstructured meshes. This is accomplished here with the development of a novel graph neural network (GNN) autoencoding architecture with demonstrations on complex fluid flow applications. To address the first goal of interpretability, the GNN autoencoder achieves reduction in the number nodes in the encoding stage through an adaptive graph reduction procedure. Further, this reduction procedure essentially amounts to flowfieldconditioned node sampling and sensor identification, and produces interpretable latent graph representations tailored to the flowfield reconstruction task in the form of so-called masked fields. These masked fields allow the user to (a) visualize where in physical space a given latent graph is active, and (b) interpret the time-evolution of the latent graph connectivity in accordance with the time-evolution of unsteady flow features (e.g. recirculation zones, shear layers) in the domain. To address the goal of unstructured mesh compatibility, the autoencoding architecture utilizes a series of multi-scale message passing (MMP) layers, each of which models information exchange among node neighborhoods at various lengthscales. The MMP layer, which augments standard single-scale message passing with learnable coarsening operations, allows the decoder to more efficiently reconstruct the flowfield from the identified regions in the masked fields. Analysis of latent graphs produced by the autoencoder for various model settings are conducted using unstructured snapshot data sourced from large-eddy simulations in a backward-facing step (BFS) flow configuration with an OpenFOAM-based flow solver at high Reynolds numbers.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Self-supervised learning of spatiotemporal thermal signatures in additive manufacturing using reduced order physics models and transformers

Microstructure control via additive manufacturing has enormous potential as manufacturers, materials scientists, and designers alike seek to exploit novel fabrication technologies to improve component performance. Recent works have demonstrated the feasibility of producing materials with controlled microstructures across various length scales. However, the experimental approach towards exploring the process-structure space can be laborious and costly. This is particularly true if also considering scan pattern optimization which is well suited for processes such as powder bed fusion electron beam melting. In this work we propose an approach for encoding additive manufacturing layer-wise thermal response signatures using self-supervised representation learning. Thermal simulations from a reduced order model are utilized to estimate the spatiotemporal response during printing. A machine learning framework, using video-transformers, is utilized to efficiently distill spatiotemporal patterns into a compact latent space representation. This latent state representation encodes the relevant physics which is then utilized to establish a data-driven process-structure model for an additively manufactured Ni-based superalloy. In conclusion, the proposed methodology could potentially be used towards in-situ process monitoring, scan pattern experimental design, and component qualification.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

An artificial intelligence’s interpretation of complex high-resolution in situ transmission electron microscopy data

Complicated nano- and atomic-scale processes with sub-angstrom spatial resolution and millisecond time resolution visualized by in situ transmission electron microscopy (TEM) are often highly dynamical and time consuming to analyze and interpret. Here, we report how variational autoencoders (VAEs) can provide an artificial intelligence’s interpretation of high-resolution in situ TEM data by condensing and deconvoluting complicated atomic-scale dynamics into a latent space with reduced dimensionality. We designed a VAE model with high latent dimensions capable of deconvoluting information from complex high-resolution TEM data. We demonstrate how this model, with high latent dimensions trained on atomically resolved TEM images of lead sulfide (PbS) nanocrystals, is able to capture movements and perturbations of periodic lattices in both simulated and real in situ TEM data. Importantly, the VAE model shows the capability of detecting and deconvoluting dynamical nanoscale physical processes, such as the rotation of crystal lattices and intra-particle ripening during the annealing of semiconductor nanocrystals.

47 OTHER INSTRUMENTATION↗
DOE OSTI2021

Variational Autoencoders for Learning Nonlinear Dynamics of Physical Systems

We develop data-driven methods for incorporating physical information for priors to learn parsimonious representations of nonlinear systems arising from parameterized PDEs and mechanics. Our approach is based on Variational Autoencoders (VAEs) for learning nonlinear state space models from observations. We develop ways to incorporate geometric and topological priors through general manifold latent space representations. We investigate the performance of our methods for learning low dimensional representations for the nonlinear Burgers equation and constrained mechanical systems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Long–short-term memory encoder–decoder with regularized hidden dynamics for fault detection in industrial processes

The ability of recurrent neural networks (RNN) to model nonlinear dynamics of high dimensional process data has enabled data-driven RNN-based fault detection algorithms. Previous studies have focused on detecting faults by identifying the discrepancies in data distribution between the faulty and normal data, as reflected in prediction errors generated by RNN models. However, in industrial processes, variations in data distribution can also result from changes in normal control setpoints and compensatory control adjustments in response to disturbances, making it hard to differentiate between normal and faulty conditions. This paper proposes a fault detection method utilizing a long short-term memory (LSTM) encoder–decoder structure with regularized hidden dynamics and reversible instance normalization (RevIN) to compactly represent high-dimensional measurements for effective monitoring. During training, the hidden states of the model are regularized to form a low-dimensional latent space representation of the original multivariate time series data. As a result, the prediction errors of the latent states can be used to monitor the abnormal dynamic variations, while the reconstruction errors of the measured variables are used to monitor the abnormal static variations. Furthermore, the proposed indices can reflect operating conditions, even when the distribution of test data changes, which helps distinguish faults from normal adjustments and disturbances that controllers can settle. Here, data from numerical simulation and the Tennessee Eastman process are used to illustrate the effectiveness of the proposed fault detection method.

42 ENGINEERING↗
DOE OSTI2023

A Graphical Model for Fusing Diverse Microbiome Data

This paper develops a Bayesian graphical model for fusing disparate types of count data. The motivating application is the study of bacterial communities from diverse high-dimensional features, in this case, transcripts, collected from different treatments. In such datasets, there are no explicit correspondences between the communities and each corresponds to different factors, making data fusion challenging. We introduce a flexible multinomial-Gaussian generative model for jointly modeling such count data. This latent variable model jointly characterizes the observed data through a common multivariate Gaussian latent space that parameterizes the set of multinomial probabilities of the transcriptome counts. The covariance matrix of the latent variables induces a covariance matrix of co-dependencies between all the transcripts, effectively fusing multiple data sources. We present a computationally scalable variational Expectation-Maximization (EM) algorithm for inferring the latent variables and the parameters of the model. Here, the inferred latent variables provide a common dimensionality reduction for visualizing the data and the inferred parameters provide a predictive posterior distribution. In addition to simulation studies that demonstrate the variational EM procedure, we apply our model to a bacterial microbiome dataset.

59 BASIC BIOLOGICAL SCIENCES↗