Search NASA⌕ Search

SEARCH · Search NASA

Results for “Latent space learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Navigating Transition-Metal Chemical Space: Artificial Intelligence for First-Principles Design

Conspectus The variability of chemical bonding in open-shell transition-metal complexes not only motivates their study as functional materials and catalysts but also challenges conventional computational modeling tools. Here, tailoring ligand chemistry can alter preferred spin or oxidation states as well as electronic structure properties and reactivity, creating vast regions of chemical space to explore when designing new materials atom by atom. Although first-principles density functional theory (DFT) remains the workhorse of computational chemistry in mechanism deduction and property prediction, it is of limited use here. DFT is both far too computationally costly for widespread exploration of transition-metal chemical space and also prone to inaccuracies that limit its predictive performance for localized d electrons in transition-metal complexes. These challenges starkly contrast with the well-trodden regions of small-organic-molecule chemical space, where the analytical forms of molecular mechanics force fields and semiempirical theories have for decades accelerated the discovery of new molecules, accurate DFT functional performance has been demonstrated, and gold-standard methods from correlated wavefunction theory can predict experimental results to chemical accuracy. The combined promise of transition-metal chemical space exploration and lack of established tools has mandated a distinct approach. In this Account, we outline the path we charted in exploration of transition-metal chemical space starting from the first machine learning (ML) models (i.e., artificial neural network and kernel ridge regression) and representations for the prediction of open-shell transition-metal complex properties. The distinct importance of the immediate coordination environment of the metal center as well as the lack of low-level methods to accurately predict structural properties in this coordination environment first motivated and then benefited from these ML models and representations. Once developed, the recipe for prediction of geometric, spin state, and redox potential properties was straightforwardly extended to a diverse range of other properties, including in catalysis, computational “feasibility”, and the gas separation properties of periodic metal–organic frameworks. Interpretation of selected features most important for model prediction revealed new ways to encapsulate design rules and confirmed that models were robustly mapping essential structure–property relationships. Encountering the special challenge of ensuring that good model performance could generalize to new discovery targets motivated investigation of how to best carry out model uncertainty quantification. Distance-based approaches, whether in model latent space or in carefully engineered feature space, provided intuitive measures of the domain of applicability. With all of these pieces together, ML can be harnessed as an engine to tackle the large-scale exploration of transition-metal chemical space needed to satisfy multiple objectives using efficient global optimization methods. In practical terms, bringing these artificial intelligence tools to bear on the problems of transition-metal chemical space exploration has resulted in ML-model assessments of large, multimillion compound spaces in minutes and validated new design leads in weeks instead of decades.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

The Importance of Being Adaptable: An Exploration of the Power and Limitations of Domain Adaptation for Simulation-Based Inference with Galaxy Clusters

The application of deep machine learning methods in astronomy has exploded in the last decade, with new models showing remarkably improved performance on benchmark tasks. Not nearly enough attention is given to understanding the models' robustness, especially when the test data are systematically different from the training data, or "out of domain." Domain shift poses a significant challenge for simulation-based inference, where models are trained on simulated data but applied to real observational data. In this paper, we explore domain shift and test domain adaptation methods for a specific scientific case: simulation-based inference for estimating galaxy cluster masses from X-ray profiles. We build datasets to mimic simulation-based inference: a training set from the Magneticum simulation, a scatter-augmented training set to capture uncertainties in scaling relations, and a test set derived from the IllustrisTNG simulation. We demonstrate that the Test Set is out of domain in subtle ways that would be difficult to detect without careful analysis. We apply three deep learning methods: a standard neural network (NN), a neural network trained on the scatter-augmented input catalogs, and a Deep Reconstruction-Regression Network (DRRN), a semi-supervised deep model engineered to address domain shift. Although the NN improves results by 17% in the Training Data, it performs 40% worse on the out-of-domain Test Set. Surprisingly, the Scatter-Augmented Neural Network (SANN) performs similarly. While the DRRN is successful in mapping the training and Test Data onto the same latent space, it consistently underperforms compared to a straightforward Yx scaling relation. These results serve as a warning that simulation-based inference must be handled with extreme care, as subtle differences between training simulations and observational data can lead to unforeseen biases creeping into the results.

Ntampaka, Michelle [Baltimore, Space Telescope Sci↗
DOE OSTI2024

Polynomial chaos expansions on principal geodesic Grassmannian submanifolds for surrogate modeling and uncertainty quantification

In this work we introduce a manifold learning-based surrogate modeling framework for uncertainty quantification in high-dimensional stochastic systems. Our first goal is to perform data mining on the available simulation data to identify a set of low-dimensional (latent) descriptors that efficiently parameterize the response of the high-dimensional computational model. To this end, we employ Principal Geodesic Analysis on the Grassmann manifold of the response to identify a set of disjoint principal geodesic submanifolds, of possibly different dimension, that captures the variation in the data. Since operations on the Grassmann require the data to be concentrated, we propose an adaptive algorithm based on Riemannian K-means and the minimization of the sample Fréchet variance on the Grassmann manifold to identify “local” principal geodesic submanifolds that represent different system behavior across the parameter space. Polynomial chaos expansion is then used to construct a mapping between the random input parameters and the projection of the response on these local principal geodesic submanifolds. Here, the method is demonstrated on four test cases, a toy-example that involves points on a hypersphere, a Lotka-Volterra dynamical system, a continuous-flow stirred-tank chemical reactor system, and a two-dimensional Rayleigh-Bénard convection problem.

42 ENGINEERING↗
DOE OSTI2024

Highly sensitive 2D X-ray absorption spectroscopy via physics informed machine learning

Abstract Improving the spatial and spectral resolution of 2D X-ray near-edge absorption structure (XANES) has been a decade-long pursuit to probe local chemical reactions at the nanoscale. However, the poor signal-to-noise ratio in the measured images poses significant challenges in quantitative analysis, especially when the element of interest is at a low concentration. In this work, we developed a post-imaging processing method using deep neural network to reliably improve the signal-to-noise ratio in the XANES images. The proposed neural network model could be trained to adapt to new datasets by incorporating the physical features inherent in the latent space of the XANES images and self-supervised to detect new features in the images and achieve self-consistency. Two examples are presented in this work to illustrate the model’s robustness in determining the valence states of Ni and Co in the LiNi x Mn y Co 1-x-y O 2 systems with high confidence.

36 MATERIALS SCIENCE↗
DOE OSTI2024

PDF DECODER ANALYSIS CODE

SF-24-038"PDFdecoder", as a new application to explore parametrizations of parton distribution functions (PDFs) of the proton or other hadrons. The PDFs are fundamental quantities in particle physics which are necessary inputs to precise theoretical predictions for experiments at the Large Hadron Collider (LHC) and other facilities. As such, understanding how the PDFs are parametrized and associated uncertainties is a pressing need. The specific problem PDFdecoder confronts is the need of having a tractable and interpretably machine-learning (ML) framework to parametrize the PDFs and their uncertainties so as to understand how a given preferred parametrization is obtained. This problem has not been significantly addressed in the current literature. While other groups have used ML-based approaches to parametrize PDFs in the form of feed-forward neural networks, the question of tractability has not been explored in a PDF context. Our solution makes significant progress in this problem by using an array of encoder-decoder (essentially, autoencoder) architectures with varying constraints to the intermediate latent spaces based on interpretable physics. As a consequence, the trained models can be used as generative networks to produce interpretable predictions for the PDFs in a way that can be refined and studied further.

Hobbs, Timothy↗
DOE OSTI2021

Predictability of Localized Plasmonic Responses in Nanoparticle Assemblies

Design of nanoscale structures with desired optical properties is a key task for nanophotonics. In this work, the correlative relationship between local nanoparticle geometries and their plasmonic responses is established using encoder-decoder neural networks. In the im2spec network, the relationship between local particle geometries and local spectra is established via encoding the observed geometries to a small number of latent variables and subsequently decoding into plasmonic spectra; in the spec2im network, the relationship is reversed. Surprisingly, these reduced descriptions allow high-veracity predictions of local responses based on geometries for fixed compositions and surface chemical states. Analysis of the latent space distributions and the corresponding decoded and closest (in latent space) encoded images yields insight into the generative mechanisms of plasmonic interactions in the nanoparticle arrays. Ultimately, this approach creates a path toward determining configurations that yield the spectrum closest to the desired one, paving the way for stochastic design of nanoplasmonic structures.

nanoparticle arrays↗
DOE OSTI2022

Inverse Analysis with Variational Autoencoders: A Comparison of Shallow and Deep Networks

Inverse problems are applied to determine unknown properties by matching observational data with a physical model that often takes many parameters as input. To overcome the underconstrained nature of inverse problems and achieve good performance, an approach is presented involving regularization with a technique known as a variational autoencoder (VAE), which is trained to map a high-dimensional parameter space with a complex structure to a low-dimensional latent space with a simple structure. We apply this approach to unconditioned realizations of the parameters (heterogeneous hydraulic fields) for a hydrogeological inverse problem. Two types of hydraulic conductivity fields are used to evaluate the characterization for the different levels of heterogeneity complexity of the physical inputs. This approach keeps the computational cost of generating the training data low. The reason is unconditioned realizations neither rely on the observational data used to perform the inverse analysis nor require any groundwater flow model forward runs. In addition, this approach applies regularization on a low-dimensional latent space from the VAE and increases optimization efficiency through automatic differentiation. Furthermore, two different neural network (NN) structures are tested for their utility in using the VAE for inverse analysis. The performance of a deep, convolutional neural network strongly depends on the dimensionality of the latent space, which requires tuning. In contrast, a shallow, dense neural network provides consistently accurate characterization without tuning. Furthermore, our approach evaluates the advantages of the shallow, dense neural network over the deep, convolutional one and enables future application to a wide range of inverse problems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Uncertainty-aware mixed-variable machine learning for materials design

Abstract Data-driven design shows the promise of accelerating materials discovery but is challenging due to the prohibitive cost of searching the vast design space of chemistry, structure, and synthesis methods. Bayesian optimization (BO) employs uncertainty-aware machine learning models to select promising designs to evaluate, hence reducing the cost. However, BO with mixed numerical and categorical variables, which is of particular interest in materials design, has not been well studied. In this work, we survey frequentist and Bayesian approaches to uncertainty quantification of machine learning with mixed variables. We then conduct a systematic comparative study of their performances in BO using a popular representative model from each group, the random forest-based Lolo model (frequentist) and the latent variable Gaussian process model (Bayesian). We examine the efficacy of the two models in the optimization of mathematical functions, as well as properties of structural and functional materials, where we observe performance differences as related to problem dimensionality and complexity. By investigating the machine learning models’ predictive and uncertainty estimation capabilities, we provide interpretations of the observed performance differences. Our results provide practical guidance on choosing between frequentist and Bayesian uncertainty-aware machine learning models for mixed-variable BO in materials design.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Nonlinear proper orthogonal decomposition for convection-dominated flows

Autoencoder techniques find increasingly common use in reduced order modeling as a means to create a latent space. This reduced order representation offers a modular data-driven modeling approach for nonlinear dynamical systems when integrated with a time series predictive model. In this Letter, we put forth a nonlinear proper orthogonal decomposition (POD) framework, which is an end-to-end Galerkin-free model combining autoencoders with long short-term memory networks for dynamics. By eliminating the projection error due to the truncation of Galerkin models, a key enabler of the proposed nonintrusive approach is the kinematic construction of a nonlinear mapping between the full-rank expansion of the POD coefficients and the latent space where the dynamics evolve. We test our framework for model reduction of a convection-dominated system, which is generally challenging for reduced order models. Our approach not only improves the accuracy, but also significantly reduces the computational cost of training and testing.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2023

Lorentz group equivariant autoencoders

Abstract There has been significant work recently in developing machine learning (ML) models in high energy physics (HEP) for tasks such as classification, simulation, and anomaly detection. Often these models are adapted from those designed for datasets in computer vision or natural language processing, which lack inductive biases suited to HEP data, such as equivariance to its inherent symmetries. Such biases have been shown to make models more performant and interpretable, and reduce the amount of training data needed. To that end, we develop the Lorentz group autoencoder (LGAE), an autoencoder model equivariant with respect to the proper, orthochronous Lorentz group $$\textrm{SO}^+(3,1)$$ SO + ( 3 , 1 ) , with a latent space living in the representations of the group. We present our architecture and several experimental results on jets at the LHC and find it outperforms graph and convolutional neural network baseline models on several compression, reconstruction, and anomaly detection metrics. We also demonstrate the advantage of such an equivariant model in analyzing the latent space of the autoencoder, which can improve the explainability of potential anomalies discovered by such ML models.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2024

Unsupervised physics-informed disentanglement of multimodal data

Here, we introduce physics-informed multimodal autoencoders (PIMA) - a variational inference framework for discovering shared information in multimodal datasets. Individual modalities are embedded into a shared latent space and fused through a product-of-experts formulation, enabling a Gaussian mixture prior to identify shared features. Sampling from clusters allows cross-modal generative modeling, with a mixture-of-experts decoder that imposes inductive biases from prior scientific knowledge and thereby imparts structured disentanglement of the latent space. This approach enables cross-modal inference and the discovery of features in high-dimensional heterogeneous datasets. Consequently, this approach provides a means to discover fingerprints in multimodal scientific datasets and to avoid traditional bottlenecks related to high-fidelity measurement and characterization of scientific datasets.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Database development and exploration of process–microstructure relationships using variational autoencoders

The paper demonstrates graphical representation of a large database containing process–microstructure relationships using an unsupervised machine learning algorithm. Correlating microstructural features to processing is an essential first step to answer the difficult problem of process sequence design. Here, a large database of 346,200 orientation distribution functions resulting from a variety of process sequences is constructed, where each sequence comprises up to four stages of tension, compression and rolling along different directions in various permutations. This open-source database is constructed for collaborative development of process design algorithms. The paper demonstrates a novel application of the large database: graphical representation of texture–process relationships. A variational autoencoder is used to reduce the entire database to a two dimensional latent space where variations in processes and properties can be visualized. Using proximity analysis in this latent space, we can quickly unearth multiple process solutions to the problem of texture or property design.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Variational AutoEncoders Reveal Intensifying GPP Extremes in Continental United States based on CESM2 Simulations

Climate extremes significantly impact terrestrial carbon cycle dynamics, necessitating robust methods for detecting and analyzing anomalous behavior in plant productivity. This study presents a novel application of variational autoencoders (VAE) for identifying extreme events in gross primary productivity (GPP) from Community Earth System Model version 2 simulations across four AR6 regions in the Continental United States. We compare VAE-based anomaly detection with traditional singular spectral analysis (SSA) methods across three time periods: 1850-80, 1950-80, and 2050-80 under SSP5-8.5 scenario. The VAE architecture employs three dense layers and a latent space with input sequence length of 12 months, training on normalized GPP time series to reconstruct the GPP and identify anomalies based on reconstruction errors. Extreme events are defined using 5th percentile thresholds applied to both VAE and SSA anomalies. Results demonstrate strong regional agreement between VAE and SSA methods in spatial patterns of extreme event frequencies, despite VAE consistently producing higher threshold values (179-756 GgC for VAE vs. 100-784 GgC for SSA across regions and periods). Both methods reveal increasing magnitudes and frequencies of negative carbon cycle extremes toward 2050-80, particularly in Western and Central North America. The VAE approach shows comparable performance to established SSA techniques while offering computational advantages and enhanced capability for capturing non-linear temporal dependencies in carbon cycle variability. This research demonstrates the potential of deep learning approaches for extremes detection and provides a foundation for improved understanding of future carbon cycle risks under future conditions.

Sharma, Bharat [ORNL] (ORCID:0000000266982487)↗
DOE OSTI2024

PreMevE‐MEO: Predicting Ultra‐Relativistic Electrons Using Observations From GPS Satellites

Abstract Ultra‐relativistic electrons with energies greater than or equal to two megaelectron‐volt (MeV) pose a major radiation threat to spaceborne electronics, and thus specifying those highly energetic electrons has a significant meaning to space weather communities. Here we report the latest progress in developing our predictive model for MeV electrons in the outer radiation belt. The new version, primarily driven by electron measurements made along medium‐Earth‐orbits (MEO), is called PREdictive MEV Electron (PreMevE)‐MEO model that nowcasts ultra‐relativistic electron flux distributions across the whole outer belt. Model inputs include >2 MeV electron fluxes observed in MEOs by a fleet of GPS satellites as well as electrons measured by one Los Alamos satellite in the geosynchronous orbit. We developed an innovative Sparse Multi‐Inputs Latent Ensemble NETwork (SmileNet) which combines convolutional neural networks with transformers, and we used long‐term in situ electron data from NASA's Van Allen Probes mission to train, validate, optimize, and test the model. It is shown that PreMevE‐MEO can provide hourly nowcasts with high model performance efficiency and high correlation with observations. This prototype PreMevE‐MEO model demonstrates the feasibility of making high‐fidelity predictions driven by observations from longstanding space infrastructure in MEO, thus has great potential of growing into an invaluable space weather operational warning tool.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

An artificial intelligence’s interpretation of complex high-resolution in situ transmission electron microscopy data

Complicated nano- and atomic-scale processes with sub-angstrom spatial resolution and millisecond time resolution visualized by in situ transmission electron microscopy (TEM) are often highly dynamical and time consuming to analyze and interpret. Here, we report how variational autoencoders (VAEs) can provide an artificial intelligence’s interpretation of high-resolution in situ TEM data by condensing and deconvoluting complicated atomic-scale dynamics into a latent space with reduced dimensionality. We designed a VAE model with high latent dimensions capable of deconvoluting information from complex high-resolution TEM data. We demonstrate how this model, with high latent dimensions trained on atomically resolved TEM images of lead sulfide (PbS) nanocrystals, is able to capture movements and perturbations of periodic lattices in both simulated and real in situ TEM data. Importantly, the VAE model shows the capability of detecting and deconvoluting dynamical nanoscale physical processes, such as the rotation of crystal lattices and intra-particle ripening during the annealing of semiconductor nanocrystals.

47 OTHER INSTRUMENTATION↗
DOE OSTI2022

Variational encoder geostatistical analysis (VEGAS) with an application to large scale riverine bathymetry

Estimation of riverbed profiles, also known as bathymetry, plays a vital role in many applications, such as safe and efficient inland navigation, prediction of bank erosion, land subsidence, and flood risk management. The high cost and complex logistics of direct bathymetry surveys, i.e, depth imaging, have encouraged the use of indirect measurements such as surface flow velocities. However, estimating high-resolution bathymetry from indirect measurements is an inverse problem that can be computationally challenging. Here, we propose a reduced-order model (ROM) based approach that utilizes a variational autoencoder (VAE), a type of deep neural network with a narrow layer in the middle, to compress bathymetry and flow velocity information and accelerate bathymetry inverse problems from flow velocity measurements. In our application, the shallow-water equations (SWE) with appropriate boundary conditions (BCs), e.g., the discharge and/or the free surface elevation, constitute the forward problem, to predict flow velocity. Then, ROMs of the SWEs are constructed on a nonlinear manifold of low dimensionality through a variational encoder and the bathymetry inversion problem is derived on the low-dimensional latent space in a Hierarchical Bayesian setting. Further, the reformulation allows variational inference with a small number (e.g., $\mathscr{O}$ (100) of ROM runs and efficient uncertainty quantification. We have tested our inversion approach on a one-mile reach of the Savannah River, GA, USA. Once the neural network is trained (offline stage), the proposed technique can perform the inversion operation orders of magnitude faster than traditional inversion methods that are commonly based on linear projections, such as principal component analysis (PCA), or the principal component geostatistical approach (PCGA). Furthermore, tests show that the algorithm can estimate the bathymetry with good accuracy even with sparse flow velocity measurements.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Bi-fidelity variational auto-encoder for uncertainty quantification

Quantifying the uncertainty of quantities of interest (QoIs) from physical systems is a primary objective in model validation. However, achieving this goal entails balancing the need for computational efficiency with the requirement for numerical accuracy. To address this trade-off, we propose a novel bi-fidelity formulation of variational auto-encoders (BF-VAE) designed to estimate the uncertainty associated with a QoI from low-fidelity (LF) and high-fidelity (HF) samples of the QoI. Here, this model allows for the approximation of the statistics of the HF QoI by leveraging information derived from its LF counterpart. Specifically, we design a bi-fidelity auto-regressive model in the latent space which is integrated within the VAE’s probabilistic encoder–decoder structure. An effective algorithm is proposed to maximize the variational lower bound of the HF log-likelihood in the presence of limited HF data, resulting in the synthesis of HF realizations with a reduced computational cost. Additionally, we introduce the concept of the bi-fidelity information bottleneck (BF-IB) to provide an information-theoretic interpretation of the proposed BF-VAE model. Our numerical results demonstrate that the BF-VAE leads to considerably improved accuracy, as compared to a VAE trained using only HF data, when limited HF data is available.

42 ENGINEERING↗
DOE OSTI2025

Mitigating spectral bias in neural operators via high-frequency scaling for physical systems

Neural operators have emerged as powerful surrogates for modeling complex physical problems. However, they suffer from spectral bias making them oblivious to high-frequency modes, which are present in multiscale physical systems. Therefore, they tend to produce over-smoothed solutions, which is particularly problematic in modeling turbulence and for systems with intricate patterns and sharp gradients such as multi-phase flow systems. In this work, we introduce a new approach named high-frequency scaling (HFS) to mitigate spectral bias in convolutional-based neural operators. By integrating HFS with proper variants of UNet, we demonstrate a higher prediction accuracy by mitigating spectral bias in single and two-phase flow problems. Unlike Fourierbased techniques, HFS is directly applied to the latent space, thus eliminating the computational cost associated with the Fourier transform. Additionally, we investigate alternative spectral bias mitigation through a diffusion model conditioned on neural operators. While the diffusion model integrated with the standard neural operator may still suffer from significant errors, these errors are substantially reduced when the diffusion model is integrated with a HFS-enhanced neural operator.

97 MATHEMATICS AND COMPUTING↗