Search NASA⌕ Search

SEARCH · Search NASA

Results for “latent variable models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Evaluation of the Near-Surface Variables in the HRRR Weather Model Using Observations from the ARM SGP Site

Abstract The performance of version 4 of the NOAA High-Resolution Rapid Refresh (HRRR) numerical weather prediction model for near-surface variables, including wind, humidity, temperature, surface latent and sensible fluxes, and longwave and shortwave radiative fluxes, is examined over the Atmospheric Radiation Measurement (ARM) Southern Great Plains (SGP) region. The study evaluated the model’s bias and bias-corrected mean absolute error relative to the observations on different time scales. Forecasts of near-surface geophysical variables at five SGP sites (HRRR at 3-km scale) were found to agree well with observations, but some consistent observation–forecast differences also occurred. Sensible and latent heat fluxes are the most challenging variables to be reproduced. The diurnal cycle is the main temporal scale affecting observation–forecast differences of the near-surface variables, and almost all of the variables showed different biases throughout the diurnal cycle. Results show that the overestimation of downward shortwave and the underestimation of downward longwave radiative flux are the two major biases found in this study. The timing and magnitude of downward longwave flux, wind speed, and sensible and latent heat fluxes are also different with contributions from model representations, data assimilation limitations, and differences in scales between HRRR and SGP sites. The positive bias in downward shortwave and negative bias in longwave radiation suggests that the model is underestimating cloud fraction in the study domain. The study concludes by showing a brief comparison with version 3 of the HRRR and shows that version 4 has better performance in almost all near-surface variables. Significance Statement A correct representation of the near-surface variables is important for numerical weather prediction models. This study investigates the capability of the latest NOAA High-Resolution Rapid Refresh (HRRRv4) model in simulating the near-surface variables by comparing against the Atmospheric Radiation Measurement (ARM) Southern Great Plains (SGP) in situ observations. Among others, we find that the surface heat fluxes, such as sensible and latent heat fluxes, are the most difficult variables to be reproduced. This study also shows that the diurnal cycle has the dominant impact on the model’s performance, which means the majority of the outputted near-surface variables have the strong diurnal cycle in their bias errors.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Integrated Design of Ultradurable, Low CO 2 Alternative Binder Systems via Machine Learning

This ARPA-E project developed a machine learning tool to use in formulation design of cementitious binders for concrete having 50% less embodied CO 2 and possessing twice the durability compared to concrete based on ordinary portland cement (OPC) binders. The technical focus was on limestone/calcined clay cement (LC3), the leading replacement for OPC. Here, hierarchical machine learning (HML) was used to model the flowability, set time, strength, and durability of LC3 concrete. This methodology identifies latent variables derived from domain knowledge and empirical models that develop an accurate model for a response surface from small datasets. For the flowability metric, particle packing was a dominant factor, while strength and durability were both strongly determined by the fraction of metakaolin and the water:solids ratio. Under constraints of water:binder ratio, material performance metrics, embodied CO 2 , and cost per tonne of OPC, multi-objective optimization was used to design binders parameterized by the mineral composition replacing OPC, particle size distributions, and water:solids ratio. The trained algorithm was able to predict multiple mixes met these performance criteria, and experimental testing validated the predictions. The model demonstrated here is relevant for North America, where pure kaolin deposits are found broadly. The approach is being taken forward into commercial application by Ansatz AI, a materials informatics company founded by PI Washburn and co-PI Poczos. Through collaborations with the cement and concrete industry, and funding from SBIR programs, a commercial software will be developed in future research.

36 MATERIALS SCIENCE↗
DOE OSTI2024

ML-based Dimension Reduction Strategies

Deep learning (DL)--based surrogate models have achieved success in various applications in carbon capture and storage (CCS). However, the model training on high-dimensional spaces is computationally expensive and impractical for large-scale and complex geological models, because the models usually contain hundreds of thousands to millions of grid cells, each with a set of parameters. Furthermore, the high cost of generating training data with sufficient variation is another limitation of model training on high-dimensional spaces, which may result in overfitting and reduce the model efficiency and prediction performance. We proposed the workflow incorporating dimension reduction methods and deep learning models, which aim to extract the latent variables of input parameters and output state variables, and then build the mapping function at the latent spaces. The proposed workflow can significantly reduce the computational complexity in solving both forward and inverse problems compared to models trained on high-dimensional spaces. Dimensionality reduction models showed great potential in workflows for fast reservoir simulation, history matching, prior model generation, visualization, and more, ultimately enhancing DL model performance in related SMART Work Packages.

Hosseini, Seyyed↗
DOE OSTI2020

Single Episode Policy Transfer in Reinforcement Learning

Transfer and adaptation to new unknown environmental dynamics is a key challenge for reinforcement learning (RL). An even greater challenge is performing near-optimally in a single attempt at test time, possibly without access to dense rewards, which is not addressed by current methods that require multiple experience rollouts for adaptation. To achieve single episode transfer in a family of environments with related dynamics, we propose a general algorithm that optimizes a probe and an inference model to rapidly estimate underlying latent variables of test dynamics, which are then immediately used as input to a universal control policy. This modular approach enables integration of state-of-the-art algorithms for variational inference or RL. Moreover, our approach does not require access to rewards at test time, allowing it to perform in settings where existing adaptive approaches cannot. In diverse experimental domains with a single episode test constraint, we report our method significantly outperforms existing adaptive approaches and shows favorable performance against baselines for robust transfer.

42 ENGINEERING↗
DOE OSTI2020

Directivity Modes of Earthquake Populations with Unsupervised Learning

We present a novel approach for resolving modes of rupture directivity in large populations of earthquakes. A seismic spectral decomposition technique is used to first produce relative measurements of radiated energy for earthquakes in a spatially compact cluster. The azimuthal distribution of energy for each earthquake is then assumed to result from one of several distinct modes of rupture propagation. Rather than fitting a kinematic rupture model to determine the most likely mode of rupture propagation, we instead treat the modes as latent variables and learn them with a Gaussian mixture model. The mixture model simultaneously determines the number of events that best identify with each mode. The technique is demonstrated on four datasets in California, each with compact clusters of several thousand earthquakes with comparable slip mechanisms. We show that the datasets naturally decompose into distinct rupture propagation modes that correspond to different rupture directions, and the fault plane is unambiguously identified for all cases. We find that these small earthquakes exhibit unilateral ruptures 63–73% of the time on average. Here, the results provide important observational constraints on the physics of earthquakes and faults.

58 GEOSCIENCES↗
DOE OSTI2022

Unsupervised probabilistic models for sequential Electronic Health Records

We develop an unsupervised probabilistic model for heterogeneous Electronic Health Record (EHR) data. Utilizing a mixture model formulation, our approach directly models sequences of arbitrary length, such as medications and laboratory results. This allows for subgrouping and incorporation of the dynamics underlying heterogeneous data types. The model consists of a layered set of latent variables that encode underlying structure in the data. These variables represent subject subgroups at the top layer, and unobserved states for sequences in the second layer. We train this model on episodic data from subjects receiving medical care in the Kaiser Permanente Northern California integrated healthcare delivery system. The resulting properties of the trained model generate novel insight from these complex and multifaceted data. In addition, we show how the model can be used to analyze sequences that contribute to assessment of mortality likelihood.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

tLaSDI: Thermodynamics-informed latent space dynamics identification

Here we propose a latent space dynamics identification method, namely tLaSDI, that embeds the first and second principles of thermodynamics. The latent variables are learned through an autoencoder as a nonlinear dimension reduction model. The latent dynamics are constructed by a neural network-based model that precisely preserves certain structures for the thermodynamic laws through the GENERIC formalism. An abstract error estimate is established, which provides a new loss formulation involving the Jacobian computation of autoencoder. The autoencoder and the latent dynamics are simultaneously trained to minimize the new loss. Computational examples demonstrate the effectiveness of tLaSDI, which exhibits robust generalization ability, even in extrapolation. In addition, an intriguing correlation is empirically observed between a quantity from tLaSDI in the latent space and the behaviors of the full-state solution.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Thermodynamics-informed latent space dynamics identification

This software showcases a latent space dynamics identification method, namely tLaSDI, that embeds the first and second principles of thermodynamics. The latent variables are learned through an autoencoder as a nonlinear dimension reduction model. The latent dynamics are constructed by a neural network-based model that precisely preserves certain structures for the thermodynamic laws through the GENERIC formalism. An abstract error estimate is established, which provides a new loss formulation involving the Jacobian computation of autoencoder. The autoencoder and the latent dynamics are simultaneously trained to minimize the new loss. Computational examples demonstrate the effectiveness of tLaSDI, which exhibits robust generalization ability, even in extrapolation. In addition, an intriguing correlation is empirically observed between a quantity from tLaSDI in the latent space and the behaviors of the full-state solution.

Cheung, Siu Wun↗
DOE OSTI2022

Towards sharp error analysis of extended Lagrangian molecular dynamics

The extended Lagrangian molecular dynamics (XLMD) method provides a useful framework for reducing the computational cost of a class of molecular dynamics simulations with constrained latent variables. The XLMD method relaxes the constraints by introducing a fictitious mass ε for the latent variables and solving a set of singularly perturbed ordinary differential equations. While favorable numerical performance of XLMD has been demonstrated in several different contexts in the past decade, mathematical analysis of the method remains scarce. Here, we propose the first error analysis of the XLMD method in the context of a classical polarizable force field model. While the dynamics with respect to the atomic degrees of freedom are general and nonlinear, the key mathematical simplification of the polarizable force field model is that the constraints on the latent variables are given by a linear system of equations. We prove that when the initial value of the latent variables is compatible in a sense that we define, XLMD converges as the fictitious mass ε is made small with $\mathscr{O}$(ε) error for the atomic degrees of freedom and with $\mathscr{O}$($\sqrt{ε}$) error for the latent variables, when the dimension of the latent variable d' is 1. Furthermore, when the initial value of the latent variables is improved to be optimally compatible in a certain sense, we prove that the convergence rate can be improved to $\mathscr{O}$(ε) for the latent variables as well. Numerical results verify that both estimates are sharp not only for d'=1, but also for arbitrary d'. In the setting of general d', we do obtain convergence, but with the non-sharp rate of $\mathscr{O}$($\sqrt{ε}$) for both the atomic and latent variables.

74 ATOMIC AND MOLECULAR PHYSICS↗
DOE OSTI2021

Domain knowledge-informed, process-mapping AI graph for designing Fe-based alloys

<span style="font-family: Calibri, sans-serif; font-size: 12pt;">Continuous improvement in efficiency of a power plant relies on designing materials for use at increasingly higher temperature and/or pressure, for 100,000s hours of operation. Due to complexity, non-linearity and high-dimensionality of the problem, traditional Machine Learning (ML) approaches require unreasonably large datasets for the data-driven model development. Science-based material and process engineering complements hard data with, sometimes soft and intuitive, empirical domain knowledge. Artificial Intelligence (AI) was used in this study to incorporate such knowledge into computational graph architecture (process-mimicking artificial neuron design, causal layer and graph structures, ensemble modeling of latent states) and learning procedures (variable transformation, fuzzy physics pre-training and freezing of deep layers, virtual microstructure representation, and adversarial multi-objective optimization). The first alloys design pathways suggested by the AI tool (pyroMind) passed a preliminary engineering review on soundness and transparency.</span>

Romanov, Vyacheslav↗
DOE OSTI2022

Autoencoder Neural Network for chemically reacting systems

Incorporating detailed chemical kinetic models is critical for accurate simulations of reacting flows. However, detailed models involve a large number of thermochemical (TC) state variables. Solving the governing equations to evolve these TC variables becomes impractical for real-world applications. In this work, we propose an autoencoder (AE) neural network (NN)-based reduced model to accelerate such simulations. The AE NN is first trained to find a low-dimensional latent representation of the TC states. Then, the evolving state of a chemical system can be tracked by solving the equations of the latent variables instead of the original TC equations. We demonstrate the reduced model in a syngas CO/H 2 combustion system, using training data collected from canonical perfectly stirred reactors (PSRs). It is found that the AE model can reduce the dimension of the combustion system from 12 to 2 while maintaining low reconstruction error and excellent elemental mass conservation for the test dataset. In the a posteriori test, the combustion states obtained from solving the two latent equations are compared to those from solving the 12 equations of the full model. The AE reduced method is found to be able to capture the diverse combustion states on the top two branches of the S-curve well including the extinction turning point, but with higher prediction errors for states near the ignition turning point.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Modeling MTS pyrolysis and SiC deposition kinetics using principal component analysis and neural networks

Accurate chemical kinetics modeling is crucial for improving the efficiency of chemical processing and synthesis of ceramic matrix composites. Detailed kinetic models are computationally expensive due to the large number of transported chemical species, while the simplified physics-based models, such as single-step global mechanisms, are efficient but often overlook key chemical intermediates and pathways. Recent deep learning approaches promise accurate and cost-effective models. Yet, they require additional closures for the transported nonlinear latent variables, complicating integration with existing solvers. In this work, we develop a hybrid linear—nonlinear reduced model for silicon carbide deposition from methyltrichlorosilane precursor by combining principal component analysis (PCA) and autoencoder (AE) neural network (NN) approaches. PCA is used to identify a smaller set of linear transport variables, enabling direct reuse of conventional transport solvers. NNs then reconstruct the full chemical state from these reduced variables. We demonstrate the method on a chemical vapor deposition reactor—comprising a gas-phase pyrolysis plug flow reactor and a heterogeneous surface reactor—over a wide range of temperatures, pressures, and residence times. Our PCA–AE model achieves high accuracy with only five transported scalars, achieving an eightfold cost reduction compared to detailed mechanisms, in both a priori (using data from the test set only) and a posteriori (coupled with a differential equation solver). In conclusion, notable errors arise primarily near training domain boundaries and for long residence times, indicating the need for domain shift indicators and better long-horizon predictions in future reduced chemistry model development.

autoencoder neural networks↗
DOE OSTI2024

CoarsenConf: Equivariant Coarsening with Aggregated Attention for Molecular Conformer Generation

Molecular conformer generation (MCG) is an important task in cheminformatics and drug discovery. The ability to efficiently generate low-energy 3D structures can avoid expensive quantum mechanical simulations, leading to accelerated virtual screenings and enhanced structural exploration. Several generative models have been developed for MCG, but many struggle to consistently produce high-quality conformers for meaningful downstream applications. To address these issues, we introduce CoarsenConf, which coarse-grains molecular graphs based on torsional angles and integrates them into an SE(3)-equivariant hierarchical variational autoencoder. Through equivariant coarse-graining, we aggregate the fine-grained atomic coordinates of subgraphs connected via rotatable bonds, creating a variable-length coarse-grained latent representation. Our model uses a novel aggregated attention mechanism to restore fine-grained coordinates from the coarse-grained latent representation, enabling efficient generation of accurate conformers. Furthermore, we evaluate the chemical and biochemical quality of our generated conformers on multiple downstream applications, including property prediction and large-scale oracle-based protein docking. Overall, CoarsenConf generates more accurate conformer ensembles compared to prior generative models.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

A Variational Autoencoder Model Toward Molecular Structure Representation Learning of Fuels

Here, in this work, a Variational Autoencoder (VAE)-based data-driven modeling framework is developed with the overarching goal of enabling fuel design. The VAE model is trained on a large dataset with several chemical species to learn a compressed latent space molecular representation. Chemical structure in the form of Simplified Molecular Input Line Entry System (SMILES) string is fed as input, encoded into the VAE latent space, and decoded back to the SMILES string using Long Short-Term Memory (LSTM) networks. Complexities of the VAE training loss function are thoroughly examined by varying the weightage (beta (𝜷) parameter) of the latent space regularization term, thereby assessing the balance between reconstruction accuracy and validity, and focusing on both accurate molecular structure reconstruction and latent space consistency. Two different strategies for 𝜷 variation are evaluated: linear annealing and cyclic annealing. In addition, the impact of total correlation adjustment and hierarchical priors is also studied with regard to the balance between reconstruction fidelity and latent space regularization, and potential issues such as posterior collapse, over-regularization, and poor disentanglement of latent variables. Overall, the best performance of the model is achieved with hierarchical priors and incrementally increasing 𝜷 from 0 to a threshold value of 0.25 over 75 epochs. The generative VAE model can be readily coupled with Quantitative Structure–Property Relationship (QSPR) analysis to develop an integrated end-to-end framework for fuel-property prediction and molecular design of novel promising fuels.

fuel design↗
DOE OSTI2023

STPPHAWKES

SAND2024-08541O STTPHAWKES software solution is a Bayesian estimation approach to a Hawkes process model that treats missing data as a latent variable in the estimation. The open-source software will be released to the R CRAN repository and Sandia National Labs’ Github. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Rowe, Stephen↗
DOE OSTI2022

Descriptor Aided Bayesian Optimization for Many-Level Qualitative Variables With Materials Design Applications

Abstract Engineering design often involves qualitative and quantitative design variables, which requires systematic methods for the exploration of these mixed-variable design spaces. Expensive simulation techniques, such as those required to evaluate optimization objectives in materials design applications, constitute the main portion of the cost of the design process and underline the need for efficient search strategies—Bayesian optimization (BO) being one of the most widely adopted. Although recent developments in mixed-variable Bayesian optimization have shown promise, the effects of dimensionality of qualitative variables have not been well studied. High-dimensional qualitative variables, i.e., with many levels, impose a large design cost as they typically require a larger dataset to quantify the effect of each level on the optimization objective. We address this challenge by leveraging domain knowledge about underlying physical descriptors, which embody the physics of the underlying physical phenomena, to infer the effect of unobserved levels that have not been sampled yet. We show that physical descriptors can be intuitively embedded into the latent variable Gaussian process approach—a mixed-variable GP modeling technique—and used to selectively explore levels of qualitative variables in the Bayesian optimization framework. This physics-informed approach is particularly useful when one or more qualitative variables are high dimensional (many-level) and the modeling dataset is small, containing observations for only a subset of levels. Through a combination of mathematical test functions and materials design applications, our method is shown to be robust to certain types of incomplete domain knowledge and significantly reduces the design cost for problems with high-dimensional qualitative variables.

Engineering↗
DOE OSTI2023

Sensitivity of Near-Surface Variables in the RUC Land Surface Model in the Weather Research and Forecasting Model

In this study, we investigate the parametric sensitivity of near-surface variables, such as sensible heat flux, latent heat flux, ground heat flux, hub-height wind speed and land surface temperature, to the parameters used in the Rapid Update Cycle (RUC) land surface model (LSM) during a wintertime and summertime period. The model simulations are compared with observations collected from the second Wind Forecast Improvement Project (WFIP2) field campaign. The results suggest that parameters related to snow/ice and thermal processes can have significant impact on the simulated near-surface variables. Out of the 11 examined parameters, only 6 of them have considerable influences on the model behaviors and explain about 60 ~ 80 % of the estimated total variance of the simulated variables. In addition, the magnitude of the parametric sensitivity varies with season. For instance, parameters associated with snow/ice processes are dominant during the wintertime whereas those associated with thermal processes are more important during the summertime. Furthermore, the impact of the identified parameters on the simulated variables is highly related to the topography. There is a high degree of sensitivity to the parameter values over the slope region. This points out the importance of collecting field observations over steep areas to better quantity the appropriate values of key parameters. Overall, our findings provide a better understanding of the RUC LSM behavior associated with parameter uncertainties and can be used to improve the forecasting skill of land surface processes via calibration of the most uncertain model parameters.

17 WIND ENERGY↗
DOE OSTI2022

Latent-Space Dynamics for Prediction and Fault Detection in Geothermal Power Plant Operations

This paper presents a latent-space dynamic neural network (LSDNN) model for the multi-step-ahead prediction and fault detection of a geothermal power plant’s operation. The model was trained to learn the dynamics of the power generation process from multivariate time-series data and the effects of exogenous variables, such as control adjustment and ambient temperature. In the LSDNN model, an encoder–decoder architecture was designed to capture cross-correlation among different measured variables. In addition, a latent space dynamic structure was proposed to propagate the dynamics in the latent space to enable prediction. The prediction power of the LSDNN was utilized for monitoring a geothermal power plant and detecting abnormal events. The model was integrated with principal component analysis (PCA)-based process monitoring techniques to develop a fault-detection procedure. The performance of the proposed LSDNN model and fault detection approach was demonstrated using field data collected from a geothermal power plant.

15 GEOTHERMAL ENERGY↗