HyperWeave: Scaling Diffusion Models to Billions of Spatio-Temporal Points from Multiphysics PDEs
N/A
97 - MATHEMATICS AND COMPUTING↗
SEARCH · Search NASA
Results for “spatio-temporal model”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
A Statistician’s Overview of Physics-Informed Neural Networks for Spatio-Temporal Data
The recent success of deep neural network models with physical constraints (so-called, Physics-Informed Neural Networks, PINNs) has led to renewed interest in the incorporation of mechanistic information in predictive models. Statisticians and others have long been interested in this problem, which has led to several practical and innovative solutions dating back decades. In this overview, we focus on the problem of data-driven prediction and inference of dynamic spatio-temporal processes that include mechanistic information, such as would be available from partial differential equations, with a strong focus on the quantification of uncertainty associated with data, process, and parameters. Here, we give a brief review of several paradigms and focus our attention on Bayesian implementations given they naturally accommodate uncertainty quantification. We then show that it is straight-forward to include the Bayesian PINN (B-PINN) within the Bayesian hierarchical model (BHM) framework that has long been considered for modeling dynamic spatio-temporal processes. Such a BHM-PINN is illustrated via a simulation study in which a latent nonlinear Burgers’ equation PDE governs the dynamics of Poisson distributed spatio-temporal data. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.
Bayesian↗
Dataset for "Climatic and socioeconomic drivers of water use and their spatio-temporal patterns for small and mid-sized cities in the Contiguous United States"
This dataset contains all code for calibrating and analyzing machine learning models for "Climatic and socioeconomic drivers of water use and their spatio-temporal patterns for small and mid-sized cities in the Contiguous United States". Please unzip the folders and follow the instructions from 'README.txt'. Required python modulessklearn=1.2.2numpy=1.23.3xgboost=2.0.2joblib=1.2.0 Required R libraryshapFlex:devtools::install_github("nredell/shapFlex")library(shapFlex)
Dave, Hari [Civil and Environmental Engineering De↗
Evaluating Mean State Cloud Properties in the Simple Cloud‐Resolving E3SM Atmosphere Model (SCREAM)
Accurately simulating clouds remains a key challenge in global climate models, primarily because cloud formation involves sub-grid processes that are parameterized and crudely represented in models. This study examines the performance of DOE's Simple Cloud-Resolving Energy Exascale Earth System (E3SM) Atmosphere Model (SCREAM) in simulating cloud properties and their spatio-temporal distribution by comparing against satellite observations. Two horizontal resolutions of SCREAM (3 and 12 km) are examined, and both depict a realistic spatial structure of mean-state cloud cover but underestimate its global mean magnitude. SCREAM 3 km reasonably reproduces the distribution of mean-state cloud properties across various cloud optical thickness and cloud-top pressure regimes, with performance comparable to CMIP5 and CMIP6 ensemble and marginally outperforming SCREAM 12 km. Still, SCREAM 3 km tends to underpredict low clouds and optically thin clouds, highlighting the need for continued improvement in representing unresolved processes. This study provides a basis for confidence in the representation of clouds in SCREAM, as simulating mean-state clouds is a necessary prerequisite for trusting its cloud responses to changes in aerosols and greenhouse gases.
Geosciences↗
Improving the Quasi‐Biennial Oscillation via a Surrogate‐Accelerated Multi‐Objective Optimization
Accurate simulation of the quasi-biennial oscillation (QBO) is challenging due to uncertainties in representing convectively generated gravity waves. We develop an end-to-end uncertainty quantification workflow that calibrates these gravity wave processes in E3SM for a realistic QBO. Central to our approach is a domain knowledge-informed, compressed representation of high-dimensional spatio-temporal wind fields. By employing a parsimonious statistical model that learns the fundamental frequency from complex observations, we extract interpretable and physically meaningful quantities capturing key attributes. Building on this, we train a probabilistic surrogate model that approximates the fundamental characteristics of the QBO as functions of critical physics parameters governing gravity wave generation. Leveraging the Karhunen–Loève decomposition, our surrogate efficiently represents these characteristics as a set of orthogonal features, capturing cross-correlations among multiple physics quantities evaluated at different pressure levels and enabling rapid surrogate-based inference at a fraction of the computational cost of full-scale simulations. Finally, we analyze the inverse problem using a multi-objective approach. Our study reveals a tension between amplitude and period that constrains the QBO representation, precluding a single optimal solution. To navigate this, we quantify the bi-criteria trade-off and generate a set of Pareto optimal parameter values that balance the conflicting objectives. This integrated workflow improves the fidelity of QBO simulations and offers a versatile template for uncertainty quantification in complex geophysical models.
54 ENVIRONMENTAL SCIENCES↗
Efficient data-driven regression for reduced-order modeling of spatial pattern formation
We present an efficient data-driven regression approach for constructing reduced-order models (ROMs) of reaction-diffusion systems exhibiting pattern formation. The ROMs are learned non-intrusively from available training data of physically accurate numerical simulations. The method can be applied to general nonlinear systems through the use of polynomial model form, while not requiring knowledge of the underlying physical model, governing equations, or numerical solvers. The process of learning ROMs is posed as a low-cost least-squares problem in a reduced-order subspace identified via Proper Orthogonal Decomposition (POD). Numerical experiments on classical pattern-forming systems–including the Schnakenberg and Mimura–Tsujikawa models–demonstrate that higher-order surrogate models significantly improve prediction accuracy while maintaining low computational cost. The proposed method provides a flexible, non-intrusive model reduction framework, well suited for the analysis of complex spatio-temporal pattern formation phenomena.
Data-driven modeling↗
Hybrid learning techniques for scientific data reduction with performance guarantees
The research initiatives supported by the U.S. Department of Energy (DOE) Grant DE-SC0022265 are fundamentally aimed at pioneering advanced machine learning (ML) techniques for scientific data compression within high-performance computing (HPC) environments. This comprehensive body of work addresses the critical challenge posed by the exponential growth of data generated by scientific simulations in domains such as fusion energy, climate modeling, and computational fluid dynamics (CFD). A core objective is to develop compression algorithms that achieve substantial data reduction—often by orders of magnitude—while rigorously ensuring the fidelity of both the primary data (PD) and scientifically crucial derived quantities of interest (QoI). The methodologies deployed under this grant integrate sophisticated deep learning architectures, prominently featuring autoencoders, advanced generative models like conditional diffusion, and hybrid learning techniques. Key innovations include the development of Guaranteed Autoencoders (GAE) and the Guaranteed Conditional Diffusion with Tensor Correction (GCDTC) framework, which provide explicit, instance-level error bounds on reconstructed data. Furthermore, specialized strategies such as nonlinear constraint satisfaction are employed to preserve the integrity of QoI, a vital requirement for the trustworthiness of downstream scientific analyses. This research also focuses on the design and implementation of scalable, GPU-accelerated software pipelines that seamlessly integrate into existing HPC workflows, ensuring both computational efficiency and practical applicability. The CAESAR framework, for example, unifies foundation and generative models to create an adaptive and efficient compression solution for spatio-temporal scientific data. Collectively, these efforts represent a significant advancement in mitigating the scientific data deluge, enabling more effective data management, accelerated scientific discovery, and optimized utilization of HPC resources.
97 MATHEMATICS AND COMPUTING↗
Final report- UFL - RAPIDS2: A SciDAC Institute for Computer Science, Data, and Artificial Intelligence
The research initiatives supported by the U.S. Department of Energy (DOE) Grant DE-SC0022265 are fundamentally aimed at pioneering advanced machine learning (ML) techniques for scientific data compression within high-performance computing (HPC) environments. This comprehensive body of work addresses the critical challenge posed by the exponential growth of data generated by scientific simulations in domains such as fusion energy, climate modeling, and computational fluid dynamics (CFD). A core objective is to develop compression algorithms that achieve substantial data reduction—often by orders of magnitude—while rigorously ensuring the fidelity of both the primary data (PD) and scientifically crucial derived quantities of interest (QoI). The methodologies deployed under this grant integrate sophisticated deep learning architectures, prominently featuring autoencoders, advanced generative models like conditional diffusion, and hybrid learning techniques. Key innovations include the development of Guaranteed Autoencoders (GAE) and the Guaranteed Conditional Diffusion with Tensor Correction (GCDTC) framework, which provide explicit, instance-level error bounds on reconstructed data. Furthermore, specialized strategies such as nonlinear constraint satisfaction are employed to preserve the integrity of QoI, a vital requirement for the trustworthiness of downstream scientific analyses. This research also focuses on the design and implementation of scalable, GPU-accelerated software pipelines that seamlessly integrate into existing HPC workflows, ensuring both computational efficiency and practical applicability. The CAESAR framework, for example, unifies foundation and generative models to create an adaptive and efficient compression solution for spatio-temporal scientific data. Collectively, these efforts represent a significant advancement in mitigating the scientific data deluge, enabling more effective data management, accelerated scientific discovery, and optimized utilization of HPC resources.
97 MATHEMATICS AND COMPUTING↗
Implications of Sea Breeze Circulations on boundary layer aerosols in the southern coastal Texas region
The Sea Breeze Circulation (SBC) influences atmospheric processes at multiple scales in coastal regions. Understanding how SBCs impact the aerosol number budget and aerosol–cloud interaction processes is essential. This study investigates sea breeze–aerosol interactions (SAIs) during 46 summertime SBC events using data from the TRacking Aerosol Convection Interactions Experiment (TRACER) field campaign across urban (main) and rural (supplemental) coastal sites in southern Texas. Weather Research and Forecasting model coupled with Chemistry (WRF-Chem) simulations complement observations to explore spatio-temporal meteorological controls on boundary layer aerosols. During the summertime, Sea Breeze Fronts (SBF) penetrating inland transported cool, moist air over the land, introducing air masses with distinct properties compared to the preexisting continental air. These SAIs cause variability in number concentrations of up to a factor of two, with events typically lasting ∼ 5 h before returning to background conditions. SAI impact on aerosols varies with site proximity to water and the preceding sea breeze (SB) history, primarily affecting the marine-influenced accumulation mode. The main site, influenced by both Galveston Bay and the Gulf of Mexico, reflects a stronger marine influence. In contrast, a supplemental site, at a similar shoreline distance but exposed only to the Gulf of Mexico and typically upstream of the urban core, samples SB air that has traversed land and partially regained continental characteristics. Simulations show that the regional SAIs extend ∼ 50 km inland and reach up to the boundary layer height. SAIs further decrease cloud condensation nuclei relevant aerosol number concentrations in ∼ 20 % of events during SBF passage.
54 ENVIRONMENTAL SCIENCES↗
Learning turbulent flows with generative models for super resolution and sparse flow reconstruction
Neural operators are promising surrogates for dynamical systems but when trained with standard L 2 losses they tend to oversmooth fine-scale turbulent structures. Here, we show that combining operator learning with generative modeling overcomes this limitation. We consider three practical turbulent-flow challenges where conventional neural operators fail: spatio-temporal super-resolution, forecasting, and sparse flow reconstruction. For Schlieren jet super-resolution, an adversarially trained neural operator (adv-NO) reduces the energy-spectrum error by 15 × while preserving sharp gradients at neural operator-like inference cost. For 3D homogeneous isotropic turbulence, adv-NO trained on only 160 timesteps from a single trajectory forecasts accurately for five eddy-turnover times and offers 114 × wall-clock speed-up at inference than the baseline diffusion-based forecasters, enabling near-real-time rollouts. For reconstructing cylinder wake flows from highly sparse Particle Tracking Velocimetry-like inputs, a conditional generative model infers full 3D velocity and pressure fields with correct phase alignment and statistics. These advances enable accurate reconstruction and forecasting at low compute cost, bringing near-real-time analysis and control within reach in experimental and computational fluid mechanics.
Fluid dynamics↗
Causality-respecting adaptive refinement for PINNs: enabling precise interface evolution in phase field modeling
Physics-informed neural networks (PINNs) have emerged as a powerful tool for solving physical systems described by partial differential equations (PDEs). However, their accuracy in dynamical systems, particularly those involving sharp moving boundaries with complex initial morphologies, remains a challenge. Here, this study introduces an approach combining residual-based adaptive refinement (RBAR) with causality-informed training to enhance the performance of PINNs in solving spatio-temporal PDEs. Our method employs a three-step iterative process: initial causality-based training, RBAR-guided domain refinement, and subsequent causality training on the refined mesh. Applied to the Allen-Cahn equation, a widely-used model in phase field simulations, our approach demonstrates significant improvements in solution accuracy and computational efficiency over traditional PINNs. Notably, we observe an ‘overshoot and relocate’ phenomenon in dynamic cases with complex morphologies, showcasing the method’s adaptive error correction capabilities. This synergistic interaction between RBAR and causality training enables accurate capture of interface evolution, even in challenging scenarios where traditional PINNs fail. Our framework not only resolves the limitations of uniform refinement strategies but also provides a generalizable methodology for solving a broad range of spatio-temporal PDEs. The enhanced performance of the RBAR–causality combined framework demonstrates its strong potential for advancing PINN-based modeling of physical systems characterized by complex, evolving interfaces.
Allen-Cahn equations↗
Distributed Cross-Channel Hierarchical Aggregation for Foundation Models
Vision-based scientific foundation models hold significant promise for advancing scientific discovery and innovation. This potential stems from their ability to aggregate images from diverse sources—such as varying physical groundings or data acquisition systems—and to learn spatio-temporal correlations using transformer architectures. However, tokenizing and aggregating images can be compute-intensive, a challenge not fully addressed by current distributed methods. In this work, we introduce the Distributed Cross-Channel Hierarchical Aggregation (D-CHAG) approach designed for datasets with a large number of channels across image modalities. Our method is compatible with any model-parallel strategy and any type of vision transformer architecture, significantly improving computational efficiency. We evaluated D-CHAG on hyperspectral imaging and weather forecasting tasks. When integrated with tensor parallelism and model sharding, our approach achieved up to a 75% reduction in memory usage and more than doubled sustained throughput on up to 1,024 AMD GPUs on the Frontier Supercomputer.
Tsaris, Aristeidis (aris) [ORNL] (ORCID:0000000277↗
Molecular fluctuations inhibit intermittency in compressible turbulence
In the standard picture of fully developed turbulence, highly intermittent hydrodynamic fields are nonlinearly coupled across scales, where local energy cascades from large scales into dissipative vortices and large density gradients. Microscopically, however, constituent fluid molecules are in constant thermal (Brownian) motion, but the role of molecular fluctuations in large-scale turbulence is largely unknown, and with rare exceptions, it has historically been considered irrelevant at scales larger than the molecular mean free path. Recent theoretical and computational investigations have shown that molecular fluctuations can impact energy cascade at Kolmogorov length scales. Here, we show that molecular fluctuations not only modify energy spectrum at wavelengths larger than the Kolmogorov length in compressible turbulence, but also significantly inhibit spatio-temporal intermittency across the entire dissipation range. Using large-scale direct numerical simulations of computational fluctuating hydrodynamics, we demonstrate that the extreme intermittency characteristic of turbulence models is replaced by nearly Gaussian statistics in the dissipation range. These results demonstrate that the compressible Navier–Stokes equations should be augmented with molecular fluctuations to accurately predict turbulence statistics across the dissipation range. Our findings have significant consequences for turbulence modelling in applications such as astrophysics, reactive flows and hypersonic aerodynamics, where dissipation-range turbulence is approximated by closure models.
compressible turbulence↗
Time-dependent-bases with local CUR decomposition method for accelerating turbulent combustion simulations
Here, this study presents a novel reduced-order modeling framework, Time-Dependent Bases with Local CUR decomposition (TDB-L-CUR), designed to efficiently and accurately approximate the species transport equations in reacting flow simulations. The method extends the existing TDB-CUR approach for chemically reacting flows (Jung et al. Comput. Methods Appl. Mech. Engrg. 437 (2025) 117758), which leverages matrix decomposition techniques to form a global-in-space, time-dependent low-dimensional manifold. While TDB-CUR performs well in homogeneous systems, it may be less well-suited to spatially heterogeneous systems such as turbulent flames, where higher-rank approximations are typically required. The proposed TDB-L-CUR framework introduces two methodological extensions to the baseline approach. First, it applies unsupervised clustering to partition the physical domain into distinct regions, enabling spatially localized manifold construction, thereby reducing the rank required for the reduced-order representation. Second, it incorporates a computational singular perturbation (CSP)-based scheme for identifying and penalizing fast species, allowing for spatio-temporally adaptive mitigation of chemical stiffness. The proposed framework is validated on a hierarchy of test cases, including a one-dimensional premixed flame, a two-dimensional nonpremixed ignition case with vortex interaction, and a three-dimensional turbulent premixed flame. TDB-L-CUR significantly improves accuracy over TDB-CUR while further reducing computational cost. The fully on-the-fly formulation of TDB-L-CUR (i.e., requiring no offline training or prior knowledge) makes it a robust and scalable tool for reduced-order modeling of reactive flows.
Local manifold↗
Big Data Meets Geothermal Exploration (CRADA Final Report)
As part of the Cyclotron Road program, Zanskar Geothermal & Minerals, Inc. investigated the application of micro-earthquake and ambient noise seismology methods to imaging and characterizing the structural characteristics and hydrothermal flux of subsurface faults. Significant advances in what could be resolved were enabled by two major developments in seismology: 1) the availability of large-n arrays of low-cost seismometers, and 2) the availability of increased computational power and semi-automated data reduction algorithms. In tandem, these advances may improve the signal-to-noise ratio and spatial precision of the data collected and enable higher-resolution characterization of subsurface fracture systems and their spatio-temporal evolution. These tools supported efforts to reduce dry-hole risk and to improve wellfield productivity for geothermal resource development. In particular, two applications of these advances were evaluated: 1) fracture-seismic imaging, which was used to detect ambient emissions from fluid-filled fractures, and 2) reservoir tomography, which used information about travel paths, source locations, and source parameters of micro-earthquakes to identify areas of enhanced permeability. Integration of these methods provided guidance for siting wells and served as prior constraints for reservoir models, informing forecasts of power potential and production and injection strategies aimed at minimizing temperature decline and improving overall resource productivity.
15 GEOTHERMAL ENERGY↗
Compactly‐Supported Nonstationary Kernels for Computing Exact Gaussian Processes on Big Data
The Gaussian process (GP) is a widely used method for analyzing large-scale data sets, including spatio-temporal measurements of nonlinear processes that are now commonplace in the environmental sciences. Traditional implementations of GPs involve stationary kernels (also termed covariance functions) that limit their flexibility, and exact methods for inference that prevent application to data sets with more than about 10,000 points. Modern approaches to address stationarity assumptions generally fail to accommodate large data sets, while all attempts to address scalability focus on approximating the Gaussian likelihood, which can involve subjectivity and lead to inaccuracies. In this work, we explicitly derive an alternative kernel that can discover and encode both sparsity and nonstationarity. We embed the kernel within a fully Bayesian GP model and leverage high-performance computing resources to enable the analysis of massive data sets. We demonstrate the favorable performance of our novel kernel relative to existing exact and approximate GP methods across a variety of synthetic data examples. Furthermore, we conduct space–time prediction based on more than 1 million measurements of daily maximum temperature and verify that our results outperform state-of-the-art methods in the Earth sciences. More broadly, having access to exact GPs that use ultra-scalable, sparsity-discovering, nonstationary kernels allows GP methods to truly compete with a wide variety of machine learning methods.
Gaussian processes↗
The ABCs of On-Demand Transit (ODT)
On-demand mobility - also referred to as on-demand transit (ODT) - is a form of public mobility that is flexible with respect to where and when service is provided, and ODT deployments have increased significantly in recent years. Transit agencies are becoming increasingly interested in ODT, and due to differing definitions and various service design and business model options, it can be difficult to learn about the emerging industry. This work provides an overview of the definitions of ODT, recent trends internationally and in the U.S., ODT's benefits and challenges (particularly compared to fixed-route transit), three primary service design options, system costs and funding considerations, and a metrics framework for evaluating ODT systems to ensure continued successful performance. Identified benefits include increased service areas, short ride and wait times, increased user flexibility, potential to reduce energy consumption and emissions through shared trips and smaller, right-sized vehicles, increased safety and comfort through door-to-door service, and rich data streams including granular spatio-temporal data that can be analyzed to continuously improve the service. Challenges include scaling ODT service up as small increases in ridership require additional supply to keep service quality high, serving peak times including keeping low wait times, the lack of fixed schedule being challenging for commuters, integrating ODT services with nearby transit systems, and equity for riders without smartphones who cannot track the vehicle in a mobile app. Finally, an overview of seven ODT case studies (in Texas, Missouri, New York, and Ontario, Canada) performed by NREL and related analysis of travel time, energy and emissions, costs, and equity are presented. Initial key findings include: ODT can be cost- and energy-effective compared to fixed-route transit, ODT serves more people than other transit options, and ODT system deployments can be followed by rapid growth.
24 POWER TRANSMISSION AND DISTRIBUTION↗
Bayesian Physics Informed Spatio-Temporal Network for Streamflow Data Imputation
Reliable reconstruction of incomplete streamflow records is critical for improving hydrological forecasting, flood preparedness, and water resource management. However, large observational gaps and uncertainties in governing physical parameters limit the accuracy of traditional statistical and machinelearning imputation frameworks. To address these challenges, we develop a Bayesian Physics-Informed Spatio-Temporal Network (BPI-STNet) that jointly captures spatial and temporal dependencies while enforcing hydrologic consistency through embedded physical constraints. The framework integrates a GraphSAGE-LSTM architecture to model spatial connectivity across gauges and temporal flow dynamics, coupled with a Bayesian update mechanism to estimate uncertain parameters in a simplified water-balance framework. Unlike conventional physics-informed networks that rely on sampling-based posterior estimation, BPI-STNet derives an analytic solution to the inverse problem, allowing closed-form Bayesian updates of uncertain parameters Λ={α,β,k} using Gaussian priors and likelihoods. Applied to daily observations from the Susquehanna River Basin (1980-2022), BPI-STNet achieves substantial improvements over a purely data-driven RGNN baseline, which reduced RMSE by 23 % and MAE by 9 %, and achieving an average NSE values up to 0.96. The results demonstrate that coupling Bayesian inference with physics-informed learning yields physically consistent, uncertainty-aware reconstructions that preserve the temporal persistence and statistical distribution of observed flows. The proposed framework establishes a generalizable paradigm for data-sparse hydrologic systems where both data fidelity and physical interpretability are essential.
Krishnan Kutty Ambika, Anukesh [ORNL] (ORCID:00000↗