Search NASA⌕ Search

SEARCH · Search NASA

Results for “autoregressive model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

35 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

A hybrid surrogate modeling framework for the Digital Twin of a Fluoride-salt-cooled High-temperature Reactor (FHR)

While nuclear energy is a non-greenhouse-gas emitting energy source, expensive operational costs due to the high-level of safety requirements decreases their competitiveness in the sustainable energy market. Advanced reactor concepts paired with Digital Twins aim to increase the commercialization gains of nuclear energy by reducing operational costs, increasing reactor reliability and enhancing power generation. To support Digital Twin tasks such as real-time autonomous control, proactive maintenance monitoring or optimizing power demand operations, a fast and accurate virtual representation of the Nuclear Power Plant (NPP) is required. The computational cost of high-fidelity, physics-based models are unsuitable for real-time analysis or scalability. Here, in this work, a hybrid surrogate modeling framework is developed fora Fluoride-salt-cooled High-temperature Reactor (FHR) that leverages physics-inspired models for key reactor components and uses data-driven methods for rapid system state space prediction. The Xenon reactivity feedback model is integrated to inform the surrogate model about the reactor core and the homologous pump theory model is the basis for representing pump degradation. Using a detailed, two dimensional thermal hydraulics model to generate data on the FHR, we train a network of Vectorized Autoregressive Moving-Average with eXogenous input (VARMAX) models to predict the remaining state values. The result is a surrogate model that provides a detailed reactor state representation of 41 system states and a pump degradation analysis. The framework is applied to Load Follows profiles, yielding high accuracy and a speedup that is more than 4000x faster compared to the higher- fidelity thermal hydraulics model, enabling real-time operational intelligence and applications in long horizon predictions. While the surrogate model framework is demonstrated for the particular case of FHR, the hybrid physical/data-driven modeling approach including the network of surrogates and the underlying modularity has the potential to be applied to other physical asset systems.

Digital Twins↗
DOE OSTI2025

Machine learning modeling and model predictive control of a closed-circuit reverse osmosis system

Closed-circuit reverse osmosis (CCRO) offers a flexible and energy-efficient alternative to conventional reverse osmosis by operating in a semi-batch mode that recycles brine, enabling higher recovery rates and reduced specific energy consumption (SEC). However, developing accurate, system-level dynamic models for CCRO remains challenging due to its nonlinear, multi-phase operation and sensitivity to variable feed water conditions. Traditional modeling approaches, such as NARMAX (nonlinear autoregressive moving average with exogenous inputs), often struggle to generalize across varying inlet feed concentrations, necessitating frequent parameter re-estimation and limiting their utility for real-time control applications. To address these limitations, we developed a long short-term memory (LSTM) neural network model trained on an extensive experimental data set from a CCRO pilot plant. The model accepts three inputs, feed flow rate, recirculation flow rate, and initial feed conductivity, and predicts three key outputs: reject conductivity, feed pump power draw, and recirculation pump power draw. We validated the LSTM model against experimental data, demonstrating its ability to distinguish between different feed conductivities and adapt to variable flow rates. Subsequently, we incorporated the LSTM model within a nonlinear model predictive control (MPC) scheme and conducted closed-loop simulations to optimize the integrated SEC (iSEC). In conclusion, the results project up to a 6% reduction in iSEC by using MPC to optimize performance over the entire experiment duration, without requiring any random excitation for data collection or parameter re-estimation.

Desalination↗
DOE OSTI2026

Predicting initial trans-membrane pressure across cycles in the ultrafiltration process using random forest

With growing freshwater scarcity, direct potable reuse (DPR) systems that reclaim wastewater for drinking are becoming increasingly important for sustainable water supply. Reliable operation requires minimizing downtime in ultrafiltration (UF) units, where membrane fouling leads to elevated trans-membrane pressure (TMP). This study develops data-driven regression models based on random forest (RF) and autoregressive (AR) approaches to forecast the initial TMP at the start of each UF filtration cycle in a pilot-scale DPR system. The RF model consistently outperforms baseline methods, including historical mean, last observation carried forward, and AR models, across multiple forecast horizons, achieving the lowest root mean square error. To evaluate how different classes of process variables contribute to TMP dynamics over time, we examine the feature importance of independent input variables across multiple forecast horizons. This analysis provides insight into the temporal relevance of operational and sensor-derived features, guiding control and monitoring strategies. Additionally, the impact of hyperparameter tuning on TMP prediction performance is assessed for both direct and recursive RF modelling approaches. The proposed RF framework establishes a robust foundation for predictive monitoring and real-time optimization of UF operations, supporting sustainable and reliable water reuse.

direct potable reuse↗
DOE OSTI2024

Spatio–Temporal Machine Learning for Regional to Continental Scale Terrestrial Hydrology

Integrated hydrologic models can simulate coupled surface and subsurface processes but are computationally expensive to run at high resolutions over large domains. Here we develop a novel deep learning model to emulate subsurface flows simulated by the integrated ParFlow–CLM model across the contiguous US. We compare convolutional neural networks like ResNet and UNet run autoregressively against our novel architecture called the Forced SpatioTemporal RNN (FSTR). The FSTR model incorporates separate encoding of initial conditions, static parameters, and meteorological forcings, which are fused in a recurrent loop to produce spatiotemporal predictions of groundwater. We evaluate the model architectures on their ability to reproduce 4D pressure heads, water table depths, and surface soil moisture over the contiguous US at 1 km resolution and daily time steps over the course of a full water year. The FSTR model shows superior performance to the baseline models, producing stable simulations that capture both seasonal and event–scale dynamics across a wide array of hydroclimatic regimes. The emulators provide over 1,000× speedup compared to the original physical model, which will enable new capabilities like uncertainty quantification and data assimilation for integrated hydrologic modeling that were not previously possible. Our results demonstrate the promise of using specialized deep learning architectures like FSTR for emulating complex process–based models without sacrificing fidelity.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Convolutional L2LFlows: generating accurate showers in highly granular calorimeters using convolutional normalizing flows

Abstract In the quest to build generative surrogate models as computationally efficient alternatives to rule-based simulations, the quality of the generated samples remains a crucial frontier. So far, normalizing flows have been among the models with the best fidelity. However, as the latent space in such models is required to have the same dimensionality as the data space, scaling up normalizing flows to high dimensional datasets is not straightforward. The prior L2LFlows approach successfully used a series of separate normalizing flows and sequence of conditioning steps to circumvent this problem. In this work, we extend L2LFlows to simulate showers with a 9-times larger profile in the lateral direction. To achieve this, we introduce convolutional layers and U-Net-type connections, move from masked autoregressive flows to coupling layers, and demonstrate the successful modelling of showers in the ILD Electromagnetic Calorimeter as well as Dataset 3 from the public CaloChallenge dataset.

Instruments & Instrumentation↗
DOE OSTI2024

A Semi‐Mechanistic Model for Partitioning Evapotranspiration Reveals Transpiration Dominates the Water Flux in Drylands

Abstract Popular evapotranspiration (ET) partitioning methods make assumptions that might not be well‐suited to dryland ecosystems, such as high sensitivity of plant water‐use efficiency (WUE) to vapor pressure deficit (VPD). Our objectives were to (a) create an ET partitioning model that can produce fine‐scale estimates of transpiration (T) in drylands, and (b) use this approach to evaluate how climate controls T and WUE across ecosystem types and timescales along a dryland aridity gradient. We developed a novel, semi‐mechanistic ET partitioning method using a Bayesian approach that constrains abiotic evaporation using process‐based models, and loosely constrains time‐varying WUE within an autoregressive framework. We used this method to estimate daily T and weekly WUE across seven dryland ecosystem types and found that T dominates ET across the aridity gradient. Then, we applied cross‐wavelet coherence analysis to evaluate the temporal coherence between focal response variables (WUE and T/ET) and environmental variables. At yearly scales, we found that WUE at less arid, higher elevation sites was primarily limited by atmospheric moisture demand, and WUE at more arid, lower elevation sites was primarily limited by moisture supply. At sub‐yearly timescales, WUE and VPD were sporadically correlated. Hence, ecosystem‐scale dryland WUE is not always sensitive to changes in VPD at short timescales, despite this being a common assumption in many ET partitioning models. This new ET partitioning method can be used in dryland ecosystems to better understand how climate influences physically and biologically driven water fluxes.

Environmental Sciences & Ecology↗
DOE OSTI2025

Unrolled Video Super-Resolution Network with Autoregressive Prior for the Case of Known Motion

Real-time detection and classification of distant objects is necessary for many national security applications. However, when objects are far from the sensor, they occupy only a small number of pixels in the captured video, limiting the amount of visual detail available for recognition. State-of-the-art classification methods typically rely on high-resolution (HR) video streams to capture characteristic object features, but obtaining such detail is challenging for distant objects that occupy only a few pixels. This motivates the development of video super-resolution (VSR) methods that enhance object classification by recovering fine details from low-pixel representations. Current VSR methods rely either on model-based optimization, which is interpretable but computationally expensive, or on learning-based approaches, which are efficient and high-performing but often lack flexibility and interpretability. In this report, we propose an end-to-end trainable unrolled VSR network, UVSRNet, which super-resolves each frame in a video by exploiting sub-pixel motion between neighboring low-resolution (LR) frames as well as incorporating high-frequency detail from previously super-resolved frames. In particular, by unrolling a plug-and-play (PnP) half-quadratic splitting (HQS) algorithm, we leverage a model-based data-fitting module alongside a learning-based autoregressive prior module. This combination yields a method that maintains the flexibility and interpretability of model-based methods while achieving the performance advantages of learning-based methods.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗
DOE OSTI2024

Unveiling the Potential of MeshGraphNets for Predicting Subsurface Evolution in Carbon Storage Projects

This is the conference paper accompanying an oral presentation “Unveiling the Potential of MeshGraphNets for Predicting Subsurface Evolution in Carbon Storage Projects” at the 17th International Conference on Greenhouse Gas Control Technologies GHGT-17 held in Calgary, Canada, October 20-24 , 2024. Carbon capture and storage (CCS) technology is critical for mitigating climate change but requires effective subsurface reservoir management to ensure safe containment of injected CO2. Accurate predictions of reservoir pressure and saturation are essential for assessing long-term CCS performance. Traditional numerical simulations, while effective, are computationally intensive, time-consuming, and constrained by data discretization. Previous work has shown the effectiveness of MeshGraphNets (MGN), a graph-based machine learning framework, as an innovative alternative for predicting reservoir behavior. MGN leverages graph neural networks (GNNs) and mesh representations to model complex geological formations, offering superior adaptability across different discretizations and reservoir configurations. Classic MGN implementations utilize an autoregressive technique to predict future behavior based on current predictions, but this technique is hampered by error accumulation over time. To enhance the model accuracy in time-series predictions, this study implemented a multi-step rollout strategy that integrates autoregressive predictions during training to stabilize prediction of saturation over time. Using the Illinois Basin – Decatur Project (IBDP) dataset, comprising 100 simulations of CO2 injection, pressure, and saturation changes, the framework demonstrated its ability to learn spatial dependencies and temporal dynamics. With inputs including permeabilities, porosities, and injection rates, MGN accurately predicted CO2 plume evolution over time, even with limited training data. Moreover, the addition of a multi-step rollout procedure during training improved the ability of MGN to predict stably over time by ~15%. This research positions MGN, enhanced with multi-step rollout capabilities, as a robust and efficient tool for CCS applications. It advances the field by enabling precise, computationally efficient predictions of reservoir behavior, providing a foundation for the broader adoption of machine learning frameworks in CCS and other geoscience domains.

Holcomb, Paul↗
DOE OSTI2025

Reweighting configurations generated by transferable, machine learned models for protein sidechain backmapping

Multiscale modeling requires the linking of models at different levels of detail, with the goal of gaining accelerations from lower fidelity models while recovering fine details from higher resolution models. Communication across resolutions is particularly important in modeling soft matter, where tight couplings exist between molecular-level details and mesoscale structures. While multiscale modeling of biomolecules has become a critical component in exploring their structure and self-assembly, backmapping from coarse-grained to fine-grained, or atomistic, representations presents a challenge, despite recent advances through machine learning. A major hurdle, especially for strategies utilizing machine learning, is that backmappings can only approximately recover the atomistic ensemble of interest. We demonstrate conditions for which backmapped configurations may be reweighted to exactly recover the desired atomistic ensemble. By training separate decoding models for each sidechain type, we develop an algorithm based on normalizing flows and geometric algebra attention to autoregressively propose backmapped configurations for any protein sequence. Critical for reweighting with modern protein force fields, our trained models include all hydrogen atoms in the backmapping and make probabilities associated with atomistic configurations directly accessible. We also demonstrate, however, that reweighting is extremely challenging despite state-of-the-art performance on recently developed metrics and generation of configurations with low energies in atomistic protein force fields. Through detailed analysis of configurational weights, we show that machine-learned backmappings must not only generate configurations with reasonable energies, but also correctly assign relative probabilities under the generative model. These are broadly important considerations in generative modeling of atomistic molecular configurations.

Monroe, Jacob I. [Univ. of Arkansas, Fayetteville,↗
DOE OSTI2024

Comparison of MeshGraphNet Techniques for Subsurface Behavior Prediction during CO2 Sequestration

Carbon sequestration is a vital part of the effort to mitigate anthropogenic climate change. Previously, we have shown that Graph Neural Networks (GNNs) provide the ability to extract meaningful insights during prediction of subsurface behavior in carbon storage projects. However, these models have struggled with long-term prediction accuracy due to error accumulation caused by autoregressive prediction. This research leverages the Illinois Basin – Decatur Project (IBDP) dataset to examine strategies for minimizing loss over time in a MeshGraphNet GNN model to improve reliability of predictions while minimizing inferencing time.

Holcomb, Paul↗
DOE OSTI2025

A physics-constrained neural ordinary differential equations approach for robust learning of stiff chemical kinetics

The high computational cost associated with solving for detailed chemistry poses a significant challenge for predictive computational fluid dynamics (CFD) simulations of turbulent reacting flows. While deep learning techniques have been explored to develop faster surrogate models, they often fail to integrate reliably with CFD solvers. This instability arises because traditional deep learning approaches optimize for training error without ensuring compatibility with ordinary differential equation (ODE) solvers, resulting in accumulation of errors over time. Recently, neuralODE (NODE) based approaches have been shown to be a promising technique to emulate and accelerate detailed chemistry computations. Here, in the present work, we extend this NODE framework for stiff chemical kinetics by incorporating mass conservation constraints directly into the loss function during training. This ensures that the total mass as well as the individual elemental species masses are conserved in an a-posteriori manner. Proof-of-concept studies are performed with the novel physics-constrained NODE (PC-NODE) approach for homogeneous autoignition of hydrogen-air mixture over a range of composition and thermodynamic conditions. It is demonstrated that the PC-NODE framework not only improves the physical consistency of the resulting data-driven model with respect to mass conservation criteria, but also improves training efficiency. PC-NODE is shown to achieve 2–100× speedup relative to the hydrogen-air detailed chemical mechanism depending on the type of the ODE solver (implicit or explicit) used during autoregressive inference tests. Lastly, a-posteriori studies are performed wherein the trained PC-NODE model is coupled with a CFD solver. It is shown that higher accuracy is achieved with PC-NODE relative to the purely data-driven NODE approach. Moreover, PC-NODE also exhibits robustness and generalizability to unseen initial conditions from within (interpolative capability) as well as outside (extrapolative capability) the training regime.

computational combustion↗
DOE OSTI2024

Streamlining Ocean Dynamics Modeling with Fourier Neural Operators: A Multiobjective Hyperparameter and Architecture Optimization Approach

Training an effective deep learning model to learn ocean processes involves careful choices of various hyperparameters. We leverage DeepHyper’s advanced search algorithms for multiobjective optimization, streamlining the development of neural networks tailored for ocean modeling. The focus is on optimizing Fourier neural operators (FNOs), a data-driven model capable of simulating complex ocean behaviors. Selecting the correct model and tuning the hyperparameters are challenging tasks, requiring much effort to ensure model accuracy. DeepHyper allows efficient exploration of hyperparameters associated with data preprocessing, FNO architecture-related hyperparameters, and various model training strategies. We aim to obtain an optimal set of hyperparameters leading to the most performant model. Moreover, on top of the commonly used mean squared error for model training, we propose adopting the negative anomaly correlation coefficient as the additional loss term to improve model performance and investigate the potential trade-off between the two terms. The numerical experiments show that the optimal set of hyperparameters enhanced model performance in single timestepping forecasting and greatly exceeded the baseline configuration in the autoregressive rollout for long-horizon forecasting up to 30 days. Utilizing DeepHyper, we demonstrate an approach to enhance the use of FNO in ocean dynamics forecasting, offering a scalable solution with improved precision.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Neural Scaling Laws for Jet Generation

Recently observed empirical scaling laws describe the performance of foundation-type models as three independent key quantities -- dataset size, compute, and model parameters -- are modified. Extracting these scaling laws informs the training of large complex models for which the tuning of hyperparameters in traditional ways is not feasible. This work for the first time explores if scaling laws can also be observed for the task of particle jet generation -- both relevant as a pre-training objective for foundation models and as in-situ simulation by itself. We indeed replicate the key logarithmic scaling law behavior for model-size scaling. Beyond studying the next token prediction validation loss of the generative model, we also study the sliced Wasserstein distance of five physical quantities that are not immediately available to the model during training. Our study shows that this quantity is monotonically related to the next token prediction validation loss, meaning that this loss is indeed a good proxy for the physics performance. For the scaling with dataset size and compute, we observe substantially weaker scaling behavior of both the loss and the sliced Wasserstein distance. We analyze this behavior by introducing the concept of a learnable window, and argue that autoregressive next token prediction on jet constituents exhibits comparatively rapid saturation relative to language-model studies. We discuss possible origins of this behavior, including the stochastic nature of QCD radiation and differences between generative and supervised learning tasks in collider physics.

Amram, Oz [Fermilab]↗
DOE OSTI2025

Simulation-Based Inference for Neutrino Interaction Model Tuning

This project demonstrates, for the first time, the application of simulation-based inference (SBI) techniques to tune neutrino–nucleus interaction models. Using a mock dataset based on the MicroBooNE tuning of the GENIE event generator, our approach employs a Neural Posterior Estimator (NPE) with Masked Autoregressive Flows (MAF) to infer the posterior distributions of key GENIE parameters directly from simulated histograms. The workflow provides a scalable and amortized framework for performing likelihood-free inference in high-dimensional parameter spaces, offering a pathway to more efficient and uncertainty-aware model tuning for next-generation neutrino experiments such as DUNE and SBND.

Tame-Narvaez, KarlaMaria [Fermi National Accelerat↗
DOE OSTI2025

Enhancing Short-Range Weather Forecasts through Temporal Variation Encoding: A Multiperiod Embedding Approach

Machine learning (ML) techniques have emerged as promising approaches to improve regional weather forecast accuracy and reliability through data-driven methods. We propose a novel ML-based weather forecasting model, the Multiperiod Embed Net (MPENet). A key distinguishing feature of MPENet is its explicit utilization of the inherent cyclic nature in weather dynamics, unlike the autoregressive strategies commonly used in other ML weather forecasting approaches. Critical cyclic structures are identified via Fourier analyses of dynamic time series. Cyclicity in the convolutional representation is achieved by transforming one-dimensional time series of meteorological variables into two-dimensional tensors based on identified periods. This approach enables the model to leverage intrinsic weather patterns, enhancing regional forecast performance. To demonstrate the effectiveness of MPENet, we conduct a comparative analysis with Nvidia’s FourCastNet. Both models are trained on High-Resolution Rapid Refresh (HRRR) data from 2015 to 2022, over a 192 km × 192 km region in Tennessee. The comparisons are performed locally at two specific locations known to have different weather dynamics due to orographic effects: Crossville, on the relatively flat Cumberland Plateau with fewer topographic airflow disruptions, and Oak Ridge, in the ridge-and-valley region, where airflow is heavily influenced by surrounding valleys and mountains. Our results indicate that FourCastNet achieves strong accuracy at very short lead times, while MPENet maintains competitive skill and shows advantages in capturing temporal evolution over longer periods. Cross-correlation analyses of MPENet and FourCastNet predictions with the HRRR data suggest that encoding critical cyclicity into the network architecture leads to improvements in the forecasting skill.

Artificial intelligence↗
DOE OSTI2025

Intercomparison of Deep Learning Model Architectures for Atmospheric River Prediction

With a rapid surge in the application of machine learning (ML) for a diverse range of tasks in climate science, the present study addresses a challenge for climate scientists when selecting the optimal ML or deep learning (DL) architecture for a given application. In particular, a DL intercomparison study was performed with a focus on forecasting the position of atmospheric rivers (ARs) on short-range time scales (up to 5-day lead times). AR predictions from multiple DL architectures, including various types of convolutional autoencoders and a vision transformer (ViT), were compared against ECMWF ERA5 reanalysis and hindcasts from a global climate model. DL models with similar trainable parameters were trained on ERA5 reanalysis data and AR positions derived from a thresholding algorithm to ensure a fair comparison among the DL models. Each model’s performance and accuracy in forecasting AR location and key input fields within a 5-day window were assessed using metrics of root-mean-square error, anomaly correlation, and mean intersection over union. The ViT architecture outperformed other autoencoder models in most of the metrics. Incorporating additional meteorological fields only yielded slight improvements in forecasting certain fields at longer lead times. The results also suggest that a smaller number of input time steps or smaller number of autoregressive steps can achieve better prediction skills, while also improving the overall computational efficiency. This research offers valuable insights into the strengths and weaknesses of different DL techniques for AR forecasting, hopefully guiding the development of improved models for forecasting this phenomenon.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2026

Improving Spectral Resolution from Real-time Evolution for Correlated Systems

Abstract The quality of numerically simulated spectra using real-time evolution methods for strongly correlated systems is affected by both the length of simulation time and the system size, limiting resolution in both frequency and momentum. In this work, we propose a computationally cheap, linear autoregressive machine learning-based framework to extend short-time and short-distance results over a wider range. We use the proposed method to extend the lesser Green’s function for both the Hubbard model and the much more computationally challenging Hubbard-extended Holstein model. This technique significantly improves both the frequency and momentum resolution of the single-particle removal spectrum $${\mathcal{A}}(k,\omega )$$ A ( k , ω ) , allowing the observation of otherwise obscured spectral features due to electron-phonon coupling.

Tang, Ta↗