Search NASA⌕ Search

SEARCH · Search NASA

Results for “Autoregressive model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Dilated causal convolutional neural networks for forecasting zone airflow to estimate short-term energy consumption

Here this paper investigates the use of dilated causal convolutional neural networks for fine- grained temporal forecasting of building zone states. Specifically, we build and evaluate models using a small set of exogenous features (e.g., external temperature) to autoregressively predict zone airflow setpoints every minute for a 24-hour prediction window. We carefully explore the trade-off between generality and specificity in these models, training and evaluating them based on zone, zone type, month, season, and combinations thereof. When evaluated for a commercial office building in Eastern Washington with 16 zones served by variable air volume air handling units, we find that the highest performance comes from a zone-specific, season-agnostic approach; with it, we obtain an R 2 of 0.704 (averaged over zones) and an average normalized root mean square error (nRMSE) of 0.111. In contrast, the most general model (trained across all zones and seasons) yields an R 2 of only 0.416 and a nRMSE of 0.168, while a baseline zone-specific reduced order model obtains 0.443 R 2 and 0.159 nRMSE. We also report on factors affecting airflow forecasting performance, on the ability of models trained on a specific zone to generalize to other zones, and on the capability of those models trained on a specific month to generalize to other months.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2024

A Semi‐Mechanistic Model for Partitioning Evapotranspiration Reveals Transpiration Dominates the Water Flux in Drylands

Abstract Popular evapotranspiration (ET) partitioning methods make assumptions that might not be well‐suited to dryland ecosystems, such as high sensitivity of plant water‐use efficiency (WUE) to vapor pressure deficit (VPD). Our objectives were to (a) create an ET partitioning model that can produce fine‐scale estimates of transpiration (T) in drylands, and (b) use this approach to evaluate how climate controls T and WUE across ecosystem types and timescales along a dryland aridity gradient. We developed a novel, semi‐mechanistic ET partitioning method using a Bayesian approach that constrains abiotic evaporation using process‐based models, and loosely constrains time‐varying WUE within an autoregressive framework. We used this method to estimate daily T and weekly WUE across seven dryland ecosystem types and found that T dominates ET across the aridity gradient. Then, we applied cross‐wavelet coherence analysis to evaluate the temporal coherence between focal response variables (WUE and T/ET) and environmental variables. At yearly scales, we found that WUE at less arid, higher elevation sites was primarily limited by atmospheric moisture demand, and WUE at more arid, lower elevation sites was primarily limited by moisture supply. At sub‐yearly timescales, WUE and VPD were sporadically correlated. Hence, ecosystem‐scale dryland WUE is not always sensitive to changes in VPD at short timescales, despite this being a common assumption in many ET partitioning models. This new ET partitioning method can be used in dryland ecosystems to better understand how climate influences physically and biologically driven water fluxes.

Environmental Sciences & Ecology↗
DOE OSTI2025

Unrolled Video Super-Resolution Network with Autoregressive Prior for the Case of Known Motion

Real-time detection and classification of distant objects is necessary for many national security applications. However, when objects are far from the sensor, they occupy only a small number of pixels in the captured video, limiting the amount of visual detail available for recognition. State-of-the-art classification methods typically rely on high-resolution (HR) video streams to capture characteristic object features, but obtaining such detail is challenging for distant objects that occupy only a few pixels. This motivates the development of video super-resolution (VSR) methods that enhance object classification by recovering fine details from low-pixel representations. Current VSR methods rely either on model-based optimization, which is interpretable but computationally expensive, or on learning-based approaches, which are efficient and high-performing but often lack flexibility and interpretability. In this report, we propose an end-to-end trainable unrolled VSR network, UVSRNet, which super-resolves each frame in a video by exploiting sub-pixel motion between neighboring low-resolution (LR) frames as well as incorporating high-frequency detail from previously super-resolved frames. In particular, by unrolling a plug-and-play (PnP) half-quadratic splitting (HQS) algorithm, we leverage a model-based data-fitting module alongside a learning-based autoregressive prior module. This combination yields a method that maintains the flexibility and interpretability of model-based methods while achieving the performance advantages of learning-based methods.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗
DOE OSTI2024

Unveiling the Potential of MeshGraphNets for Predicting Subsurface Evolution in Carbon Storage Projects

This is the conference paper accompanying an oral presentation “Unveiling the Potential of MeshGraphNets for Predicting Subsurface Evolution in Carbon Storage Projects” at the 17th International Conference on Greenhouse Gas Control Technologies GHGT-17 held in Calgary, Canada, October 20-24 , 2024. Carbon capture and storage (CCS) technology is critical for mitigating climate change but requires effective subsurface reservoir management to ensure safe containment of injected CO2. Accurate predictions of reservoir pressure and saturation are essential for assessing long-term CCS performance. Traditional numerical simulations, while effective, are computationally intensive, time-consuming, and constrained by data discretization. Previous work has shown the effectiveness of MeshGraphNets (MGN), a graph-based machine learning framework, as an innovative alternative for predicting reservoir behavior. MGN leverages graph neural networks (GNNs) and mesh representations to model complex geological formations, offering superior adaptability across different discretizations and reservoir configurations. Classic MGN implementations utilize an autoregressive technique to predict future behavior based on current predictions, but this technique is hampered by error accumulation over time. To enhance the model accuracy in time-series predictions, this study implemented a multi-step rollout strategy that integrates autoregressive predictions during training to stabilize prediction of saturation over time. Using the Illinois Basin – Decatur Project (IBDP) dataset, comprising 100 simulations of CO2 injection, pressure, and saturation changes, the framework demonstrated its ability to learn spatial dependencies and temporal dynamics. With inputs including permeabilities, porosities, and injection rates, MGN accurately predicted CO2 plume evolution over time, even with limited training data. Moreover, the addition of a multi-step rollout procedure during training improved the ability of MGN to predict stably over time by ~15%. This research positions MGN, enhanced with multi-step rollout capabilities, as a robust and efficient tool for CCS applications. It advances the field by enabling precise, computationally efficient predictions of reservoir behavior, providing a foundation for the broader adoption of machine learning frameworks in CCS and other geoscience domains.

Holcomb, Paul↗
DOE OSTI2025

Reweighting configurations generated by transferable, machine learned models for protein sidechain backmapping

Multiscale modeling requires the linking of models at different levels of detail, with the goal of gaining accelerations from lower fidelity models while recovering fine details from higher resolution models. Communication across resolutions is particularly important in modeling soft matter, where tight couplings exist between molecular-level details and mesoscale structures. While multiscale modeling of biomolecules has become a critical component in exploring their structure and self-assembly, backmapping from coarse-grained to fine-grained, or atomistic, representations presents a challenge, despite recent advances through machine learning. A major hurdle, especially for strategies utilizing machine learning, is that backmappings can only approximately recover the atomistic ensemble of interest. We demonstrate conditions for which backmapped configurations may be reweighted to exactly recover the desired atomistic ensemble. By training separate decoding models for each sidechain type, we develop an algorithm based on normalizing flows and geometric algebra attention to autoregressively propose backmapped configurations for any protein sequence. Critical for reweighting with modern protein force fields, our trained models include all hydrogen atoms in the backmapping and make probabilities associated with atomistic configurations directly accessible. We also demonstrate, however, that reweighting is extremely challenging despite state-of-the-art performance on recently developed metrics and generation of configurations with low energies in atomistic protein force fields. Through detailed analysis of configurational weights, we show that machine-learned backmappings must not only generate configurations with reasonable energies, but also correctly assign relative probabilities under the generative model. These are broadly important considerations in generative modeling of atomistic molecular configurations.

Monroe, Jacob I. [Univ. of Arkansas, Fayetteville,↗
DOE OSTI2024

Comparison of MeshGraphNet Techniques for Subsurface Behavior Prediction during CO2 Sequestration

Carbon sequestration is a vital part of the effort to mitigate anthropogenic climate change. Previously, we have shown that Graph Neural Networks (GNNs) provide the ability to extract meaningful insights during prediction of subsurface behavior in carbon storage projects. However, these models have struggled with long-term prediction accuracy due to error accumulation caused by autoregressive prediction. This research leverages the Illinois Basin – Decatur Project (IBDP) dataset to examine strategies for minimizing loss over time in a MeshGraphNet GNN model to improve reliability of predictions while minimizing inferencing time.

Holcomb, Paul↗
DOE OSTI2024

Toward Improving the Generation Quality of Autoregressive Slot VAEs

Unconditional scene inference and generation are challenging to learn jointly with a single compositional model. Despite encouraging progress on models that extract object-centric representations (“slots”) from images, unconditional generation of scenes from slots has received less attention. This is primarily because learning the multiobject relations necessary to imagine coherent scenes is difficult. We hypothesize that most existing slot-based models have a limited ability to learn object correlations. We propose two improvements that strengthen object correlation learning. The first is to condition the slots on a global, scene-level variable that captures higher-order correlations between slots. Second, we address the fundamental lack of a canonical order for objects in images by proposing to learn a consistent order to use for the autoregressive generation of scene objects. Specifically, we train an autoregressive slot prior to sequentially generate scene objects following a learned order. Ordered slot inference entails first estimating a randomly ordered set of slots using existing approaches for extracting slots from images, then aligning those slots to ordered slots generated autoregressively with the slot prior. Our experiments across three multiobject environments demonstrate clear gains in unconditional scene generation quality. Detailed ablation studies are also provided that validate the two proposed improvements.

MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Short–Term Forecasting of Wind Gusts at Airports Across CONUS Using Machine Learning

Short–term forecasting of wind gusts, particularly those of higher intensity, is of great societal importance but is challenging due to the presence of multiple gust generation mechanisms. Wind gust observations from eight high–passenger–volume airports across the continental United States (CONUS) are summarized and used to develop predictive models of wind gust occurrence and magnitude. These short–term (same hour) forecast models are built using multiple logistic and linear regression, as well as artificial neural networks (ANNs) of varying complexity. A suite of 19 upper–air predictors drawn from the ERA5 reanalysis and an autoregressive (AR) term are used. Stepwise procedures instruct predictor selection, and resampling is used to quantify model stability. All models are developed separately for the warm (April–September) and cold (October–March) seasons. Results show that ANNs of 3–5 hidden layers (HLs) generally exhibit higher hit rates than logistic regression models and also improve skill with respect to wind gust magnitudes. However, deeper networks with more HLs increase false alarm rates in occurrence models and mean absolute error in magnitude models due to model overfitting. For model skill, inclusion of the AR term is critical while the majority of the remaining skill derives from wind speeds and lapse rates. A predictive ceiling is also clearly demonstrated, particularly for the strong and damaging gust magnitudes, which appears to be partially due to ERA5 predictor characteristics and the presence of mixed wind climates.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

A physics-constrained neural ordinary differential equations approach for robust learning of stiff chemical kinetics

The high computational cost associated with solving for detailed chemistry poses a significant challenge for predictive computational fluid dynamics (CFD) simulations of turbulent reacting flows. While deep learning techniques have been explored to develop faster surrogate models, they often fail to integrate reliably with CFD solvers. This instability arises because traditional deep learning approaches optimize for training error without ensuring compatibility with ordinary differential equation (ODE) solvers, resulting in accumulation of errors over time. Recently, neuralODE (NODE) based approaches have been shown to be a promising technique to emulate and accelerate detailed chemistry computations. Here, in the present work, we extend this NODE framework for stiff chemical kinetics by incorporating mass conservation constraints directly into the loss function during training. This ensures that the total mass as well as the individual elemental species masses are conserved in an a-posteriori manner. Proof-of-concept studies are performed with the novel physics-constrained NODE (PC-NODE) approach for homogeneous autoignition of hydrogen-air mixture over a range of composition and thermodynamic conditions. It is demonstrated that the PC-NODE framework not only improves the physical consistency of the resulting data-driven model with respect to mass conservation criteria, but also improves training efficiency. PC-NODE is shown to achieve 2–100× speedup relative to the hydrogen-air detailed chemical mechanism depending on the type of the ODE solver (implicit or explicit) used during autoregressive inference tests. Lastly, a-posteriori studies are performed wherein the trained PC-NODE model is coupled with a CFD solver. It is shown that higher accuracy is achieved with PC-NODE relative to the purely data-driven NODE approach. Moreover, PC-NODE also exhibits robustness and generalizability to unseen initial conditions from within (interpolative capability) as well as outside (extrapolative capability) the training regime.

computational combustion↗
DOE OSTI2024

Streamlining Ocean Dynamics Modeling with Fourier Neural Operators: A Multiobjective Hyperparameter and Architecture Optimization Approach

Training an effective deep learning model to learn ocean processes involves careful choices of various hyperparameters. We leverage DeepHyper’s advanced search algorithms for multiobjective optimization, streamlining the development of neural networks tailored for ocean modeling. The focus is on optimizing Fourier neural operators (FNOs), a data-driven model capable of simulating complex ocean behaviors. Selecting the correct model and tuning the hyperparameters are challenging tasks, requiring much effort to ensure model accuracy. DeepHyper allows efficient exploration of hyperparameters associated with data preprocessing, FNO architecture-related hyperparameters, and various model training strategies. We aim to obtain an optimal set of hyperparameters leading to the most performant model. Moreover, on top of the commonly used mean squared error for model training, we propose adopting the negative anomaly correlation coefficient as the additional loss term to improve model performance and investigate the potential trade-off between the two terms. The numerical experiments show that the optimal set of hyperparameters enhanced model performance in single timestepping forecasting and greatly exceeded the baseline configuration in the autoregressive rollout for long-horizon forecasting up to 30 days. Utilizing DeepHyper, we demonstrate an approach to enhance the use of FNO in ocean dynamics forecasting, offering a scalable solution with improved precision.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Neural Scaling Laws for Jet Generation

Recently observed empirical scaling laws describe the performance of foundation-type models as three independent key quantities -- dataset size, compute, and model parameters -- are modified. Extracting these scaling laws informs the training of large complex models for which the tuning of hyperparameters in traditional ways is not feasible. This work for the first time explores if scaling laws can also be observed for the task of particle jet generation -- both relevant as a pre-training objective for foundation models and as in-situ simulation by itself. We indeed replicate the key logarithmic scaling law behavior for model-size scaling. Beyond studying the next token prediction validation loss of the generative model, we also study the sliced Wasserstein distance of five physical quantities that are not immediately available to the model during training. Our study shows that this quantity is monotonically related to the next token prediction validation loss, meaning that this loss is indeed a good proxy for the physics performance. For the scaling with dataset size and compute, we observe substantially weaker scaling behavior of both the loss and the sliced Wasserstein distance. We analyze this behavior by introducing the concept of a learnable window, and argue that autoregressive next token prediction on jet constituents exhibits comparatively rapid saturation relative to language-model studies. We discuss possible origins of this behavior, including the stochastic nature of QCD radiation and differences between generative and supervised learning tasks in collider physics.

Amram, Oz [Fermilab]↗
DOE OSTI2021

Accelerating phase-field-based microstructure evolution predictions via surrogate models trained by machine learning methods

Abstract The phase-field method is a powerful and versatile computational approach for modeling the evolution of microstructures and associated properties for a wide variety of physical, chemical, and biological systems. However, existing high-fidelity phase-field models are inherently computationally expensive, requiring high-performance computing resources and sophisticated numerical integration schemes to achieve a useful degree of accuracy. In this paper, we present a computationally inexpensive, accurate, data-driven surrogate model that directly learns the microstructural evolution of targeted systems by combining phase-field and history-dependent machine-learning techniques. We integrate a statistically representative, low-dimensional description of the microstructure, obtained directly from phase-field simulations, with either a time-series multivariate adaptive regression splines autoregressive algorithm or a long short-term memory neural network. The neural-network-trained surrogate model shows the best performance and accurately predicts the nonlinear microstructure evolution of a two-phase mixture during spinodal decomposition in seconds, without the need for “on-the-fly” solutions of the phase-field equations of motion. We also show that the predictions from our machine-learned surrogate model can be fed directly as an input into a classical high-fidelity phase-field model in order to accelerate the high-fidelity phase-field simulations by leaping in time. Such machine-learned phase-field framework opens a promising path forward to use accelerated phase-field simulations for discovering, understanding, and predicting processing–microstructure–performance relationships.

36 MATERIALS SCIENCE↗
DOE OSTI2023

AutoTG: Reinforcement Learning-Based Symbolic Optimization for AI-Assisted Power Converter Design

Power converters are pervasive in modern electronic component design. They can be found in all electronic devices from household appliances and cellphone chargers to vehicles. Currently, designing new circuit topologies is hard because it requires human expertise based on experience and is difficult to automate. However, artificial-intelligence-assisted design can significantly facilitate the development of new power converters and/or improve the final result. Intelligently designed highly efficient power converters can have a significant effect on many important attributes, such as power efficiency, layout size, cost, heat dissemination, energy requirements, etc. We propose Autonomous Topology Generator (AutoTG), a reinforcement-learning-based framework that generates power converter topology candidates based on user specifications, optimized for user preferences. By modeling power converter design as a symbolic optimization problem, we sequentially sample components in an autoregressive manner until new topologies are formed, providing both the topology specification and the sizing (magnitude of each component parameter) of the proposed power converter. Here, we provide an empirical evaluation and show that AutoTG is able to generate varied high-efficiency topologies within component restrictions based on user input and show that previously unknown topologies can be found for further evaluation.

(AI)-based design↗
DOE OSTI2020

A Time Series Sustainability Assessment of a Partial Energy Portfolio Transition

Energy portfolios are overwhelmingly dependent on fossil fuel resources that perpetuate the consequences associated with climate change. Therefore, it is imperative to transition to more renewable alternatives to limit further harm to the environment. This study presents a univariate time series prediction model that evaluates sustainability outcomes of partial energy transitions. Future electricity generation at the state-level is predicted using exponential smoothing and autoregressive integrated moving average (ARIMA). The best prediction results are then used as an input for a sustainability assessment of a proposed transition by calculating carbon, water, land, and cost footprints. Missouri, USA was selected as a model testbed due to its dependence on coal. Of the time series methods, ARIMA exhibited the best performance and was used to predict annual electricity generation over a 10-year period. The proposed transition consisted of a one-percent annual decrease of coal’s portfolio share to be replaced with an equal share of solar and wind supply. The sustainability outcomes of the transition demonstrate decreases in carbon and water footprints but increases in land and cost footprints. Decision makers can use the results presented here to better inform strategic provisioning of critical resources in the context of proposed energy transitions.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2022

Technical note: Using long short-term memory models to fill data gaps in hydrological monitoring networks

Abstract. Quantifying the spatiotemporal dynamics in subsurface hydrological flows over a long time window usually employs a network of monitoring wells. However, such observations are often spatially sparse with potential temporal gaps due to poor quality or instrument failure. In this study, we explore the ability of recurrent neural networks to fill gaps in a spatially distributed time-series dataset. We use a well network that monitors the dynamic and heterogeneous hydrologic exchanges between the Columbia River and its adjacent groundwater aquifer at the U.S. Department of Energy's Hanford site. This 10-year-long dataset contains hourly temperature, specific conductance, and groundwater table elevation measurements from 42 wells with gaps of various lengths. We employ a long short-term memory (LSTM) model to capture the temporal variations in the observed system behaviors needed for gap filling. The performance of the LSTM-based gap-filling method was evaluated against a traditional autoregressive integrated moving average (ARIMA) method in terms of error statistics and accuracy in capturing the temporal patterns of river corridor wells with various dynamics signatures. Our study demonstrates that the ARIMA models yield better average error statistics, although they tend to have larger errors during time windows with abrupt changes or high-frequency (daily and subdaily) variations. The LSTM-based models excel in capturing both high-frequency and low-frequency (monthly and seasonal) dynamics. However, the inclusion of high-frequency fluctuations may also lead to overly dynamic predictions in time windows that lack such fluctuations. The LSTM can take advantage of the spatial information from neighboring wells to improve the gap-filling accuracy, especially for long gaps in system states that vary at subdaily scales. While LSTM models require substantial training data and have limited extrapolation power beyond the conditions represented in the training data, they afford great flexibility to account for the spatial correlations, temporal correlations, and nonlinearity in data without a priori assumptions. Thus, LSTMs provide effective alternatives to fill in data gaps in spatially distributed time-series observations characterized by multiple dominant frequencies of variability, which are essential for advancing our understanding of dynamic complex systems.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Simulation-Based Inference for Neutrino Interaction Model Tuning

This project demonstrates, for the first time, the application of simulation-based inference (SBI) techniques to tune neutrino–nucleus interaction models. Using a mock dataset based on the MicroBooNE tuning of the GENIE event generator, our approach employs a Neural Posterior Estimator (NPE) with Masked Autoregressive Flows (MAF) to infer the posterior distributions of key GENIE parameters directly from simulated histograms. The workflow provides a scalable and amortized framework for performing likelihood-free inference in high-dimensional parameter spaces, offering a pathway to more efficient and uncertainty-aware model tuning for next-generation neutrino experiments such as DUNE and SBND.

Tame-Narvaez, KarlaMaria [Fermi National Accelerat↗
DOE OSTI2025

Enhancing Short-Range Weather Forecasts through Temporal Variation Encoding: A Multiperiod Embedding Approach

Machine learning (ML) techniques have emerged as promising approaches to improve regional weather forecast accuracy and reliability through data-driven methods. We propose a novel ML-based weather forecasting model, the Multiperiod Embed Net (MPENet). A key distinguishing feature of MPENet is its explicit utilization of the inherent cyclic nature in weather dynamics, unlike the autoregressive strategies commonly used in other ML weather forecasting approaches. Critical cyclic structures are identified via Fourier analyses of dynamic time series. Cyclicity in the convolutional representation is achieved by transforming one-dimensional time series of meteorological variables into two-dimensional tensors based on identified periods. This approach enables the model to leverage intrinsic weather patterns, enhancing regional forecast performance. To demonstrate the effectiveness of MPENet, we conduct a comparative analysis with Nvidia’s FourCastNet. Both models are trained on High-Resolution Rapid Refresh (HRRR) data from 2015 to 2022, over a 192 km × 192 km region in Tennessee. The comparisons are performed locally at two specific locations known to have different weather dynamics due to orographic effects: Crossville, on the relatively flat Cumberland Plateau with fewer topographic airflow disruptions, and Oak Ridge, in the ridge-and-valley region, where airflow is heavily influenced by surrounding valleys and mountains. Our results indicate that FourCastNet achieves strong accuracy at very short lead times, while MPENet maintains competitive skill and shows advantages in capturing temporal evolution over longer periods. Cross-correlation analyses of MPENet and FourCastNet predictions with the HRRR data suggest that encoding critical cyclicity into the network architecture leads to improvements in the forecasting skill.

Artificial intelligence↗
DOE OSTI2021

Self-Supervised Anomaly Detection via Neural Autoregressive Flows with Active Learning

Many self-supervised methods have been proposed with the target of image anomaly detection. These methods often rely on the paradigm of data augmentation with predefined transformations such as flipping, cropping, and rotations. However, it is not straightforward to apply these techniques for non-image data, such as time series or tabular data, while the performance of the existing deep approaches has been under our expectation on tasks beyond images. In this work, we propose a novel active learning (AL) scheme that relied on neural autoregressive flows (NAF) for self-supervised anomaly detection, specifically on small-scale data. Unlike other generative models such as GANs or VAEs, flow-based models allow to explicitly learn the probability density and thus can assign accurate likelihoods to normal data which makes it usable to detect anomalies. The proposed NAF-AL method is achieved by efficiently generating random samples from latent space and transforming them into feature space along with likelihoods via invertible mapping. The samples with lower likelihoods are selected and further checked by outlier detection using Mahalanobis distance. The augmented samples incorporating with normal samples are used for training a better detector so as to approach decision boundaries. Compared with random transformations, NAF-AL can be interpreted as a likelihood-oriented data augmentation that is more efficient and robust. Extensive experiments show that our approach outperforms existing baselines on multiple time series and tabular datasets, and a real-world application in advanced manufacturing, with significant improvement on anomaly detection accuracy and robustness over the state-of-the-art.

Zhang, Jiaxin↗