Search NASA⌕ Search

SEARCH · Search NASA

Results for “autoregressive model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Convolutional L2LFlows: generating accurate showers in highly granular calorimeters using convolutional normalizing flows

Abstract In the quest to build generative surrogate models as computationally efficient alternatives to rule-based simulations, the quality of the generated samples remains a crucial frontier. So far, normalizing flows have been among the models with the best fidelity. However, as the latent space in such models is required to have the same dimensionality as the data space, scaling up normalizing flows to high dimensional datasets is not straightforward. The prior L2LFlows approach successfully used a series of separate normalizing flows and sequence of conditioning steps to circumvent this problem. In this work, we extend L2LFlows to simulate showers with a 9-times larger profile in the lateral direction. To achieve this, we introduce convolutional layers and U-Net-type connections, move from masked autoregressive flows to coupling layers, and demonstrate the successful modelling of showers in the ILD Electromagnetic Calorimeter as well as Dataset 3 from the public CaloChallenge dataset.

Instruments & Instrumentation↗
DOE OSTI2024

A Semi‐Mechanistic Model for Partitioning Evapotranspiration Reveals Transpiration Dominates the Water Flux in Drylands

Abstract Popular evapotranspiration (ET) partitioning methods make assumptions that might not be well‐suited to dryland ecosystems, such as high sensitivity of plant water‐use efficiency (WUE) to vapor pressure deficit (VPD). Our objectives were to (a) create an ET partitioning model that can produce fine‐scale estimates of transpiration (T) in drylands, and (b) use this approach to evaluate how climate controls T and WUE across ecosystem types and timescales along a dryland aridity gradient. We developed a novel, semi‐mechanistic ET partitioning method using a Bayesian approach that constrains abiotic evaporation using process‐based models, and loosely constrains time‐varying WUE within an autoregressive framework. We used this method to estimate daily T and weekly WUE across seven dryland ecosystem types and found that T dominates ET across the aridity gradient. Then, we applied cross‐wavelet coherence analysis to evaluate the temporal coherence between focal response variables (WUE and T/ET) and environmental variables. At yearly scales, we found that WUE at less arid, higher elevation sites was primarily limited by atmospheric moisture demand, and WUE at more arid, lower elevation sites was primarily limited by moisture supply. At sub‐yearly timescales, WUE and VPD were sporadically correlated. Hence, ecosystem‐scale dryland WUE is not always sensitive to changes in VPD at short timescales, despite this being a common assumption in many ET partitioning models. This new ET partitioning method can be used in dryland ecosystems to better understand how climate influences physically and biologically driven water fluxes.

Environmental Sciences & Ecology↗
DOE OSTI2025

Unrolled Video Super-Resolution Network with Autoregressive Prior for the Case of Known Motion

Real-time detection and classification of distant objects is necessary for many national security applications. However, when objects are far from the sensor, they occupy only a small number of pixels in the captured video, limiting the amount of visual detail available for recognition. State-of-the-art classification methods typically rely on high-resolution (HR) video streams to capture characteristic object features, but obtaining such detail is challenging for distant objects that occupy only a few pixels. This motivates the development of video super-resolution (VSR) methods that enhance object classification by recovering fine details from low-pixel representations. Current VSR methods rely either on model-based optimization, which is interpretable but computationally expensive, or on learning-based approaches, which are efficient and high-performing but often lack flexibility and interpretability. In this report, we propose an end-to-end trainable unrolled VSR network, UVSRNet, which super-resolves each frame in a video by exploiting sub-pixel motion between neighboring low-resolution (LR) frames as well as incorporating high-frequency detail from previously super-resolved frames. In particular, by unrolling a plug-and-play (PnP) half-quadratic splitting (HQS) algorithm, we leverage a model-based data-fitting module alongside a learning-based autoregressive prior module. This combination yields a method that maintains the flexibility and interpretability of model-based methods while achieving the performance advantages of learning-based methods.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗
DOE OSTI2024

Unveiling the Potential of MeshGraphNets for Predicting Subsurface Evolution in Carbon Storage Projects

This is the conference paper accompanying an oral presentation “Unveiling the Potential of MeshGraphNets for Predicting Subsurface Evolution in Carbon Storage Projects” at the 17th International Conference on Greenhouse Gas Control Technologies GHGT-17 held in Calgary, Canada, October 20-24 , 2024. Carbon capture and storage (CCS) technology is critical for mitigating climate change but requires effective subsurface reservoir management to ensure safe containment of injected CO2. Accurate predictions of reservoir pressure and saturation are essential for assessing long-term CCS performance. Traditional numerical simulations, while effective, are computationally intensive, time-consuming, and constrained by data discretization. Previous work has shown the effectiveness of MeshGraphNets (MGN), a graph-based machine learning framework, as an innovative alternative for predicting reservoir behavior. MGN leverages graph neural networks (GNNs) and mesh representations to model complex geological formations, offering superior adaptability across different discretizations and reservoir configurations. Classic MGN implementations utilize an autoregressive technique to predict future behavior based on current predictions, but this technique is hampered by error accumulation over time. To enhance the model accuracy in time-series predictions, this study implemented a multi-step rollout strategy that integrates autoregressive predictions during training to stabilize prediction of saturation over time. Using the Illinois Basin – Decatur Project (IBDP) dataset, comprising 100 simulations of CO2 injection, pressure, and saturation changes, the framework demonstrated its ability to learn spatial dependencies and temporal dynamics. With inputs including permeabilities, porosities, and injection rates, MGN accurately predicted CO2 plume evolution over time, even with limited training data. Moreover, the addition of a multi-step rollout procedure during training improved the ability of MGN to predict stably over time by ~15%. This research positions MGN, enhanced with multi-step rollout capabilities, as a robust and efficient tool for CCS applications. It advances the field by enabling precise, computationally efficient predictions of reservoir behavior, providing a foundation for the broader adoption of machine learning frameworks in CCS and other geoscience domains.

Holcomb, Paul↗
DOE OSTI2025

Reweighting configurations generated by transferable, machine learned models for protein sidechain backmapping

Multiscale modeling requires the linking of models at different levels of detail, with the goal of gaining accelerations from lower fidelity models while recovering fine details from higher resolution models. Communication across resolutions is particularly important in modeling soft matter, where tight couplings exist between molecular-level details and mesoscale structures. While multiscale modeling of biomolecules has become a critical component in exploring their structure and self-assembly, backmapping from coarse-grained to fine-grained, or atomistic, representations presents a challenge, despite recent advances through machine learning. A major hurdle, especially for strategies utilizing machine learning, is that backmappings can only approximately recover the atomistic ensemble of interest. We demonstrate conditions for which backmapped configurations may be reweighted to exactly recover the desired atomistic ensemble. By training separate decoding models for each sidechain type, we develop an algorithm based on normalizing flows and geometric algebra attention to autoregressively propose backmapped configurations for any protein sequence. Critical for reweighting with modern protein force fields, our trained models include all hydrogen atoms in the backmapping and make probabilities associated with atomistic configurations directly accessible. We also demonstrate, however, that reweighting is extremely challenging despite state-of-the-art performance on recently developed metrics and generation of configurations with low energies in atomistic protein force fields. Through detailed analysis of configurational weights, we show that machine-learned backmappings must not only generate configurations with reasonable energies, but also correctly assign relative probabilities under the generative model. These are broadly important considerations in generative modeling of atomistic molecular configurations.

Monroe, Jacob I. [Univ. of Arkansas, Fayetteville,↗
NASA NTRS2001

Do nonlinearities play a significant role in short term, beat-to-beat variability?

Numerous studies of short-term beat-to-beat variability in cardiovascular signals have not resolved the debate about the completeness of linear analysis techniques. This aim of this paper is to evaluate further the role of nonlinearities in short-term, beat-to-beat variability. We compared linear autoregressive moving average (ARMA) and nonlinear neural network (NN) models for predicting instantaneous heart rate (HR) and mean arterial blood pressure (BP) from past HR and BP. To evaluate these models, we used HR and BP time series from the MIMIC database. Experimental results indicate that NN-based nonlinearities do not play a significant role and suggest that ARMA linear analysis techniques provide adequate characterization of the system dynamics responsible for generating short-term, beat-to-beat variability.

NASA Discipline Cardiopulmonary↗
DOE OSTI2024

Comparison of MeshGraphNet Techniques for Subsurface Behavior Prediction during CO2 Sequestration

Carbon sequestration is a vital part of the effort to mitigate anthropogenic climate change. Previously, we have shown that Graph Neural Networks (GNNs) provide the ability to extract meaningful insights during prediction of subsurface behavior in carbon storage projects. However, these models have struggled with long-term prediction accuracy due to error accumulation caused by autoregressive prediction. This research leverages the Illinois Basin – Decatur Project (IBDP) dataset to examine strategies for minimizing loss over time in a MeshGraphNet GNN model to improve reliability of predictions while minimizing inferencing time.

Holcomb, Paul↗
NASA NTRS1991

Total ozone trends deduced from Nimbus 7 TOMS data

The Total Ozone Mapping Spectrometer (TOMS) on the Nimbus 7 satellite has been measuring the total column amount of ozone over the globe for more than 11 years. Recent improvements in the data analysis have led to a technique for determining and removing drift in the calibration such that the data at the end of the record are precise to + or - 1.3 percent (2-sigma) relative to the data at the beginning of the record. A statistical model, including terms for seasonal variation, linear trend, quasi-biennial oscillation, solar cycle and second-order autoregressive noise has been fit to the TOMS time series of total ozone data. The linear trend obtained when this statistical model is fit to the TOMS data averaged between 65 N and 65 S latitudes is -0.26 + or - 0.14 percent/year or -3 percent over the 11.6 year time period from November 1978 to May 1990. The trend is near zero (0.0002 + or - 0.2 percent/year) at the equator and increases toward both poles.

Stolarski, Richard S.↗
DOE OSTI2025

A physics-constrained neural ordinary differential equations approach for robust learning of stiff chemical kinetics

The high computational cost associated with solving for detailed chemistry poses a significant challenge for predictive computational fluid dynamics (CFD) simulations of turbulent reacting flows. While deep learning techniques have been explored to develop faster surrogate models, they often fail to integrate reliably with CFD solvers. This instability arises because traditional deep learning approaches optimize for training error without ensuring compatibility with ordinary differential equation (ODE) solvers, resulting in accumulation of errors over time. Recently, neuralODE (NODE) based approaches have been shown to be a promising technique to emulate and accelerate detailed chemistry computations. Here, in the present work, we extend this NODE framework for stiff chemical kinetics by incorporating mass conservation constraints directly into the loss function during training. This ensures that the total mass as well as the individual elemental species masses are conserved in an a-posteriori manner. Proof-of-concept studies are performed with the novel physics-constrained NODE (PC-NODE) approach for homogeneous autoignition of hydrogen-air mixture over a range of composition and thermodynamic conditions. It is demonstrated that the PC-NODE framework not only improves the physical consistency of the resulting data-driven model with respect to mass conservation criteria, but also improves training efficiency. PC-NODE is shown to achieve 2–100× speedup relative to the hydrogen-air detailed chemical mechanism depending on the type of the ODE solver (implicit or explicit) used during autoregressive inference tests. Lastly, a-posteriori studies are performed wherein the trained PC-NODE model is coupled with a CFD solver. It is shown that higher accuracy is achieved with PC-NODE relative to the purely data-driven NODE approach. Moreover, PC-NODE also exhibits robustness and generalizability to unseen initial conditions from within (interpolative capability) as well as outside (extrapolative capability) the training regime.

computational combustion↗
DOE OSTI2024

Streamlining Ocean Dynamics Modeling with Fourier Neural Operators: A Multiobjective Hyperparameter and Architecture Optimization Approach

Training an effective deep learning model to learn ocean processes involves careful choices of various hyperparameters. We leverage DeepHyper’s advanced search algorithms for multiobjective optimization, streamlining the development of neural networks tailored for ocean modeling. The focus is on optimizing Fourier neural operators (FNOs), a data-driven model capable of simulating complex ocean behaviors. Selecting the correct model and tuning the hyperparameters are challenging tasks, requiring much effort to ensure model accuracy. DeepHyper allows efficient exploration of hyperparameters associated with data preprocessing, FNO architecture-related hyperparameters, and various model training strategies. We aim to obtain an optimal set of hyperparameters leading to the most performant model. Moreover, on top of the commonly used mean squared error for model training, we propose adopting the negative anomaly correlation coefficient as the additional loss term to improve model performance and investigate the potential trade-off between the two terms. The numerical experiments show that the optimal set of hyperparameters enhanced model performance in single timestepping forecasting and greatly exceeded the baseline configuration in the autoregressive rollout for long-horizon forecasting up to 30 days. Utilizing DeepHyper, we demonstrate an approach to enhance the use of FNO in ocean dynamics forecasting, offering a scalable solution with improved precision.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Neural Scaling Laws for Jet Generation

Recently observed empirical scaling laws describe the performance of foundation-type models as three independent key quantities -- dataset size, compute, and model parameters -- are modified. Extracting these scaling laws informs the training of large complex models for which the tuning of hyperparameters in traditional ways is not feasible. This work for the first time explores if scaling laws can also be observed for the task of particle jet generation -- both relevant as a pre-training objective for foundation models and as in-situ simulation by itself. We indeed replicate the key logarithmic scaling law behavior for model-size scaling. Beyond studying the next token prediction validation loss of the generative model, we also study the sliced Wasserstein distance of five physical quantities that are not immediately available to the model during training. Our study shows that this quantity is monotonically related to the next token prediction validation loss, meaning that this loss is indeed a good proxy for the physics performance. For the scaling with dataset size and compute, we observe substantially weaker scaling behavior of both the loss and the sliced Wasserstein distance. We analyze this behavior by introducing the concept of a learnable window, and argue that autoregressive next token prediction on jet constituents exhibits comparatively rapid saturation relative to language-model studies. We discuss possible origins of this behavior, including the stochastic nature of QCD radiation and differences between generative and supervised learning tasks in collider physics.

Amram, Oz [Fermilab]↗
NASA NTRS1991

Detection of target distance in the presence of an interfering reflection using a frequency-stepped double side-band suppressed carrier microwave radar system

A technique for detecting the distance to a highly reflective target in the presence of an interesting reflection using a frequency-stepped double-sideband suppressed carrier (DSBSC) microwave-millimeter-wave radar system is analytically derived. The main result of the analysis shows that the measured group delays produced by the DSBSC system possess a periodicity inversely proportional to the difference between the time delays to the target and interferer, independent of the signal-to-interference ratio (SIR). Simulation results are presented in the context of electron plasma density range estimation using a block diagram communications CAD tool. A unique and accurate plasma model is introduced. A high-resolution spectral estimation technique based on an autoregressive time series analysis is applied to the measured group delays, and it is shown that accurate target distance estimates may be obtained, independent of SIR.

Ybarra, Gary A.↗
NASA NTRS1995

Interdecadal Variations in the Alaska Gyre

Climatic dynamic topography variations in the Alaska gyre during the period 1968-1990 are described with an objective analysis of more than 12000 STD and XBT stations, and COADS wind stress data Interannual the dynamic height and SST variations were correlated and were consistent with recently described large-scale climatic shifts in the North Pacific. The gyre was centered more to the east, circulation appeared stronger, and SST was lower during the early to mid-1970s than during the 1980s. The Aleutian low (NP and PNA indices) intensified during the interim, but the response did not appear as a gyre spinup. Instead, the associated wind stress anomalies forced a slowly varying dynamic height anomaly across the eastern and northern part of the gyre through Ekman convergence, which had the effect of displacing the gyre's low somewhat to the WSW in the 1980s. The wind curl spectrum was white, and the slow oceanic response was modeled as stochastic-forced climate variability with a simple first-order Markov autoregression process. Forcing was assumed to be Ekman pumping of the pycnocline, and the damping coefficient was estimated from the data to be approx. 1 yr. A hindcast with observed winds gave estimated dynamic height patterns similar to those observed, with a canonical correlation of 0.79 at 99% confidence. This response was weak in the western half of the gyre, where slow baroclinic variability may have been influenced by long Rossby wave propagation. A simple autoregression simulation using artificial white noise forcing shows the evolution of decadal variations similar in nature to those observed. This result, along with the low frequency correlation between dynamic height and SST, suggests that the upper-ocean climatic variability in this region is primarily wind forced.

Lagerloef, Gary S.E.↗
NASA NTRS1981

Parameter estimation of large flexible aerospace structures with application to the control of the Maypole Deployable Reflector

Systems such as the Maypole deployable reflector have a distributed parameter nature. The flexible column and hoop structure and the circular antenna of 30-100 meter diameter which it supports are described by partial, rather than ordinary, differential equations. Progress completed in reduced order modelling andd controller design and digital parameter estimation and control is summarized. Topics covered include depolyment and on-orbit operation; quasi-static (steady state) operation; dynamic distributed parameter system; autoregressive moving average identification; frequency domain procedures; direct or implicit active control; adaptive observers; parameter estimation using a linear reinforcement learning factor; feedback control; and reduced order modeling for nonlinear systems.

Balas, M. J.↗
NASA NTRS1987

The application of time series models to cloud field morphology analysis

A modeling method for the quantitative description of remotely sensed cloud field images is presented. A two-dimensional texture modeling scheme based on one-dimensional time series procedures is adopted for this purpose. The time series procedure used is the seasonal autoregressive, moving average (ARMA) process in Box and Jenkins. Cloud field properties such as directionality, clustering and cloud coverage can be retrieved by this method. It has been demonstrated that a cloud field image can be quantitatively defined by a small set of parameters and synthesized surrogates can be reconstructed from these model parameters. This method enables cloud climatology to be studied quantitatively.

Chin, Roland T.↗
DOE OSTI2025

Simulation-Based Inference for Neutrino Interaction Model Tuning

This project demonstrates, for the first time, the application of simulation-based inference (SBI) techniques to tune neutrino–nucleus interaction models. Using a mock dataset based on the MicroBooNE tuning of the GENIE event generator, our approach employs a Neural Posterior Estimator (NPE) with Masked Autoregressive Flows (MAF) to infer the posterior distributions of key GENIE parameters directly from simulated histograms. The workflow provides a scalable and amortized framework for performing likelihood-free inference in high-dimensional parameter spaces, offering a pathway to more efficient and uncertainty-aware model tuning for next-generation neutrino experiments such as DUNE and SBND.

Tame-Narvaez, KarlaMaria [Fermi National Accelerat↗
NASA NTRSDate not supplied

Representing the Sub-Grid Heterogeneity of Surface Precipitation in A General Circulation Model

Precipitation variability on spatial scales smaller than a typical general circulation model (GCM) grid box is often neglected, with the grid-mean precipitation rate being applied uniformly to underlying surface tiles. This reduces the extrema seen by the surface, with corresponding reductions in surface runoff and altered land-atmosphere fluxes. Here we present a novel approach to stochastically distribute precipitation across sub-grid surface tiles within a GCM. Based on 4 km Stage IV precipitation data, the scheme parameterizes the dry area fraction as a function of grid mean precipitation rate, and defines the relative distribution of intensities across non-dry surface tiles. To incorporate memory and mimic the persistence of precipitating storms, the relative intensity assigned to each sub-grid tile is determined by an autoregressive process. Using single column experiments, the scheme is shown to reproduce observed precipitation statistics at the scale of model surface tiles. We also document impacts on surface hydrology and energy partitioning, with notable increases in precipitation runoff, surface temperature variance, and the Bowen ratio.

GCM↗
NASA NTRS1992

Stratospheric ozone profile and total ozone trends derived from the SAGE I and SAGE II data

Global trends in both stratospheric column ozone and as a function of altitude are derived on the basis of SAGE I/II ozone data from the period 1979-1991. A statistical model containing quasi-biennial, seasonal, and semiannual oscillations, a linear component, and a first-order autoregressive noise process was fit to the time series of SAGE I/II monthly zonal mean data. The linear trend in column ozone above 17-km altitude, averaged between 65 deg S and 65 deg N, is -0.30 +/-0.19 percent/yr, or -3.6 percent over the time period February 1979 through April 1991. The data show that the column trend above 17 km is nearly zero in the tropics and increases towards the high latitudes with values of -0.6 percent/yr at 60 deg S and -0.35 percent/yr at 60 deg N. Both these results are in agreement with the recent TOMS results. The profile trend analyses show that the column ozone losses are occurring below 25 km, with most of the loss coming from the region between 17 and 20 km. Negative trend values on the order of -2 percent/yr are found at 17 km in midlatitudes.

Mccormick, M. P.↗