Search NASA⌕ Search

SEARCH · Search NASA

Results for “non-stationary data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Uncertainty guided online ensemble for non-stationary data streams in fusion science

Machine Learning (ML) is poised to play a pivotal role in the development and operation of next-generation fusion devices. Fusion data shows non-stationary behavior with distribution drifts, resulted by both experimental evolution and machine wear-and-tear. ML models assume stationary distribution and fail to maintain performance when encountered with such non-stationary data streams. Online learning techniques have been leveraged in other domains, however it has been largely unexplored for fusion applications. In this paper, we investigate online learning for continuous adaptation to drifting data streams in the prediction of Toroidal Field (TF) coils deflection at the DIII-D fusion facility. We further address the short-term performance degradation inherent to standard online learning, which arises because ground truth is unavailable at prediction time. To mitigate this issue, we propose an uncertainty-guided online ensemble framework. The method leverages the Deep Gaussian Process Approximation (DGPA) for calibrated uncertainty estimation and uses these uncertainty measures to guide a meta-algorithm that aggregates predictions from learners trained over different historical horizons. Our results show that online learning reduces prediction error by 80% compared to a static model. The online ensemble and the proposed uncertainty-guided ensemble further reduce error by approximately 6%, and 10% respectively, relative to standard single-model online learning, while also providing calibrated uncertainty estimates to support operational decision-making.

AI↗

Uncertainty based Online Ensemble on Non-Stationary Data for Fusion Science

Machine Learning (ML) is poised to play a pivotal role in the development and operation of next-generation fusion devices. Fusion data shows non-stationary behavior due to drifts in the data. The drifts can arise from both experimental evolution and machine wear-and-tear. ML models assume stationary distribution and fail to maintain performance when encountered with non-stationary data streams.Online learning can be used to continuously adapt the models with new data as it is acquired. However, traditional online learning can suffer from short-term performance degradation, as ground truth are not available before making the prediction. To address this challenge, we propose uncertainty aware ensemble approach for online learning. We use Deep Gaussian Process Approximation (DGPA) technique for calibrated uncertainty estimation and use the uncertainty values to guide a meta-algorithm that produces predictions based on ensemble of learners. Moreover, DGPA also provides uncertainty estimation along with the predictions for decision makers. This paper demonstrates that the proposed method outperforms traditional online learning approach, and a naive ensemble without uncertainty guidance by about 7% and 6%, respectively, on B-coil deflection prediction at DIII-D Fusion Facility.

Rajput, Kishansingh [Thomas Jefferson National Acc↗

Uncertainty based Online Ensemble on Non-Stationary Data for Fusion Science

Machine Learning (ML) is poised to play a pivotal role in the development and operation of next-generation fusion devices. Fusion data shows non-stationary behavior due to drifts in the data. The drifts can arise from both experimental evolution and machine wear-and-tear. ML models assume stationary distribution and fail to maintain performance when encountered with non-stationary data streams.Online learning can be used to continuously adapt the models with new data as it is acquired. However, traditional online learning can suffer from short-term performance degradation, as ground truth are not available before making the prediction. To address this challenge, we propose uncertainty aware ensemble approach for online learning. We use Deep Gaussian Process Approximation (DGPA) technique for calibrated uncertainty estimation and use the uncertainty values to guide a meta-algorithm that produces predictions based on ensemble of learners. Moreover, DGPA also provides uncertainty estimation along with the predictions for decision makers. This paper demonstrates that the proposed method outperforms traditional online learning approach, and a naive ensemble without uncertainty guidance by about 7% and 6%, respectively, on B-coil deflection prediction at DIII-D Fusion Facility.

Rajput, Kishansingh [Thomas Jefferson National Acc↗

Continual Learning for Particle Accelerators

Particle accelerators operate under dynamically changing conditions, which often lead to data distribution drifts. These drifts pose significant challenges for Machine Learning (ML) models, which typically fail to maintain performance when faced with such non-stationary data. In particle accelerators, the primary sources of these data drifts include changes in accelerator settings and non-measured parameters such as machine degradation and environmental factors. Previous research has proposed conditional models to handle multiple beam configurations effectively; however, it is challenging to train the ML models on all possible configuration settings. Additionally, conditional models alone can not address performance degradation caused by drifts due to non-measured factors. These limitations contribute to a significant gap between ML development and its deployment in real-world operational settings. To bridge this gap, in this paper, we identify some of the key areas within particle accelerators where continual learning can help mitigate drift-induced performance degradation. In addition, we present a practical use case where a conditional Auto-Encoder model coupled with memory-based continual learning has been employed to demonstrate stable performance even when underlying data drifts.

Schram, Malachi [Thomas Jefferson National Acceler↗

Continual Learning for Particle Accelerators

Particle accelerators operate under dynamically changing conditions, which often lead to data distribution drifts. These drifts pose significant challenges for Machine Learning (ML) models, which typically fail to maintain performance when faced with such non-stationary data. In particle accelerators, the primary sources of these data drifts include changes in accelerator settings and non-measured parameters such as machine degradation and environmental factors. Previous research has proposed conditional models to handle multiple beam configurations effectively; however, it is challenging to train the ML models on all possible configuration settings. Additionally, conditional models alone can not address performance degradation caused by drifts due to non-measured factors. These limitations contribute to a significant gap between ML development and its deployment in real-world operational settings. To bridge this gap, in this paper, we identify some of the key areas within particle accelerators where continual learning can help mitigate drift-induced performance degradation. In addition, we present a practical use case where a conditional Auto-Encoder model coupled with memory-based continual learning has been employed to demonstrate stable performance even when underlying data drifts

Rajput, Kishansingh [Thomas Jefferson National Acc↗

Online evolutionary neural architecture search for multivariate non-stationary time series forecasting

Time series forecasting (TSF) is one of the most important tasks in data science. TSF models are usually pre-trained with historical data and then applied on future unseen datapoints. However, real-world time series data is usually non-stationary and models trained offline usually face problems from data drift. Models trained and designed in an offline fashion can not quickly adapt to changes quickly or be deployed in real-time. To address these issues, this work presents the Online NeuroEvolution-based Neural Architecture Search (ONE-NAS) algorithm, which is a novel neural architecture search method capable of automatically designing and dynamically training recurrent neural networks (RNNs) for online forecasting tasks. Without any pre-training, ONE-NAS utilizes populations of RNNs that are continuously updated with new network structures and weights in response to new multivariate input data. ONE-NAS is tested on real-world, large-scale multivariate wind turbine data as well as the univariate Dow Jones Industrial Average (DJIA) dataset. These results demonstrate that ONE-NAS outperforms traditional statistical time series forecasting methods, including online linear regression, fixed long short-term memory (LSTM) and gated recurrent unit (GRU) models trained online, as well as state-of-the-art, online ARIMA strategies. Additionally, results show that utilizing multiple populations of RNNs which are periodically repopulated provide significant performance improvements, allowing this online neural network architecture design and training to be successful.

97 MATHEMATICS AND COMPUTING↗

Uncertainty quantification of machine learning models to improve streamflow prediction under changing climate and environmental conditions

Machine learning (ML) models, and Long Short-Term Memory (LSTM) networks in particular, have demonstrated remarkable performance in streamflow prediction and are increasingly being used by the hydrological research community. However, most of these applications do not include uncertainty quantification (UQ). ML models are data driven and can suffer from large extrapolation errors when applied to changing climate/environmental conditions. UQ is required to quantify the influence of data noises on model predictions and avoid overconfident projections in extrapolation. In this work, we integrate a novel UQ method, called PI3NN, with LSTM networks for streamflow prediction. PI3NN calculates Prediction Intervals by training 3 Neural Networks. It can precisely quantify the predictive uncertainty caused by the data noise and identify out-of-distribution (OOD) data in a non-stationary condition to avoid overconfident predictions. We apply the PI3NN-LSTM method in the snow-dominant East River Watershed in the western US and in the rain-driven Walker Branch Watershed in the southeastern US. Results indicate that for the prediction data which have similar features as the training data, PI3NN precisely quantifies the predictive uncertainty with the desired confidence level; and for the OOD data where the LSTM network fails to make accurate predictions, PI3NN produces a reasonably large uncertainty indicating that the results are not trustworthy and should avoid overconfidence. PI3NN is computationally efficient, robust in performance, and generalizable to various network structures and data with no distributional assumptions. It can be broadly applied in ML-based hydrological simulations for credible prediction.

54 ENVIRONMENTAL SCIENCES↗

Subordinated Gaussian Processes for Solar Irradiance

Traditionally the power grid has been a one-way street with power flowing from large transmission-connected generators through the distribution network to consumers. This paradigm is changing with the introduction of distributed renewable energy resources (DERs), and with it, the way the grid is managed. There is currently a dearth of high fidelity solar irradiance datasets available to help grid researchers understand how expansion of DERs could affect future power system operations. Realistic simulations of by-the-second solar irradiances are needed to study how DER variability affects the grid. Irradiance data are highly non-stationary and non-Gaussian, and even modern time series models are challenged by their distributional properties. We develop a subordinated non-Gaussian stochastic model whose simulations realistically capture the distribution and dependence structure in measured irradiance. We illustrate our approach on a fine resolution dataset from Hawaii, where our approach outperforms standard nonlinear time series models.

MATHEMATICS AND COMPUTING,SOLAR ENERGY↗

FLYING SERVING: On-the-Fly Parallelism Switching for Large Language Model Serving

Production LLM serving must simultaneously deliver high throughput, low latency, and sufficient context capacity under non-stationary traffic and mixed request requirements. Data parallelism (DP) maximizes throughput by running independent replicas, while tensor parallelism (TP) reduces per-request latency and pools memory for long-context inference. However, existing serving stacks typically commit to a static parallelism configuration at deployment; adapting to bursts, priorities, or long-context requests is often disruptive and slow. We present Flying Serving, a vLLM-based system that enables online DP-TP switching without restarting engine workers. Flying Serving makes reconfiguration practical by virtualizing the state that would otherwise force data movement: (i) a zero-copy Model Weights Manager that exposes TP shard views on demand, (ii) a KV Cache Adaptor that preserves request KV state across DP/TP layouts, (iii) an eagerly initialized Communicator Pool to amortize collective setup, and (iv) a deadlock-free scheduler that coordinates safe transitions under execution skew. Across three popular LLMs and realistic serving scenarios, Flying Serving improves performance by up to 4.79 × under high load and 3.47 × under low load while supporting latency- and memory-driven requests.

Gao, Shouwei [ORNL]↗

Continual Learning for Production-Level Machine Learning in Particle Accelerators

Particle accelerators operate in complex environments where data distribution can change dynamically, leading to data drifts that significantly challenge Machine Learning (ML) models. These non-stationary conditions often cause ML models to deteriorate in performance, making it difficult to maintain reliable predictions in operation. The primary sources of data drifts are changes in accelerator settings and changes in equipment performance which cannot be measured directly. To bridge this gap between ML development and long-term deployment in operational settings, we identify key areas within particle accelerators where continual learning can help mitigate drift-induced performance degradation. We will provide a practical guide on selecting the appropriate method given resource constraints and desired stability plasticity trade offs. As a concrete example, we will present a real-world use case for anomaly detection to predict errant beams at the Spallation Neutron Source accelerator, where continual learning has been employed to demonstrate stable performance on drifting data streams. We will present practical challenges, lessons learned, and the results from the deployed ML model.

Rajput, Kishansingh [Thomas Jefferson National Acc↗

Data-driven projection pursuit adaptation of polynomial chaos expansions for dependent high-dimensional parameters

Uncertainty quantification (UQ) and inference involving a large number of parameters are valuable tools for problems associated with heterogeneous and non-stationary behaviors. The difficulty with these problems is exacerbated when these parameters are statistically dependent requiring statistical characterization over joint measures. Probabilistic modeling methodologies stand as effective tools in the realms of UQ and inference. Among these, polynomial chaos expansions (PCE), when adapted to low-dimensional quantities of interest (QoI), provide effective yet accurate approximations for these QoI in terms of an adapted orthogonal basis. These adaptation techniques have been cast as projection pursuits in Gaussian Hilbert space in what has been referred to as a projection pursuit adaptation (PPA) by Xiaoshu Zeng and Roger Ghanem (2023). The PPA method efficiently identifies an optimal low-dimensional space for representing the QoI and simultaneously evaluates an optimal PCE within that space. The quality of this approximation clearly depends on the size of the training dataset, which is typically a function of the adapted reduced dimension. Here, the complexity of the problem is thus mediated by the complexity of the low-dimensional quantity of interest and not the complexity of the high-dimensional parameter space.

Data-driven↗

Exact Gaussian processes for massive datasets via non-stationary sparsity-discovering kernels

Abstract A Gaussian Process (GP) is a prominent mathematical framework for stochastic function approximation in science and engineering applications. Its success is largely attributed to the GP’s analytical tractability, robustness, and natural inclusion of uncertainty quantification. Unfortunately, the use of exact GPs is prohibitively expensive for large datasets due to their unfavorable numerical complexity of $$O(N^3)$$ O ( N 3 ) in computation and $$O(N^2)$$ O ( N 2 ) in storage. All existing methods addressing this issue utilize some form of approximation—usually considering subsets of the full dataset or finding representative pseudo-points that render the covariance matrix well-structured and sparse. These approximate methods can lead to inaccuracies in function approximations and often limit the user’s flexibility in designing expressive kernels. Instead of inducing sparsity via data-point geometry and structure, we propose to take advantage of naturally-occurring sparsity by allowing the kernel to discover—instead of induce—sparse structure. The premise of this paper is that the data sets and physical processes modeled by GPs often exhibit natural or implicit sparsities, but commonly-used kernels do not allow us to exploit such sparsity. The core concept of exact, and at the same time sparse GPs relies on kernel definitions that provide enough flexibility to learn and encode not only non-zero but also zero covariances. This principle of ultra-flexible, compactly-supported, and non-stationary kernels, combined with HPC and constrained optimization, lets us scale exact GPs well beyond 5 million data points.

97 MATHEMATICS AND COMPUTING↗

Unsupervised acoustic detection of fatigue-induced damage modes from wind turbine blades

This paper proposes a new in-situ damage detection approach for wind turbine blades, which leverages blade-internal non-stationary acoustic pressure fluctuations caused by the mechanical loading as the main source of excitation. This acoustic excitation was leveraged for the detection of fatigue-related damage modes on a full-scale wind turbine blade undergoing edgewise fatigue testing. An unsupervised, data-driven structural health monitoring strategy was developed to learn the normal cavity-internal acoustic sequences generated by the blade’s load cycles and to detect damage-related anomalies in the context of those sequences. A linear cepstral-coefficient based feature set was used to characterize the cavity-internal acoustics and LSTM-autoencoders were trained to accurately reconstruct healthy-case sequences. The reconstruction error was then used to characterize anomalous acoustic patterns within the blade cavity. The technique was able to detect a damage event earlier than a strain-based system by 120,000 load cycles.

17 WIND ENERGY↗

Evaluating the contribution of plant metabolic pathways in the light to the ATP:NADPH demand using a meta-analysis of isotopically non-stationary metabolic flux analyses

Abstract Balancing the ATP: NADPH demand from plant metabolism with supply from photosynthesis is essential for preventing photodamage and operating efficiently, so understanding its drivers is important for integrating metabolism with the light reactions of photosynthesis and for bioengineering efforts that may radically change this demand. It is often assumed that the C3 cycle and photorespiration consume the largest amount of ATP and reductant in illuminated leaves and as a result mostly determine the ATP: NADPH demand. However, the quantitative extent to which other energy consuming metabolic processes contribute in large ways to overall ATP: NADPH demand remains unknown. Here, we used the metabolic flux networks of numerous recently published isotopically non-stationary metabolic flux analyses (INST-MFA) to evaluate flux through the C3 cycle, photorespiration, the oxidative pentose phosphate pathway, the tricarboxylic acid cycle, and starch/sucrose synthesis and characterize broad trends in the demand of energy across different pathways and compartments as well as in the overall ATP:NADPH demand. These data sets include a variety of species including Arabidopsis thaliana , Nicotiana tabacum , and Camelina sativa as well as varying environmental factors including high/low light, day length, and photorespiratory levels. Examining these datasets in aggregate reveals that ultimately the bulk of the energy flux occurred in the C3 cycle and photorespiration, however, the energy demand from these pathways did not determine the ATP: NADPH demand alone. Instead, a notable contribution was revealed from starch and sucrose synthesis which might counterbalance photorespiratory demand and result in fewer adjustments in mechanisms which balance the ATP deficit.

59 BASIC BIOLOGICAL SCIENCES↗

Asymptotic coefficients of the attached-eddy model derived from an adiabatic atmosphere

The attached-eddy model (AEM) predicts that the mean streamwise velocity and streamwise velocity variance profiles follow a logarithmic shape, while the vertical velocity variance remains invariant with height in the overlap region of high Reynolds number wall-bounded turbulent flows. Moreover, the AEM coefficients are presumed to attain asymptotically constant values at very high Reynolds numbers. Here, the AEM predictions are examined using sonic anemometer measurements in the near-neutral atmospheric surface layer, with a focus on the logarithmic behaviour of the streamwise velocity variance. Utilizing an extensive 210-day dataset collected from a 62 m meteorological tower located in the Eastern Snake River Plain, Idaho, USA, the inertial sublayer is first identified by analysing the measured momentum flux and mean velocity profiles. The logarithmic behaviour of the streamwise velocity variance and the associated ‘−1’ scaling of the streamwise velocity energy spectra are then investigated. The findings indicate that the Townsend–Perry coefficient (A 1 ) is influenced by mild non-stationarity that manifests itself as a Reynolds number dependence. After excluding non-stationary runs, and requiring the bulk Reynolds number defined using the atmospheric boundary layer height to be larger than 4 × 10 7 , the inferred A 1 converges to values ranging between 1 and 1.25, consistent with laboratory experiments. Furthermore, nine benchmark cases selected through a restrictive quality control reveal a close relation between the ‘−1’ scaling in the streamwise velocity energy spectrum and the logarithmic behaviour of streamwise velocity variance. However, additional data are required to determine whether the plateau value of the pre-multiplied streamwise velocity energy spectrum is identical to A 1 .

atmospheric flows↗

Non-stationary precipitation design standards for stormwater infrastructure modernization at USAF installations

The resilience of defense infrastructure systems to a changing climate is critical for national security. Climate induced recurrent flooding is already impacting over 20 U.S. Air Force installations, underscoring the urgency of revisiting precipitation standards and stormwater infrastructure design. Despite growing scientific knowledge and an expanding set of tools for updating outdated precipitation standards based on the assumption of climate stationarity, the adoption of climate informed analyses remain limited in practice. This study utilizes an existing framework to update Intensity (or Depth)-Duration-Frequency (DDF) curves using an ensemble of future climate projections. Change factors in precipitation estimates are derived and applied to six USAF installations across the U.S. The analysis is further extended to evaluate the implications of climate-informed DDFs on stormwater infrastructure performance and flood analysis at Tyndall AFB. Results indicate that the current design precipitation estimates are likely to become obsolete in all six USAF bases by the end of the century. The wide range of change factors across 32 GCM ensembles highlights the need to integrate uncertainty and evolving scientific data into infrastructure planning. The study also finds that the impacts of a changing climate vary spatially and temporally, emphasizing the value of localized analysis for infrastructure decision-making. The work advances ongoing DoD and societal efforts to implement adaptation strategies aimed at enhancing infrastructure resilience.

Intensity-duration-frequency curves↗

Universal method for the optimization of HDC coating uniformity on non-planar, non-stationary substrates for inertial confinement fusion targets

The thickness uniformity of chemical vapor deposited (CVD) diamond coatings on non-planar, non-stationary substrates depends on both the intrinsic instantaneous coating thickness distribution (ICTD) of the coating conditions used and, if applicable, on the frequency of substrate reorientation. While important for many CVD diamond applications, the relative impact of the ICTD and substrate reorientation on the coating thickness uniformity has not been studied. In this work, we systematically investigate the effect of these factors for microwave-plasma chemical vapor deposition (MPCVD) of diamond (referred to as high density carbon (HDC) in the inertial confinement fusion (ICF) community) coatings on spherical, rolling substrates. This coating technique is used to fabricate capsules for ICF experiments, which require extreme coating uniformity with <0.3 % thickness variation (so-called Mode 1 or M1) to ensure symmetric compression of imploding targets. To extract the otherwise unobservable reorientation timescale (Δt), Monte Carlo simulations were performed using experimental ICTD data as input. This combined approach confirms scaling relationships between the substrate reorientation timescale as well as coating thickness and coating uniformity, as expected from a 3D random walk. Simulations confirm that M1 is Rayleigh-distributed and scales as (Δt) 1/2 , consistent with the randomization of two angles that determine orientation of a sphere. We also demonstrate that, under the conditions studied, Δt is the dominant factor in determining thickness uniformity while the intrinsic ICTD has minimal impact. Finally, experiments show that Δt can be affected by total batch size under constant agitation conditions due to space constraints that limit the capsule reorientation kinetics. In conclusion, this study highlights the utility of a combined experiment-simulation approach as a general methodology for understanding and improving coating uniformity on non-planar, non-stationary substrates.

Capsule↗