Search NASA⌕ Search

SEARCH · Search NASA

Results for “forecast skill”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Attribution of NAO Predictive Skill Beyond 2 Weeks in Boreal Winter

Weeks 3–6 averaged winter North Atlantic Oscillation (NAO) predictive skill in a state-of-the-art coupled climate prediction system is attributed to two principle sources: upper and lower boundary conditions linked to the stratosphere and El Niño-Southern Oscillation (ENSO), respectively. A 20-member ensemble of 45-day reforecasts over 1999–2015 is utilized, together with uninitialized simulations with the atmospheric component of the prediction system forced with observed radiative forcing and lower boundary conditions. NAO forecast skill for lead times out to 6 weeks is higher following extreme stratospheric polar vortex conditions (weak and strong vortex events) compared to neutral states. Enhanced weeks 3-6 NAO predictive skill for weak vortex events results primarily from stratospheric downward coupling to the troposphere, while enhanced skill for strong vortex events can be partly attributed to lower boundary forcing related to the ENSO phenomenon. Implications for forecast system development and improvement are discussed.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Optimizing Earthquake Nowcasting With Machine Learning: The Role of Strain Hardening in the Earthquake Cycle

Abstract Nowcasting is a term originating from economics, finance, and meteorology. It refers to the process of determining the uncertain state of the economy, markets or the weather at the current time by indirect means. In this paper, we describe a simple two‐parameter data analysis that reveals hidden order in otherwise seemingly chaotic earthquake seismicity. One of these parameters relates to a mechanism of seismic quiescence arising from the physics of strain‐hardening of the crust prior to major events. We observe an earthquake cycle associated with major earthquakes in California, similar to what has long been postulated. An estimate of the earthquake hazard revealed by this state variable time series can be optimized by the use of machine learning in the form of the Receiver Operating Characteristic skill score. The ROC skill is used here as a loss function in a supervised learning mode. Our analysis is conducted in the region of 5° × 5° in latitude‐longitude centered on Los Angeles, a region which we used in previous papers to build similar time series using more involved methods (Rundle & Donnellan, 2020, https://doi.org/10.1029/2020EA001097 ; Rundle, Donnellan et al., 2021, https://doi.org/10.1029/2021EA001757 ; Rundle, Stein et al., 2021, https://doi.org/10.1088/1361-6633/abf893 ). Here we show that not only does the state variable time series have forecast skill, the associated spatial probability densities have skill as well. In addition, use of the standard ROC and Precision (PPV) metrics allow probabilities of current earthquake hazard to be defined in a simple, straightforward, and rigorous way.

58 GEOSCIENCES↗
DOE OSTI2022

Convolutional neural networks for intra-hour solar forecasting based on sky image sequences

Accurate and timely solar forecasts play an increasingly critical role in power systems. Compared to longer forecasting timescales, very short-term solar forecasting has lagged behind in both research and practice. In this paper, we propose deep convolutional neural networks (CNNs) to provide operational intra-hour (10-minute-ahead to 60-minute-ahead) solar forecasts. We develop two CNN structures inspired by a widely-used CNN architecture. The CNNs are tailored to our solar forecasting regression tasks and rely solely on sky image sequences. Case studies based on six years of data (over 150,000 data points) demonstrate that the best CNN model has forecast skill scores of 20%-39% over the naive persistence of cloudiness benchmark, even at these very short timescales. The CNNs also have consistently superior performance when compared to shallow machine learning models with meteorological predictors, where the improvement averages around 7%. The sensitivity analyses show that the sky image length, resolution, and weather conditions have impacts on the deep learning model accuracy. In our intra-hour problem with specific setups, two sky images with a 10-minute 128 x 128 resolution yield the most accurate forecasts. Current limitations, future work, and deployment challenges and solutions are also discussed.

14 SOLAR ENERGY↗
DOE OSTI2022

Towards physics-inspired data-driven weather forecasting: integrating data assimilation with a deep spatial-transformer-based U-NET in a case study with ERA5

Abstract. There is growing interest in data-driven weather prediction (DDWP), e.g., using convolutional neural networks such as U-NET that are trained on data from models or reanalysis. Here, we propose three components, inspired by physics, to integrate with commonly used DDWP models in order to improve their forecast accuracy. These components are (1) a deep spatial transformer added to the latent space of U-NET to capture rotation and scaling transformation in the latent space for spatiotemporal data, (2) a data-assimilation (DA) algorithm to ingest noisy observations and improve the initial conditions for next forecasts, and (3) a multi-time-step algorithm, which combines forecasts from DDWP models with different time steps through DA, improving the accuracy of forecasts at short intervals. To show the benefit and feasibility of each component, we use geopotential height at 500 hPa (Z500) from ERA5 reanalysis and examine the short-term forecast accuracy of specific setups of the DDWP framework. Results show that the spatial-transformer-based U-NET (U-STN) clearly outperforms the U-NET, e.g., improving the forecast skill by 45 %. Using a sigma-point ensemble Kalman (SPEnKF) algorithm for DA and U-STN as the forward model, we show that stable, accurate DA cycles are achieved even with high observation noise. This DDWP+DA framework substantially benefits from large (O(1000)) ensembles that are inexpensively generated with the data-driven forward model in each DA cycle. The multi-time-step DDWP+DA framework also shows promise; for example, it reduces the average error by factors of 2–3. These results show the benefits and feasibility of these three components, which are flexible and can be used in a variety of DDWP setups. Furthermore, while here we focus on weather forecasting, the three components can be readily adopted for other parts of the Earth system, such as ocean and land, for which there is a rapid growth of data and need for forecast and assimilation.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Supercell Low-Level Mesocyclones: Origins of Inflow and Vorticity

Abstract The development and intensification of low-level mesocyclones in supercell thunderstorms have often been attributed, at least in part, to augmented streamwise vorticity generated baroclinically in the forward flank of supercells. However, the ambient streamwise vorticity of the environment (often quantified via storm-relative helicity), especially near the ground, is particularly skillful at discriminating between nontornadic and tornadic supercells. This study investigates whether the origins of the inflow air into supercell low-level mesocyclones, both horizontally and vertically, can help explain the dynamical role of environmental versus storm-generated vorticity in the development of low-level mesocyclone rotation. Simulations of supercells, initialized with wind profiles common to supercell environments observed in nature, show that the air bound for the low-level mesocyclone primarily originates from the ambient environment (rather than from along the forward flank) and from very close to the ground, often in the lowest 200–400 m of the atmosphere. Given that the near-ground environmental air comprises the bulk of the inflow into low-level mesocyclones, this likely explains the forecast skill of environmental streamwise vorticity in the lowest few hundred meters of the atmosphere. The low-level mesocyclone does not appear to require much augmentation from the development of additional horizontal vorticity in the forward flank. Instead, the dominant contributor to vertical vorticity within the low-level mesocyclone is from the environmental horizontal vorticity. This study provides further context to the ongoing discussion regarding the development of rotation within supercell low-level mesocyclones. Significance Statement Supercell thunderstorms produce the majority of tornadoes, and a defining characteristic of supercells is their rotating updraft, known as the “mesocyclone.” When the mesocyclone is stronger at lower altitudes, the likelihood of tornadoes increases. The purpose of this study is to understand if the rotation of the mesocyclone in supercells is due to horizontal spin present in the ambient environment or whether additional horizontal spin generated by the storm itself primarily drives this rotation. Our results suggest that inflow air into supercells and low-level mesocyclone rotation are mainly due to the properties of the environmental inflow air, especially near the ground. This hopefully provides further context to how our community views the development of low-level mesocyclones in supercells.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

The Largest Ever Recorded Heatwave—Characteristics and Attribution of the Antarctic Heatwave of March 2022

An unprecedented heatwave impacted East Antarctica in March 2022, peaking at 39°C above climatology, the largest temperature anomaly ever recorded globally. We investigate the causes of the heatwave, the impact of climate change, and a climate model's ability in simulating such an event. The heatwave, which was skillfully forecast, resulted from a highly anomalous large-scale circulation pattern that advected an Australian airmass to East Antarctica in 4 days and produced record atmospheric heat fluxes. Southern Ocean sea surface temperatures anomalies had a minimal impact on the heatwave's amplitude. Simulations from a climate model fail to simulate such a large temperature anomaly mostly due to biases in its large-scale circulation variability, showcasing a pathway for future model improvement in simulating extreme heatwaves. The heatwave was made 2°C warmer by climate change, and end of 21st century heatwaves may be an additional 5–6°C warmer, raising the prospect of near-melting temperatures over the interior of East Antarctica.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Prediction of the SYM-H Index Using a Bayesian Deep Learning Method With Uncertainty Quantification

We propose a novel deep learning framework, named SYMHnet, which employs a graph neural network and a bidirectional long short-term memory network to cooperatively learn patterns from solar wind and interplanetary magnetic field parameters for short-term forecasts of the SYM-H index based on 1- and 5-min resolution data. SYMHnet takes, as input, the time series of the parameters' values provided by NASA's Space Science Data Coordinated Archive and predicts, as output, the SYM-H index value at time point t + w hours for a given time point t where w is 1 or 2. By incorporating Bayesian inference into the learning framework, SYMHnet can quantify both aleatoric (data) uncertainty and epistemic (model) uncertainty when predicting future SYM-H indices. Experimental results show that SYMHnet works well at quiet time and storm time, for both 1- and 5-min resolution data. The results also show that SYMHnet generally performs better than related machine learning methods. For example, SYMHnet achieves a forecast skill score (FSS) of 0.343 compared to the FSS of 0.074 of a recent gradient boosting machine (GBM) method when predicting SYM-H indices (1 hr in advance) in a large storm (SYM-H = -393 nT) using 5-min resolution data. When predicting the SYM-H indices (2 hr in advance) in the large storm, SYMHnet achieves an FSS of 0.553 compared to the FSS of 0.087 of the GBM method. In addition, SYMHnet can provide results for both data and model uncertainty quantification, whereas the related methods cannot.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

Large‐Scale Statistically Meaningful Patterns (LSMPs) Associated With Precipitation Extremes Over Northern California

Abstract We analyze large‐scale statistically meaningful patterns (LSMPs) that precede extreme precipitation (PEx) events over Northern California (NorCal). We find LSMPs by applying k‐means clustering to the two leading principal components of daily 500 hPa geopotential height anomalies two days before the onset, from October to March during 1948–2015. Statistical significance testing based on Monte Carlo simulations suggests a minimum of four statistically distinguished LSMP clusters. The four LSMP clusters are characterized as Northwest continental negative height anomaly, Eastward positive “Pacific‐North American Pattern (PNA),” Westward negative “PNA,” and Prominent Alaskan ridge. These four clusters, shown in multiple variables, evolve very differently and have differing links to the Arctic and tropical Pacific regions. Using binary forecast skill measures and a new copula‐based framework for predicting PEx events, we find LSMP indices that are useful predictors of NorCal PEx events, with moisture‐based variables being the best predictors of PEx events at least 6 days before the onset, and the lower atmospheric variables being better than their upper atmospheric counterparts any day in advance tested. To ensure statistical rigor, the LSMPs analyzed here (with the modified acronym) include local tests of both significance and consistency, which are not always featured in the literature on large‐scale meteorological patterns.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2026

Observed Land Surface Influence on Atmospheric Heat and Moisture Profiles During Interstorms

Land-atmospheric (L-A) feedbacks have historically been studied using models whose structure and parameterizations influence outcomes and insights. The representation of L-A feedbacks based on observations alone remains an ongoing challenge for understanding boundary layer development and precipitation. To address this gap, we use ground-based passive remote sensing and in-situ observations to present an analysis of the atmosphere during 103 interstorm soil moisture drydown events spanning nine warm seasons (2016–2024) in the U.S. Southern Great Plains region. By separating events based on local L-A coupling signals and characterizing the profiles of atmospheric heat and moisture to surface energy flux behavior, we investigate the physical mechanisms linking land surface processes to boundary layer development. We find that during interstorm drydowns, the atmospheric column follows a consistent pattern: moisture increases within the boundary layer, peaks near its top, and declines rapidly above, while warming occurs through the depth. Drydowns that shift toward evaporation produce stronger and deeper thermodynamic responses than cases dominated by sensible heating, which are weaker and shallower. Additionally, moisture is accumulated faster within the boundary layer during shorter drydowns, with longer drydowns representing slower, moisture-limited growth. Drydowns with wetter initial soil moisture will sustain stronger moistening within and above the boundary layer, accelerating buoyancy growth and convective potential toward the next storm. These results provide observational evidence linking surface flux evolution to boundary layer thermodynamics and offer a process-level benchmark for evaluating coupled L-A representations in models and demonstrating the influence of soil moisture on short-term weather forecasting skill.

Zhang, M. S. [Massachusetts Inst. of Technology (M↗
DOE OSTI2023

Memory-based parameterization with differentiable solver: Application to Lorenz ’96

Physical parameterizations (or closures) are used as representations of unresolved subgrid processes within weather and global climate models or coarse-scale turbulent models, whose resolutions are too coarse to resolve small-scale processes. These parameterizations are typically grounded on physically based, yet empirical, representations of the underlying small-scale processes. Machine learning-based parameterizations have recently been proposed as an alternative solution and have shown great promise to reduce uncertainties associated with the parameterization of small-scale processes. Yet, those approaches still show some important mismatches that are often attributed to the stochasticity of the considered process. This stochasticity can be due to coarse temporal resolution, unresolved variables, or simply to the inherent chaotic nature of the process. To address these issues, we propose a new type of parameterization (closure), which is built using memory-based neural networks, to account for the non-instantaneous response of the closure and to enhance its stability and prediction accuracy. We apply the proposed memory-based parameterization, with differentiable solver, to the Lorenz ’96 model in the presence of a coarse temporal resolution and show its capacity to predict skillful forecasts over a long time horizon of the resolved variables compared to instantaneous parameterizations. This approach paves the way for the use of memory-based parameterizations for closure problems.

Mathematics↗
DOE OSTI2023

Data assimilation in operator algebras

We develop an algebraic framework for sequential data assimilation of partially observed dynamical systems. In this framework, Bayesian data assimilation is embedded in a nonabelian operator algebra, which provides a representation of observables by multiplication operators and probability densities by density operators (quantum states). In the algebraic approach, the forecast step of data assimilation is represented by a quantum operation induced by the Koopman operator of the dynamical system. Moreover, the analysis step is described by a quantum effect, which generalizes the Bayesian observational update rule. Projecting this formulation to finite-dimensional matrix algebras leads to computational schemes that are i) automatically positivity-preserving and ii) amenable to consistent data-driven approximation using kernel methods for machine learning. Moreover, these methods are natural candidates for implementation on quantum computers. Applications to the Lorenz 96 multiscale system and the El Niño Southern Oscillation in a climate model show promising results in terms of forecast skill and uncertainty quantification.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Machine learning methods for weather forecasting

SAND2025-14466O This repository contains code for developing, training, and evaluating machine learning models for weather and climate forecasting, including forecast skill assessment, feature importance analysis, and reproducible workflows for model comparison. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Holthuijzen, Maike [Sandia National Lab. (SNL-CA),↗
DOE OSTI2022

The Impact of Constrained Data Assimilation on the Forecasts of Three Convection Systems During the ARM MC3E Field Campaign

A constrained data assimilation (CDA) system based on the ensemble variational (EnVar) method and physical constraints of mass and water conservations is evaluated through three convective cases during the Midlatitude Continental Convective Clouds Experiment (MC3E) of the Atmospheric Radiation Measurement (ARM) program. Compared to the original data assimilation (ODA), the CDA is shown to perform better in the forecasted state variables and simulated precipitation. The CDA is also shown to greatly mitigate the loss of forecast skills in observation denial experiments when radar radial winds are withheld in the assimilation. Modifications to the algorithm and sensitivities of the CDA to the calculation of the time tendencies in the constraints are described.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Probabilistic Machine Learning and Data Assimilation

This white paper responds to Focal Area 1. The associated portfolio of research activities is well-suited to DOE’s asset mix of HPC platforms, climate expertise, climate simulation codes, and AI expertise, which creates an opportunity to use manifold-finding probabilistic AI methods to create more powerful data assimilation techniques that increase the fidelity and forecasting skill of Earth System Prediction.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Sensitivity of Near-Surface Variables in the RUC Land Surface Model in the Weather Research and Forecasting Model

In this study, we investigate the parametric sensitivity of near-surface variables, such as sensible heat flux, latent heat flux, ground heat flux, hub-height wind speed and land surface temperature, to the parameters used in the Rapid Update Cycle (RUC) land surface model (LSM) during a wintertime and summertime period. The model simulations are compared with observations collected from the second Wind Forecast Improvement Project (WFIP2) field campaign. The results suggest that parameters related to snow/ice and thermal processes can have significant impact on the simulated near-surface variables. Out of the 11 examined parameters, only 6 of them have considerable influences on the model behaviors and explain about 60 ~ 80 % of the estimated total variance of the simulated variables. In addition, the magnitude of the parametric sensitivity varies with season. For instance, parameters associated with snow/ice processes are dominant during the wintertime whereas those associated with thermal processes are more important during the summertime. Furthermore, the impact of the identified parameters on the simulated variables is highly related to the topography. There is a high degree of sensitivity to the parameter values over the slope region. This points out the importance of collecting field observations over steep areas to better quantity the appropriate values of key parameters. Overall, our findings provide a better understanding of the RUC LSM behavior associated with parameter uncertainties and can be used to improve the forecasting skill of land surface processes via calibration of the most uncertain model parameters.

17 WIND ENERGY↗
DOE OSTI2021

MicroPulse DIAL (MPD) – a diode-laser-based lidar architecture for quantitative atmospheric profiling

Abstract. Continuous water vapor and temperature profiles are critically needed for improved understanding of the lower atmosphere and potential advances in weather forecasting skill. Ground-based, national-scale profiling networks are part of a suite of instruments to provide such observations; however, the technological method must be cost-effective and quantitative. We have been developing an active remote sensing technology based on a diode-laser-based lidar technology to address this observational need. Narrowband, high-spectral-fidelity diode lasers enable accurate and calibration-free measurements requiring a minimal set of assumptions based on direct absorption (Beer–Lambert law) and a ratio of two signals. These well-proven quantitative methods are known as differential absorption lidar (DIAL) and high-spectral-resolution lidar (HSRL). This diode-laser-based architecture, characterized by less powerful laser transmitters than those historically used for atmospheric studies, can be made eye-safe and robust. Nevertheless, it also requires solar background suppression techniques such as narrow-field-of-view receivers with an ultra-narrow bandpass to observe individual photons backscattered from the atmosphere. We discuss this diode-laser-based lidar architecture's latest generation and analyze how it addresses a national-scale profiling network's need to provide continuous thermodynamic observations. The work presented focuses on general architecture changes that pertain to both the water vapor and the temperature profiling capabilities of the MicroPulse DIAL (MPD). However, the specific subcomponent testing and instrument validation presented are for the water vapor measurements only. A fiber-coupled seed laser transmitter optimization is performed and shown to meet all of the requirements for the DIAL technique. Further improvements – such as a fiber-coupled near-range receiver, the ability to perform quality control via automatic receiver scanning, advanced multi-channel scalar capabilities, and advanced processing techniques – are discussed. These new developments increase narrowband DIAL technology readiness and are shown to allow higher-quality water vapor measurements closer to the surface via preliminary intercomparisons within the MPD network itself and with radiosondes.

Spuler, Scott M.↗
DOE OSTI2025

Evaluation of the 2022 West Nile virus forecasting challenge, USA

Abstract Background West Nile virus (WNV) is the most common cause of mosquito-borne disease in the continental USA, with an average of ~1200 severe, neuroinvasive cases reported annually from 2005 to 2021 (range 386–2873). Despite this burden, efforts to forecast WNV disease to inform public health measures to reduce disease incidence have had limited success. Here, we analyze forecasts submitted to the 2022 WNV Forecasting Challenge, a follow-up to the 2020 WNV Forecasting Challenge. Methods Forecasting teams submitted probabilistic forecasts of annual West Nile virus neuroinvasive disease (WNND) cases for each county in the continental USA for the 2022 WNV season. We assessed the skill of team-specific forecasts, baseline forecasts, and an ensemble created from team-specific forecasts. We then characterized the impact of model characteristics and county-specific contextual factors (e.g., population) on forecast skill. Results Ensemble forecasts for 2022 anticipated a season at or below median long-term WNND incidence for nearly all (> 99%) counties. More counties reported higher case numbers than anticipated by the ensemble forecast median, but national caseload (826) was well below the 10-year median (1386). Forecast skill was highest for the ensemble forecast, though the historical negative binomial baseline model and several team-submitted forecasts had similar forecast skill. Forecasts utilizing regression-based frameworks tended to have more skill than those that did not and models using climate, mosquito surveillance, demographic, or avian data had less skill than those that did not, potentially due to overfitting. County-contextual analysis showed strong relationships with the number of years that WNND had been reported and permutation entropy (historical variability). Evaluations based on weighted interval score and logarithmic scoring metrics produced similar results. Conclusions The relative success of the ensemble forecast, the best forecast for 2022, suggests potential gains in community ability to forecast WNV, an improvement from the 2020 Challenge. Similar to the previous challenge, however, our results indicate that skill was still limited with general underprediction despite a relative low incidence year. Potential opportunities for improvement include refining mechanistic approaches, integrating additional data sources, and considering different approaches for areas with and without previous cases. Graphical Abstract

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

The use of soil moisture and Standardized Evaporative Stress Ratio (SESR) anomalies for increased lead time of the development flash drought and heat waves

Drought and rapid intensification of drought (i.e., flash drought) can have significant impacts on the natural environment and the economy, particularly when coupled with a period(s) of extreme heat (Otkin et al., 2018). The development and/or intensification of drought falls squarely in the sub-seasonal to seasonal (S2S) time frame, which dictates that drought development, intensification, and propagation could be an output of a subseasonal or seasonal forecast. Current S2S forecasts have poor skill for forecasting the development of extreme drought and heat waves.

54 ENVIRONMENTAL SCIENCES↗