Search NASA⌕ Search

SEARCH · Search NASA

Results for “Prediction skill”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Quantifying sources of subseasonal prediction skill in CESM2

Abstract Subseasonal prediction fills the gap between weather forecasts and seasonal outlooks. There is evidence that predictability on subseasonal timescales comes from a combination of atmosphere, land, and ocean initial conditions. Predictability from the land is often attributed to slowly varying changes in soil moisture and snowpack, while predictability from the ocean is attributed to sources such as the El Niño Southern Oscillation. Here we use a set of subseasonal reforecast experiments with CESM2 to quantify the respective roles of atmosphere, land, and ocean initial conditions on subseasonal prediction skill over land. These reveal that the majority of prediction skill for global surface temperature in weeks 3–4 comes from the atmosphere, while ocean initial conditions become important after week 4, especially in the Tropics. In the CESM2 subseasonal prediction system, the land initial state does not contribute to surface temperature prediction skill in weeks 3–6 and climatological land conditions lead to higher skill, disagreeing with our current understanding. However, land-atmosphere coupling is important in week 1. Subseasonal precipitation prediction skill also comes primarily from the atmospheric initial condition, except for the Tropics, where after week 4 the ocean state is more important.

Richter, Jadwiga H. (ORCID:0000000170480781)↗
DOE OSTI2022

Quantifying the Effect of Climate Change on Midlatitude Subseasonal Prediction Skill Provided by the Tropics

Abstract Subseasonal timescales (∼2 weeks–2 months) are known for their lack of predictability, however, specific Earth system states known to have a strong influence on these timescales can be harnessed to improve prediction skill (known as “forecasts of opportunity”). As the climate continues warming, it is hypothesized these states may change and consequently, their importance for subseasonal prediction may also be impacted. Here, we examine changes to midlatitude subseasonal prediction skill provided by the tropics under anthropogenic warming using artificial neural networks to quantify skill. The network is tasked to predict the sign of the 500 hPa geopotential height for historical and future time periods in the Community Earth System Model Version 2 ‐ Large Ensemble across the Northern Hemisphere at a 3 week lead using tropical precipitation. We show prediction skill changes substantially in key midlatitude regions and these changes appear linked to changes in seasonal variability with the largest differences in accuracy occurring during forecasts of opportunity.

Mayer, Kirsten J.↗
DOE OSTI2024

Incorrect computation of Madden-Julian oscillation prediction skill

The Madden–Julian oscillation (MJO) is a major tropical weather system and one of the largest sources of predictability for subseasonal-to-seasonal weather forecasts. Skillful prediction of the MJO has been a highly active area of research due to its large socio-economic impacts. Silini et al., herein S21, developed a machine learning model to predict the MJO, which they claimed to have an MJO prediction skill of 26–27 days over all seasons and 45 days for December–February (DJF) winter. If true, this would make the skill of their model competitive with that of the state-of-the-art dynamical MJO prediction systems at 20–35 days. However, here we show that the MJO prediction was calculated incorrectly in S21, which spuriously increased the performance of their model. Correctly computed skill of their model was substantially lower than that reported in S21; the skill for all seasons drops to 11–12 days and the skill for forecasts initialized during DJF drops to 15 days. Our findings clarify that the S21 machine learning model is not competitive with state-of-the-art numerical weather prediction models in predicting the MJO.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Co-Evolving Climate Models under Uncertainty to Improve Predictive Skill

Focal Area: Predictive modeling through the use of AI techniques and AI-derived model components; the use of AI and other tools to design a prediction system comprising of a hierarchy of models Science Challenge: Advances in modeling of climate and improved observational capabilities have led to great improvements in understanding large-scale historical climate effects. It however remains a challenge to reliably predict the risk of fine-scale regional climate events on human systems even as such risks are on the rise because warmer and wetter climates are more prone to hydrometeorological extremes,

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Attribution of NAO Predictive Skill Beyond 2 Weeks in Boreal Winter

Weeks 3–6 averaged winter North Atlantic Oscillation (NAO) predictive skill in a state-of-the-art coupled climate prediction system is attributed to two principle sources: upper and lower boundary conditions linked to the stratosphere and El Niño-Southern Oscillation (ENSO), respectively. A 20-member ensemble of 45-day reforecasts over 1999–2015 is utilized, together with uninitialized simulations with the atmospheric component of the prediction system forced with observed radiative forcing and lower boundary conditions. NAO forecast skill for lead times out to 6 weeks is higher following extreme stratospheric polar vortex conditions (weak and strong vortex events) compared to neutral states. Enhanced weeks 3-6 NAO predictive skill for weak vortex events results primarily from stratospheric downward coupling to the troposphere, while enhanced skill for strong vortex events can be partly attributed to lower boundary forcing related to the ENSO phenomenon. Implications for forecast system development and improvement are discussed.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Machine Learning for Daily Forecasts of Arctic Sea Ice Motion: An Attribution Assessment of Model Predictive Skill

Physics-based simulations of Arctic sea ice are highly complex, involving transport between different phases, length scales, and time scales. Resultantly, numerical simulations of sea ice dynamics have a high computational cost and model uncertainty. We employ data-driven machine learning (ML) to make predictions of sea ice motion. The ML models are built to predict present-day sea ice velocity given present-day wind velocity and previous-day sea ice concentration and velocity. Models are trained using reanalysis winds and satellite-derived sea ice properties. We compare the predictions of three different models: persistence (PS), linear regression (LR), and a convolutional neural network (CNN). We quantify the spatiotemporal variability of the correlation between observations and the statistical model predictions. Additionally, we analyze model performance in comparison to variability in properties related to ice motion (wind velocity, ice velocity, ice concentration, distance from coast, bathymetric depth) to understand the processes related to decreases in model performance. Results indicate that a CNN makes skillful predictions of daily sea ice velocity with a correlation up to 0.81 between predicted and observed sea ice velocity, while the LR and PS implementations exhibit correlations of 0.78 and 0.69, respectively. The correlation varies spatially and seasonally: lower values occur in shallow coastal regions and during times of minimum sea ice extent. LR parameter analysis indicates that wind velocity plays the largest role in predicting sea ice velocity on 1-day time scales, particularly in the central Arctic. Regions where wind velocity has the largest LR parameter are regions where the CNN has higher predictive skill than the LR.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Volcanic forcing degrades multiyear-to-decadal prediction skill in the tropical Pacific

Volcanic aerosol forcing can affect global climate, but its role in climate prediction remains poorly understood. We isolate the impact of volcanic eruptions on multiyear-to-decadal climate prediction skill by comparing two suites of initialized decadal hindcasts conducted with and without historical volcanic forcing. Unexpectedly, the inclusion of volcanic forcing in the prediction system significantly degrades the forecast skill of detrended multiyear-to-decadal sea surface temperature (SST) variability in the central-eastern tropical Pacific. The ensemble mean hindcasts produce multiyear-to-decadal tropical Pacific SST cooling in response to large tropical volcanic eruptions through thermodynamic and El Niño–Southern Oscillation (ENSO)–like dynamic processes. However, in observations, these eruptions coincided with tropical Pacific warming, which is well predicted by the no-volcano hindcasts and, hence, is likely related to internal climate variability. Improved model representation of volcanic response and its interaction with internal climate variability is required to advance prediction of tropical Pacific decadal variability and associated global impacts.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Attribution of North American Subseasonal Precipitation Prediction Skill

The skill of NOAA’s official monthly U.S. precipitation forecasts (issued in the middle of the prior month) has historically been low, having shown modest skill over the southern United States, but little or no skill over large portions of the central United States. The goal of this study is to explain the seasonal and regional variations of the North American subseasonal (weeks 3–6) precipitation skill, specifically the reasons for its successes and its limitations. The performances of multiple recent-generation model reforecasts over 1999–2015 in predicting precipitation are compared to uninitialized simulation skill using the atmospheric component of the forecast systems. This parallel analysis permits attribution of precipitation skill to two distinct sources: one due to slowly evolving ocean surface boundary states and the other to faster time-scale initial atmospheric weather states. A strong regionality and seasonality in precipitation forecast performance is shown to be analogous to skill patterns dictated by boundary forcing constraints alone. The correspondence is found to be especially high for the North American pattern of the maximum monthly skill that is achieved in the reforecast. The boundary forcing of most importance originates from tropical Pacific SST influences, especially those related to El Niño–Southern Oscillation. Furthermore, we discuss physical constraints that may limit monthly precipitation skill and interpret the performance of existing models in the context of plausible upper limits.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Quantifying UAS Observation Error Variance Used in Data Assimilation Systems and Its Impact on Predictive Skill

Observation error determines the weights of the observations and background state used in data assimilation to generate analyses. Quantifying observation error is critical for the optimal assimilation of observational data sets. Uncrewed Aircraft System (UAS) observations have shown potential benefits in filling observational gaps in the lower atmosphere; however, characterization of their error characteristics has been limited. To optimize the use of UAS observations in numerical weather prediction, UAS observation error is estimated based on the 3‐cornered hat diagnostic approach which uses three independent estimates of the atmospheric state. This approach is applied to data from the 2018 Lower Atmospheric Profiling Studies at Elevation‐a Remotely‐piloted Aircraft Team Experiment field campaign using collocated UAS and rawinsonde observations along with output from a set of convection‐permitting model simulations. The estimated observation error values for UAS temperature, wind, and relative humidity measurements were found to be only weakly dependent on height AGL with mean values equal to 0.5°C, 0.8 m s −1 , and 3%, respectively. Only the newly estimated observation error for temperature differed from that previously used to assimilate commercial aircraft observations into global models (1.0°C). However, using this reduced temperature observation error produced more accurate mesoscale analyses and forecasts of both terrain‐driven flows and convection initiation generated by colliding outflow boundaries within the San Luis Valley of Colorado.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

The predictive skill of convolutional neural networks models for disease forecasting

In this paper we investigate the utility of one-dimensional convolutional neural network (CNN) models in epidemiological forecasting. Deep learning models, in particular variants of recurrent neural networks (RNNs) have been studied for ILI (Influenza-Like Illness) forecasting, and have achieved a higher forecasting skill compared to conventional models such as ARIMA. In this study, we adapt two neural networks that employ one-dimensional temporal convolutional layers as a primary building block—temporal convolutional networks and simple neural attentive meta-learners—for epidemiological forecasting. We then test them with influenza data from the US collected over 2010-2019. We find that epidemiological forecasting with CNNs is feasible, and their forecasting skill is comparable to, and at times, superior to, plain RNNs. Thus CNNs and RNNs bring the power of nonlinear transformations to purely data-driven epidemiological models, a capability that heretofore has been limited to more elaborate mechanistic/compartmental disease models.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

Predictive Skill of Deep Learning Models Trained on Limited Sequence Data

In this report we investigate the utility of one-dimensional convolutional neural network (CNN) models in epidemiological forecasting. Deep learning models, especially variants of recurrent neural networks (RNNs) have been studied for influenza forecasting, and have achieved higher forecasting skill compared to conventional models such as ARIMA models. In this study, we adapt two neural networks that employ one-dimensional temporal convolutional layers as a primary building block temporal convolutional networks and simple neural attentive meta-learner for epidemiological forecasting and test them with influenza data from the US collected over 2010-2019. We find that epidemiological forecasting with CNNs is feasible, and their forecasting skill is comparable to, and at times, superior to, RNNs. Thus CNNs and RNNs bring the power of nonlinear transformations to purely data-driven epidemiological models, a capability that heretofore has been limited to more elaborate mechanistic/compartmental disease models.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2020

Skillful multiyear predictions of ocean acidification in the California Current System

The California Current System (CCS) sustains economically valuable fisheries and is particularly vulnerable to ocean acidification, due to its natural upwelling of carbon-enriched waters that generate corrosive conditions for local ecosystems. Here we use a novel suite of retrospective, initialized ensemble forecasts with an Earth system model (ESM) to predict the evolution of surface pH anomalies in the CCS. We show that the forecast system skillfully predicts observed surface pH variations a year in advance over a naive forecasting method, with the potential for skillful prediction up to five years in advance. Skillful predictions of surface pH are mainly derived from the initialization of dissolved inorganic carbon anomalies that are subsequently transported into the CCS. Our results demonstrate the potential for ESMs to provide skillful predictions of ocean acidification on large scales in the CCS. Initialized ESMs could also provide boundary conditions to improve high-resolution regional forecasting systems.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Improved Predictability of the Indian Ocean Dipole Using a Stochastic Dynamical Model Compared to the North American Multimodel Ensemble Forecast

This study assesses the predictive skill of eight North American Multimodel Ensemble (NMME) models in predicting the Indian Ocean dipole (IOD). We find that the forecasted ensemble-mean IOD–El Niño–Southern Oscillation (ENSO) relationship deteriorates away from the observed relationship with increasing lead time, which might be one reason that limits the IOD predictive skill in coupled models. We are able to improve the IOD predictive skill using a recently developed stochastic dynamical model (SDM) forced by forecasted ENSO conditions. The results are consistent with the previous result that operational IOD predictability beyond persistence at lead times beyond one season is mostly controlled by ENSO predictability and the signal-to-noise ratio of the Indo-Pacific climate system. The multimodel ensemble (MME) investigated here is found to be of superior skill compared to each individual model at most lead times. Importantly, the skill of the SDM IOD predictions forced with forecasted ENSO conditions were either similar or better than those of the MME IOD forecasts. Moreover, the SDM forced with observed ENSO conditions exhibits significantly higher IOD prediction skill than the MME at longer lead times, suggesting the large potential skill increase that could be achieved by improving operational ENSO forecasts. We find that both cold and warm biases of the predicted Niño-3.4 index may cause false alarms of negative and positive IOD events, respectively, in NMME models. Many false alarms for IOD forecasts at lead times longer than one season in the original forecasts disappear or are significantly reduced in the SDM forced by forecasted ENSO conditions.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Evaluating skill in predicting the Interdecadal Pacific Oscillation in initialized decadal climate prediction hindcasts in E3SMv1 and CESM1 using two different initialization methods and a small set of start years

Abstract It is a daunting challenge to conduct initialized hindcasts with enough ensemble members and associated start years to form a drifted climatology from which to compute the anomalies necessary to quantify the skill of the hindcasts when compared to observations. This limits the ability to experiment with case studies and other applications where only a few initial years are needed. Here we run a set of hindcasts with CESM1 and E3SMv1 using two different initialization methods for a limited set of start years and use the respective uninitialized free-running historical simulations to form the model climatologies. Since the drifts from the observed initial states in the hindcasts toward the uninitialized model state are large and rapid, after a few years the drifted initialized models approach the uninitialized model climatological errors. Therefore, hindcasts from the limited start years can use the uninitialized climatology to represent the drifted model states after about lead year 3, providing a means to compute forecast anomalies in the absence of a large hindcast sample. There is comparable skill for predicting spatial patterns of multi-year Pacific sea surface temperature anomalies in the domain of the Interdecadal Pacific Oscillation using this method compared to the conventional methodology with a large hindcast data set, though there is a model dependence to the drifts in the two initialization methods.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

CMIP6 skill at predicting interannual to multi-decadal summer monsoon precipitation variability

Monsoons affect the economy, agriculture, and human health of two thirds of the world’s population. Therefore, predicting variations in monsoon precipitation is societally important. We explore the ability of climate models from the sixth phase of the Climate Model Intercomparison Project to predict summer monsoon precipitation variability by using hindcasts from the Decadal Climate Prediction Project (Component A). The multi-model ensemble-mean shows significant skill at predicting summer monsoon precipitation from one year to 6–9 years ahead. However, this skill is dependent on the model, monsoon domain, and lead-time. In general, the skill of the multi-model ensemble-mean prediction is low in year 1 but increases for longer-lead times and is largely consistent with externally forced changes. The best captured region is northern Africa for the 2–5 and 6–9 year forecast lead times. In contrast, there is no significant skill using the ensemble-mean over East and South Asia and, furthermore, there is significant spread in skill among models for these domains. By sub-sampling the ensemble we show that the difference in skill between models is tied to the simulation of the externally forced response over East and South Asia, with models with a more skilful forced response capable of better predictions. A further contribution is from skilful prediction of Pacific Ocean temperatures for the South Asian summer monsoon at longer lead-times. Therefore, these results indicate that predictions of the East and South Asian monsoons could be significantly improved.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Subseasonal Earth System Prediction with CESM2

Abstract Prediction systems to enable Earth system predictability research on the subseasonal time scale have been developed with the Community Earth System Model, version 2 (CESM2) using two configurations that differ in their atmospheric components. One system uses the Community Atmosphere Model, version 6 (CAM6) with its top near 40 km, referred to as CESM2(CAM6). The other employs the Whole Atmosphere Community Climate Model, version 6 (WACCM6) whose top extends to ∼140 km, and it includes fully interactive tropospheric and stratospheric chemistry [CESM2(WACCM6)]. Both systems are utilized to carry out subseasonal reforecasts for the 1999–2020 period following the Subseasonal Experiment’s (SubX) protocol. Subseasonal prediction skill from both systems is compared to those of the National Oceanic and Atmospheric Administration CFSv2 and European Centre for Medium-Range Weather Forecasts (ECMWF) operational models. CESM2(CAM6) and CESM2(WACCM6) show very similar subseasonal prediction skill of 2-m temperature, precipitation, the Madden–Julian oscillation, and North Atlantic Oscillation to its previous version and to the NOAA CFSv2 model. Overall, skill of CESM2(CAM6) and CESM2(WACCM6) is a little lower than that of the ECMWF system. In addition to typical output provided by subseasonal prediction systems, CESM2 reforecasts provide comprehensive datasets for predictability research of multiple Earth system components, including three-dimensional output for many variables, and output specific to the mesosphere and lower-thermosphere (MLT) region from CESM2(WACCM6). It is shown that sudden stratosphere warming events, and the associated variability in the MLT, can be predicted ∼10 days in advance. Weekly real-time forecasts and reforecasts with CESM2(CAM6) and CESM2(WACCM6) are freely available. Significance Statement We describe here the design and prediction skill of two subseasonal prediction systems based on two configurations of the Community Earth System Model, version 2 (CESM2): CESM2 with the Community Atmosphere Model, version 6 [CESM2(CAM6)] and CESM 2 with Whole Atmosphere Community Climate Model, version 6 [CESM2(WACCM6)] as its atmospheric component. These two systems provide a foundation for community-model based subseasonal prediction research. The CESM2(WACCM6) system provides a novel capability to explore the predictability of the stratosphere, mesosphere, and lower thermosphere. Both CESM2(CAM6) and CESM2(WACCM6) demonstrate subseasonal surface prediction skill comparable to that of the NOAA CFSv2 model, and a little lower than that of the ECMWF forecasting system. CESM2 reforecasts provide a comprehensive dataset for predictability research of multiple aspects of the Earth system, including the whole atmosphere up to 140 km, land, and sea ice. Weekly real-time forecasts, reforecasts, and models are publicly available.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

A Bayesian Deep Learning Approach to Near-Term Climate Prediction

Since model bias and associated initialization shock are serious shortcomings that reduce prediction skills in state-of-the-art decadal climate prediction efforts, we pursue a complementary machine-learning-based approach to climate prediction. The example problem setting we consider consists of predicting natural variability of the North Atlantic sea surface temperature on the interannual timescale in the pre-industrial control simulation of the Community Earth System Model. While previous works have considered the use of recurrent networks such as convolutional LSTMs and reservoir computing networks in this and other similar problem settings, we currently focus on the use of feedforward convolutional networks. In particular, we find that a feedforward convolutional network with a Densenet architecture is able to outperform a convolutional LSTM in terms of predictive skill. Next, we go on to consider a probabilistic formulation of the same network based on Stein variational gradient descent and find that in addition to providing useful measures of predictive uncertainty, the probabilistic (Bayesian) version improves on its deterministic counterpart in terms of predictive skill. Finally, we characterize the reliability of the ensemble of machine learning models obtained in the probabilistic setting by using analysis tools developed in the context of ensemble numerical weather prediction.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Predictability of the early summer surface air temperature over Western South Asia

Variability of the Surface Air Temperature (SAT) over the Western South Asia (WSA) region leads to frequent heatwaves during the early summer (May-June) season. The present study uses the European Centre for Medium-Range Weather Forecast’s fifth-generation seasonal prediction system, SEAS5, from 1981 to 2022 based on April initial conditions (1-month lead) to assess the SAT predictability during early summer season. The goal is to evaluate the SEAS5’s ability to predict the El Niño-Southern Oscillation (ENSO) related interannual variability and predictability of the SAT over WSA, which is mediated through upper-level (200-hPa) geopotential height anomalies. This teleconnection leads to anomalously warm surface conditions over the region during the negative ENSO phase, as observed in the reanalysis and SEAS5. We evaluate SEAS5 prediction skill against two observations and three reanalyses datasets. The SEAS5 SAT prediction skill is higher with high spatial resolution observations and reanalysis datasets compared to the ones with low-resolution. Overall, SEAS5 shows reasonable skill in predicting SAT and its variability over the WSA region. Moreover, the predictability of SAT during La Niña is comparable to El Niño years over the WSA region.

54 ENVIRONMENTAL SCIENCES↗