Search NASA⌕ Search

SEARCH · Search NASA

Results for “Prediction skill”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Quantifying sources of subseasonal prediction skill in CESM2

Abstract Subseasonal prediction fills the gap between weather forecasts and seasonal outlooks. There is evidence that predictability on subseasonal timescales comes from a combination of atmosphere, land, and ocean initial conditions. Predictability from the land is often attributed to slowly varying changes in soil moisture and snowpack, while predictability from the ocean is attributed to sources such as the El Niño Southern Oscillation. Here we use a set of subseasonal reforecast experiments with CESM2 to quantify the respective roles of atmosphere, land, and ocean initial conditions on subseasonal prediction skill over land. These reveal that the majority of prediction skill for global surface temperature in weeks 3–4 comes from the atmosphere, while ocean initial conditions become important after week 4, especially in the Tropics. In the CESM2 subseasonal prediction system, the land initial state does not contribute to surface temperature prediction skill in weeks 3–6 and climatological land conditions lead to higher skill, disagreeing with our current understanding. However, land-atmosphere coupling is important in week 1. Subseasonal precipitation prediction skill also comes primarily from the atmospheric initial condition, except for the Tropics, where after week 4 the ocean state is more important.

Richter, Jadwiga H. (ORCID:0000000170480781)↗
NASA NTRS2020

Prediction skill of the 2012 U.S. Great Plains flash drought in Subseasonal Experiment (SubX) models

Rapid-onset droughts, known as flash droughts, can have devastating impacts on agriculture, water resources, and ecosystems. The ability to predict flash droughts in advance would greatly enhance our preparation for them and potentially mitigate their impacts. Here, we investigate the prediction skill of the extreme 2012 flash drought over the U.S. Great Plains at subseasonal lead times (3 weeks or more in advance) in global forecast systems participating in the Subseasonal Experiment (SubX). An additional comprehensive set of subseasonal hindcasts with NASA’s GEOS model, a SubX model with relatively high prediction skill, was performed to investigate the separate contributions of atmospheric and land initial conditions to flash drought prediction skill. The results show that the prediction skill of the SubX models is quite variable. While skillful predictions are restricted to within the first two forecast weeks in most models, skill is considerably better (3-4 weeks or more) for certain models and initialization dates. The enhanced prediction skill is found to originate from two robust sources: 1) accurate soil moisture initialization once dry soil conditions are established, and 2) the satisfactory representation of quasi-stationary cross-Pacific Rossby wave trains that lead to the rapid intensification of flash droughts. Evidence is provided that the importance of soil moisture initialization applies more generally to central U.S. summer flash droughts. Our results corroborate earlier findings that accurate soil moisture initialization is important for skillful subseasonal forecasts and highlight the need for additional research on the sources and predictability of drought-inducing quasi-stationary atmospheric circulation anomalies.

Anthony M DeAngelis↗
NASA NTRS2020

Prediction Skill of the 2012 U.S. Great Plains Flash Drought in Subseasonal Experiment (SubX) Models

Flash droughts refer to droughts that develop much more rapidly than normal (i.e., on the order of weeks to a few months). Such droughts can have devastating impacts on agriculture, water resources, and ecosystems. The ability to predict flash droughts in advance would greatly enhance our preparation for them and potentially mitigate their impacts. We investigated the prediction skill of U.S. flash droughts at subseasonal lead times in global forecast systems participating in the Subseasonal Experiment (SubX) project. An additional comprehensive set of hindcasts with NASA?s GEOSv2.1, a model with relatively high prediction skill, was performed to investigate the separate contributions of atmospheric and land initial conditions to flash drought prediction skill. Here we focus on results for the 2012 Great Plains flash drought, noting that the findings based on this event are generally applicable to other U.S. flash droughts. The prediction skill of the SubX models is quite variable. While the skill is limited to less than 2 weeks in most models, it is considerably higher (3-4 weeks or more) for certain models and initialization dates. The enhanced prediction skill is found to originate from two robust sources: 1) accurate soil moisture initialization, and 2) the satisfactory representation of quasi-stationary cross-North Pacific Rossby wave trains that lead to the rapid intensification of flash droughts. Our results corroborate earlier findings that accurate soil moisture initialization is important for skillful subseasonal forecasts and highlight the need for additional research on the sources and predictability of drought-inducing quasi-stationary Rossby waves.

DeAngelis, Anthony M.↗
DOE OSTI2022

Quantifying the Effect of Climate Change on Midlatitude Subseasonal Prediction Skill Provided by the Tropics

Abstract Subseasonal timescales (∼2 weeks–2 months) are known for their lack of predictability, however, specific Earth system states known to have a strong influence on these timescales can be harnessed to improve prediction skill (known as “forecasts of opportunity”). As the climate continues warming, it is hypothesized these states may change and consequently, their importance for subseasonal prediction may also be impacted. Here, we examine changes to midlatitude subseasonal prediction skill provided by the tropics under anthropogenic warming using artificial neural networks to quantify skill. The network is tasked to predict the sign of the 500 hPa geopotential height for historical and future time periods in the Community Earth System Model Version 2 ‐ Large Ensemble across the Northern Hemisphere at a 3 week lead using tropical precipitation. We show prediction skill changes substantially in key midlatitude regions and these changes appear linked to changes in seasonal variability with the largest differences in accuracy occurring during forecasts of opportunity.

Mayer, Kirsten J.↗
DOE OSTI2024

Incorrect computation of Madden-Julian oscillation prediction skill

The Madden–Julian oscillation (MJO) is a major tropical weather system and one of the largest sources of predictability for subseasonal-to-seasonal weather forecasts. Skillful prediction of the MJO has been a highly active area of research due to its large socio-economic impacts. Silini et al., herein S21, developed a machine learning model to predict the MJO, which they claimed to have an MJO prediction skill of 26–27 days over all seasons and 45 days for December–February (DJF) winter. If true, this would make the skill of their model competitive with that of the state-of-the-art dynamical MJO prediction systems at 20–35 days. However, here we show that the MJO prediction was calculated incorrectly in S21, which spuriously increased the performance of their model. Correctly computed skill of their model was substantially lower than that reported in S21; the skill for all seasons drops to 11–12 days and the skill for forecasts initialized during DJF drops to 15 days. Our findings clarify that the S21 machine learning model is not competitive with state-of-the-art numerical weather prediction models in predicting the MJO.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Co-Evolving Climate Models under Uncertainty to Improve Predictive Skill

Focal Area: Predictive modeling through the use of AI techniques and AI-derived model components; the use of AI and other tools to design a prediction system comprising of a hierarchy of models Science Challenge: Advances in modeling of climate and improved observational capabilities have led to great improvements in understanding large-scale historical climate effects. It however remains a challenge to reliably predict the risk of fine-scale regional climate events on human systems even as such risks are on the rise because warmer and wetter climates are more prone to hydrometeorological extremes,

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Attribution of NAO Predictive Skill Beyond 2 Weeks in Boreal Winter

Weeks 3–6 averaged winter North Atlantic Oscillation (NAO) predictive skill in a state-of-the-art coupled climate prediction system is attributed to two principle sources: upper and lower boundary conditions linked to the stratosphere and El Niño-Southern Oscillation (ENSO), respectively. A 20-member ensemble of 45-day reforecasts over 1999–2015 is utilized, together with uninitialized simulations with the atmospheric component of the prediction system forced with observed radiative forcing and lower boundary conditions. NAO forecast skill for lead times out to 6 weeks is higher following extreme stratospheric polar vortex conditions (weak and strong vortex events) compared to neutral states. Enhanced weeks 3-6 NAO predictive skill for weak vortex events results primarily from stratospheric downward coupling to the troposphere, while enhanced skill for strong vortex events can be partly attributed to lower boundary forcing related to the ENSO phenomenon. Implications for forecast system development and improvement are discussed.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Machine Learning for Daily Forecasts of Arctic Sea Ice Motion: An Attribution Assessment of Model Predictive Skill

Physics-based simulations of Arctic sea ice are highly complex, involving transport between different phases, length scales, and time scales. Resultantly, numerical simulations of sea ice dynamics have a high computational cost and model uncertainty. We employ data-driven machine learning (ML) to make predictions of sea ice motion. The ML models are built to predict present-day sea ice velocity given present-day wind velocity and previous-day sea ice concentration and velocity. Models are trained using reanalysis winds and satellite-derived sea ice properties. We compare the predictions of three different models: persistence (PS), linear regression (LR), and a convolutional neural network (CNN). We quantify the spatiotemporal variability of the correlation between observations and the statistical model predictions. Additionally, we analyze model performance in comparison to variability in properties related to ice motion (wind velocity, ice velocity, ice concentration, distance from coast, bathymetric depth) to understand the processes related to decreases in model performance. Results indicate that a CNN makes skillful predictions of daily sea ice velocity with a correlation up to 0.81 between predicted and observed sea ice velocity, while the LR and PS implementations exhibit correlations of 0.78 and 0.69, respectively. The correlation varies spatially and seasonally: lower values occur in shallow coastal regions and during times of minimum sea ice extent. LR parameter analysis indicates that wind velocity plays the largest role in predicting sea ice velocity on 1-day time scales, particularly in the central Arctic. Regions where wind velocity has the largest LR parameter are regions where the CNN has higher predictive skill than the LR.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Volcanic forcing degrades multiyear-to-decadal prediction skill in the tropical Pacific

Volcanic aerosol forcing can affect global climate, but its role in climate prediction remains poorly understood. We isolate the impact of volcanic eruptions on multiyear-to-decadal climate prediction skill by comparing two suites of initialized decadal hindcasts conducted with and without historical volcanic forcing. Unexpectedly, the inclusion of volcanic forcing in the prediction system significantly degrades the forecast skill of detrended multiyear-to-decadal sea surface temperature (SST) variability in the central-eastern tropical Pacific. The ensemble mean hindcasts produce multiyear-to-decadal tropical Pacific SST cooling in response to large tropical volcanic eruptions through thermodynamic and El Niño–Southern Oscillation (ENSO)–like dynamic processes. However, in observations, these eruptions coincided with tropical Pacific warming, which is well predicted by the no-volcano hindcasts and, hence, is likely related to internal climate variability. Improved model representation of volcanic response and its interaction with internal climate variability is required to advance prediction of tropical Pacific decadal variability and associated global impacts.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2013

Decadal Prediction Skill in the GEOS-5 Forecast System

A suite of decadal predictions has been conducted with the NASA Global Modeling and Assimilation Office's (GMAO's) GEOS-5 Atmosphere-Ocean general circulation model. The hind casts are initialized every December 1st from 1959 to 2010, following the CMIP5 experimental protocol for decadal predictions. The initial conditions are from a multivariate ensemble optimal interpolation ocean and sea-ice reanalysis, and from GMAO's atmospheric reanalysis, the modern-era retrospective analysis for research and applications. The mean forecast skill of a three-member-ensemble is compared to that of an experiment without initialization but also forced with observed greenhouse gases. The results show that initialization increases the forecast skill of North Atlantic sea surface temperature compared to the uninitialized runs, with the increase in skill maintained for almost a decade over the subtropical and mid-latitude Atlantic. On the other hand, the initialization reduces the skill in predicting the warming trend over some regions outside the Atlantic. The annual-mean Atlantic meridional overturning circulation index, which is defined here as the maximum of the zonally-integrated overturning stream function at mid-latitude, is predictable up to a 4-year lead time, consistent with the predictable signal in upper ocean heat content over the North Atlantic. While the 6- to 9-year forecast skill measured by mean squared skill score shows 50 percent improvement in the upper ocean heat content over the subtropical and mid-latitude Atlantic, prediction skill is relatively low in the sub-polar gyre. This low skill is due in part to features in the spatial pattern of the dominant simulated decadal mode in upper ocean heat content over this region that differ from observations. An analysis of the large-scale temperature budget shows that this is the result of a model bias, implying that realistic simulation of the climatological fields is crucial for skillful decadal forecasts.

Decadal Prediction↗
NASA NTRS2018

Prediction Skill of U.S. Flash Droughts in Subseasonal Experiment (SubX) Model Hindcasts

Droughts that establish themselves over a short period of time (weeks to a few months), referred to as flash droughts, can have devastating impacts on agriculture, water resources, and ecosystems. The ability to predict such droughts in advance would greatly enhance our preparation for them and potentially reduce their impacts. The sub-seasonal time scale at which flash droughts occur emphasizes the importance of producing forecasts at weekly or finer intervals that extend beyond the numerical weather prediction time frame. Here we assess the ability of eight global forecast systems, each participating in the Sub-seasonal Experiment project (SubX), to predict key features associated with rapidly developing droughts over the United States during the last two decades. MERRA2 reanalysis is used as observations. Prediction skill for temperature and precipitation anomalies during these events is limited to the first 1-2 weeks after initialization for most hindcasts. However, there are some hindcasts in which large anomalies are well predicted 3-4 weeks or more in advance. The physical mechanisms that are key to the development of surface anomalies, including quasi-stationary atmospheric waves, were also evaluated. Most hindcasts were unable to capture the development or progression of such drought-inducing circulation features more than 1-2 weeks in advance.

Deangelis, Anthony M.↗
NASA NTRS2012

Decadal Prediction Skill in the GEOS-5 Forecast System

A suite of decadal predictions has been conducted with the NASA Global Modeling and Assimilation Office?s GEOS-5 Atmosphere-Ocean General Circulation Model (AOGCM). The hindcasts are initialized every December from 1959 to 2010 following the CMIP5 experimental protocol for decadal predictions. The initial conditions are from a multi-variate ensemble optimal interpolation ocean and sea-ice reanalysis, and from the atmospheric reanalysis (MERRA, the Modern-Era Retrospective Analysis for Research and Applications) generated using the GEOS-5 atmospheric model. The forecast skill of a three-member-ensemble mean is compared to that of an experiment without initialization but forced with observed CO2. The results show that initialization acts to increase the forecast skill of Northern Atlantic SST compared to the uninitialized runs, with the increase in skill maintained for almost a decade over the subtropical and mid-latitude Atlantic. The annual-mean Atlantic Meridional Overturning Circulation (AMOC) index is predictable up to a 5-year lead time, consistent with the predictable signal in upper ocean heat content over the Northern Atlantic. While the skill measured by Mean Squared Skill Score (MSSS) shows 50% improvement up to 10-year lead forecast over the subtropical and mid-latitude Atlantic, however, prediction skill is relatively low in the subpolar gyre, due in part to the fact that the spatial pattern of the dominant simulated decadal mode in upper ocean heat content over this region appears to be unrealistic. An analysis of the large-scale temperature budget shows that this is the result of a model bias, implying that realistic simulation of the climatological fields is crucial for skillful decadal forecasts.

Ham, Yoo-Geun↗
NASA NTRS2018

Prediction Skill of the MJO Teleconnection Signals in the NASA GEOS Subseasonal Reforecasts

Tropical-extratropical teleconnections are considered key to advancing subseasonal prediction. The Madden Julian oscillation (MJO), characterized by large scale convective envelopes propagating along the tropical Indo-Pacific sector, is known to modulate midlatitude circulation and associated weather patterns. Although there is a general consensus on the MJO's influence on the midlatitude circulation, which is thought to be due to modulations of the North Atlantic Oscillation (NAO) and the Pacific North America (PNA) pattern, relatively less is known about the predictability of these teleconnection signals in dynamical forecast models. The composite evolution of the midlatitude circulation anomalies and associated wave train structure as the delayed response to tropical heating are reported in many studies that have examined reanalyses and long model simulations. However, it is yet to be determined whether they lend any beneficial subseasonal forecast skill, especially to weekly mean surface temperature and precipitation over North America. Investigating useful predictable signals from the MJO teleconnections is also complicated by the fact that the MJO is a moving heat source with an approximate periodicity of 30-60 days, and that the structure and amplitude of the midlatitude response can be sensitive to the longitudinal positioning of the heating anomaly as well as the propagation speed of the MJO. The objective of this study is to investigate the impact of MJO teleconnections on forecast accuracy at 2-3 week lead over North America, with an emphasis on the above-mentioned lesser known aspects of these teleconnections. To this end, we utilize a suite of subseasonal reforecasts performed with the latest NASA GEOS-5 seasonal-to-subseasonal (S2S) system. These reforecasts were performed as part of the NOAA SubX project, wherein the NASA GEOS-5 atmosphere-ocean coupled model was run at degree horizontal resolution, initialized every 5 days for the period 1999-2016. The GEOS-5 model shows skillful predictions of the MJO, with the correlation coefficient based on the real-time multivariate MJO (RMM) index staying at or above 0.5 up to forecast lead 26-36 days. The system is thus a useful tool for investigating MJO teleconnection processes.

Achuthavarier, Deepthi↗
NASA NTRS2011

Predicting Hydrological Drought: Relative Contributions of Soil Moisture and Snow Information to Seasonal Streamflow Prediction Skill

in this study we examine how knowledge of mid-winter snow accumulation and soil moisture conditions contribute to our ability to predict streamflow months in advance. A first "synthetic truth" analysis focuses on a series of numerical experiments with multiple sophisticated land surface models driven with a dataset of observations-based meteorological forcing spanning multiple decades and covering the continental United States. Snowpack information by itself obviously contributes to the skill attained in streamflow prediction, particularly in the mountainous west. The isolated contribution of soil moisture information, however, is found to be large and significant in many areas, particularly in the west but also in region surrounding the Great Lakes. The results are supported by a supplemental, observations-based analysis using (naturalized) March-July streamflow measurements covering much of the western U.S. Additional forecast experiments using start dates that span the year indicate a strong seasonality in the skill contributions; soil moisture information, for example, contributes to kill at much longer leads for forecasts issued in winter than for those issued in summer.

Koster, R.↗
DOE OSTI2022

Attribution of North American Subseasonal Precipitation Prediction Skill

The skill of NOAA’s official monthly U.S. precipitation forecasts (issued in the middle of the prior month) has historically been low, having shown modest skill over the southern United States, but little or no skill over large portions of the central United States. The goal of this study is to explain the seasonal and regional variations of the North American subseasonal (weeks 3–6) precipitation skill, specifically the reasons for its successes and its limitations. The performances of multiple recent-generation model reforecasts over 1999–2015 in predicting precipitation are compared to uninitialized simulation skill using the atmospheric component of the forecast systems. This parallel analysis permits attribution of precipitation skill to two distinct sources: one due to slowly evolving ocean surface boundary states and the other to faster time-scale initial atmospheric weather states. A strong regionality and seasonality in precipitation forecast performance is shown to be analogous to skill patterns dictated by boundary forcing constraints alone. The correspondence is found to be especially high for the North American pattern of the maximum monthly skill that is achieved in the reforecast. The boundary forcing of most importance originates from tropical Pacific SST influences, especially those related to El Niño–Southern Oscillation. Furthermore, we discuss physical constraints that may limit monthly precipitation skill and interpret the performance of existing models in the context of plausible upper limits.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Quantifying UAS Observation Error Variance Used in Data Assimilation Systems and Its Impact on Predictive Skill

Observation error determines the weights of the observations and background state used in data assimilation to generate analyses. Quantifying observation error is critical for the optimal assimilation of observational data sets. Uncrewed Aircraft System (UAS) observations have shown potential benefits in filling observational gaps in the lower atmosphere; however, characterization of their error characteristics has been limited. To optimize the use of UAS observations in numerical weather prediction, UAS observation error is estimated based on the 3‐cornered hat diagnostic approach which uses three independent estimates of the atmospheric state. This approach is applied to data from the 2018 Lower Atmospheric Profiling Studies at Elevation‐a Remotely‐piloted Aircraft Team Experiment field campaign using collocated UAS and rawinsonde observations along with output from a set of convection‐permitting model simulations. The estimated observation error values for UAS temperature, wind, and relative humidity measurements were found to be only weakly dependent on height AGL with mean values equal to 0.5°C, 0.8 m s −1 , and 3%, respectively. Only the newly estimated observation error for temperature differed from that previously used to assimilate commercial aircraft observations into global models (1.0°C). However, using this reduced temperature observation error produced more accurate mesoscale analyses and forecasts of both terrain‐driven flows and convection initiation generated by colliding outflow boundaries within the San Luis Valley of Colorado.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

The predictive skill of convolutional neural networks models for disease forecasting

In this paper we investigate the utility of one-dimensional convolutional neural network (CNN) models in epidemiological forecasting. Deep learning models, in particular variants of recurrent neural networks (RNNs) have been studied for ILI (Influenza-Like Illness) forecasting, and have achieved a higher forecasting skill compared to conventional models such as ARIMA. In this study, we adapt two neural networks that employ one-dimensional temporal convolutional layers as a primary building block—temporal convolutional networks and simple neural attentive meta-learners—for epidemiological forecasting. We then test them with influenza data from the US collected over 2010-2019. We find that epidemiological forecasting with CNNs is feasible, and their forecasting skill is comparable to, and at times, superior to, plain RNNs. Thus CNNs and RNNs bring the power of nonlinear transformations to purely data-driven epidemiological models, a capability that heretofore has been limited to more elaborate mechanistic/compartmental disease models.

59 BASIC BIOLOGICAL SCIENCES↗