Search NASA⌕ Search

SEARCH · Search NASA

Results for “forecast skill”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 325 records · Page 18
All sourcesNASA NTRSDOE OSTI
NASA NTRS2020

Subseasonal-to-Seasonal Hindcast Skill Assessment of Ridging Events Related to Drought Over the Western United States

Persistent atmospheric ridging events centered near the western United States are associated with widespread precipitation deficits and meteorological drought. Due to the relatively low skill of- dynamical models in forecasting precipitation on subseasonal‐to‐seasonal (S2S) time scales across this region, forecasts of ridging are explored in this study as a potential bridge for early warning drought prediction. To assess skill, we evaluate deterministic and probabilistic S2S hindcasts (out to 6 week lead time) of ridging events and geopotential height anomalies over the western United States in five ensemble hindcast systems. Prediction skill for ridging events is shown to be highly variable across models. For some models, longer‐time‐averaged patterns of geopotential height anomalies across the first 6 weeks can be skillfully simulated, when evaluated probabilistically against climatology. The most skillful models show modest skill in forecasting above normal ridging occurrences at lead times of Weeks 3–4 and Weeks 5–6,with some sensitivity to the specific ridge location and method for determining skill. Using the European Centre for Medium‐Range Weather Forecasts (ECMWF) model as a case study, longer lead time forecast busts are shown to often occur under extended periods of highly active La Nina‐like tropical convection. Model errors at simulating these tropical convection features may have a disproportionately large impact and degrade downstream forecasts over the western United States. Despite the documented model shortcomings, our results highlight an opportunity to target skillful ridging forecasts from dynamical models for improving early warning drought forecasting at S2S lead times.

S2S↗
NASA NTRS2019

Near Real-Time Sub/Seasonal Prediction of Aerosol at NASA Global Modeling and Assimilation Office

A new version of the coupled modeling and analysis system used to produce near real time subseasonal to seasonal forecasts was released over a year ago by the NASA/Goddard Global Modeling and Assimilation Office. The model runs at approximately 1/2 degree globally in the atmosphere and ocean, contains a realistic description of the cryosphere, and includes an interactive aerosol model. The data assimilation used to produce initial conditions is weakly coupled, in which the atmosphere-only assimilated state is coupled to an ocean data assimilation system using a Local Ensemble Transform Kalman Filter. Here will briefly describe the new system, and show results of aerosol-derived air quality from an extensive series of retrospective forecasts. The interactive aerosol is shown to improve seasonal time scale prediction skill during some "forecasts of oppurtunity". Plans for a future version of the system with predicted biomass burning from fires will also be discussed.

Molod, Andrea M.↗
DOE OSTI2022

Evaluating wind speed and power forecasts for wind energy applications using an open-source and systematic validation framework

Building on the verification and validation work developed under the Second Wind Forecast Improvement Project, this work exhibits the value of a consistent procedure to evaluate wind power forecasts. We established an open-source Python code base tailored for wind speed and wind power forecast validation, WE-Validate. The code base can evaluate model forecasts with observations in a coherent manner. To demonstrate the systematic validation framework of WE-Validate, we designed and hosted a forecast evaluation benchmark exercise. We invited forecast providers in industry and academia to participate and submit forecasts for two case studies. We then evaluated the submissions with WE-Validate. Our findings suggest that ensemble means have reasonable skills in time series forecasting, whereas they are often inferior to single ensemble members in wind ramp forecasting. Adopting a voting scheme in ramp forecasting that allows ensemble members to detect ramps independently leads to satisfactory skill scores. Throughout this document, we also emphasize the importance of using statistically robust and resistant metrics as well as equitable skill scores in forecast evaluation.

17 WIND ENERGY↗
NASA NTRS1985

Results of an observing system simulation experiment based on the proposed Windsat instrument

Two fraternal twin experiments were conducted as part of this study. A data impact experiment using real data (runs 1 and 2) was conducted to assess the impact that rawin wind observations have on both a 5-day assimilation and a single 5-day forecast generated at the end of each assimilation. An observing system simulation experiment (OSSE) using simulated observations (runs 3, 4, and 5) was conducted in order to first calibrate the OSSE results and second to use this calibration to estimate the 'real world' impact from the contribution of global 3-dimensional wind profiles generated from a space-based lidar system known as Windsat. Each of the three runs in the seond experiment were also 5-day assimilation runs with a 5-day forecast initialized from the last 6-hour update cycle of the assimilation. The data impact study revealed a consistent positive impact when rawin winds were added back into an otherwise complete FGGE data set. Both the 6-hour and 5-day forecasts were improved at all levels, in both hemispheres, and for both the wind and the geopotential height fields. Similar results were obtained from the two parallel simulation runs, 3 and 4. Together with the results from runs 1 and 2, calibration coefficients were generated so as to 'correct' the results determined from the addition of Windsat winds (run 5). The Windsat simulation showed a positive improvement in all cases studied. Even though only the tropics were enhanced with these wind observations, hemispheric rms errors were decreased in both the assimilation and 5-day forecast. The 6-hour forecasts of zonal wind from the assimilation run were improved by as much as 50 pct. on the average, and the single forecast showed an average improvement of near 30 pct. Even though these calibrated values are considered too optimistic, the skill of the forecast generated from this run extended the useful forecast period by 18-24 hours.

Arnold, Charles P., Jr.↗
NASA NTRS2005

Assimilation of ozone profiles from the Improved Limb Atmospheric Spectrometer: Study on Antarctic Ozone - 2

Ozone data from the Improved Limb Atmospheric Spectrometer-II (ILAS-II) were included in addition to other satellite observations in the ozone assimilation system at the Global Modeling and Assimilation Office (GMAO) of NASA/Goddard. The control run assimilated data from NOAA 16 Solar Backscatter Ultraviolet/2 (SBUV/2) and Polar Ozone and Aerosol Measurement III (POAM III) instruments. Persistent impacts over Antarctica and transient impacts over northern middle and high latitudes are seen from April to October 2003, when ILAS-II provided good coverage. The largest improvements with respect to independent ozone sonde data are seen over the South Pole station. Ozone analyses and forecasts from the assimilation of SBUV/2, POAM III and ILAS-II data b e used to investigate the transport of ozone to southern middle latitudes following the breakup of the Antarctic vortex. The quality of analyses and forecasts is evaluated by comparison with independent Stratospheric Aerosol and Gas Experiment III (SAGE III) ozone data near 46degs. Anomaly correlations between SAGE III data and forecasts'exceed 0.6 for up to five to seven days at 30,50, and 70 ma. The loss of skill with advancing forecast length is related to dynamical errors due to an excessively persistent vortex in longer forecasts, which hampers the transport of low ozone air into middle latitudes.

Stajner, Ivanka↗
DOE OSTI2022

Quantifying the Effect of Climate Change on Midlatitude Subseasonal Prediction Skill Provided by the Tropics

Abstract Subseasonal timescales (∼2 weeks–2 months) are known for their lack of predictability, however, specific Earth system states known to have a strong influence on these timescales can be harnessed to improve prediction skill (known as “forecasts of opportunity”). As the climate continues warming, it is hypothesized these states may change and consequently, their importance for subseasonal prediction may also be impacted. Here, we examine changes to midlatitude subseasonal prediction skill provided by the tropics under anthropogenic warming using artificial neural networks to quantify skill. The network is tasked to predict the sign of the 500 hPa geopotential height for historical and future time periods in the Community Earth System Model Version 2 ‐ Large Ensemble across the Northern Hemisphere at a 3 week lead using tropical precipitation. We show prediction skill changes substantially in key midlatitude regions and these changes appear linked to changes in seasonal variability with the largest differences in accuracy occurring during forecasts of opportunity.

Mayer, Kirsten J.↗
NASA NTRS1990

Forecasting atmospheric angular momentum and length-of-day using operational meteorological models

Forecasts of zonal wind fields produced by the medium-range forecast model of the U.S. National Meteorological Center are used to create predictions of the atmosphere's angular momentum at lead times of 1-10 days. The skill of these forecasts, which are of interest to those concerned with monitoring changes in the length-of-day for navigational purposes, is assessed, and the regions in the atmosphere that contribute most importantly to forecast errors are identified.

Rosen, R. D.↗
NASA NTRS2009

Adjoint-Based Diagnostics in Atmospheric Data Assimilation

With the adjoint of a data assimilation system, the impact of any or all assimilated observations on measures of forecast or analysis skill can be estimated accurately and efficiently. The approach is especially well-suited for assessing the impact of hyper-spectral satellite instruments on numerical weather forecasts because it easily allows aggregation of results in terms of individual data types, channels or locations, all computed simultaneously based on a single pass of the Adjoint- system. This talk 'provides a general overview of the use of Adjoint-based diagnostic tools in data assimilation. We will focus on the theoretical basis and practical implementation of these tools, as well as their application to specific problems. Adjoint-based Impact calculations will be compared with results from standard observing system experiments (OSEs) and their application to related problems, such as the calibration of Observing System Simulation Experiments (OSSEs) will be discussed.

Gelaro, Ronald↗
DOE OSTI2023

Assessing decadal variability of subseasonal forecasts of opportunity using explainable AI

Abstract Identifying predictable states of the climate system allows for enhanced prediction skill on the generally low-skill subseasonal timescale via forecasts with higher confidence and accuracy, known as forecasts of opportunity. This study takes a neural network approach to explore decadal variability of subseasonal predictability, particularly during forecasts of opportunity. Specifically, this work quantifies subseasonal prediction skill provided by the tropics within the Community Earth System Model Version 2 (CESM2) Large Ensemble and assesses how this skill evolves on decadal timescales. Utilizing the networks’ confidence and explainable artificial intelligence, physically meaningful sources of predictability associated with periods of enhanced skill are identified. Using these networks, we find that tropically-driven subseasonal predictability varies on decadal timescales during forecasts of opportunity. Further, we investigate the drivers of the low frequency modulation of the tropical-extratropical teleconnection and discuss the implications. Analysis is extended to ECMWF Reanalysis v5 data, revealing that the relationships learned within the CESM2-Large Ensemble holds in modern reanalysis data. These results indicate that the neural networks are capable of identifying predictable decadal states of the climate system within CESM2 that are useful for making confident, accurate subseasonal precipitation predictions in the real world.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2017

GEOS-5 Seasonal Forecast System

Ensembles of numerical forecasts based on perturbed initial conditions have long been used to improve estimates of both weather and climate forecasts. The Goddard Earth Observing System (GEOS) Atmosphere-Ocean General Circulation Model, Version 5 (GEOS-5 AOGCM) Seasonal-to-Interannual Forecast System has been used routinely by the GMAO since 2008, the current version since 2012. A coupled reanalysis starting in 1980 provides the initial conditions for the 9 month experimental forecasts. Once a month, sea surface temperature from a suite of 11 ensemble forecasts is contributed to the North American Multi-Model Ensemble (NMME) consensus project, which compares and distributes seasonal forecasts of ENSO events. Since June 2013, GEOS-5 forecasts of the Arctic sea-ice distribution were provided to the Sea-Ice Outlook project. The seasonal forecast output data includes surface fields, atmospheric and ocean fields, as well as sea ice thickness and area, and soil moisture variables. The current paper aims to document the characteristics of the GEOS-5 seasonal forecast system and to highlight forecast biases and skills of selected variables (sea surface temperature, air temperature at 2 m, precipitation and sea ice extent) to be used as a benchmark for the future GMAO seasonal forecast systems and to facilitate comparison with other global seasonal forecast systems.

GEOS-5↗
DOE OSTI2022

Improving Seasonal Forecast Using Probabilistic Deep Learning

The path toward realizing the potential of seasonal forecasting and its socioeconomic benefits relies on improving general circulation model (GCM) based dynamical forecast systems. To improve dynamical seasonal forecasts, it is crucial to set up forecast benchmarks, and clarify forecast limitations posed by model initialization errors, formulation deficiencies, and internal climate variability. With huge costs in generating large forecast ensembles, and limited observations for forecast verification, the seasonal forecast benchmarking and diagnosing task proves challenging. Here, we develop a probabilistic deep learning-based statistical forecast methodology, drawing on a wealth of climate simulations to enhance seasonal forecast capability and forecast diagnosis. By explicitly modeling the internal climate variability and GCM formulation differences, the proposed Conditional Generative Forecasting (CGF) methodology enables bypassing crucial barriers in dynamical forecast, and offers a top-down viewpoint to examine how complicated GCMs encode the seasonal predictability information. We apply the CGF methodology for global seasonal forecast of precipitation and 2 m air temperature, based on a unique data set consisting 52,201 years of climate simulation. Results show that the CGF methodology can faithfully represent the seasonal predictability information encoded in GCMs. We successfully apply this learned relationship in real-world seasonal forecast, achieving competitive performance compared to dynamical forecasts. Using this CGF as benchmark, we reveal the impact of insufficient forecast spread sampling that limits the skill of the considered dynamical forecast system. Finally, we introduce different strategies for composing ensembles using the CGF methodology, highlighting the potential for leveraging the strengths of multiple GCMs to achieve advantgeous seasonal forecast.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Probabilistic Day-Ahead Forecasting Using an Analog Ensemble Approach for Wind Farm Grid Services

Wind resource assessment and wind power forecasting are used in research and industry to anticipate future power output at scales ranging from individual wind turbines to entire wind farms. Probabilistic day-ahead wind forecasting is useful for anticipating how a wind farm could potentially participate in the day-ahead market by providing upper and lower bounds for expected power generation, thus informing grid operators of its uncertainty. Understanding this uncertainty is part of a larger project focused on building a platform that combines efforts in weather forecasting, aerodynamic and economic modeling to create maximum value of a wind plant to better provide services to the grid. This effort is also known as the Atmosphere to Electrons to Grid (A2E2G) project. One method for producing a probabilistic forecast is through the analog ensemble approach (Delle Monache et al., 2011). This method leverages historical forecasts and their corresponding observations as a training data set from which future forecasts can be made. For some future forecast, the most similar historical forecasts (analogs) are identified on a regular time basis such as once per a 3-hour window. The most similar analogs, based on a metric such as root mean square error (RMSE), are recorded and their corresponding verifying observations are used as an ensemble member for this future forecast. Prior work in this area demonstrates improvements over raw Numerical Weather Prediction (NWP) forecasts and shows skill similar to techniques such as logistic regression and machine learning (Delle Monache et al., 2013; Alessandrini et al., 2015). Here, we take the High-Resolution Rapid Refresh model (HRRR) day-ahead forecast (0-36 hours) to create a probabilistic day-ahead forecast using an analog ensemble approach. The HRRR has an hourly temporal resolution, with a spatial resolution of 3 km. The 12 UTC HRRR model run is downloaded every day for one year from August 2019 - July 2020, with the first 11 months serving as a bank of analogs from which the forecasting algorithm can create a probabilistic forecast. Once downloaded, the original HRRR forecast is temporally interpolated to 5-minutes, aligning with both the temporal resolution of the observations as well as the timescale relevant for day-ahead power forecasts. The forecast is validated at the M2 tower at the Flatirons Campus of the National Renewable Energy Laboratory (NREL) at a typical wind turbine height of 80 m. Variables such as wind speed, wind direction, and turbulence intensity are incorporated into the probabilistic forecast model and weighted according to their relative importance to the forecast. Based on metrics such as mean bias error (MBE), mean absolute error (MAE), and root mean square error, the analog ensemble forecast outperforms the raw HRRR forecast during the testing period of July 2020. Figure 1 illustrates an example day-ahead forecast compared against the verifying observations. The general variability and ramps are captured throughout the day, with potential to further improve the analog ensemble model through machine learning techniques.

numerical weather prediction↗
NASA NTRS2015

Sea Ice Outlook for September 2015 June Report - NASA Global Modeling and Assimilation Office

The recent decline in perennial sea ice cover in Arctic Ocean is a topic of enormous scientific interest and has relevance to a broad variety of scientific disciplines and human endeavors including biological and physical oceanography, atmospheric circulation, high latitude ecology, the sustainability of indigenous communities, commerce, and resource exploration. A credible seasonal prediction of sea ice extent would be of substantial use to many of the stakeholders in these fields and may also reveal details on the physical processes that result in the current trends in the ice cover. Forecasts are challenging due in part to limitations in the polar observing network, the large variability in the climate system, and an incomplete knowledge of the significant processes. Nevertheless it is a useful to understand the current capabilities of high latitude seasonal forecasting and identify areas where such forecasts may be improved. Since 2008 the Arctic Research Consortium of the United States (ARCUS) has conducted a seasonal forecasting contest in which the average Arctic sea ice extent for the month of September (the month of the annual extent minimum) is predicted from available forecasts in early June, July, and August. The competition is known as the Sea Ice Outlook (SIO) but recently came under the auspices of the Sea Ice Prediction Network (SIPN), and multi-agency funded project to evaluate the SIO. The forecasts are submitted based on modeling, statistical, and heuristic methods. Forecasts of Arctic sea ice extent from the GMAO are derived from seasonal prediction system of the NASA Goddard Earth Observing System model, version 5 (GEOS 5) coupled atmosphere and ocean general circulation model (AOGCM). The projections are made in order to understand the relative skill of the forecasting system and to determine the effects of future improvements to the system. This years prediction is for a September average Arctic ice extent of 5.030.41 million km2.

SIPN↗
DOE OSTI2020

Improved Predictability of the Indian Ocean Dipole Using a Stochastic Dynamical Model Compared to the North American Multimodel Ensemble Forecast

This study assesses the predictive skill of eight North American Multimodel Ensemble (NMME) models in predicting the Indian Ocean dipole (IOD). We find that the forecasted ensemble-mean IOD–El Niño–Southern Oscillation (ENSO) relationship deteriorates away from the observed relationship with increasing lead time, which might be one reason that limits the IOD predictive skill in coupled models. We are able to improve the IOD predictive skill using a recently developed stochastic dynamical model (SDM) forced by forecasted ENSO conditions. The results are consistent with the previous result that operational IOD predictability beyond persistence at lead times beyond one season is mostly controlled by ENSO predictability and the signal-to-noise ratio of the Indo-Pacific climate system. The multimodel ensemble (MME) investigated here is found to be of superior skill compared to each individual model at most lead times. Importantly, the skill of the SDM IOD predictions forced with forecasted ENSO conditions were either similar or better than those of the MME IOD forecasts. Moreover, the SDM forced with observed ENSO conditions exhibits significantly higher IOD prediction skill than the MME at longer lead times, suggesting the large potential skill increase that could be achieved by improving operational ENSO forecasts. We find that both cold and warm biases of the predicted Niño-3.4 index may cause false alarms of negative and positive IOD events, respectively, in NMME models. Many false alarms for IOD forecasts at lead times longer than one season in the original forecasts disappear or are significantly reduced in the SDM forced by forecasted ENSO conditions.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2007

Objective Lightning Probability Forecasting for Kennedy Space Center and Cape Canaveral Air Force Station, Phase II

This report describes the work done by the Applied Meteorology Unit (AMU) to update the lightning probability forecast equations developed in Phase I. In the time since the Phase I equations were developed, new ideas regarding certain predictors were formulated and a desire to make the tool more automated was expressed by 45 WS forecasters. Five modifications were made to the data: 1) increased the period of record from 15 to 17 years, 2) modified the valid area to match the lighting warning areas, 3) added the 1000 UTC CCAFS sounding to the other soundings in determining the flow regime, 4) used a different smoothing function for the daily climatology, and 5) determined the optimal relative humidity (RH) layer to use as a predictor. The new equations outperformed the Phase I equations in several tests, and improved the skill of the forecast over the Phase I equations by 8%. A graphical user interface (GUI) was created in the Meteorological Interactive Data Display System (MIDDS) that gathers the predictor values for the equations automatically. The GUI was transitioned to operations in May 2007 for the 2007 warm season.

Lambert, Winifred↗
NASA NTRS2003

Diabatic Initialization of Mesoscale Models in the Southeastern United States: Can 0 to 12h Warm Season QPF be Improved?

It is well known that numerical warm season quantitative precipitation forecasts lack significant skill for numerous reasons. Some are related to the model--it may lack physical processes required to realistically simulate convection or the numerical algorithms and dynamics employed may not be adequate. Others are related to initialization-mesoscale features play an important role in convective initialization and atmospheric observation systems are incapable of properly depicting the three-dimensional stability structure at the mesoscale. The purpose of this study is to determine if a mesoscale model initialized with a diabatic initialization scheme can improve short-term (0 to 12h) warm season quantitative precipitation forecasts in the Southeastern United States. The Local Analysis and Prediction System (LAPS) developed at the Forecast System Laboratory is used to diabatically initialize the Pennsylvania State University/National center for Atmospheric Research (PSUNCAR) Mesoscale Model version 5 (MM5). The SPORT Center runs LAPS operationally on an hourly cycle to produce analyses on a 15 km covering the eastern 2/3 of the United States. The 20 km National Centers for Environmental Prediction (NCEP) Rapid Update Cycle analyses are used for the background fields. Standard observational data are acquired from MADIS with GOES/CRAFT Nexrad data acquired from in-house feeds. The MM5 is configured on a 140 x 140 12 km grid centered on Huntsville Alabama. Preliminary results indicate that MM5 runs initialized with LAPS produce improved 6 and 12h QPF threat scores compared with those initialized with the NCEP RUC.

Lapenta, William M.↗
NASA NTRS2024

Tropical Cyclones in the GEOS-S2S-2 Subseasonal Forecasts

This paper analyzes the climatology, prediction skill, and predictability of tropical cyclones (TCs) in NASA’s Global Earth Observing System Subseasonal to Seasonal (GEOS-S2S) forecast system version 2. GEOS reasonably simulates the number and spatial distribution of TCs compared to observations except in the Atlantic where the model simulates too few TCs due to low genesis rates in the Caribbean Sea and Gulf of Mexico. The environmental conditions, diagnosed through a genesis potential index, do not clearly explain model biases in the genesis rates, especially in the Atlantic. At the storm-scale, GEOS reforecasts replicate several key aspects of the thermodynamic and dynamic structure of observed TCs, such as a warm core and the secondary circulation. The model, however, fails to simulate an off-center eyewall when evaluating vertical velocity, precipitation and moisture. The analysis of prediction skill of TC genesis and occurrence shows that GEOS has comparable skill to other global models in WMO S2S archive and that its skill could be further improved by increasing the ensemble size. After calibration, GEOS forecasts are skillful in the Western North Pacific and Southern Indian Ocean up to 20 days in advance. A model-based predictability analysis demonstrates the importance of the Madden-Julian Oscillation (MJO) as a source of predictability of TC occurrence beyond the 14 day lead-time. Forecasts initialized under strong MJO conditions show evidence of predictability beyond week 3. However, due to model biases in the forecast distribution there are notable gaps between MJO-related prediction skill and predictability which require further study.

Jorge L. Garcia-Franco↗
NASA NTRS1990

The relationship of marine stratus to synoptic conditions

The marine stratus which persistently covered most of the eastern Pacific Ocean, had large clear areas during the FIRE Intensive Field Operations (IFO) in 1987. Clear zones formed inside the large oceanic cloud mass on almost every day during the IFO. The location and size of the clear zones varied from day to day implying that they were related to dynamic weather conditions and not to oceanic conditions. Forecasting of cloud cover for aircraft operations during the IFO was directed towards predicting when and where the clear and broken zones would form inside the large marine stratus cloud mass. The clear zones often formed to the northwest of the operations area and moved towards it. However, on some days the clear zones appeared to form during the day in the operations area as part of the diurnal cloud burn off. The movement of the clear zones from day to day were hard to follow because of the large diurnal changes in cloud cover. Clear and broken cloud zones formed during the day only to distort in shape and fill during the following night. The field forecasters exhibited some skill in predicting when the clear and broken cloud patterns would form in the operations area. They based their predictions on the analysis and simulations of the models run by NOAA's Numeric Meteorological Center. How the atmospheric conditions analyzed by one NOAA/NMC model related to the cloud cover is discussed.

Wylie, Donald P.↗