Search NASA⌕ Search

SEARCH · Search NASA

Results for “forecast skill”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Seasonal forecasting skill for the High Mountain Asia region in the Goddard Earth Observing System

Seasonal variability of the global hydrologic cycle directly impacts human activities, including hazard assessment and mitigation, agricultural decisions, and water resources management. This is particularly true across the High Mountain Asia (HMA) region, where availability of water resources can change depending on local seasonality of the hydrologic cycle. Forecasting the atmospheric states and surface conditions, including hydrometeorologically relevant variables, at subseasonal-to-seasonal (S2S) lead times of weeks to months is an area of active research and development. NASA's Goddard Earth Observing System (GEOS) S2S prediction system has been developed with this research goal in mind. Here, we benchmark the forecast skill of GEOS-S2S (version 2) hydrometeorological forecasts at 1–3-month lead times in the HMA region, including a portion of the Indian subcontinent, during the retrospective forecast period, 1981–2016. To assess forecast skill, we evaluate 2 m air temperature, total precipitation, fractional snow cover, snow water equivalent, surface soil moisture, and terrestrial water storage forecasts against the Modern-Era Retrospective analysis for Research and Applications, Version 2 (MERRA-2) and independent reanalysis data, satellite observations, and data fusion products. Anomaly correlation is highest when the forecasts are evaluated against MERRA-2 and particularly in variables with long memory in the climate system, likely due to the similar initial conditions and model architecture used in GEOS-S2S and MERRA-2. When compared to MERRA-2, results for the 1-month forecast skill range from an anomaly correlation of R anom =0.18 for precipitation to R anom =0.62 for soil moisture. Anomaly correlations are consistently lower when forecasts are evaluated against independent observations; results for the 1-month forecast skill range from R anom =0.13 for snow water equivalent to R anom =0.24 for fractional snow cover. We find that, generally, hydrometeorological forecast skill is dependent on the forecast lead time, the memory of the variable within the physical system, and the validation dataset used. Overall, these results benchmark the GEOS-S2S system's ability to forecast HMA hydrometeorology.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Relating CMIP5 Model Biases to Seasonal Forecast Skill in the Tropical Pacific

Abstract We examine links between tropical Pacific mean state biases and El Niño/Southern Oscillation forecast skill, using model‐analog hindcasts of sea surface temperature (SST; 1961–2015) and precipitation (1979–2015) at leads of 0–12 months, generated by 28 different models from the fifth phase of the Coupled Model Intercomparison Project (CMIP5). Model‐analog forecast skill has been demonstrated to match or even exceed traditional assimilation‐initialized forecast skill in a given model. Models with the most realistic mean states and interannual variability for SST, precipitation, and 10‐m zonal winds in the equatorial Pacific also generate the most skillful precipitation forecasts in the central equatorial Pacific and the best SST forecasts at 6‐month or longer leads. These results show direct links between model climatological biases and seasonal forecast errors, demonstrating that model‐analog hindcast skill—that is, how well a model can capture the observed evolution of tropical Pacific anomalies—is an informative El Niño/Southern Oscillation metric for climate simulations.

Ding, Hui↗
DOE OSTI2021

Evaluating precipitation, streamflow, and inundation forecasting skills during extreme weather events: A case study for an urban watershed

Integrated forecasting systems for precipitation, streamflow, and floodplain inundation are of critical importance in mitigating the impacts of destructive floods caused by extreme weather events. However, the skills of streamflow and floodplain inundation forecasts derived from various Quantitative Precipitation Forecasts (QPF) require a greater level of understanding. In this paper, a set of QPF developed by the National Weather Service (NWS) were used to drive a flood modeling system obtained utilizing offline coupling of a physics-based distributed hydrological model, the Distributed Hydrology Soil and Vegetation Model (DHSVM), and a hydrodynamic model, the Two-dimensional Runoff Inundation Toolkit for Operational Needs (TRITON). This flood modeling system was used to produce forecasts of streamflow and floodplain inundation maps during three major flood events in the Brays Bayou Watershed (Houston, Texas, USA) for a range of QPF durations (6–72 h). Then, to investigate the effects of increasing QPF durations on the forecasts, the forecasting skills of precipitation, streamflow, and floodplain inundation were quantified. The results show that: 1) QPF skills for more intense and sustained events such as hurricanes and tropical storms are higher than for shorter, less intense events; 2) while QPF and streamflow forecasting skills decrease as QPF durations increase, inundation forecasts under longer QPF durations (24 or 72 h) show higher skills; 3) extending the maximum QPF duration in operational hydrologic modeling from 24 h (under normal circumstances) to 72 h (for extreme events) may increase the skills of long lead time forecasts for large-scale events like Hurricane Harvey.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Five-day track forecast skills of WRF model for the western North Pacific tropical cyclones

In this study, the characteristics of simulated tropical cyclones (TCs) over the western North Pacific by a regional model (the WRF Model) are verified. We utilize 12-km horizontal grid spacing, and simulations are integrated for 5 days from model initialization. A total of 125 forecasts are divided into five clusters through the k-means clustering method. The TCs in the cluster 1 and 2 (group 1), which includes many TCs moving northward in the subtropical region, generally have larger track errors than for TCs in cluster 3 and 4 (group 2). The optimal steering vector is used to examine the difference in the track forecast skill between these two groups. The bias in the steering vector between the model and analysis data is found to be more substantial for group 1 TCs than group 2 TCs. The larger steering vector difference for group 1 TCs indicates that environmental fields tend to be poorly simulated in group 1 TC cases. Furthermore, the residual terms, including the storm-scale process, asymmetric convection distribution, or beta-related effect, are also larger for group 1 TCs than group 2 TCs. Therefore, it is probable that the large track forecast error for group 1 TCs is a result of unreasonable simulations of environmental wind fields and residual processes in the midlatitudes.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Multi-decadal variation of ENSO forecast skill since the late 1800s

Diagnosing El Niño-Southern Oscillation (ENSO) predictability within operational forecast models is hindered by computational expense and the need for initialization with three-dimensional fields generated by global data assimilation. We instead examine multi-year ENSO predictability since the late 1800s using the model-analog technique, which has neither limitation. We first draw global coupled model states from pre-industrial control simulations, from the Coupled Model Intercomparison Project Phase 6, that are chosen to initially match observed monthly sea surface temperature and height anomalies in the Tropics. Their subsequent 36-month model evolution are the hindcasts, whose 20th century ENSO skill is comparable to twice-yearly hindcasts generated by a state-of-the-art European operational forecasting system. Despite the so-called spring predictability barrier, present throughout the record, there is substantial second-year ENSO skill, especially after 1960. Overall, ENSO exhibited notably high values of both amplitude and skill towards the end of the 19th century, and again in recent decades.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Machine learning for postprocessing ensemble streamflow forecasts

Skillful streamflow forecasts can inform decisions in various areas of water policy and management. We integrate numerical weather prediction ensembles, distributed hydrological model, and machine learning to generate ensemble streamflow forecasts at medium-range lead times (1–7 days). We demonstrate the application of machine learning as postprocessor for improving the quality of ensemble streamflow forecasts. Our results show that the machine learning postprocessor can improve streamflow forecasts relative to low-complexity forecasts (e.g., climatological and temporal persistence) as well as standalone hydrometeorological modeling and neural network. The relative gain in forecast skill from postprocessor is generally higher at medium-range timescales compared to shorter lead times; high flows compared to low–moderate flows, and the warm season compared to the cool ones. Overall, our results highlight the benefits of machine learning in many aspects for improving both the skill and reliability of streamflow forecasts.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

An Observationally Trained Markov Model for MJO Propagation

A Markovian stochastic model is developed for studying the propagation of the Madden-Julian Oscillation (MJO). This model represents the daily changes in real time multivariate MJO (RMM) indices as random functions of their current state and background conditions. The probability distribution function of the RMM changes is obtained using a machine learning algorithm trained to maximize MJO forecast skills using observed daily indices of RMM and different modes of variability. Skillful forecasts are obtained for lead times between 8 and 27 days. Large ensemble simulations by the stochastic model show that with monsoonal changes in the background state, MJO propagation across the Maritime Continent (MC) is most likely to be disrupted in boreal spring and summer when MJO events propagate from favorable conditions over the Indian Ocean to unfavorable ones over the MC, and predictability is higher during spring and summer when MJO activity is away from the MC region.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

The predictive skill of convolutional neural networks models for disease forecasting

In this paper we investigate the utility of one-dimensional convolutional neural network (CNN) models in epidemiological forecasting. Deep learning models, in particular variants of recurrent neural networks (RNNs) have been studied for ILI (Influenza-Like Illness) forecasting, and have achieved a higher forecasting skill compared to conventional models such as ARIMA. In this study, we adapt two neural networks that employ one-dimensional temporal convolutional layers as a primary building block—temporal convolutional networks and simple neural attentive meta-learners—for epidemiological forecasting. We then test them with influenza data from the US collected over 2010-2019. We find that epidemiological forecasting with CNNs is feasible, and their forecasting skill is comparable to, and at times, superior to, plain RNNs. Thus CNNs and RNNs bring the power of nonlinear transformations to purely data-driven epidemiological models, a capability that heretofore has been limited to more elaborate mechanistic/compartmental disease models.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Evaluation of Subseasonal-to-Seasonal (S2S) precipitation forecast from the North American Multi-Model ensemble phase II (NMME-2) over the contiguous U.S.

The second phase of the North America Multi-Model Ensemble (NMME-2) provides globally available Subseasonal-to-Seasonal (S2S) precipitation forecasts with a daily resolution. The S2S precipitation forecasts are getting increasing attention for their potentials in providing hydrometeorological forcing information for water resources planning at an extended range. However, the forecast skills of many existing S2S forecast products will significantly decrease when the lead time increases, hindering their applicability for watershed-scale hydrologic modeling. Therefore, forecast validation and large-scale evaluation are of great importance for water resources planning and hydrological applications. In this study, we comprehensively evaluate the S2S precipitation forecasts from the NMME-2 dataset over the contiguous United States (CONUS) and during the study period from 1982 to 2011. Three aspects of precipitation forecast capabilities are compared and analyzed: bias, skill scores, and the ability to predict extreme precipitation events. The Parameter-elevation Regressions on Independent Slopes Model (PRISM) is used as ground truth reference. Differs from other regional forecast validation study, we further examined and analyzed the dependences of NMME-2 precipitation forecast skills according to different seasonality, geographical locations, and lead times. Results show that the forecast biases are not sensitive to lead times but are seasonally dependent of all NMME-2 models. Overestimations are found in the Western U.S. in cooler seasons while underestimations are observed in the central regions of the U.S. in warmer seasons. The forecast skill of all individual NMME-2 models generally decreases as increases of lead times. The simple model averaging (SMA) of five NMME-2 models demonstrates a higher forecast skill than any individual NMME-2 models. Spatially, the highest forecast skill scores are observed at coastal areas in the Western U.S. with an one-week lead time. As compared to the historical resampled forecasts, NMME-2 also shows better performance in predicting extreme precipitation events above 99% percentiles and below 1% percentiles with higher probability of detections and lower false alarm ratios. Finally, the obtained results suggest the great potentials of NMME-2 precipitation forecasts in assisting ensemble hydrologic forecasts at the S2S scale over the CONUS.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Open Source Evaluation Framework for Solar Forecasting

The Solar Forecast Arbiter is an open-source evaluation framework for solar forecasting. The framework enables evaluations of solar irradiance, solar power, and net-load forecasts that are impartial, repeatable and auditable. The Solar Forecast Arbiter addresses stakeholder-informed use cases including evaluation of forecast skill, comparisons to reference data sets, private forecast trials, and evaluation of probabilistic forecast skill. The framework includes a data validation toolkit, reference data sources, data privacy protocols, and benchmark forecast capabilities for intra-hour and day ahead forecast horizons. Reports and metrics communicate the relative merits of the test and benchmark forecasts. The reports are created from standardized templates and include graphics for qualitatively evaluating deterministic and probabilistic forecasts and standard metrics for quantitatively evaluating forecasts. The Solar Forecast Arbiter is designed to support all solar forecasting stakeholders, including Solar Forecasting 2 Topic Area 2 and Topic Area 3 teams.

14 SOLAR ENERGY↗
DOE OSTI2023

Skillful Multi‐Month Predictions of Ecosystem Stressors in the Surface and Subsurface Ocean

Abstract Anthropogenic carbon emissions and associated climate change are driving rapid warming, acidification, and deoxygenation in the ocean, which increasingly stress marine ecosystems. On top of long‐term trends, short term variability of marine stressors can have major implications for marine ecosystems and their management. As such, there is a growing need for predictions of marine ecosystem stressors on monthly, seasonal, and multi‐month timescales. Previous studies have demonstrated the ability to make reliable predictions of the surface ocean physical and biogeochemical state months to years in advance, but few studies have investigated forecast skill of multiple stressors simultaneously or assessed the forecast skill below the surface. Here, we use the Community Earth System Model (CESM) Seasonal to Multiyear Large Ensemble (SMYLE) along with novel observation‐based biogeochemical and physical products to quantify the predictive skill of dissolved inorganic carbon (DIC), dissolved oxygen, and temperature in the surface and subsurface ocean. CESM SMYLE demonstrates high physical and biogeochemical predictive skill multiple months in advance in key oceanic regions and frequently outperforms persistence forecasts. We find up to 10 months of skillful forecasts, with particularly high skill in the Northeast Pacific (Gulf of Alaska and California Current Large Marine Ecosystems) for temperature, surface DIC, and subsurface oxygen. Our findings suggest that dynamical marine ecosystem prediction could support actionable advice for decision making.

Environmental Sciences & Ecology↗
DOE OSTI2023

Predicting spring phenology in deciduous broadleaf forests: NEON phenology forecasting community challenge

Accurate models are important to predict how global climate change will continue to alter plant phenology and near-term ecological forecasts can be used to iteratively improve models and evaluate predictions that are made a priori. The Ecological Forecasting Initiative's National Ecological Observatory Network (NEON) Forecasting Challenge, is an open challenge to the community to forecast daily greenness values, measured through digital images collected by the PhenoCam Network at NEON sites before the data are collected. For the first round of the challenge, which is presented here, we forecasted canopy greenness throughout the spring at eight deciduous broadleaf sites to investigate when, where, and for what model type phenology forecast skill is highest. A total of 192,536 predictions were submitted, representing eighteen models, including a persistence and a day of year mean null models. We found that overall forecast skill was highest when forecasting earlier in the greenup curve compared to the end, for shorter lead times, for sites that greened up earlier, and when submitting forecasts during times other than near budburst. The models based on day of year historical mean had the highest predictive skill across the challenge period. In this first round of the challenge, by synthesizing across forecasts, we started to elucidate what factors affect the predictive skill of near-term phenology forecasts.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

An ensemble data assimilation modeling system for operational outdoor microalgae growth forecasting

Microalgae have received increasing attention as a potential feedstock for biofuel or biobased products. Forecasting the microalgae growth is beneficial for managers in planning pond operations and harvesting decisions. This study proposed a biomass forecasting system comprised of the Huesemann Algae Biomass Growth Model (BGM), the Modular Aquatic Simulation System in Two Dimensions (MASS2), ensemble data assimilation (DA), and numerical weather prediction Global Ensemble Forecast System (GEFS) ensemble meteorological forecasts. The novelty of this study is to seek the use of ensemble DA to improve both BGM and MASS2 model initial conditions with the assimilation of biomass and water temperature measurements and consequently improve short-term biomass forecasting skills. This study introduces the theory behind the proposed integrated biomass forecasting system, with an application undertaken in pseudo-real-time in three outdoor ponds cultured with Chlorella sorokiniana in Delhi, California, United States. Results from all three case studies demonstrate that the biomass forecasting system improved the short-term (i.e., 7-day) biomass forecasting skills by about 60% on average, comparing to forecasts without using the ensemble DA method. Given the satisfactory performances achieved in this study, it is probable that the integrated BGM-MASS2-DA forecasting system can be used operationally to inform managers in making pond operation and harvesting planning decisions.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

Forecasting Remote Atmospheric Responses to Decadal Kuroshio Stability Transitions

Based on observational estimates and global ocean eddy-resolving coupled retrospective initialized predictions, we show that Kuroshio Extension variability affects rainfall variability along the west coast of North America. We show that the teleconnection between the current undulations and downstream rainfall can lead to improved subseasonal to seasonal predictions of precipitation over California, and we demonstrate that capturing these teleconnections requires coupled systems with sufficient ocean resolution (i.e., eddy-resolving), especially over time scales longer than one season. The improved forecast skill is diagnosed in terms of 35 years of retrospective initialized ensemble forecasts with an ocean eddy-resolving and an ocean eddy-parameterized coupled model. Not only does the ocean eddy-resolving model show sensitivity to Kuroshio Extension variability in terms of western North America precipitation, but the ocean eddy-resolving forecasts also show improved forecast skill compared to the ocean eddy-parameterized model. Additionally, the ocean eddy-parameterized coupled model shows no sensitivity to Kuroshio Extension variability. We also find near-decadal variability associated with a progression of a lower-tropospheric height dipole around the North Pacific and how these height anomalies lead to wind-driven Rossby waves that affect the eddy activity in the Kuroshio Extension with a time lag on the order of four years. This decadal-scale variability (~10 years) opens the possibility of multiyear predictability of western North American rainfall.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Evaluation of FluSight influenza forecasting in the 2021–22 and 2022–23 seasons with a new target laboratory-confirmed influenza hospitalizations

Accurate forecasts can enable more effective public health responses during seasonal influenza epidemics. For the 2021–22 and 2022–23 influenza seasons, 26 forecasting teams provided national and jurisdiction-specific probabilistic predictions of weekly confirmed influenza hospital admissions for one-to-four weeks ahead. Forecast skill is evaluated using the Weighted Interval Score (WIS), relative WIS, and coverage. Six out of 23 models outperform the baseline model across forecast weeks and locations in 2021–22 and 12 out of 18 models in 2022–23. Averaging across all forecast targets, the FluSight ensemble is the 2nd most accurate model measured by WIS in 2021–22 and the 5th most accurate in the 2022–23 season. Forecast skill and 95% coverage for the FluSight ensemble and most component models degrade over longer forecast horizons. In this work we demonstrate that while the FluSight ensemble was a robust predictor, even ensembles face challenges during periods of rapid change.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

Predictive Skill of Deep Learning Models Trained on Limited Sequence Data

In this report we investigate the utility of one-dimensional convolutional neural network (CNN) models in epidemiological forecasting. Deep learning models, especially variants of recurrent neural networks (RNNs) have been studied for influenza forecasting, and have achieved higher forecasting skill compared to conventional models such as ARIMA models. In this study, we adapt two neural networks that employ one-dimensional temporal convolutional layers as a primary building block temporal convolutional networks and simple neural attentive meta-learner for epidemiological forecasting and test them with influenza data from the US collected over 2010-2019. We find that epidemiological forecasting with CNNs is feasible, and their forecasting skill is comparable to, and at times, superior to, RNNs. Thus CNNs and RNNs bring the power of nonlinear transformations to purely data-driven epidemiological models, a capability that heretofore has been limited to more elaborate mechanistic/compartmental disease models.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2021

Real-time ensemble microalgae growth forecasting with data assimilation

Accurate short-range (e.g., 7-day) microalgae growth forecasts will be beneficial for both production and harvesting of microalgae. This study developed an operational microalgae growth forecasting system with ensemble data assimilation (DA). The forecasting system was validated against observed Monoraphidium minutum 26B-AM growth in two outdoor pond cultures located in Mesa, Arizona, U.S. We first examined the relative roles of uncertainty in the meteorological forecast and initial conditions (i.e., algal concentration at the time of forecast) in the microalgae 7-day forecast and found initial conditions dominated the microalgae forecasting skill, suggesting the importance of implementing DA to improve initial condition characterization. To correct the systematic bias in biomass simulations, we developed a particle filter with bias estimation (PFBE) DA method to estimate biases and correct the model forecast. We found the DA forecasting system could improve the 7-day microalgae forecasting skill by about 85% on average compared to model forecasts without DA. These results suggest the potential accuracy of biomass growth forecasts may be sufficient to inform real-time operational decisions, such as harvesting planning, for commercial-scale microalgae production.

59 BASIC BIOLOGICAL SCIENCES↗