Search NASA⌕ Search

SEARCH · Search NASA

Results for “forecast evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Error-Level-Controlled Synthetic Forecasts for Renewable Generation

Renewable energy resources, including solar and wind energy, play a significant role in sustainable energy systems. However, the inherent uncertainty and intermittency of renewable generation pose challenges to the safe and efficient operation of power systems. Recognizing the importance of short-term (hours ahead) renewable generation forecasting in power systems operation, it becomes crucial to address the potential inaccuracies in these forecasts. To systematically evaluate the performance of controllers in the presence of imperfect forecasts, we generate synthetic forecasts using actual renewable generation profiles (one from solar and one from wind). These synthetic forecasts incorporate different levels of statistical error, allowing us to control and manipulate the accuracy of the predictions. The primary objective is to employ synthetic forecasts with controlled yet realistic error levels to systematically investigate how controllers adapt to variations in forecast accuracy, providing valuable insights into their robustness and effectiveness under real-world conditions.

Array↗
NASA NTRS2017

Short-Term Forecasts Using NU-WRF for the Winter Olympics 2018

The NASA Unified-Weather Research and Forecasting model (NU-WRF) will be included for testing and evaluation in the forecast demonstration project (FDP) of the International Collaborative Experiment −PyeongChang 2018 Olympic and Paralympic (ICE-POP) Winter Games. An international array of radar and supporting ground based observations together with various forecast and now-cast models will be operational during ICE-POP. In conjunction with personnel from NASA's Goddard Space Flight Center, the NASA Short-term Prediction Research and Transition (SPoRT) Center is developing benchmark simulations for a real-time NU-WRF configuration to run during the FDP. ICE-POP observational datasets will be used to validate model simulations and investigate improved model physics and performance for prediction of snow events during the research phase (RDP) of the project The NU-WRF model simulations will also support NASA Global Precipitation Measurement (GPM) Mission ground-validation physical and direct validation activities in relation to verifying, testing and improving satellite-based snowfall retrieval algorithms over complex terrain.

forecasting↗
NASA NTRS1993

An evaluation of the real-time tropical cyclone forecast skill of the Navy Operational Global Atmospheric Prediction System in the western North Pacific

The paper evaluates the meteorological quality and operational utility of the Navy Operational Global Atmospheric Prediction System (NOGAPS) in forecasting tropical cyclones. It is shown that the model can provide useful predictions of motion and formation on a real-time basis in the western North Pacific. The meterological characteristics of the NOGAPS tropical cyclone predictions are evaluated by examining the formation of low-level cyclone systems in the tropics and vortex structure in the NOGAPS analysis and verifying 72-h forecasts. The adjusted NOGAPS track forecasts showed equitable skill to the baseline aid and the dynamical model. NOGAPS successfully predicted unusual equatorward turns for several straight-running cyclones.

Fiorino, Michael↗
DOE OSTI2026

FFTSF: Revisiting Sub-Seasonal Streamflow Forecasting with Simple Feedforward Network

Accurate short-to-subseasonal streamflow forecasts are vital for water management, including flood preparedness, drought mitigation, hydropower scheduling, and ecosystem protection. However, extending a forecast beyond a few days remains challenging due to complexity of hydrological processes. While recent self-attention based transformer architectures such as iTransformer have gained traction in time-series forecasting, these models suffer from several critical limitations: (1) significant computational overhead that scales quadratically with sequence length, (2) vulnerability to overfitting on limited hydrological datasets, (3) degraded performance on long-horizon forecasts due to attention decay, and (4) excessive architectural complexity that hampers interpretability and operational deployment. In this study, we propose a simple Feedforward Time Series Forecasting (FFTSF) network that directly addresses these limitations through its lightweight architecture and long-range forecasting capabilities. We evaluate FFTSF across 178 USGS stream gauges spanning diverse climate regimes by forecasting lead times of 1-, 7-, 14-, and 30-days. Our results demonstrate that FFTSF achieves competitive performance at short lead times (NSE of 0.778 for 1-day forecasts) while substantially outperforming complex baselines at longer forecast period, achieving the highest NSE (0.271) at 30-day forecasts with greater robustness and stability. For 30-day forecasts, FFTSF achieves a 71% improvement over NLinear, 57% improvement over DLinear and 12% improvement over the computationally intensive iTransformer while requiring fewer computational resources. Our findings reveal that architectural complexity is not necessary for hydrological forecasting, demonstrating that well-designed simple models can outperform attention mechanisms for subseasonal streamflow forecasting. The computational efficiency and consistent long-range performance of FFTSF make it suitable for water management applications where reliable extended forecasts are essential.

Krishnan Kutty Ambika, Anukesh [ORNL] (ORCID:00000↗
DOE OSTI2024

Evaluating the Accuracy of Machine Learning Forecasts

To improve the accuracy of forecasting in machine learning, we must investigate multiple machine learning models and see how accurately they can predict values after training. We used seven machine learning models to try and get more accurate predictions. The models that were used were ARIMA, SES, MLP, CART, LightGBM, and XGBoost. We used a processed dataset from a Terminal at LAX that had the number of people traveling through terminal X every hour in March from 2015-2019. We trained our models with the dates March 6 - March 19 to predict the value for March 20th and the hours 6:00 am to 6:00 pm since those are the most popular traveling hours. By using the different models, we had varying results of accuracy when estimating the amount of people traveling through terminal X on March 20th. We know that machine learning models are helpful for forecasting and by seeing how accurately these models can predict, we can see how forecasting can be helpful for other issues. Using these methods, airports can use forecasting to predict the amount of people coming in and out and can use these predictions to prepare their resource management, operational efficiency, and overall passenger experience.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1983

An evaluation of soundings, analyses and model forecasts derived from TIROS-N and NOAA-6 satellite data

TIROS-N and NOAA-6 temperature soundings over North America during three days in January 1980, and synoptic analyses and numerical-model forecasts derived from them, are compared with conventional data and analyses from NMC's limited-area fine-mesh model (LFM). The collocated sounding comparison revealed significant errors, especially near the surface and the tropopause. Satellite-derived thermal gradients were found to be weak, and thickness-analysis difference fields to propagate eastward, suggesting that sounding errors are correlated with synoptic patterns. The same pattern of anomalies is seen in the model forecasts. More detailed determinations of the correlation detected here could be used to optimize the assimilation of satellite soundings to conventional data.

Koehler, T. L.↗
NASA NTRS1990

A preliminary evaluation of short-term thunderstorm forecasting using surface winds at Kennedy Space Center

In 1987 NASA expanded its surface wind network onto the mainland west of Kennedy Space Center, increasing the network area from nearly 800 sq km to over 1600 sq km. Here, the results of this expansion are reported using three years of wind and lightning information collected during June, July, August, and September of 1987, 1988, and 1989. The divergence-lightning relationships and the importance of wind direction are addressed, and the verification is summarized.

Watson, Andrew I.↗
NASA NTRS2007

Evaluation of Integrating the Invasive Species Forecasting System to Support National Park Service Decisions on Fire Management Activities and Invasive Plant Species Control

The USGS and NASA, in conjunction with Colorado State University, George Mason University and other partners, have developed the Invasive Species Forecasting System (ISFS), a flexible tool that capitalizes on NASA's remote sensing resource to produce dynamic habitat maps of invasive terrestrial plant species across the United States. In 2006 ISFS was adopted to generate predictive invasive habitat maps to benefit noxious plant and fire management teams in three major National Park systems: The Greater Yellowstone Area (Yellowstone / Grand Tetons National Parks), Sequoia and Kings Canyon National Park, and interior Alaskan (between Denali, Gates of The Arctic and Yukon-Charley). One of the objectives of this study is to explore how the ISFS enhances decision support apparatus in use by National Park management teams. The first step with each park system was to work closely with park managers to select top-priority invasive species. Specific species were chosen for each study area based on management priorities, availability of observational data, and their potential for invasion after fire disturbances. Once focal species were selected, sources of presence/absence data were collected from previous surveys for each species in and around the Parks. Using logistic regression to couple presence/absence points with environmental data layers, the first round of ISFS habitat suitability maps were generated for each National Park system and presented during park visits over the summer of 2006. This first engagement provided a demonstration of what the park service can expect from ISFS and initiated the ongoing dialog on how the parks can best utilized the system to enhance their decisions related to invasive species control. During the park visits it was discovered that separate "expert opinion" maps would provide a valuable baseline to compare against the ISFS model output. Opinion maps are a means of spatially representing qualitative knowledge into a quantitative two-dimensional map. Furthermore, our approach combines the qualitative expert opinion habitat maps -- with the quantitative ISFS habitat maps in a difference map that shows where the two maps agree and disagree. The objective of the difference map is to help focus future field sampling and improve model results. This paper presents a demonstration of the habitat, expert opinion, and difference map for Yellowstone National Park.

Ma, Peter↗
DOE OSTI2022

A Physics-Based DNI Model for Advancing Solar Resource Assessment and Forecasting: Preprint

Direct Normal Irradiance (DNI) is one of the most used quantities to quantify the magnitude of solar energy resource. The concept of DNI is often interpreted differently for ground measurements and solar forecasting by numerical weather prediction (NWP) models, leading to substantial bias during evaluation of DNI forecasts especially under cloudy-sky conditions. To eliminate the bias, we use the Fast All-sky Radiation Model for Solar applications with DNI (FARMS-DNI) to provide a physics-based solution of solar radiation in the circumsolar region. The FARMS-DNI is implemented in the Weather Research and Forecasting model with solar extensions (WRF-Solar) to forecast day-ahead DNI in the north America. By comparing with conventional predictions from WRF-Solar and satellite observations from the National Solar Radiation Data Base (NSRDB), we found significant improvements in our prediction of DNI.

DNI↗
NASA NTRS2001

The Evaluation of the Regional Atmospheric Modeling System in the Eastern Range Dispersion Assessment System

The Applied Meteorology Unit (AMU) evaluated the Regional Atmospheric Modeling System (RAMS) contained within the Eastern Range Dispersion Assessment System (ERDAS). ERDAS provides emergency response guidance for Cape Canaveral Air Force Station and Kennedy Space Center operations in the event of an accidental hazardous material release or aborted vehicle launch. The RAMS prognostic data are available to ERDAS for display and are used to initialize the 45th Space Wing/Range Safety dispersion model. Thus, the accuracy of the dispersion predictions is dependent upon the accuracy of RAMS forecasts. The RAMS evaluation consisted of an objective and subjective component for the 1999 and 2000 Florida warm seasons, and the 1999-2000 cool season. In the objective evaluation, the AMU generated model error statistics at surface and upper-level observational sites, compared RAMS errors to a coarser RAMS grid configuration, and benchmarked RAMS against the nationally-used Eta model. In the subjective evaluation, the AMU compared forecast cold fronts, low-level temperature inversions, and precipitation to observations during the 1999-2000 cool season, verified the development of the RAMS forecast east coast sea breeze during both warm seasons, and examined the RAMS daily thunderstorm initiation and precipitation patterns during the 2000 warm season. This report summarizes the objective and subjective verification for all three seasons.

Case, Jonathan↗
NASA NTRS2008

Application of the NASA A-Train to Evaluate Clouds Simulated by the Weather Research and Forecast Model

The CloudSat Mission, part of the NASA A-Train, is providing the first global survey of cloud profiles and cloud physical properties, observing seasonal and geographical variations that are pertinent to evaluating the way clouds are parameterized in weather and climate forecast models. CloudSat measures the vertical structure of clouds and precipitation from space through the Cloud Profiling Radar (CPR), a 94 GHz nadir-looking radar measuring the power backscattered by clouds as a function of distance from the radar. One of the goals of the CloudSat mission is to evaluate the representation of clouds in forecast models, thereby contributing to improved predictions of weather, climate and the cloud-climate feedback problem. This paper highlights potential limitations in cloud microphysical schemes currently employed in the Weather Research and Forecast (WRF) modeling system. The horizontal and vertical structure of explicitly simulated cloud fields produced by the WRF model at 4-km resolution are being evaluated using CloudSat observations in concert with products derived from MODIS and AIRS. A radiative transfer model is used to produce simulated profiles of radar reflectivity given WRF input profiles of hydrometeor mixing ratios and ambient atmospheric conditions. The preliminary results presented in the paper will compare simulated and observed reflectivity fields corresponding to horizontal and vertical cloud structures associated with midlatitude cyclone events.

Molthan, Andrew L.↗
NASA NTRS2019

GMAO Seasonal Forecast Ensemble Exploration

GMAO Sub/Seasonal prediction system (S2S) has recently been upgraded. A complete set (1981-2016) of 9-months hindcasts for the previous and current versions (S2S-1.0 and S2S-2.1 respectively) allows for the evaluation of the forecast skill and a study of various characteristics of the ensemble forecasts in particular. We compared the intra-seasonal, interannual and intra-ensemble SST variability of the two systems against the observed. Focusing on the ENSO SST indices, we analyzed the consistency of the forecasts ensembles by studying rank histograms and comparing the ensemble spread with the standard error of the estimate.The S2S-2.1 ensemble appears to be more consistent with observations in Niño1+2 region compared to S2S-1.0, while in the central equatorial Pacific ocean this measure is comparably good for both systems. The S2S-1.0 system tends to be under dispersive, while the new system is under dispersive only at very short lead times, but tends to be over dispersive at long leads and for forecasts verifying in spring in Niño 3.4 region.Overall, the new system has greater skill in predicting ENSO. The evaluation techniques tested here will be applied for testing of the next generation sub/seasonal forecast system under development.

Borovikov, Anna↗
NASA NTRS2020

Dynamical Forecasts of Tropical Terrestrial Carbon Fluxes with the NASA S2S Retrospective Forecast System

Recent advances in the ability to predict climate anomalies at sub-seasonal to seasonal (S2S) timescales allow us to explore the possibility of forecasting carbon flux anomalies. Although carbon flux forecasting is a relatively new concept, it is potentially beneficial as it can help us better understand global and regional land-atmosphere carbon feedbacks associated with climate variations and can provide guidance for future field mission design. Here we evaluate the skill of forecasted terrestrial carbon anomalies generated from meteorological anomalies produced with the NASA Global Modeling and Assimilation Office (GMAO) S2S forecast system. We focus here on three representative time periods (the most recent 2015-2016 El Nino, 2011 La Nina, and 2014 as a neutral year), with each corresponding 9-month forecast comprising four ensemble members initialized in the preceding December. The meteorological variables produced by the GMAO forecast system were bias-corrected using a climatology derived from the Modern Era Retrospective analysis for Research and Applications, version 2 (MERRA-2) before being used to drive a suite of offline simulations with the NASA Catchment-CN terrestrial biosphere model, a model that computes water-energy-carbon dynamics. Forecasts are evaluated by comparing against satellite-driven estimates of gross primary production (GPP) and inverse model estimates of net carbon flux that incorporate satellite carbon dioxide measurements. We find that the restrospectively predicted carbon fluxes in the tropics reasonably reproduce the signs and magnitudes of the observed anomalies between the 2015-2016 El Nino and the 2011 La Nina for both net flux and GPP. For instance, for the El Nino period, the magnitude of the forecasted negative GPP anomaly in the South American tropics (which undergoes anomalously warm and dry conditions) agrees with the observed GPP anomaly at leads of up to three or four months. Overall, this study demonstrates potential skill in the forecast of biospheric carbon fluxes a few months in advance, a capability that could contribute to attribution studies focusing on carbon flux variations and support innovative observation strategies in the future.

Lee, Eunjee↗
NASA NTRS2019

Uncertainty of Observation Impact Estimation in an Adjoint Model Investigated with an Observing System Simulation Experiment

Adjoint models are often used to estimate the impact of different observations on short-term forecast skill. A common difficulty with the evaluation of short term forecast quality is the choice of verification fields. The use of self-analysis fields for verification is typical but incestuous, and introduces uncertainty due to biases and errors in the analysis field. In this study, an observing system simulation experiment (OSSE) is used to explore the uncertainty in adjoint model estimations of observation impact. The availability of the true state for verification in the OSSE framework in the form of the Nature Run allows calculation of the observation impact without the uncertainties present in self-analysis verification. These impact estimates are compared to estimates calculated using self-analysis verification. The Global Earth Observing System version 5 (GEOS-5) forecast model with Gridpoint Statistical Interpolation (GSI) is used with the National Aeronautics and Space Administration Global Modeling and Assimilation Office (NASA/GMAO) OSSE capability. The adjoint model includes moist processes, with total wet energy selected as the norm for evaluation of observation impacts. The results show that there are measurable but small discrepancies in the adjoint model estimation of observation impact. In general, observations of temperature and winds tend to have overestimated impacts with self-analysis verification, while observations of humidity and moisture-affected observations tend to have underestimated impacts. The small magnitude of the differences in impact estimates supports the robustness of the adjoint method of estimating observation impacts.

N C Prive↗
NASA NTRSDate not supplied

Global Landslide Hazard Assessment for Situational Awareness (LHASA) Version 2: New Activities and Future Plans

A remote sensing-based system has been developed to characterize the potential for rainfall-triggered landslides across the globe in near real-time. The Landslide Hazard Assessment for Situational Awareness (LHASA) model uses a decision tree framework to combine a static susceptibility map derived from information on slope, rock characteristics, forest loss, distance to fault zones and distance to road networks with satellite precipitation estimates from the Global Precipitation Measurement (GPM) mission. Since 2016, the LHASA model has been providing near real-time and retrospective estimates of potential landslide activity. Results of this work are available at https://landslides.nasa.gov. In order to advance LHASA’s capabilities to characterize landslide hazards and impacts dynamically, we have implemented a new approach that leverages machine learning, new parameters, and new inventories. LHASA 2.0 uses the XGBoost machine learning model to bring in dynamic variables as well as additional static variables to better represent landslide hazard globally. Global rainfall forecasts are also being evaluated to provide a 1-3 day forecast of potential landslide activity. Additional factors such as recent seismicity and burned areas are also being considered to represent the preconditioning or changing interactions with subsequent rainfall over affected areas. A series of parameters are being tested within this structure using NASA’s Global Landslide Catalog as well as many other event-based and multi-temporal inventories mapped by the project team or provided by project partners. In addition to estimates of landslide hazard, LHASA Version 2 will incorporate dynamic estimates of exposure including population, roads and infrastructure to highlight the potential impacts that rainfall-triggered landslides. The ultimate goal of LHASA Version 2.0 is to approximate the relative probabilities of landslide hazard and exposure across different space and time scales to inform hazard assessment retrospectively over the past 20 years, in near real-time, and in the future. In addition to the hazard. This presentation will outline the new activities for LHASA Version 2.0 and present some next steps for this system.

Dalia Kirschbaum↗
NASA NTRS1988

Providing operational guidance for the development of sea breeze thunderstorms at the Kennedy Space Center - An experiment using a mesoscale numerical model

The effectiveness of a mesoscale numerical model to provide improved local forecast guidance is evaluated with respect to sea breeze convection storms at the Kennedy Space Center. The model and operational forecast guidance production are described. A case study is presented for sea breeze convection storms and lightning events on July 1, 1986. It is found that the mesoscale numerical model outperforms purely subjective predictions of sea breeze convection. The range of applications for the model are considered.

Lyons, Walter A.↗