Search NASA⌕ Search

SEARCH · Search NASA

Results for “forecast error”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 541 records · Page 30

Probabilistic Day-Ahead Forecasting Using an Analog Ensemble Approach for Wind Farm Grid Services

Wind resource assessment and wind power forecasting are used in research and industry to anticipate future power output at scales ranging from individual wind turbines to entire wind farms. Probabilistic day-ahead wind forecasting is useful for anticipating how a wind farm could potentially participate in the day-ahead market by providing upper and lower bounds for expected power generation, thus informing grid operators of its uncertainty. Understanding this uncertainty is part of a larger project focused on building a platform that combines efforts in weather forecasting, aerodynamic and economic modeling to create maximum value of a wind plant to better provide services to the grid. This effort is also known as the Atmosphere to Electrons to Grid (A2E2G) project. One method for producing a probabilistic forecast is through the analog ensemble approach (Delle Monache et al., 2011). This method leverages historical forecasts and their corresponding observations as a training data set from which future forecasts can be made. For some future forecast, the most similar historical forecasts (analogs) are identified on a regular time basis such as once per a 3-hour window. The most similar analogs, based on a metric such as root mean square error (RMSE), are recorded and their corresponding verifying observations are used as an ensemble member for this future forecast. Prior work in this area demonstrates improvements over raw Numerical Weather Prediction (NWP) forecasts and shows skill similar to techniques such as logistic regression and machine learning (Delle Monache et al., 2013; Alessandrini et al., 2015). Here, we take the High-Resolution Rapid Refresh model (HRRR) day-ahead forecast (0-36 hours) to create a probabilistic day-ahead forecast using an analog ensemble approach. The HRRR has an hourly temporal resolution, with a spatial resolution of 3 km. The 12 UTC HRRR model run is downloaded every day for one year from August 2019 - July 2020, with the first 11 months serving as a bank of analogs from which the forecasting algorithm can create a probabilistic forecast. Once downloaded, the original HRRR forecast is temporally interpolated to 5-minutes, aligning with both the temporal resolution of the observations as well as the timescale relevant for day-ahead power forecasts. The forecast is validated at the M2 tower at the Flatirons Campus of the National Renewable Energy Laboratory (NREL) at a typical wind turbine height of 80 m. Variables such as wind speed, wind direction, and turbulence intensity are incorporated into the probabilistic forecast model and weighted according to their relative importance to the forecast. Based on metrics such as mean bias error (MBE), mean absolute error (MAE), and root mean square error, the analog ensemble forecast outperforms the raw HRRR forecast during the testing period of July 2020. Figure 1 illustrates an example day-ahead forecast compared against the verifying observations. The general variability and ramps are captured throughout the day, with potential to further improve the analog ensemble model through machine learning techniques.

numerical weather prediction↗

Fitting dynamic models to the Geosat sea level observations in the tropical Pacific Ocean. I - A free wave model

Free, equatorially trapped sinusoidal wave solutions to a linear model on an equatorial beta plane are used to fit the Geosat altimetric sea level observations in the tropical Pacific Ocean. The Kalman filter technique is used to estimate the wave amplitude and phase from the data. The estimation is performed at each time step by combining the model forecast with the observation in an optimal fashion utilizing the respective error covariances. The model error covariance is determined such that the performance of the model forecast is optimized. It is found that the dominant observed features can be described qualitatively by basin-scale Kelvin waves and the first meridional-mode Rossby waves. Quantitatively, however, only 23 percent of the signal variance can be accounted for by this simple model.

Fu, Lee-Lueng↗

Variational Continuous Assimilation of TMI and SSM/I Rain Rates: Impact on GEOS-3 Hurricane Analyses and Forecasts

We describe a variational continuous assimilation (VCA) algorithm for assimilating tropical rainfall data using moisture and temperature tendency corrections as the control variable to offset model deficiencies. For rainfall assimilation, model errors are of special concern since model-predicted precipitation is based on parameterized moist physics, which can have substantial systematic errors. This study examines whether a VCA scheme using the forecast model as a weak constraint offers an effective pathway to precipitation assimilation. The particular scheme we exarnine employs a '1+1' dimension precipitation observation operator based on a 6-h integration of a column model of moist physics from the Goddard Earth Observing System (GEOS) global data assimilation system DAS). In earlier studies, we tested a simplified version of this scheme and obtained improved monthly-mean analyses and better short-range forecast skills. This paper describes the full implementation ofthe 1+1D VCA scheme using background and observation error statistics, and examines how it may improve GEOS analyses and forecasts of prominent tropical weather systems such as hurricanes. Parallel assimilation experiments with and without rainfall data for Hurricanes Bonnie and Floyd show that assimilating 6-h TMI and SSM/I surfice rain rates leads to more realistic storm features in the analysis, which, in turn, provide better initial conditions for 5-day storm track prediction and precipitation forecast. These results provide evidence that addressing model deficiencies in moisture tendency may be crucial to making effective use of precipitation information in data assimilation.

Hou, Arthur Y.↗

A Methodology for Quantifying Reliability Benefits From Improved Solar Power Forecasting in Multi-Timescale Power System Operations

Solar power forecasting improvements are mainly evaluated by statistical and economic metrics, and the practical reliability benefits of these forecasting enhancements have not yet been well quantified. This paper aims to quantify reliability benefits from solar power forecasting improvements. To systematically analyze the relationship between solar power forecasting improvements and reliability performance in power system operations, an expected synthetic reliability (ESR) metric is proposed to integrate state-of-the-art independent reliability metrics. The absolute value and standard deviation of area control errors (ACE), and the North American Electric Reliability Corporation Control Performance Standard 2 (CPS2) score are calculated through the multi-timescale scheduling simulation, including the day-ahead unit commitment (DU), real-time unit commitment (RU), real-time economic dispatch (RE), automatic generation control (AGC) sub-models. The absolute area control error in energy (AACEE), CPS2 violations, CPS2 score, and standard deviation of the raw ACE are all calculated and combined as the ESR metric. Numerical simulations show that the reliability benefits of multi-timescale power system operations are significantly increased due to the improved solar power forecasts.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Assessing Uncertainties and Approximations in Solar Heating of the Climate System

In calculating solar radiation, climate models make many simplifications, in part to reduce computational cost and enable climate modeling, and in part from lack of understanding of critical atmospheric information. Whether known errors or unknown errors, the community's concern is how these could impact the modeled climate. The simplifications are well known and most have published studies evaluating them, but with individual studies it is difficult to compare. Here, we collect a wide range of such simplifications in either radiative transfer modeling or atmospheric conditions and assess potential errors within a consistent framework on climate-relevant scales. We build benchmarking capability around a solar heating code (Solar-J) that doubles as a photolysis code for chemistry and can be readily adapted to consider other errors and uncertainties. The broad classes here include: use of broad wavelength bands to integrate over spectral features; scattering approximations that alter phase function and optical depths for clouds and gases; uncertainty in ice-cloud optics; treatment of fractional cloud cover including overlap; and variability of ocean surface albedo. We geographically map the errors in W m -2 using a full climate re-creation for January 2015 from a weather forecasting model. For many approximations assessed here, mean errors are ~2 W m -2 with greater latitudinal biases and are likely to affect a model’s ability to match the current climate state. Combining this work with previous studies, we make priority recommendations for fixing these simplifications based on both the magnitude of error and the ease or computational cost of the fix.

58 GEOSCIENCES↗

Improved Use of Satellite Imagery to Forecast Hurricanes

This project tested a novel method that uses satellite imagery to correct phase errors in the initial state for numerical weather prediction, applied to hurricane forecasts. The system was tested on hurricanes Guillermo (1997), Felicia (1997) and Iniki (1992). We compared the performance of the system with and without phase correction to a procedure that uses bogus data in the initial state, similar to current operational procedures. The phase correction keeps the hurricane on track in the analysis and is far superior to a system without phase correction. Compared to operational procedure, phase correction generates somewhat worse 3-day forecast of the hurricane track, but better forecast of intensity. It is believed that the phase correction module would work best in the context of 4-dimensional variational data assimilation. Very little modification to 4DVar would be required.

Louis, Jean-Francois↗

Predictability in the extended range

This paper describes the results of extended range predictability experiments using an efficient two-level spherical quasi-geostrophic model. The experiments have an initial rms doubling time of about two days. This growth rate, along with an initial error of about one-half the initial error of present operational models, produces an rms error equal to the climatological rms error and a correlation of 0.5 on about day 12 of the forecast. On the largest scales, this limiting point is reached shortly thereafter. The error continues to grow at a decreasing rate until at about 30 days the forecast skill is extremely small and comparable to the skill of a persistence forecast. Various time averages at various lags are examined for skill in the extended range. Filters that weighted most strongly in the initial forecast days provide increased skill.

Roads, John O.↗

Ground-based time-guidance algorithm for control of airplanes in a time-metered air traffic control environment: A piloted simulation study

The rapidly increasing costs of flight operations and the requirement for increased fuel conservation have made it necessary to develop more efficient ways to operate airplanes and to control air traffic for arrivals and departures to the terminal area. One concept of controlling arrival traffic through time metering has been jointly studied and evaluated by NASA and ONERA/CERT in piloted simulation tests. From time errors attained at checkpoints, airspeed and heading commands issued by air traffic control were computed by a time-guidance algorithm for the pilot to follow that would cause the airplane to cross a metering fix at a preassigned time. These tests resulted in the simulated airplane crossing a metering fix with a mean time error of 1.0 sec and a standard deviation of 16.7 sec when the time-metering algorithm was used. With mismodeled winds representing the unknown in wind-aloft forecasts and modeling form, the mean time error attained when crossing the metering fix was increased and the standard deviation remained approximately the same. The subject pilots reported that the airspeed and heading commands computed in the guidance concept were easy to follow and did not increase their work load above normal levels.

Knox, C. E.↗

Small-Footprint Lidar Estimations of Sagebrush Canopy Characteristics

Separating lidar returns for use in determining canopy height and shape in low-height vegetation is difficult because the vegetation canopy return is often close to the ground return in time and space. In addition, height underestimation is likely exacerbated in sparsely vegetated shrub ecosystems. This study compares lidar point-cloud data to sagebrush canopy characteristics measured in the field. It was determined that cumulative prediction error could account for as much as 35.6% of the average height and 37.4% of the average canopy area of shrubs sampled. When scaling from the individual shrub scale to coarser scales, prediction error averaged over a number of shrubs decreases as observation numbers increase. High density (in this case an average of 9.46 returns per m2), small footprint lidar (in this case a footprint diameter of 18 cm at nadir) may provide sufficient accuracy for characterizing sagebrush structure and cover and estimating biomass across landscapes.

54 ENVIRONMENTAL SCIENCES↗

Using MERRA Gridded Innovations for Quantifying Uncertainties in Analysis Fields and Diagnosing Observing System Inhomogeneities

MERRA is a NASA reanalysis for the satellite era using a major new version of the Goddard Earth Observing System Data Assimilation System Version 5 (GEOS-5). The project focuses on historical analyses of the hydrological cycle on a broad range of weather and climate time scales and places the NASA EOS suite of observations in a climate context. The characterization of uncertainty in reanalysis fields is a commonly requested feature by users of such data. While intercomparison with reference data sets is common practice for ascertaining the realism of the datasets, such studies typically are restricted to long term climatological statistics and seldom provide state dependent measures of the uncertainties involved. In principle, variational data assimilation algorithms have the ability of producing error estimates for the analysis variables (typically surface pressure, winds, temperature, moisture and ozone) consistent with the assumed background and observation error statistics. However, these "perceived error estimates" are expensive to obtain and are limited by the somewhat simplistic errors assumed in the algorithm. The observation minus forecast residuals (innovations) by-product of any assimilation system constitutes a powerful tool for estimating the systematic and random errors in the analysis fields. Unfortunately, such data is usually not readily available with reanalysis products, often requiring the tedious decoding of large datasets and not so-user friendly file formats. With MERRA we have introduced a gridded version of the observations/innovations used in the assimilation process, using the same grid and data formats as the regular datasets. Such dataset empowers the user with the ability of conveniently performing observing system related analysis and error estimates. The scope of this dataset will be briefly described. We will present a systematic analysis of MERRA innovation time series for the conventional observing system, including maximum-likelihood estimates of background and observation errors, as well as global bias estimates. Starting with the joint PDF of innovations and analysis increments at observation locations we propose a technique for diagnosing bias among the observing systems, and document how these contextual biases have evolved during the satellite era covered by MERRA.

da Silva, Arlindo↗

Forecasts of time averages with a numerical weather prediction model

Forecasts of time averages of 1-10 days in duration by an operational numerical weather prediction model are documented for the global 500 mb height field in spectral space. Error growth in very idealized models is described in order to anticipate various features of these forecasts and in order to anticipate what the results might be if forecasts longer than 10 days were carried out by present day numerical weather prediction models. The data set for this study is described, and the equilibrium spectra and error spectra are documented; then, the total error is documented. It is shown how forecasts can immediately be improved by removing the systematic error, by using statistical filters, and by ignoring forecasts beyond about a week. Temporal variations in the error field are also documented.

Roads, J. O.↗

Semiparametric modeling: Correcting low-dimensional model error in parametric models

In this paper, a semiparametric modeling approach is introduced as a paradigm for addressing model error arising from unresolved physical phenomena. Our approach compensates for model error by learning an auxiliary dynamical model for the unknown parameters. Practically, the proposed approach consists of the following steps. Given a physics-based model and a noisy data set of historical observations, a Bayesian filtering algorithm is used to extract a time-series of the parameter values. Subsequently, the diffusion forecast algorithm is applied to the retrieved time-series in order to construct the auxiliary model for the time evolving parameters. The semiparametric forecasting algorithm consists of integrating the existing physics-based model with an ensemble of parameters sampled from the probability density function of the diffusion forecast. To specify initial conditions for the diffusion forecast, a Bayesian semiparametric filtering method that extends the Kalman-based filtering framework is introduced. In difficult test examples, which introduce chaotically and stochastically evolving hidden parameters into the Lorenz-96 model, we show that our approach can effectively compensate for model error, with forecasting skill comparable to that of the perfect model.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Improving Global Reanalyses and Short Range Forecast Using TRMM and SSM/I-Derived Precipitation and Moisture Observations

Global reanalyses currently contain significant errors in the primary fields of the hydrological cycle such as precipitation, evaporation, moisture, and the related cloud fields, especially in the tropics. The Data Assimilation Office (DAO) at the NASA Goddard Space Flight Center has been exploring the use of tropical rainfall and total precipitable water (TPW) observations from the TRMM Microwave Imager (TMI) and the Special Sensor Microwave/ Imager (SSM/I) instruments to improve short-range forecast and reanalyses. We describe a "1+1"D procedure for assimilating 6-hr averaged rainfall and TPW in the Goddard Earth Observing System (GEOS) Data Assimilation System (DAS). The algorithm is based on a 6-hr time integration of a column version of the GEOS DAS, hence the "1+1"D designation. The scheme minimizes the least-square differences between the observed TPW and rain rates and those produced by the column model over the 6-hr analysis window. This 1+lD scheme, in its generalization to four dimensions, is related to the standard 4D variational assimilation but uses analysis increments instead of the initial condition as the control variable. Results show that assimilating the TMI and SSM/I rainfall and TPW observations improves not only the precipitation and moisture fields but also key climate parameters such as clouds, the radiation, the upper-tropospheric moisture, and the large-scale circulation in the tropics. In particular, assimilating these data reduce the state-dependent systematic errors in the assimilated products. The improved analysis also provides better initial conditions for short-range forecasts, but the improvements in forecast are less than improvements in the time-averaged assimilation fields, indicating that using these data types is effective in correcting biases and other errors of the forecast model in data assimilation.

Hou, Arthur Y.↗

Improving Global Reanalyses and Short-Range Forecast Using TRMM and SSM/I-Derived Precipitation and Moisture Observations

Global reanalyses currently contain significant errors in the primary fields of the hydrological cycle such as precipitation, evaporation, moisture, and the related cloud fields, especially in the tropics. The Data Assimilation Office (DAO) at the NASA Goddard Space Flight Center has been exploring the use of tropical rainfall and total precipitable water (TPW) observations from the TRMM Microwave Imager (TMI) and the Special Sensor Microwave/ Imager (SSM/I) instruments to improve short-range forecast and reanalyses. We describe a 1+1D procedure for assimilating 6-hr averaged rainfall and TPW in the Goddard Earth Observing System (GEOS) Data Assimilation System (DAS). The algorithm is based on a 6-hr time integration of a column version of the GEOS DAS, hence the 1+1D designation. The scheme minimizes the least-square differences between the observed TPW and rain rates and those produced by the column model over the 6-hr analysis window. This 1+1D scheme, in its generalization to four dimensions, is related to the standard 4D variational assimilation but uses analysis increments instead of the initial condition as the control variable. Results show that assimilating the TMI and SSW rainfall and TPW observations improves not only the precipitation and moisture fields but also key climate parameters such as clouds, the radiation, the upper-tropospheric moisture, and the large-scale circulation in the tropics. In particular, assimilating these data reduce the state-dependent systematic errors in the assimilated products. The improved analysis also provides better initial conditions for short-range forecasts, but the improvements in forecast are less than improvements in the time-averaged assimilation fields, indicating that using these data types is effective in correcting biases and other errors of the forecast model in data assimilation.

Hou, Arthur Y.↗

Comparative analysis of operational forecasts versus actual weather conditions in airline flight planning, volume 3

The impact of more timely and accurate weather data on airline flight planning with the emphasis on fuel savings is studied. This volume of the report discusses the results of Task 3 of the four major tasks included in the study. Task 3 compares flight plans developed on the Suitland forecast with actual data observed by the aircraft (and averaged over 10 degree segments). The results show that the average difference between the forecast and observed wind speed is 9 kts. without considering direction, and the average difference in the component of the forecast wind parallel to the direction of the observed wind is 13 kts. - both indicating that the Suitland forecast underestimates the wind speeds. The Root Mean Square (RMS) vector error is 30.1 kts. The average absolute difference in direction between the forecast and observed wind is 26 degrees and the temperature difference is 3 degree Centigrade. These results indicate that the forecast model as well as the verifying analysis used to develop comparison flight plans in Tasks 1 and 2 is a limiting factor and that the average potential fuel savings or penalty are up to 3.6 percent depending on the direction of flight.

Keitz, J. F.↗

Section on Observed Impacts on El Nino

Agricultural applications of El Nino forecasts are already underway in some countries and need to be evaluated or re-evaluated. For example, in Peru, El Nino forecasts have been incorporated into national planning for the agricultural sector, and areas planted with rice and cotton (cotton being the more drought-tolerant crop) are adjusted accordingly. How well are this and other such programs working? Such evaluations will contribute to the governmental and intergovernmental institutions, including the Inter-American Institute for Global Change Research and the US National Ocean and Atmospheric Agency that are fostering programs to aid the effective use of forecasts. As El Nino climate forecasting grows out of the research mode into operational mode, the research focus shifts to include the design of appropriate modes of utilization. Awareness of and sensitivity to the costs of prediction errors also grow. For example, one major forecasting model failed to predict the very large El Nino event of 1997, when Pacific sea-surface temperatures were the highest on record. Although simple correlations between El Nino events and crop yields may be suggestive, more sophisticated work is needed to understand the subtleties of the interplay among the global climate system, regional climate patterns, and local agricultural systems. Honesty about the limitations of an forecast is essential, especially when human livelihoods are at stake. An end-to-end analysis links tools and expertise from the full sequence of ENSO cause-and-effect processes. Representatives from many disciplines are needed to achieve insights, e.g, oceanographers and atmospheric scientists who predict El Nino events, climatologists who drive global climate models with sea-surface temperature predictions, agronomists who translate regional climate connections in to crop yield forecasts, and economists who analyze market adjustments to the vagaries of climate and determine the value of climate forecasts. Methods include historical studies to understand past patterns and to test hindcasts of the prediction tools, crop modeling, spatial analysis and remote sensing. This research involves expanding, deepening, and applying the understanding of physical climate to the fields of agronomy and social science; and the reciprocal understanding of crop growth and farm economics to climatology. Delivery of a regional climate forecast with no information about how the climate forecast was derived limits its effectiveness. Explanation of a region's major climate driving forces helps to place a seasonal forecast in context. Then, a useful approach is to show historical responses to previous El Nino events, and projections, with uncertainty intervals, of crop response from dynamic process crop growth models. Regional ID forecasts should be updated with real-time weather conditions. Since every El Nino event is different, it is important to track, report and advise on each new event as it unfolds. The stability of human enterprises depends on understanding both the potentialities and the limits of predictability. Farmers rely on past experience to anticipate and respond to fluctuations in the biophysical systems on which their livelihoods depend. Now scientists are improving their ability to predict some major elements of climate variability. The improvements in the reliability of El Nino forecasts are encouraging, but seasonal forecasts for agriculture are not, and will probably never be completely infallible, due to the chaotic nature of the climate system. Uncertainties proliferate as we extend beyond Pacific sea-surface temperatures to climate teleconnections and agricultural outcomes. The goal of this research is to shed as a clear light as possible on these inherent uncertainties and thus to contribute to the development of appropriate responses to El Nino and other seasonal forecasts for a range of stakeholders, which, ultimately, includes food consumers everywhere.

Rosenzweig, Cynthia↗

A novel data gaps filling method for solar PV output forecasting

This study proposes a modified gaps filling method, expanding the column mean imputation method and evaluated using randomly generated missing values comprising 5%, 10%, 15%, and 20% of the original data on power output. The XGBoost algorithm was implemented as a forecasting model using the original and processed datasets and two sources of solar radiation data, namely, Shortwave Radiation (SWR) from Advanced Himawari Imager 8 (AHI-8) and Surface Solar Radiation Downward (SSRD) from ERA5 global reanalysis data. Further, the accuracy of the two sets of forecasted power output was evaluated using Root Mean Square Error (RMSE) and Mean Absolute Error (MAE). Results show that by applying the proposed gap filling method and using SWR in forecasting solar photovoltaic (PV) output, the improvement in the RMSE and MAE values range from 12.52% to 24.30% and from 21.10% to 31.31%, respectively. Meanwhile, using SSRD, the improvement in the RMSE values range from 14.01% to 28.54% and MAE values from 22.39% to 35.53%. To further evaluate the accuracy of the proposed gap-filling method, the proposed method could be validated using different datasets and other forecasting methods. Future studies could also consider applying the said method to datasets with data gaps higher than 20%.

Energy & Fuels↗

Effects of data selection on the assimilation of AIRS data

The Atmospheric InfraRed Sounder (AIRS), flying aboard NASA's Earth Observing System (EOS) Aqua satellite with the Advanced Microwave Sounding Unit-A (AMSU-A), has been providing data for use in numerical weather prediction (NWP) and data assimilation systems (DAS) for over three years. The full AIRS data set is currently not transmitted in near-real-time (NRT) to the NWP centers. Instead, data sets with reduced spatial and spectral information are produced and made available in NRT. In this paper, we evaluate the use of different channel selections and error specifications. We achieved significant positive impact from the Aqua AIRS/AMSU-A combination in both hemispheres during our experimental time period of January 2003. The best results were obtained using a set of 156 channels that did not include any in the 6.7micron water vapor band. The latter have a large influence on both temperature and humidity analyses. If observation and background errors are not properly specified, the partitioning of temperature and humidity information from these channels will not be correct, and this can lead to a degradation in forecast skill. We found that changing the specified channel errors had a significant effect on the amount of data that entered into the analysis as a result of quality control thresholds that are related to the errors. However, changing the channel errors within a relatively small window did not significantly impact forecast skill with the 155 channel set. We also examined the effects of different types of spatial data reduction on assimilated data sets and NWP forecast skill. Whether we picked the center or the warmest AIRS pixel in a 3x3 array affected the amount of data ingested by the analysis but had a negligible impact on the forecast skill.

Joiner, Joanna↗