Search NASA⌕ Search

SEARCH · Search NASA

Results for “forecast error”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
NASA NTRS2022

Exploring the Relationship Between Temperature Forecast Errors and Earth System Variables

Accurate subseasonal weather forecasts, from two weeks up to a season, can help reduce costs and impacts related to weather and corresponding extremes. The quality of weather forecasts has improved considerably in recent decades as models represent more details of physical processes, and they benefit from assimilating comprehensive Earth observation data as well as increasing computing power. However, with ever–growing model complexity, it becomes increasingly difficult to pinpoint weaknesses in the forecast models’ process representations which is key to improving forecast accuracy. In this study, we use a comprehensive set of observation–based ecological, hydrological and meteorological variables to study their potential for explaining temperature forecast errors at the weekly time scale. For this purpose, we compute Spearman correlations between each considered variable and the forecast error obtained from the ECMWF subseasonal–to–seasonal (S2S) reforecasts at lead times of 1–6 weeks. This is done across the globe for the time period 2001–2017. The results show that temperature forecast errors globally are most strongly related with climate–related variables such as surface solar radiation and precipitation, which highlights the model’s difficulties in accurately capturing the evolution of the climate–related variables during the forecasting period. At the same time, we find particular regions in which other variables are more strongly related to forecast errors. For instance, in central Europe, eastern North America and southeastern Asia, vegetation greenness and soil moisture are relevant, while in western South America and central North America, circulation–related variables such as surface pressure relate more strongly with forecast errors. Overall, the identified relationships between forecast errors and independent Earth observations reveal promising variables on which future forecasting system development could focus by specifically considering related process representations and data assimilation.

Temperature↗
NASA NTRS2015

Spectral Analysis of Forecast Error Investigated with an Observing System Simulation Experiment

The spectra of analysis and forecast error are examined using the observing system simulation experiment (OSSE) framework developed at the National Aeronautics and Space Administration Global Modeling and Assimilation Office (NASAGMAO). A global numerical weather prediction model, the Global Earth Observing System version 5 (GEOS-5) with Gridpoint Statistical Interpolation (GSI) data assimilation, is cycled for two months with once-daily forecasts to 336 hours to generate a control case. Verification of forecast errors using the Nature Run as truth is compared with verification of forecast errors using self-analysis; significant underestimation of forecast errors is seen using self-analysis verification for up to 48 hours. Likewise, self analysis verification significantly overestimates the error growth rates of the early forecast, as well as mischaracterizing the spatial scales at which the strongest growth occurs. The Nature Run-verified error variances exhibit a complicated progression of growth, particularly for low wave number errors. In a second experiment, cycling of the model and data assimilation over the same period is repeated, but using synthetic observations with different explicitly added observation errors having the same error variances as the control experiment, thus creating a different realization of the control. The forecast errors of the two experiments become more correlated during the early forecast period, with correlations increasing for up to 72 hours before beginning to decrease.

OSSE↗
DOE OSTI2024

Solar and Wind Forecast Error Reserve Sharing in a Multi-Utility Region

As electricity systems transition to higher levels of solar and wind generation, electric system operators will likely need to hold additional reserves to manage solar and wind forecast error. Because solar and wind forecast errors tend to be weakly correlated across space, system operators can reduce their reserve requirements by sharing reserves. This paper examines the benefits of forecast error reserve sharing among balancing areas in the Southeastern United States, in scenarios in which solar and wind generation ranges from 34% to 65% of total generation. It finds that day-ahead forecast error reserve requirements increase linearly with growth in solar and wind generation capacity (6%-10% of total capacity), but that reserve sharing can significantly reduce these requirements (by 6%-29%). It finds that, in economic terms, the value of forecast error reserve sharing ($\$$0.09-$\$$1.24 billion per year, $\$$0.12-$\$$1.68/MWh of load across scenarios) tends to decline with higher levels of solar and wind generation, due to lower reserve and energy prices. Even with declines in reserve prices, forecast error reserve sharing can still provide substantial value, though with higher levels of solar, wind, and electricity storage this value is increasingly tied to avoiding scarcity prices.

14 SOLAR ENERGY↗
DOE OSTI2024

Solar and Wind Forecast Error Reserve Sharing in a Multi-Utility Region

As electricity systems transition to higher levels of solar and wind generation, electric system operators will likely need to hold additional reserves to manage the forecast error associated with these resources. Because wind and solar forecast errors tend to be poorly correlated across space, system operators can reduce their reserve requirements by sharing reserves. This paper examines the value of forecast error reserve sharing among balancing areas in the Southeast United States. It finds that forecast error reserve requirements increase linearly with growth in solar and wind generation capacity but that reserve sharing can significantly reduce physical (MW) reserve requirements (from 25%-26% to 18%-19% of average load in high solar scenarios). It finds that the value of forecast error reserve sharing declines with higher levels of solar and wind generation, due to lower wholesale energy and reserve prices. Even with declines in wholesale prices, forecast error reserve sharing can still provide substantial value (as much as $\$$400 million per year in a high solar scenario), though with higher levels of solar, wind, and electricity storage, this value is increasingly tied to avoiding scarcity prices. The results suggest the importance of coordinated capacity expansion planning for forecast error reserve sharing.

14 SOLAR ENERGY↗
NASA NTRS1997

Distortion Representation of Forecast Errors for Model Skill Assessment and Objective Analysis

We proposed a novel characterization of errors for numerical weather predictions. In its simplest form we decompose the error into a part attributable to phase errors and a remainder. The phase error is represented in the same fashion as a velocity field and is required to vary slowly and smoothly with position. A general distortion representation allows for the displacement and amplification or bias correction of forecast anomalies. Characterizing and decomposing forecast error in this way has two important applications, which we term the assessment application and the objective analysis application. For the assessment application, our approach results in new objective measures of forecast skill which are more in line with subjective measures of forecast skill and which are useful in validating models and diagnosing their shortcomings. With regard to the objective analysis application, meteorological analysis schemes balance forecast error and observational error to obtain an optimal analysis. Presently, representations of the error covariance matrix used to measure the forecast error are severely limited. For the objective analysis application our approach will improve analyses by providing a more realistic measure of the forecast error. We expect, a priori, that our approach should greatly improve the utility of remotely sensed data which have relatively high horizontal resolution, but which are indirectly related to the conventional atmospheric variables. In this project, we are initially focusing on the assessment application, restricted to a realistic but univariate 2-dimensional situation. Specifically, we study the forecast errors of the sea level pressure (SLP) and 500 hPa geopotential height fields for forecasts of the short and medium range. Since the forecasts are generated by the GEOS (Goddard Earth Observing System) data assimilation system with and without ERS 1 scatterometer data, these preliminary studies serve several purposes. They (1) provide a testbed for the use of the distortion representation of forecast errors, (2) act as one means of validating the GEOS data assimilation system and (3) help to describe the impact of the ERS 1 scatterometer data.

Hoffman, Ross N.↗
NASA NTRS1983

A stochastic-dynamic model for the spatial structure of forecast error statistics

The present investigation is concerned with the presentation of a simplified model of the spatial structure of forecast error statistics, a comparison of the model with actual numerical weather prediction results, and the extent to which simplifying assumptions made in the model are justified. A stochastic-dynamic model is derived for the spatial structure of the global atmospheric mass-field forecast error. The model states that the relative potential vorticity of the forecast error is random. The covariance function of the model's solutions is found to be governed by a simple deterministic equation. The agreement between the stochastic model and actual mass-field forecast errors fields for 12-36 h periods validates the assumptions on which the model is derived. Within this period, the difference between the potential voriticity fields of the atmosphere and of the numerical forecasts used in the comparison is well represented by white noise.

Balgovind, R.↗
DOE OSTI2021

Utilizing physics-based input features within a machine learning model to predict wind speed forecasting error

Machine learning is quickly becoming a commonly used technique for wind speed and power forecasting. Many machine learning methods utilize exogenous variables as input features, but there remains the question of which atmospheric variables are most beneficial for forecasting, especially in handling non-linearities that lead to forecasting error. This question is addressed via creation of a hybrid model that utilizes an autoregressive integrated moving-average (ARIMA) model to make an initial wind speed forecast followed by a random forest model that attempts to predict the ARIMA forecasting error using knowledge of exogenous atmospheric variables. Variables conveying information about atmospheric stability and turbulence as well as inertial forcing are found to be useful in dealing with non-linear error prediction. Streamwise wind speed, time of day, turbulence intensity, turbulent heat flux, vertical velocity, and wind direction are found to be particularly useful when used in unison for hourly and 3 h timescales. The prediction accuracy of the developed ARIMA–random forest hybrid model is compared to that of the persistence and bias-corrected ARIMA models. The ARIMA–random forest model is shown to improve upon the latter commonly employed modeling methods, reducing hourly forecasting error by up to 5 % below that of the bias-corrected ARIMA model and achieving an R 2 value of 0.84 with true wind speed.

17 WIND ENERGY↗
NASA NTRS1991

The behavior of forecast error covariances for a Kalman filter in two dimensions

The behavior of forecast error covariances in a fairly realistic setting is demonstrated via a Kalman filter algorithm. It is used to assimilate simulated data from the existing radiosonde network, from the demonstration network of 31 Doppler wind profilers in the central U.S., and from hypothetical radiometers located at five of the profiler sites. Some theoretical justification of the hypothesis advanced by Phillips (1982), and the hypothesis is used to formulate the model error covariance matrix required by the Kalman filter. The results show that assimilating the profiler wind data leads to a large reduction of forecast/analysis error in heights as well as in winds, over the profiler region and also downstream, when compared with the results of assimilating the radiosonde data alone. The forecast error covariance matrices that the Kalman filter calculates to obtain this error reduction differ considerably from those prescribed by the optimal interpolation schemes that are employed for data assimilation at operational centers.

Cohn, Stephen E.↗
NASA NTRS1998

Distortion Representation of Forecast Errors for Model Skill Assessment and Objective Analysis

We proposed a novel characterization of errors for numerical weather predictions. A general distortion representation allows for the displacement and amplification or bias correction of forecast anomalies. Characterizing and decomposing forecast error in this way has several important applications, including the model assessment application and the objective analysis application. In this project, we have focused on the assessment application, restricted to a realistic but univariate 2-dimensional situation. Specifically, we study the forecast errors of the sea level pressure (SLP), the 500 hPa geopotential height, and the 315 K potential vorticity fields for forecasts of the short and medium range. The forecasts are generated by the Goddard Earth Observing System (GEOS) data assimilation system with and without ERS-1 scatterometer data. A great deal of novel work has been accomplished under the current contract. In broad terms, we have developed and tested an efficient algorithm for determining distortions. The algorithm and constraints are now ready for application to larger data sets to be used to determine the statistics of the distortion as outlined above, and to be applied in data analysis by using GEOS water vapor imagery to correct short-term forecast errors.

Hoffman, Ross N.↗
NASA NTRS1985

The impact of data density and data error on the evolution of mesoscale forecast error

The effects of data density and data errors on the accuracy of mesoscale weather forecasts were assessed by simulations of a period in February 1979 when a snowstorm occurred along the U.S. eastern seaboard. The simulations were initiated every 12 hr and the growth of the rms error was tracked as a function of time and varying data densities. A uniform grid of instrumentation separation and perfect boundary conditions were assumed for all the simulations. A bias error representation was defined to account for systematic measurement errors. A brief summary of features of the somewhat complex storm is provided, together with the parameters of the 11 different simulations with high, medium and low data densities. The accuracy of the forecasts were proportional to the data density, although the error decreased for all the simulations over a 24 hr period. The density of the vertical data had a significant impact on the accuracy of the forecast, whereas the horizontal data density did not. Finally, pathways by which errors were transferred among variables were identified.

Warner, T. T.↗
NASA NTRS1990

Simulated forecast error and climate drift resulting from the omission of the upper stratosphere in numerical models

Using an NCAR community climate model, Version I, the forecast error growth and the climate drift resulting from the omission of the upper stratosphere are investigated. In the experiment, the control simulation is a seasonal integration of a medium horizontal general circulation model with 30 levels extending from the surface to the upper mesosphere, while the main experiment uses an identical model, except that only the bottom 15 levels (below 10 mb) are retained. It is shown that both random and systematic errors develop rapidly in the lower stratosphere with some local propagation into the troposphere in the 10-30-day time range. The random growth rate in the troposphere in the case of the altered upper boundary was found to be slightly faster than that for the initial-condition uncertainty alone. However, this is not likely to make a significant impact in operational forecast models, because the initial-condition uncertainty is very large.

Boville, Byron A.↗
NASA NTRS1996

Distortion Representation of Forecast Errors for Model Skill Assessment and Objective Analysis

We study a novel characterization of errors for numerical weather predictions. In its simplest form we decompose the error into a part attributable to phase errors and a remainder. The phase error is represented in the same fashion as a velocity field and will be required to vary slowly and smoothly with position. A general distortion representation allows for the displacement and a bias correction of forecast anomalies. In brief, the distortion is determined by minimizing the objective function by varying the displacement and bias correction fields. In the present project we use a global or hemispheric domain, and spherical harmonics to represent these fields. In this project we are initially focusing on the assessment application, restricted to a realistic but univariate 2-dimensional situation. Specifically we study the forecast errors of the 500 hPa geopotential height field for forecasts of the short and medium range. The forecasts are those of the Goddard Earth Observing System data assimilation system. Results presented show that the methodology works, that a large part of the total error may be explained by a distortion limited to triangular truncation at wavenumber 10, and that the remaining residual error contains mostly small spatial scales.

Hoffman, Ross N.↗
NASA NTRS2021

Asymmetry in Subseasonal Surface Air Temperature Forecast Error with Respect to Soil Moisture Initialization

Soil moisture (W) helps control evapotranspiration (ET), and ET variations can in turn have a distinct impact on 2-m air temperature (T2M), given that increases in evaporative cooling encourage reduced temperatures. Soil moisture is accordingly linked to T2M, and realistic soil moisture initialization has, in previous studies, been shown to improve the skill of subseasonal T2M forecasts. The relationship between soil moisture and evapotranspiration, however, is distinctly nonlinear, with ET tending to increase with soil moisture in drier conditions and to be insensitive to soil moisture variations in wetter conditions. Here, through an extensive analysis of subseasonal forecasts produced with a state-of-the-art seasonal forecast system, this nonlinearity is shown to imprint itself on T2M forecast error in the conterminous United States in two unique ways: (i) the T2M forecast bias (relative to independent observations) induced by a negative precipitation bias tends to be larger for dry initializations, and (ii) on average, the unbiased root-mean-square error (ubRMSE) tends to be larger for dry initializations. Such findings can aid in the identification of forecasts of opportunity; taken a step further, they suggest a pathway for improving bias correction and uncertainty estimation in subseasonal T2M forecasts by conditioning each on initial soil moisture state.

Air Temperature↗
DOE OSTI2023

Impacts of hybridization and forecast errors on the probabilistic capacity credit of batteries

Battery storage is increasingly identified as being among the least-cost mix of technologies in the evolving U.S. electricity mix. This study explores the marginal capacity credit of batteries using a probabilistic, reliability-based, effective firm capacity method, which we apply for multiple battery power ratings, durations, coupling types, deployment locations, and dispatch profiles within a test system that is based on the Texas Interconnection in the year 2024. We find that the capacity credits for all battery durations depend on their ability to predict the timing of reliability events. Even 1-2 h forecast errors - resulting in early or delayed battery discharging relative to the onset of a reliability event - lead pronounced capacity credit reductions, especially for 4-h duration batteries. Coupling batteries with solar mitigates the uncertainty associated with a shorter-duration battery's availability during reliability events, primarily due to the relatively high solar capacity credit in our test system. Coupled (or hybrid) system designs with oversized solar arrays, the ability to charge the coupled battery with grid energy, and larger batteries lead to the greatest capacity credit benefits of hybridization. We do not see evidence that the hybrid capacity credit exceeds the sum of the separate battery and solar capacity credits.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

Techno-economic implications and cost of forecasting errors in solar PV power production using optimized deep learning models

Accurate solar Photovoltaic (PV) power forecasting is important for enhancing both the performance and economic feasibility of PV systems. This study evaluates several deep learning models, including Dense Neural Networks (DNN), Long Short-Term Memory (LSTM), Convolutional Neural Networks (CNN), and a hybrid LSTMCNN model, for predicting PV power production one day in advance. Prior to optimization, the models exhibited relatively high errors, with the best model (DNN) achieving a Root Mean Square Error (RMSE) of 31.13 kW and a coefficient of determination (R 2 ) of 62.15 %. After employing Bayesian optimization, the LSTM-CNN model demonstrated the best performance, with the RMSE reduced to 9.79 kW and R 2 improved to 97.62 %, showcasing significant enhancement in predictive accuracy. Here, the economic evaluation considered three cases: rewards for underestimation (0.08 USD/kWh), no rewards, and penalties for both over-and underestimation (120 % of the utility tariff). In the rewards scenario, the LSTM-CNN model reduced the Levelized Cost of Electricity (LCOE) by 4 %, while in the penalty scenario, a backup diesel generator would have increased the LCOE by 49 %. Additionally, the LSTM-CNN model minimized financial losses, achieving the lowest penalties and maximizing net cash flow compared to other models, demonstrating its overall technical and economic superiority.

Deep learning↗
DOE OSTI2023

Impacts of Dispatch Strategies and Forecast Errors on the Economics of Behind-the-Meter PV-Battery Systems

To assess the economic value of batteries in hybrid PV-battery systems, one must create a dispatch profile for the battery. Many analyses of battery value assume perfect forecasts of PV generation and load, determining an upper limit on the value of the battery. Prior work that accounts for forecast uncer- tainty often does so in the context of a single dispatch algorithm, which does not provide a baseline for comparison. Furthermore, when multiple dispatch algorithms are assessed with uncertainty, the benefits considered are for diesel generation in a microgrid, not retail rate savings. This work addresses the gaps in the literature by comparing the performance of both heuristic and optimal dispatch algorithms for retail rate savings under forecast uncertainty, and provides comparisons of the robustness of these algorithms and their associated estimates of economic value. We find that using a perfect forecast can overestimate the value of hybrid PV-battery systems between 1% and 8% compared to the reality of using a day-ahead forecast, depending on the dispatch algorithm used. Thus, accounting for forecast uncertainty in system design and analysis will significantly improve the accuracy of modeled system values.

batteries↗
DOE OSTI2023

Impacts of Dispatch Strategies and Forecast Errors on the Economics of Behind-the-Meter PV-Battery Systems

To assess the economic value of batteries in hybrid PV-battery systems, one must create a dispatch profile for the battery. Many analyses of battery value assume perfect forecasts of PV generation and load, determining an upper limit on the value of the battery. Prior work that accounts for forecast uncertainty often does so in the context of a single dispatch algorithm, which does not provide a baseline for comparison. Furthermore, when multiple dispatch algorithms are assessed with uncertainty, the benefits considered are for diesel generation in a microgrid, not retail rate savings. This work addresses the gaps in the literature by comparing the performance of both heuristic and optimal dispatch algorithms for retail rate savings under forecast uncertainty, and provides comparisons of the robustness of these algorithms and their associated estimates of economic value. We find that using a perfect forecast can overestimate the value of hybrid PV-battery systems between 1% and 8% compared to the reality of using a day-ahead forecast, depending on the dispatch algorithm used. Thus, accounting for forecast uncertainty in system design and analysis will significantly improve the accuracy of modeled system values.

batteries↗
DOE OSTI2023

Impacts of Dispatch Strategies and Forecast Errors on the Economics of Behind-the-Meter PV-Battery Systems: Preprint

To assess the economic value of batteries in hybrid PV-battery systems, one must create a dispatch profile for the battery. Many analyses of battery value assume perfect forecasts of PV generation and load, determining an upper limit on the value of the battery. Prior work that accounts for forecast uncertainty often does so in the context of a single dispatch algorithm, which does not provide a baseline for comparison. Furthermore, when multiple dispatch algorithms are assessed with uncertainty, the benefits considered are for diesel generation in a microgrid, not retail rate savings. This work addresses the gaps in the literature by comparing the performance of both heuristic and optimal dispatch algorithms for retail rate savings under forecast uncertainty, and provides comparisons of the robustness of these algorithms and their associated estimates of economic value. We find that using a perfect forecast can overestimate the value of hybrid PV-battery systems between 1% and 8% compared to the reality of using a day-ahead forecast, depending on the dispatch algorithm used. Thus, accounting for forecast uncertainty in system design and analysis will significantly improve the accuracy of modeled system values.

batteries↗