Search NASA⌕ Search

SEARCH · Search NASA

Results for “forecast accuracy”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
NASA NTRS2010

Convective Weather Forecast Accuracy Analysis at Center and Sector Levels

This paper presents a detailed convective forecast accuracy analysis at center and sector levels. The study is aimed to provide more meaningful forecast verification measures to aviation community, as well as to obtain useful information leading to the improvements in the weather translation capacity models. In general, the vast majority of forecast verification efforts over past decades have been on the calculation of traditional standard verification measure scores over forecast and observation data analyses onto grids. These verification measures based on the binary classification have been applied in quality assurance of weather forecast products at the national level for many years. Our research focuses on the forecast at the center and sector levels. We calculate the standard forecast verification measure scores for en-route air traffic centers and sectors first, followed by conducting the forecast validation analysis and related verification measures for weather intensities and locations at centers and sectors levels. An approach to improve the prediction of sector weather coverage by multiple sector forecasts is then developed. The weather severe intensity assessment was carried out by using the correlations between forecast and actual weather observation airspace coverage. The weather forecast accuracy on horizontal location was assessed by examining the forecast errors. The improvement in prediction of weather coverage was determined by the correlation between actual sector weather coverage and prediction. observed and forecasted Convective Weather Avoidance Model (CWAM) data collected from June to September in 2007. CWAM zero-minute forecast data with aircraft avoidance probability of 60% and 80% are used as the actual weather observation. All forecast measurements are based on 30-minute, 60- minute, 90-minute, and 120-minute forecasts with the same avoidance probabilities. The forecast accuracy analysis for times under one-hour showed that the errors in intensity and location for center forecast are relatively low. For example, 1-hour forecast intensity and horizontal location errors for ZDC center were about 0.12 and 0.13. However, the correlation between sector 1-hour forecast and actual weather coverage was weak, for sector ZDC32, about 32% of the total variation of observation weather intensity was unexplained by forecast; the sector horizontal location error was about 0.10. The paper also introduces an approach to estimate the sector three-dimensional actual weather coverage by using multiple sector forecasts, which turned out to produce better predictions. Using Multiple Linear Regression (MLR) model for this approach, the correlations between actual observation and the multiple sector forecast model prediction improved by several percents at 95% confidence level in comparison with single sector forecast.

Wang, Yao↗
NASA NTRS1992

Initial weather regimes as predictors of numerical 30-day mean forecast accuracy

Thirty 30-day mean 500-mb-height anomaly forecasts generated by the NCAR Community Climate Model (CCM) for the year 1978 are examined in order to determine if the forecast accuracy can be estimated with the initial conditions. The initial weather regimes were defined in such a way that the regimes could discriminate between the best and the worst 30-day mean forecasts run from the initial fields in this data set. On the basis of the CCM experiments, it is suggested that the accuracy of numerical 30-day mean forecasts may depend upon the accuracy with which the cyclones and their interactions with the planetary scale are predicted early in the forecast cycle, and that this accuracy may depend upon the initial conditions.

Colucci, Stephen J.↗
NASA NTRS1983

The impact of satellite soundings and their distribution on forecast accuracy

The impact of satellite data is found to vary considerably from one map to another and from one geographic region to another. However, the impact of satellite data is generally small in comparison with forecast error. On the average, over land areas (where good conventional data permit accurate verification), the impact of satellite data on 48-hr forecasts of 500-mb height is seen to be neutral. Indirect numerical evidence is found, however, that satellite data has a marked positive impact in the lower and middle troposphere in ocean regions, especially those regions where conventional data are sparse. The impact over oceans is found to be especially noticeable in maps depicting the mean differences between the satellite and the conventional-data-only maps averaged over the period of the data set (January 1-14, 1980), both for analyses and for forecasts to 48 hours.

Thomasell, A., Jr.↗
NASA NTRS2011

Effect of Cross-Correlation on Geomagnetic Forecast Accuracies

Surface geomagnetic observation can determine up to degree L = 14 time-varying spherical harmonic coefficients of the poloidal magnetic field. Assimilation of these coefficients to numerical dynamo simulation could help us understand better the dynamical processes in the Earth's outer core, and to provide more accurate forecast of geomagnetic secular variations (SV). In our previous assimilation studies, only the poloidal magnetic field in the core is corrected by the observations in the analysis. Unobservable core state variables (the toroidal magnetic field and the core velocity field) are corrected via the dynamical equations of the geodynamo. Our assimilation experiments show that the assimilated core state converges near the CMB, implying that the dynamo state is strongly constrained by surface geomagnetic observations, and is pulled closer to the truth by the data. We are now carrying out an ensemble of assimilation runs with 1000 years of geomagnetic and archeo/paleo magnetic record. In these runs the cross correlation between the toroidal and the poloidal magnetic fields is incorporated into the analysis. This correlation is derived from the physical boundary conditions of the toroidal field at the core-mantle boundary (CMB). The assimilation results are then compared with those of the ensemble runs without the cross-correlation, aiming at understanding two fundamental issues: the effect of the crosscorrelation on (1) the convergence of the core state, and (2) the SV prediction accuracies. The constrained dynamo solutions will provide valuable insights on interpreting the observed SV, e.g. the near-equator magnetic flux patches, the core-mantle interactions, and possibly other geodynamic observables.

Kuang, Weijia↗
NASA NTRS1984

The effect of compact implicit differencing in a baroclinic primitive equations model

In an effort to improve forecast accuracy, the horizontal accuracy of gridpoint forecast models at a number of numerical weather prediction (NWP) centers was increased from second order to fourth order. The current GLAS model uses second-order explicit finite-difference formulas in the vertical and fourth-order explicit finite-difference formulas in the horizontal. Experiments are described which indicate the increased forecast accuracy gained by use of compact fourth-order finite differences in a simple baroclinic model. Tables show an increase in forecast accuracy by a factor of 100-150 by use of fourth-order instead of second-order vertical discretization.

Augenbaum, J. M.↗
NASA NTRS2004

An Evaluation of the Accuracy of Meteor Shower Forecasts

Brought into being by the recent Leonid meteor storms, meteor shower forecasts are now regarded by many spacecraft projects as necessary inputs into the planning of spacecraft operations. We compare the shower forecasts made by various researchers over the past six years to actual shower observations in an attempt to create an overall picture of forecast accuracy, specifically focusing on the three aspects most important to space vehicles: 1) the time of shower maximum, 2) the half-width (duration), and 3) the maximum Zenith Hourly Rate (ZHR). It will be noted that, while the times of maxima are generally predicted to within several minutes, the peak ZHRs are often overestimated and shower half-widths are frequently not even calculated. The difficulties involved in converting shower ZHRs into the meteoroid fluxes needed to assess spacecraft risk are also discussed.

Cooke, W.↗
DOE OSTI2022

Use of Physics to Improve Solar Forecast: Part II, Machine Learning and Model Interpretability

Machine learning (ML) models have been applied to forecast solar energy; however, they often lack clarity of interpretability and underlying physics. This work addresses such challenges by developing a hierarchy of ML models that gradually introduce predictors to improve the forecast accuracy based on a physics-based framework. Three ML models (ARIMA, LSTM, and XGBoost) are examined and compared with four physics-informed persistence models reported in Part I and the simple persistence model to assess the improvement of different models. The 7-year measurements at the U.S. Department of Energy's Atmospheric Radiation Measurement's Southern Great Plains Central Facility site are used for forecasts and evaluations. The results reveal that the step-by-step introduction of predictors leads to different improvements for models at different hierarchical levels. Comparison of the ML models with persistence models shows that LSTM and XGBoost outperform all the persistence models, with LSTM having the overall best performance; however, ARIMA underperforms the four physics-informed persistence models. This study demonstrates the importance and utility of incorporating physics into ML models in improving forecast accuracy by introducing a hierarchy of physics-based predictors, distinguishing predictor contributions, and enhancing the ML interpretability. The combined use of Global Horizontal Irradiance (GHI) and Direct Normal Irradiance (DNI) significantly improves the forecast accuracy compared to using individual irradiances alone because the pair contains more information on cloud-radiation interactions.

interpretability↗
NASA NTRS1980

Application of snowcovered area to runoff forecasting in selected basins of the Sierra Nevada, California

The author has identified the following significant results. Direct overlay onto 1:1,000,000 prints takes about one third the time of 1:500,000 zone transfer scope analysis using transparencies, but the consistency of the transparencies reduce the time for data analysis. LANDSAT data received on transparencies is better and more easily interpreted than the near real-time data from Quick Look, or imagery from other sources such as NOAA. The greatest potential for water supply forecasting is probably in improving forecast accuracy and in expanding forecast services during the period of snowmelt. Problems of transient snow line and uncertainties in future weather are the main reasons that snow cover area appears to offer little in water supply forecast accuracy improvement during the peroid snowpack accumulation.

Brown, A. J.↗
NASA NTRS2005

Applications of Principled Search Methods in Climate Influences and Mechanisms

Forest and grass fires cause economic losses in the billions of dollars in the U.S. alone. In addition, boreal forests constitute a large carbon store; it has been estimated that, were no burning to occur, an additional 7 gigatons of carbon would be sequestered in boreal soils each century. Effective wildfire suppression requires anticipation of locales and times for which wildfire is most probable, preferably with a two to four week forecast, so that limited resources can be efficiently deployed. The United States Forest Service (USFS), and other experts and agencies have developed several measures of fire risk combining physical principles and expert judgment, and have used them in automated procedures for forecasting fire risk. Forecasting accuracies for some fire risk indices in combination with climate and other variables have been estimated for specific locations, with the value of fire risk index variables assessed by their statistical significance in regressions. In other cases, the MAPSS forecasts [23, 241 for example, forecasting accuracy has been estimated only by simulated data. We describe alternative forecasting methods that predict fire probability by locale and time using statistical or machine learning procedures trained on historical data, and we give comparative assessments of their forecasting accuracy for one fire season year, April- October, 2003, for all U.S. Forest Service lands. Aside from providing an accuracy baseline for other forecasting methods, the results illustrate the interdependence between the statistical significance of prediction variables and the forecasting method used.

Glymour, Clark↗
NASA NTRSDate not supplied

Simulating Microwave and Radar Signals in Severe Weather Conditions

The development of radiative transfer simulators for radar and microwave signals, spanning a range of frequencies from 10 to 800 GHz, is paramount for enhancing weather forecasting accuracy, particularly for severe weather events. These tools facilitate the assimilation of microwave and radar observations into numerical weather prediction models, thereby improving accuracy of weather forecasts. Additionally, they directly simulate radar signals crucial for autonomous vehicle operation, with frequencies commonly used in radar systems such as 24, 74, 77, and 79 GHz. However, these frequencies are susceptible to weather phenomena like severe rain and snow, which can significantly impact vehicle safety and performance. These simulation tools, including radiative transfer simulators, serve a dual purpose. Firstly, they enhance forecasts for severe weather events, contributing to autonomous vehicle safety by providing early warnings and risk assessments. Secondly, they enable the direct simulation of radar signals in autonomous vehicle driving systems, allowing researchers and engineers to evaluate radar system performance under various weather conditions. In addition to the radar signal simulator, this abstract discusses the incorporation of advanced scattering properties developed using the discrete dipole approximation (DDA). The DDA technique enhances scattering calculations for frozen hydrometeors at microwave frequencies, thereby improving the accuracy of radar signal simulations and enabling more accurate assessments of radar system performance in adverse weather conditions. In summary, this abstract explores the development and utilization of comprehensive simulation tools, emphasizing their significance in simulating microwave and radar signals and improving weather forecasts. Special attention is given to the simulation of radar signals at critical frequencies for autonomous vehicle sensing and navigation, addressing challenges posed by severe weather phenomena and their effects on signal propagation and detection

Isaac Moradi↗
NASA NTRS1984

A quasi-Lagrangian diagnostic case study of the effect of satellite sounding data assimilation on model cyclone prediction

A quasi-Lagrangian diagnostic study was performed to compare weather forecast accuracy with and without use of satellite data. Attention was focused on the mass and angular momentum budgets of the cyclone. The diagnostic study isolated the effects of physical processes within a cyclone and the effects of transport processes moving with the cyclone. Satellite data furnished different initialization values for horizontal mass and eddy angular momentum. Colder temperatures and weaker stabilities in the NW quadrant lower layers in the budget volume and increased stability in the middle and lower layers of another quadrant were obtained when the satellite data were considered. The addition of satellite data was concluded to have enhanced the accuracy of the forecast.

Vergin, J. M.↗
NASA NTRS2019

Evaluating Flood Forecasting System Performance in Cambodia During the 2019 Flood Season

Every year, Cambodia experiences both flash and prolonged riverine flooding as a result of monsoon rains and typhoons. Floods occur frequently in various parts of the region, and result in significant economic cost. Flood forecasting systems are designed to mitigate economic and social impacts, enabling people to prepare for extreme events. However, in order for forecasts to be used effectively, an assessment of the available forecasting systems is needed. In addition, following the onset of flooding, satellite imagery is used to generate flood maps to aid in response efforts. This study demonstrates the performance of regional and global flood forecasting systems over the 2019 flood season. To do this, we assess the flood forecast accuracy at different forecast lead times through a series of forecast verification metrics at gauge locations in Cambodia. We then compare the flood forecast performance to Sentinel 1 flood maps produced by the Hydrological Remote Sensing Analysis of Floods (HYDRAFloods) tool currently being co-developed by SERVIR-Mekong in collaboration with the Myanmar Department of Disaster Management. This assessment of the flood forecasting systems’ performance and comparison to generated flood maps helps provide context to forecasters and disaster managers as they make improvements to their models. Additionally, these results provide support to forecast users as they evaluate the strengths and weaknesses of different systems for taking action.

Nauman, Claire M.↗
NASA NTRS1979

Impact of improved information on the structure of world grain trade

The benefits to be derived by the United States from improvements in global grain crop forecasting capability are discussed. The improvements in forecasting accuracy, which are a result of the use of satellite technology in conjunction with existing ground based estimating procedures are described. The degree of forecasting accuracy to be obtained from satellite technology is also examined. Specific emphasis is placed on wheat production in seven countries/regions: the United States; Canada; Argentina; Australia; Western Europe; the USSR; and all other countries in a group.

Source record↗
DOE OSTI2022

Towards physics-inspired data-driven weather forecasting: integrating data assimilation with a deep spatial-transformer-based U-NET in a case study with ERA5

Abstract. There is growing interest in data-driven weather prediction (DDWP), e.g., using convolutional neural networks such as U-NET that are trained on data from models or reanalysis. Here, we propose three components, inspired by physics, to integrate with commonly used DDWP models in order to improve their forecast accuracy. These components are (1) a deep spatial transformer added to the latent space of U-NET to capture rotation and scaling transformation in the latent space for spatiotemporal data, (2) a data-assimilation (DA) algorithm to ingest noisy observations and improve the initial conditions for next forecasts, and (3) a multi-time-step algorithm, which combines forecasts from DDWP models with different time steps through DA, improving the accuracy of forecasts at short intervals. To show the benefit and feasibility of each component, we use geopotential height at 500 hPa (Z500) from ERA5 reanalysis and examine the short-term forecast accuracy of specific setups of the DDWP framework. Results show that the spatial-transformer-based U-NET (U-STN) clearly outperforms the U-NET, e.g., improving the forecast skill by 45 %. Using a sigma-point ensemble Kalman (SPEnKF) algorithm for DA and U-STN as the forward model, we show that stable, accurate DA cycles are achieved even with high observation noise. This DDWP+DA framework substantially benefits from large (O(1000)) ensembles that are inexpensively generated with the data-driven forward model in each DA cycle. The multi-time-step DDWP+DA framework also shows promise; for example, it reduces the average error by factors of 2–3. These results show the benefits and feasibility of these three components, which are flexible and can be used in a variety of DDWP setups. Furthermore, while here we focus on weather forecasting, the three components can be readily adopted for other parts of the Earth system, such as ocean and land, for which there is a rapid growth of data and need for forecast and assimilation.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS1998

Evaluation of the 29-km Eta Model: Objective Verification at Three Selected Stations - Part 1

This paper describes an objective verification of the National Centers for Environmental Prediction (NCEP) 29-km eta model from May 1996 through January 1998. The evaluation was designed to assess the model's surface and upper-air point forecast accuracy at three selected locations during separate warm (May - August) and cool (October - January) season periods. In order to enhance sample sizes available for statistical calculations, the objective verification includes two consecutive warm and cool season periods. Systematic model deficiencies comprise the larger portion of the total error in most of the surface forecast variables that were evaluated. The error characteristics for both surface and upper-air forecasts vary widely by parameter, season, and station location. At upper levels, a few characteristic biases are identified. Overall however, the upper-level errors are more nonsystematic in nature and could be explained partly by observational measurement uncertainty. With a few exceptions, the upper-air results also indicate that 24-h model error growth is not statistically significant. In February and August 1997, NCEP implemented upgrades to the eta model's physical parameterizations that were designed to change some of the model's error characteristics near the surface. The results shown in this paper indicate that these upgrades led to identifiable and statistically significant changes in forecast accuracy for selected surface parameters. While some of the changes were expected, others were not consistent with the intent of the model updates and further emphasize the need for ongoing sensitivity studies and localized statistical verification efforts. Objective verification of point forecasts is a stringent measure of model performance, but when used alone, is not enough to quantify the overall value that model guidance may add to the forecast process. Therefore, results from a subjective verification of the meso-eta model over the Florida peninsula are discussed in the companion paper by Manobianco and Nutter. Overall verification results presented here and in part two should establish a reasonable benchmark from which model users and developers may pursue the ongoing eta model verification strategies in the future.

Nutter, Paul A.↗
DOE OSTI2020

Analysis of Random Forest Modeling Strategies for Multi-Step Wind Speed Forecasting

Although the random forest (RF) model is a powerful machine learning tool that has been utilized in many wind speed/power forecasting studies, there has been no consensus on optimal RF modeling strategies. This study investigates three basic questions which aim to assist in the discernment and quantification of the effects of individual model properties, namely: (1) using a standalone RF model versus using RF as a correction mechanism for the persistence approach, (2) utilizing a recursive versus direct multi-step forecasting strategy, and (3) training data availability on model forecasting accuracy from one to six hours ahead. These questions are investigated utilizing data from the FINO1 offshore platform and Atmospheric Radiation Measurement (ARM) Southern Great Plains (SGP) C1 site, and testing results are compared to the persistence method. At FINO1, due to the presence of multiple wind farms and high inter-annual variability, RF is more effective as an error-correction mechanism for the persistence approach. The direct forecasting strategy is seen to slightly outperform the recursive strategy, specifically for forecasts three or more steps ahead. Finally, increased data availability (up to ~8 equivalent years of hourly training data) appears to continually improve forecasting accuracy, although changing environmental flow patterns have the potential to negate such improvement. We hope that the findings of this study will assist future researchers and industry professionals to construct accurate, reliable RF models for wind speed forecasting.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Design considerations for a Digital Twin built to improve nitrification performance at a water resource recovery facility

A Digital Twin built around Activated Sludge Model No. 1 was deployed at a full-scale water resource recovery facility. Its design included a waste rate recommender system based on automatic scenario analyses, where influent loads and waste rates are varied to determine their impact on nitrification. At the same frequency as these scenario analyses, scheduled auto-calibrations allow for nitrifier maximum specific growth rate (μmax-NITO) soft sensing, the only kinetic parameter shown to require adjustment if the objective is aeration tank effluent ammonia forecasting accuracy. By integrating temperature forecasting over the next three sludge ages, this Digital Twin approach creates opportunities for advancing waste rate decisions in anticipation of seasonal temperature changes, optimizing ammonia control authority under varying influent loads, and furnishing valuable insights for future capital projects requiring nitrifier kinetic understanding and modelling.

54 ENVIRONMENTAL SCIENCES↗