Search NASA⌕ Search

SEARCH · Search NASA

Results for “forecast error”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 397 records · Page 22

Analyzing and Exploring Training Recipes for Large-Scale Transformer-Based Weather Prediction

Abstract The rapid rise of deep learning (DL) in numerical weather prediction (NWP) has led to a proliferation of models which forecast atmospheric variables with comparable or superior skill than traditional physics-based NWP. However, among these leading DL models, there is a wide variance in both the training settings and architecture used. Further, the lack of thorough ablation studies makes it hard to discern which components are most critical to success. In this work, we show that it is possible to attain high forecast skill even with relatively off-the-shelf architectures, simple training procedures, and moderate compute budgets. Specifically, we train a minimally modified Swin Transformer V2 (SwinV2) on ERA5 data and find that it attains superior skill in terms of mean-square errors of deterministic forecasts when compared against the European Centre for Medium-Range Weather Forecasts’ Integrated Forecasting System (IFS). Almost all DL–NWP systems share a core set of hyperparameters and design decisions. To aid and expedite future DL–NWP research, we present an in-depth, systematic exploration of different loss functions, model sizes and depths, patch sizes, and multistep training objectives. We also examine the model performance with metrics beyond the typical accuracy (ACC) and RMSE and investigate how the performance scales with model size. Through our open-source code, scoring pipelines, and models, we share our findings on key aspects of the training pipeline. These ablations reduce the necessity for expensive hyperparameter tuning and lower the barrier to entry for future DL–NWP research. Significance Statement This study investigates the potential of using large-scale transformer-based models for weather prediction, showing that it is possible to achieve high forecast accuracy with simpler, off-the-shelf architectures. By training a minimally modified SwinV2 transformer on ERA5 data, we show that the model achieves competitive forecast skill in terms of mean-square error for key variables, outperforming the European Centre for Medium-Range Weather Forecasts’ Integrated Forecasting System (IFS) at all lead times. Our findings suggest that effective training strategies, such as multistep fine-tuning and channel-weighted losses, significantly enhance the model’s performance. However, we also highlight that these improvements come with trade-offs in other areas, such as ensemble spread and high-frequency spatial detail. This work highlights the promise of deep learning in improving weather forecasts, which could lead to better preparedness and response to weather events, ultimately benefiting society by providing more reliable weather predictions.

Willard, Jared D. [Lawrence Berkeley National Labo↗

Scaling Observation Error for Optimal Assimilation of CCI SST Data into a Regional HYCOM EnOI System

South Africa currently possesses no operational ocean forecasting system for the purpose of predicting ocean state variables including temperature,salinity and velocity. Substantial initial efforts towards this goal have been made and resulted in a system using a regional Hybrid Coordinate Ocean Model (HYCOM) along with the Ensemble Optimal Interpolation (EnOI)assimilation scheme. Assimilating only sea surface temperature (SST) observations from the Operational Sea Surface Temperature and Sea Ice Analysis (OSTIA) product into the system resulted in a degraded forecast. Aiming to address this, Climate Change Initiative (CCI) SSTs are assimilated into the system in an effort to improve the forecast skill. Observation errors in the assimilated product are used in the EnOI to determine whether more confidence should be placed in the model or observations in producing the analysis, but overconfidence in observations can shock the model and result in failure. To tweak the impact of the assimilation, a scaling factor is applied in the assimilation code. A scaling factor of 25 was found to produce a favourable result with lowest mean root mean square error (RMSE;1.098°C) between the model and observations over time. Postulating the error to be overconfident, a floor value is introduced in order to set a minimum value for the observation error thereby reducing confidence in the observations. These experiments fared less favourably with a floor value of 0.5 and a scaling factor of 15 producing the best mean RMSE (1.118°C).

Luyt, Hermann↗

Simultaneous Estimation of Model State Variables and Observation and Forecast Biases Using a Two-Stage Hybrid Kalman Filter

In this paper, we present a two-stage hybrid Kalman filter to estimate both observation and forecast bias in hydrologic models, in addition to state variables. The biases are estimated using the discrete Kalman filter, and the state variables using the ensemble Kalman filter. A key issue in this multi-component assimilation scheme is the exact partitioning of the difference between observation and forecasts into state, forecast bias and observation bias updates. Here, the error covariances of the forecast bias and the unbiased states are calculated as constant fractions of the biased state error covariance, and the observation bias error covariance is a function of the observation prediction error covariance. In a series of synthetic experiments, focusing on the assimilation of discharge into a rainfall-runoff model, it is shown that both static and dynamic observation and forecast biases can be successfully estimated. The results indicate a strong improvement in the estimation of the state variables and resulting discharge as opposed to the use of a bias-unaware ensemble Kalman filter. Furthermore, minimal code modification in existing data assimilation software is needed to implement the method. The results suggest that a better performance of data assimilation methods should be possible if both forecast and observation biases are taken into account.

Data Assimilation↗

Near-Real-Time Forecast of Satellite-Based Soil Moisture Using Long Short-Term Memory with an Adaptive Data Integration Kernel

Nowcasts, or near-real-time (NRT) forecasts, of soil moisture based on the Soil Moisture Active and Passive (SMAP) mission could provide substantial value for a range of applications including hazards monitoring and agricultural planning. To provide such a NRT forecast with high fidelity, we enhanced a time series deep learning architecture, long short-term memory (LSTM), with a novel data integration (DI) kernel to assimilate the most recent SMAP observations as soon as they become available. The kernel is adaptive in that it can accommodate irregular observational schedules. Testing over the CONUS, this NRT forecast product showcases predictions with unprecedented accuracy when evaluated against subsequent SMAP retrievals. It showed smaller error than NRT forecasts reported in the literature, especially at longer forecast latency. The comparative advantage was due to LSTM’s structural improvements, as well as its ability to utilize more input variables and more training data. The DI-LSTM was compared to the original LSTM model that runs without data integration, referred to as the projection model here. We found that the DI procedure removed the autocorrelated effects of forcing errors and errors due to processes not represented in the inputs, for example, irrigation and floodplain/lake inundation, as well as mismatches due to unseen forcing conditions. The effects of this purely data-driven DI kernel are discussed for the first time in the geosciences. Furthermore, this work presents an upper-bound estimate for the random component of the SMAP retrieval error.

54 ENVIRONMENTAL SCIENCES↗

Predictions of zonal wind and angular momentum by the NMC medium-range forecast model during 1985-89

This paper investigates the quality of weather predictions of the atmosphere's relative angular momentum (M) made by the most recent version of the NMC medium-range forecast model (MRF88) during December 1985-1989. It was found that, compared with older versions of MRF, bias errors in the MRF88 forecasts of M became more prominent, while random errors were not affected. Both types of errors in the M forecasts could be traced to problems with forecasts in the zonal mean zonal wind in the tropics.

Rosen, Richard D.↗

Improving Multiday Solar Wind Speed Forecasts

We analyze the residual errors for the Wang-Sheeley-Arge (WSA) solar wind speed forecasts as a function of the photospheric magnetic field expansion factor (fp) and the minimum separation angle (d) in the photosphere between the footpoints of open field lines and the nearest coronal hole boundary. We find the map of residual speed errors are systematic when examined as a function of fp and d. We use these residual error maps to apply corrections to the model speeds. We test this correction approach using 3-day lead time speed forecasts for an entire year of observations and model results. Our methods can readily be applied to develop corrections for the remaining WSA forecast lead times which range from 1 to 7 days in 1-day increments. Since the solar wind density, temperature, and the interplanetary magnetic field strength all correlate well with the solar wind speed, the improved accuracy of solar wind speed forecasts enables the production of multiday forecasts of the solar wind density, temperature, pressure, and interplanetary field strength, and geophysical indices. These additional parameters would expand the usefulness of Air Force Data Assimilative Photospheric Flux Transport-WSA forecasts for space weather clients.

H A Elliot↗

Optimal interpolation and the Kalman filter

The estimation theory of stochastic-dynamic systems is described and used in a numerical study of optimal interpolation. The general form of data assimilation methods is reviewed. The Kalman-Bucy, KB filter, and optimal interpolation (OI) filters are examined for effectiveness in performance as gain matrices using a one-dimensional form of the shallow-water equations. Control runs in the numerical analyses were performed for a ten-day forecast in concert with the OI method. The effects of optimality, initialization, and assimilation were studied. It was found that correct initialization is necessary in order to localize errors, especially near boundary points. Also, the use of small forecast error growth rates over data-sparse areas was determined to offset inaccurate modeling of correlation functions near boundaries.

Cohn, S.↗

Investigation of Stochastic Unit Commitment to Enable Advanced Flexibility Measures for High Shares of Solar PV

As the share of solar photovoltaics (PV) in the power system increases, there is a growing need for flexibility from multiple, possibly interdependent sources to adjust to PV's variability, uncertainty, and diurnal dependence. This paper investigates how stochastic unit commitment leveraging probabilistic solar forecasts can support other flexibility measures under high solar shares. We consider two flexibility measures relevant to day-ahead scheduling: battery energy time-shifting and solar ancillary service provision. Unit commitment and economic dispatch simulations are conducted on a realistic test system based on Texas using day-ahead solar trajectories. The benefits of the two flexibility measures are pronounced when the instantaneous solar share is high, offering cost savings of 10%-20% in the spring. For a Texas-sized system, this translates to hundreds of millions of dollars in cost savings once the installed PV capacity enables instantaneous solar shares regularly exceeding 40%. Using probabilistic forecasts also greatly increases the reliability of upward reserve provision from solar PV, reducing unserved reserves by 50%-100%. Both day-ahead forecast resolution and errors can impact system reliability at high solar shares, but the stochastic formulation has significant value, mitigating reliability impacts on over-forecast days.

ancillary services↗

Short-Term Forecasting of Electric Loads Using Nonlinear Autoregressive Artificial Neural Networks with Exogenous Vector Inputs

Short-term load forecasting is crucial for the operations planning of an electrical grid. Forecasting the next 24 h of electrical load in a grid allows operators to plan and optimize their resources. The purpose of this study is to develop a more accurate short-term load forecasting method utilizing non-linear autoregressive artificial neural networks (ANN) with exogenous multi-variable input (NARX). The proposed implementation of the network is new: the neural network is trained in open-loop using actual load and weather data, and then, the network is placed in closed-loop to generate a forecast using the predicted load as the feedback input. Unlike the existing short-term load forecasting methods using ANNs, the proposed method uses its own output as the input in order to improve the accuracy, thus effectively implementing a feedback loop for the load, making it less dependent on external data. Using the proposed framework, mean absolute percent errors in the forecast in the order of 1% have been achieved, which is a 30% improvement on the average error using feedforward ANNs, ARMAX and state space methods, which can result in large savings by avoiding commissioning of unnecessary power plants. The New England electrical load data are used to train and validate the forecast prediction.

30 DIRECT ENERGY CONVERSION↗

Coupling computational fluid dynamics with the high resolution rapid refresh model for forecasting dynamic line ratings

This study looks at forecasted dynamic line ratings in southern Idaho by using data from the High Resolution Rapid Refresh (HRRR) model for forecasted weather conditions. The HRRR model can provide accurate 18- hour forecasts with a 15-minute temporal resolution. Typical static ratings used for over head transmission lines use overly conservative assumptions for local weather conditions, such as using the maximum solar irradiance or temperature measured during the summer combined with low wind speed for an entire season. The HRRR forecast model used here has a high spatial resolution to provide local forecast conditions along the entire length of a transmission line. The area that is of interest in this study is in southern Idaho spanning a total of 15,000 square kilometers. The forecasted weather data is coupled with a computational fluid dynamics model of the wind in the region to improve the resolution to an even finer scale for resolving local convective cooling rates on each individual transmission line midpoint span. This high fidelity approach can be used to find the minimum ampacity limit across all given midpoints of a transmission line to determine the absolute minimum to be used for the line rating. The ability to increase the line rating above the conservative static approach with the forecasted weather data provides a large potential to alleviate congestion and provide data for the utility market transactions. Furthermore, this study shows that for the region of interest, using forecasted weather data coupled with CFD modeling will calculate DLR ampacity rating above static about 90% of the time, with only a small relative error in the forecasted ampacity over time.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Occlusion-Perturbed Deep Learning for Probabilistic Solar Forecasting via Sky Images

Solar forecasting is shifting to the probabilistic paradigm due to the inherent uncertainty within the solar resource. Input uncertainty quantification is one of the widely used and best-performing ways to model solar uncertainty. However, compared to other sources of inputs, such as numerical weather prediction models, pure sky image-based probabilistic solar forecasting lags behind. In this research, an occlusion-perturbed convolutional neural network, named the PSolarNet, is developed. The PSolarNet provides very short-term deterministic forecasts, forecast scenarios, and probabilistic forecasts of the global horizontal irradiance from sky image sequences. Case studies based on 6 years of open-source data show that the developed PSolarNet is able to generate accurate 10-minute ahead deterministic forecasts with a 5.62% normalized root mean square error, realistic and diverse forecast scenarios with a 0.966 average correlation with the actual time series, and reliable and sharp probabilistic forecasts with a 2.77% normalized continuous ranked probability score.

Bayesian model averaging↗

Occlusion-Perturbed Deep Learning for Probabilistic Solar Forecasting via Sky Images: Preprint

Solar forecasting is shifting to the probabilistic paradigm due to the inherent uncertainty within the solar resource. Input uncertainty quantification is one of the widely used and best-performing ways to model solar uncertainty. However, compared to other sources of inputs, such as numerical weather prediction models, pure sky image-based probabilistic solar forecasting lags behind. In this research, an occlusion-perturbed convolutional neural network, named the PSolarNet, is developed. The PSolarNet provides very short-term deterministic forecasts, forecast scenarios, and probabilistic forecasts of the global horizontal irradiance from sky image sequences. Case studies based on 6 years of open-source data show that the developed PSolarNet is able to generate accurate 10-minute ahead deterministic forecasts with a 5.62% normalized root mean square error, realistic and diverse forecast scenarios with a 0.966 average correlation with the actual time series, and reliable and sharp probabilistic forecasts with a 2.77% normalized continuous ranked probability score.

Bayesian model averaging↗

Cosmological measurements with general relativistic galaxy correlations

We investigate the cosmological dependence and the constraining power of large-scale galaxy correlations, including all redshift-distortions, wide-angle, lensing and gravitational potential effects on linear scales. We analyze the cosmological information present in the lensing convergence and in the gravitational potential terms describing the so-called ''relativistic effects'', and we find that, while smaller than the information contained in intrinsic galaxy clustering, it is not negligible. We investigate how neglecting them does bias cosmological measurements performed by future spectroscopic and photometric large-scale surveys such as SKA and Euclid. We perform a Fisher analysis using the CLASS code, modified to include scale-dependent galaxy bias and redshift-dependent magnification and evolution bias. Our results show that neglecting relativistic terms, especially lensing convergence, introduces an error in the forecasted precision in measuring cosmological parameters of the order of a few tens of percent, in particular when measuring the matter content of the Universe and primordial non-Gaussianity parameters. The analysis suggests a possible substantial systematic error in cosmological parameter constraints. Therefore, we argue that radial correlations and integrated relativistic terms need to be taken into account when forecasting the constraining power of future large-scale number counts of galaxy surveys.

79 ASTRONOMY AND ASTROPHYSICS↗

Global analysis and forecasts of carbon monoxide on Mars

Spatial and temporal variations in the Martian carbon monoxide (CO) cycle have been investigated through combining Compact Reconnaissance Imaging Spectrometer for Mars (CRISM) retrievals of carbon monoxide mixing ratio and Mars Climate Sounder (MCS) temperature profiles with a Martian global circulation model (GCM) to produce the first global reanalysis of the Martian CO cycle. The reanalysis reduces the root mean square error between the forecast and CRISM CO retrievals by a factor of 2–4, dependent on the time of year. Forecasts initiated from the reanalysis show an improved match to standalone CRISM CO retrievals from northern winter solstice to northern summer solstice, indicating the benefit of data assimilation in simulating the CO abundance. The northern summer solstice CO minimum between 10°S – 50°S in the CRISM CO retrievals is found to be caused by a suppression of CO-enriched air leaking from the Hellas and Argyre basins, and is also strongly influenced by the amount of carbon dioxide condensing at the time. The GCM is the first reported to simulate the local CO minimum however it is weaker in strength, as CO-enriched air is also released northward in the area of Argyre basin in the GCM because of a shifted boundary between the southern polar vortex and mid-latitudes. The reanalysis, as a result of the assimilation of MCS temperature profiles, indicates that the polar vortex boundary is northward of Argyre basin and hence no northerly transport of CO-enriched air should be present over this location. Differences in local CO abundance between the reanalysis and GCM are also evident in spatio-temporal regions where no nearby CRISM CO retrievals are available as a result of alterations in local circulation and the sublimation/condensation of carbon dioxide through the combined assimilation of CRISM CO retrievals alongside MCS temperature profiles, illustrating how constraints can be imposed indirectly on the CO cycle through the powerful technique of data assimilation.

James A. Holmes↗

Robustness of Observing System Simulation Experiments

Observing System Simulation Experiments (OSSEs) are used to investigate the potential performance of proposed new instruments on numerical weather prediction (NWP). As OSSEs involve a framework in which the atmosphere and observations are all completely simulated, it is necessary to perform validation of the OSSE to ensure that it is sufficiently realistic to provide useful experimental results. A common issue that affects the forecast skill and observation impacts is the tendency of OSSEs to have insufficient model error compared to NWP in the real world. In this work, two versions of the National Aeronautics and Space Administration Global Modeling and Assimilation Office (NASA/GMAO) NWP OSSE framework are compared, with the newer framework having more simulated forecast model error than the older framework due to changes to the NWP system. The performance of the updated OSSE is validated against corresponding behavior of the same NWP system in the real world in terms of the simulated observations, the analysis increments, forecast error, and observation impacts. OSSE results of analysis and forecast impacts for three proposed new observation systems are also compared between the older and newer frameworks to evaluate the robustness of the OSSE experiments and the role of model error in observation impacts. These three new instruments are the Geostationary eXtended Observations (GeoXO) hyperspectral infrared sounder (GXS), the Midwave Infrared Sounding of Temperature and humidity in a Constellation for Winds (MISTiC Winds) atmospheric motion vectors, and additional Global Navigation Satellite System Radio Occultations (GNSS-RO).

Observing System Simulation Experiments↗

Machine learning based algorithms for uncertainty quantification in numerical weather prediction models

Complex numerical weather prediction models incorporate a variety of physical processes, each described by multiple alternative physical schemes with specific parameters. The selection of the physical schemes and the choice of the corresponding physical parameters during model configuration can significantly impact the accuracy of model forecasts. There is no combination of physical schemes that works best for all times, at all locations, and under all conditions. It is therefore of considerable interest to understand the interplay between the choice of physics and the accuracy of the resulting forecasts under different conditions. This paper demonstrates the use of machine learning techniques to study the uncertainty in numerical weather prediction models due to the interaction of multiple physical processes. The first problem addressed herein is the estimation of systematic model errors in output quantities of interest at future times, and the use of this information to improve the model forecasts. The second problem considered is the identification of those specific physical processes that contribute most to the forecast uncertainty in the quantity of interest under specified meteorological conditions. In order to address these questions we employ two machine learning approaches, random forests and artificial neural networks. The discrepancies between model results and observations at past times are used to learn the relationships between the choice of physical processes and the resulting forecast errors. Numerical experiments are carried out with the Weather Research and Forecasting (WRF) model. The output quantity of interest is the model precipitation, a variable that is both extremely important and very challenging to forecast. The physical processes under consideration include various micro-physics schemes, cumulus parameterizations, short wave, and long wave radiation schemes. The experiments demonstrate the strong potential of machine learning approaches to aid the study of model errors.

97 MATHEMATICS AND COMPUTING↗

Causes of Missing Snowmelt Following Drought

Abstract Water management in snowy mountainous regions hinges on forecasting snowmelt runoff. However, droughts are altering snowpack‐runoff relationships with ongoing debate about the driving mechanisms. For example, in 2021 in California, less than half of predicted streamflow arrived. Mechanisms proposed for this “missing” streamflow included changes in evapotranspiration ( ET ), rainfall, and subsurface moisture conditions. Here, we demonstrate that ET in drought years generates dry subsurface conditions that reduce runoff in subsequent years. A model including this legacy of depleted moisture storage reduced median error in 2021 forecasts from 60% to 20% at 15 minimally disturbed basins and from 18% to 2% at 6 water supply basins in the Sierra Nevada (basins range in area from 5 to 23,051 km 2 and mean annual precipitation from 814 to 1,549 mm). Our findings indicate that the relationship between snowpack and runoff will evolve as plant ecosystems respond to climate change and alter subsurface water storage dynamics.

Lapides, Dana A.↗

Massively Parallel Assimilation of TOGA/TAO and Topex/Poseidon Measurements into a Quasi Isopycnal Ocean General Circulation Model Using an Ensemble Kalman Filter

A massively parallel ensemble Kalman filter (EnKF)is used to assimilate temperature data from the TOGA/TAO array and altimetry from TOPEX/POSEIDON into a Pacific basin version of the NASA Seasonal to Interannual Prediction Project (NSIPP)ls quasi-isopycnal ocean general circulation model. The EnKF is an approximate Kalman filter in which the error-covariance propagation step is modeled by the integration of multiple instances of a numerical model. An estimate of the true error covariances is then inferred from the distribution of the ensemble of model state vectors. This inplementation of the filter takes advantage of the inherent parallelism in the EnKF algorithm by running all the model instances concurrently. The Kalman filter update step also occurs in parallel by having each processor process the observations that occur in the region of physical space for which it is responsible. The massively parallel data assimilation system is validated by withholding some of the data and then quantifying the extent to which the withheld information can be inferred from the assimilation of the remaining data. The distributions of the forecast and analysis error covariances predicted by the ENKF are also examined.

Keppenne, Christian L.↗