Search NASA⌕ Search

SEARCH · Search NASA

Results for “forecast accuracy”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Decreasing wind speed extrapolation error via domain-specific feature extraction and selection

Abstract. Model uncertainty is a significant challenge in the wind energy industry and can lead to mischaracterization of millions of dollars' worth of wind resources. Machine learning methods, notably deep artificial neural networks (ANNs), are capable of modeling turbulent and chaotic systems and offer a promising tool to produce high-accuracy wind speed forecasts and extrapolations. This paper uses data collected by profiling Doppler lidars over three field campaigns to investigate the efficacy of using ANNs for wind speed vertical extrapolation in a variety of terrains, and it quantifies the role of domain knowledge in ANN extrapolation accuracy. A series of 11 meteorological parameters (features) are used as ANN inputs, and the resulting output accuracy is compared with that of both standard log-law and power-law extrapolations. It is found that extracted nondimensional inputs, namely turbulence intensity, current wind speed, and previous wind speed, are the features that most reliably improve the ANN's accuracy, providing up to a 65 % and 52 % increase in extrapolation accuracy over log-law and power-law predictions, respectively. The volume of input data is also deemed important for achieving robust results. One test case is analyzed in depth using dimensional and nondimensional features, showing that the feature nondimensionalization drastically improves network accuracy and robustness for sparsely sampled atmospheric cases.

17 WIND ENERGY↗
DOE OSTI2021

A forecast-driven decision-making model for long-term operation of a hydro-wind-photovoltaic hybrid system

Hydro-wind-photovoltaic (PV) hybrid system has the potential to increase the integration of renewable energy sources into an existing grid. For the long-term operation of the system, due to the non-storable nature of wind and PV power, it is essentially to decide the optimal long-term carryover storage of cascade reservoirs. However, it remains a challenge due to high uncertainties of long-term forecasts and complicated hydraulic/electrical relationships between cascade reservoirs. Here in this study, a forecast-driven decision-making model is proposed for the hybrid system, which converts the multi-stage long-term operation process into a two-stage operation problem (including current stage and carryover stage) to avoid using longer-horizon forecast information with lower accuracy. First, the carryover stage energy surfaces (CESs) considering the forecast uncertainties of wind, solar and hydro resources are proposed to characterize carryover stage benefit quantitatively. Then a CESs-based forward decision-making optimization model is developed to guide the long-term operation of a hydro-wind-photovoltaic hybrid system. The applications in a hydro-wind-PV hybrid system of Yalong River basin results show that: compared with conventional operation, 1) power generation increases 9.03%; 2) in terms of the carryover storages control, the reservoir impounding and drawdown timing are delayed, and the drawdown depth is increased, which can be used to formulate better reservoir operation rules.

13 HYDRO ENERGY↗
DOE OSTI2022

Evaluation of individual and ensemble probabilistic forecasts of COVID-19 mortality in the United States

This paper compares the probabilistic accuracy of short-term forecasts of reported deaths due to COVID-19 during the first year and a half of the pandemic in the United States. Results show high variation in accuracy between and within stand-alone models and more consistent accuracy from an ensemble model that combined forecasts from all eligible models. This demonstrates that an ensemble model provided a reliable and comparatively accurate means of forecasting deaths during the COVID-19 pandemic that exceeded the performance of all of the models that contributed to it. This work strengthens the evidence base for synthesizing multiple models to support public-health action.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2022

Quantifying the Effect of Climate Change on Midlatitude Subseasonal Prediction Skill Provided by the Tropics

Abstract Subseasonal timescales (∼2 weeks–2 months) are known for their lack of predictability, however, specific Earth system states known to have a strong influence on these timescales can be harnessed to improve prediction skill (known as “forecasts of opportunity”). As the climate continues warming, it is hypothesized these states may change and consequently, their importance for subseasonal prediction may also be impacted. Here, we examine changes to midlatitude subseasonal prediction skill provided by the tropics under anthropogenic warming using artificial neural networks to quantify skill. The network is tasked to predict the sign of the 500 hPa geopotential height for historical and future time periods in the Community Earth System Model Version 2 ‐ Large Ensemble across the Northern Hemisphere at a 3 week lead using tropical precipitation. We show prediction skill changes substantially in key midlatitude regions and these changes appear linked to changes in seasonal variability with the largest differences in accuracy occurring during forecasts of opportunity.

Mayer, Kirsten J.↗
DOE OSTI2025

AI Improves the Accuracy, Reliability, and Economic Value of Continental‐Scale Flood Predictions

Accurate flood early warnings are critical to minimize damage and loss of life. Current large‐scale operational forecasting systems, however, have limited accuracy, description of uncertainty, and computational efficiency. While Artificial intelligence (AI) can address these limitations in principle, the accuracy and reliability of AI forecasts have thus far proven insufficient. Here we present a novel hybrid framework that integrates AI‐based machinery termed Errorcastnet (ECN) with the National Water Model (NWM) to showcase the potential of ensemble AI flood forecasts over the contiguous U.S. ECN boosts prediction accuracy four‐ to six‐fold across lead times of 1–10 days, while providing uncertainty quantification. It also outperforms Google's state‐of‐the‐art global AI model. ECN‐based forecasts offer superior economic value (up to four‐fold) for decision‐making as compared to those from NWM alone. ECN performs well in varied ecoregions, physiography, and land management conditions. The framework is computationally efficient, enabling national‐scale ensemble forecasts in minutes.

artificial intelligence↗
DOE OSTI2020

An empirical study of moving horizon closed-loop demand response scheduling

The potential of electricity-intensive chemical plants to engage in demand response (DR) initiatives in support of power grid operations has been the subject of many conceptual studies. In this work, using an industrially-relevant model of an air separation unit, we undertake an extensive simulation study of moving horizon (MH) rescheduling approaches, where we "close the scheduling loop" based on updated information regarding disturbances such as changes in electricity prices, ambient conditions and chemical product demand. Our study produces several unexpected findings regarding the non-periodic nature of rescheduling solutions, the impact of the accuracy of disturbance forecasts on the economics of DR scheduling, and on the interplay of simultaneously dealing with fluctuations on both the supply side (i.e., electricity prices) and the product demand side of the plant. Furthermore, we posit that the latter fluctuations pose significant limitations to the potential of a chemical plant to engage in DR.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

A Hybrid Framework Combining Model-Based and Data-Driven Methods for Hierarchical Decentralized Robust Dynamic State Estimation

This paper combines model-based and data-driven methods to develop a hierarchical, decentralized, robust dynamic state estimator (DSE). A two-level hierarchy is proposed, where the lower level consists of robust, model-based, decentralized DSEs. The state estimates sent from the lower level are received at the upper level, where they are filtered by a robust data-driven DSE after a principled sparse selection. This selection allows us to shrink the dimension of the problem at the upper level and hence significantly speed up the computational time. The proposed hybrid framework does not depend on the centralized infrastructure of the control centers; thus it can be completely embedded into the wide-area measurement systems. This feature will ultimately facilitate the placement of hierarchical decentralized control schemes at the phasor data concentrator locations. Also, the network model is not necessary; thus, a topology processor is not required. Finally, there is no assumption on the dynamics of the electric loads. The proposed framework is tested on the 2,000-bus synthetic Texas system, and shown to be capable of reconstructing the dynamic states of the generators with high accuracy, and of forecasting in the advent of missing data.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2022

Flow Forecasting on New England’s Great River Hydro (Abstract)

Great River Hydro (GRH) operates 13 generating stations and 3 storage-only reservoirs along the Upper Connecticut River, draining 6,266 square miles. Managing the reservoirs requires coordination over a couple days. So far seasonal flow-forecast and medium range probabilistic flow forecast during high flow conditions are leading to satisfactory management. Great River Hydro is seeking technical assistance to evaluate potential improvements in inflow forecasting and scheduling accuracy, particularly during short- to medium- duration periods (1-10 days). Improved accuracy is anticipated to enhance the efficiency with which GRH utilizes water, improving GRH’s ability to hit the best priced hours throughout the system and enhancing revenues as a result.

13 HYDRO ENERGY↗
DOE OSTI2020

Using Machine Learning to Predict Future Temperature Outputs in Geothermal Systems

Optimizing the power output, and economic value, of geothermal power plants over decades of operation is a major challenge in renewable energy. Optimizing the output requires the ability to predict the mass flow rates and the output temperatures of production wells based on the inputs of injection wells, as well as the time history of the system. Machine Learning (ML) that incorporates the known physics of geothermal systems is one possible solution to this challenge. In this work, we explore the ability of ML algorithms to predict future temperature outputs based on historical data. Considering the challenges with obtaining an empirical dataset from field data that is large enough to enable reliable ML, we propose an alternate approach: developing a high-fidelity reservoir model and using computational resources to build a dataset that enables ML. As a first step towards achieving this goal, we present preliminary results from applying ML to predict the temperature timeseries of simple modeled geothermal systems. We describe the application of relevant state-of-the-art ML approaches, such as the Long Short-Term Memory (LSTM) networks and Convolutional Neural Networks (CNN), to extract temporal structures in the model data. We assess the accuracy of the forecasts we obtain, compare the selected approaches, and share the lessons learned that would inform the process of training and utilizing ML algorithms for larger and more complex geothermal systems.

GEOTHERMAL ENERGY↗
DOE OSTI2024

Evaluating Ensemble Predictions of South Asian Monsoon Low Pressure System Genesis

Abstract Synoptic-scale vortices known as monsoon low pressure systems (LPSs) frequently produce intense precipitation and hydrological disasters in South Asia, so accurately forecasting LPS genesis is crucial for improving disaster preparedness and response. However, the accuracy of LPS genesis forecasts by numerical weather prediction models has remained unknown. Here, we evaluate the performance of two global ensemble models—the U.S. Global Ensemble Forecast System (GEFS) and the Ensemble Prediction System of the European Centre for Medium-Range Weather Forecasts (ECMWF)—in predicting LPS genesis during the years 2021–22. The GEFS successfully predicted about half the observed LPS genesis events 1–2 days in advance; the ECMWF model captured an additional 10% of observed genesis events. Both models had a false alarm ratio (FAR) of around 50% for 1–2-day lead times. In both ensembles, the control run typically exhibited a higher probability of detection (POD) of observed events and a lower FAR compared to the perturbed ensemble members. However, a consensus forecast, in which genesis is predicted when at least 20% of ensemble members forecast LPS formation, had POD values surpassing those of the control run for all lead times. Moreover, probabilistic predictions of genesis over the Bay of Bengal, where most LPSs form, were skillful, with the fraction of ensemble members predicting LPS formation over a 5-day lead time approximating the observed frequency of genesis, without any adjustment or bias correction.

Suhas, D. L.↗
DOE OSTI2021

Verification and validation of developed short-term forecasting models

Recent advancements in machine learning (ML) and artificial intelligence (AI) technologies provide an opportunity for leveraging data-driven algorithms to predict future nuclear power plant (NPP) operating conditions by using recorded plant process data. Successfully implementing these models can lead to cost-reducing, conditioned-based predictive maintenance through optimized maintenance schedules and a reduction of unnecessary maintenance activities. This report discusses the verification and validation of short-term forecasting processes (i.e., data cleaning, feature selection, model optimization, and forecasting) developed in previous reports. The verification and validation (V&V) process demonstrates the expected precision and accuracy when the ML model encounters new datasets from different systems. Shapley additive explanations were used as the primary means of feature selection across these different data set. Individual models were trained for each data set, then validated through a cross-validation procedure. In this report, two different ML models were tasked to predict variables from three different plant process data sets with varying prediction horizons. The results indicate that support vector regression (SVR) outperformed long short-term memory (LSTM) neural networks in regard to each data set and each prediction horizon in this study, but further tuning and optimization could improve long short-term memory results. However, each forecasting model showed reduced performance as the prediction horizon was extended from 1 hour to 1 day ahead. Research is ongoing to evaluate the optimal input variable space, which is based on a given set of process parameters, to further improve forecasting accuracy.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2025

UNCERTAINTY-AWARE DEEP LEARNING FRAMEWORK FOR FORECASTING COASTAL WATER LEVEL IN VIRGINIA BEACH

Coastal areas like Virginia Beach, USA, are increasingly vulnerable to flooding. To mitigate the impact of flooding, it is crucial for the City of Virginia Beach to have reliable 72-hour-ahead (3 days) forecasts of water levels at key gauge locations. To support this effort, several sensors have been installed throughout the city to monitor water levels and other environmental parameters such as wind speed, precipitation, and atmospheric pressure. Leveraging sensor data from one of these locations, we developed an uncertainty-aware deep learning model to forecast water levels. We employed deep quantile regression (DQR) to quantify variability in the predictions and examined the performance of three different model architectures. In addition to exclusively including historical data, we investigated the improvement wind forecasts provide to the accuracy of 72-hour-ahead water level predictions. The results show a twelvefold improvement in the flood forecast for a real flooding event.

Hasan, Mahmud [Thomas Jefferson National Accelerat↗
DOE OSTI2021

How New Mexico Leveraged a COVID-19 Case Forecasting Model to Preemptively Address the Healthcare Needs of the State: Quantitative Analysis

Prior to the COVID-19 pandemic, US hospitals relied on static projections of future trends for long-term planning and were only beginning to consider forecasting methods for short-term planning of staffing and other resources. With the overwhelming burden imposed by COVID-19 on the health care system, an emergent need exists to accurately forecast hospitalization needs within an actionable timeframe. Our goal was to leverage an existing COVID-19 case and death forecasting tool to generate the expected number of concurrent hospitalizations, occupied intensive care unit (ICU) beds, and in-use ventilators 1 day to 4 weeks in the future for New Mexico and each of its five health regions. We developed a probabilistic model that took as input the number of new COVID-19 cases for New Mexico from Los Alamos National Laboratory’s COVID-19 Forecasts Using Fast Evaluations and Estimation tool, and we used the model to estimate the number of new daily hospital admissions 4 weeks into the future based on current statewide hospitalization rates. The model estimated the number of new admissions that would require an ICU bed or use of a ventilator and then projected the individual lengths of hospital stays based on the resource need. By tracking the lengths of stay through time, we captured the projected simultaneous need for inpatient beds, ICU beds, and ventilators. We used a postprocessing method to adjust the forecasts based on the differences between prior forecasts and the subsequent observed data. Thus, we ensured that our forecasts could reflect a dynamically changing situation on the ground. Forecasts made between September 1 and December 9, 2020, showed variable accuracy across time, health care resource needs, and forecast horizon. Forecasts made in October, when new COVID-19 cases were steadily increasing, had an average accuracy error of 20.0%, while the error in forecasts made in September, a month with low COVID-19 activity, was 39.7%. Across health care use categories, state-level forecasts were more accurate than those at the regional level. Although the accuracy declined as the forecast was projected further into the future, the stated uncertainty of the prediction improved. Forecasts were within 5% of their stated uncertainty at the 50% and 90% prediction intervals at the 3- to 4-week forecast horizon for state-level inpatient and ICU needs. However, uncertainty intervals were too narrow for forecasts of state-level ventilator need and all regional health care resource needs. Real-time forecasting of the burden imposed by a spreading infectious disease is a crucial component of decision support during a public health emergency. Our proposed methodology demonstrated utility in providing near-term forecasts, particularly at the state level. This tool can aid other stakeholders as they face COVID-19 population impacts now and in the future.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2024

Validation of Weather Forecasting Products

Meteorological modeling plays a pivotal role in operational safety and emergency response at the Savannah River Site (SRS). This study focuses on verifying the Regional Atmospheric Modelling System (RAMS) Version 4.3 through Mean Bias Error (MBE) and Root Mean Square Error (RMSE) analyses of temperature, dew point, and wind speed over a decade. Using observed data from SRS, we assessed RAMS' accuracy, revealing seasonal biases and error trends. Results indicate RAMS' strengths in mild weather conditions but challenges during seasonal extremes and wind speed predictions due to measurement disparities. Future research aims to expand verification to other models and parameters, advocating for enhanced forecasting accuracy crucial for safeguarding personnel and community well-being.

42 ENGINEERING↗
DOE OSTI2025

Analyzing and Exploring Training Recipes for Large-Scale Transformer-Based Weather Prediction

Abstract The rapid rise of deep learning (DL) in numerical weather prediction (NWP) has led to a proliferation of models which forecast atmospheric variables with comparable or superior skill than traditional physics-based NWP. However, among these leading DL models, there is a wide variance in both the training settings and architecture used. Further, the lack of thorough ablation studies makes it hard to discern which components are most critical to success. In this work, we show that it is possible to attain high forecast skill even with relatively off-the-shelf architectures, simple training procedures, and moderate compute budgets. Specifically, we train a minimally modified Swin Transformer V2 (SwinV2) on ERA5 data and find that it attains superior skill in terms of mean-square errors of deterministic forecasts when compared against the European Centre for Medium-Range Weather Forecasts’ Integrated Forecasting System (IFS). Almost all DL–NWP systems share a core set of hyperparameters and design decisions. To aid and expedite future DL–NWP research, we present an in-depth, systematic exploration of different loss functions, model sizes and depths, patch sizes, and multistep training objectives. We also examine the model performance with metrics beyond the typical accuracy (ACC) and RMSE and investigate how the performance scales with model size. Through our open-source code, scoring pipelines, and models, we share our findings on key aspects of the training pipeline. These ablations reduce the necessity for expensive hyperparameter tuning and lower the barrier to entry for future DL–NWP research. Significance Statement This study investigates the potential of using large-scale transformer-based models for weather prediction, showing that it is possible to achieve high forecast accuracy with simpler, off-the-shelf architectures. By training a minimally modified SwinV2 transformer on ERA5 data, we show that the model achieves competitive forecast skill in terms of mean-square error for key variables, outperforming the European Centre for Medium-Range Weather Forecasts’ Integrated Forecasting System (IFS) at all lead times. Our findings suggest that effective training strategies, such as multistep fine-tuning and channel-weighted losses, significantly enhance the model’s performance. However, we also highlight that these improvements come with trade-offs in other areas, such as ensemble spread and high-frequency spatial detail. This work highlights the promise of deep learning in improving weather forecasts, which could lead to better preparedness and response to weather events, ultimately benefiting society by providing more reliable weather predictions.

Willard, Jared D. [Lawrence Berkeley National Labo↗
DOE OSTI2024

Enhancing Solar Power Forecasting with Regularized Constrained Quantile Regression Averaging and Bootstrapping Techniques

Probabilistic solar power forecasting (SPF) plays an essential role in optimizing power-grid operations by quantifying the forecast uncertainty. To improve the accuracy and robustness of probabilistic SPF, this paper introduces the regularized constrained quantile regression averaging (rCQRA) method to combine outputs from multiple PSPF models. In addition, a bootstrapping method was used to quantify model uncertainty, providing insights into the reliability and significance of each ensemble component. To evaluate its efficacy, the proposed rCQRA method is used to integrate four PSPF methods. The resulting SPF models are trained and validated using a real-world six-year dataset from a rooftop solar plant in the USA. The performance of the proposed rCQRA method is evaluated and compared with two benchmark methods under three categories of weather conditions. It is shown that the rCQRA method has superior performance in its forecast reliability, sharpness, and accuracy.

Ensemble learning, probabilistic solar power forec↗
DOE OSTI2020

Extreme Scale Infrasound Inversion and Prediction for Weather Characterization and Acute Event Detection

Accurate and timely weather predictions are critical to many aspects of society with a profound impact on our economy, general well-being, and national security. In particular, our ability to forecast severe weather systems is necessary to avoid injuries and fatalities, but also important to minimize infrastructure damage and maximize mitigation strategies. The weather community has developed a range of sophisticated numerical models that are executed at various spatial and temporal scales in an attempt to issue global, regional, and local forecasts in pseudo real time. The accuracy however depends on the time period of the forecast, the nonlinearities of the dynamics, and the target spatial resolution. Significant uncertainties plague these predictions including errors in initial conditions, material properties, data, and model approximations. To address these shortcomings, a continuous data collection occurs at an effort level that is even larger than the modeling process. It has been demonstrated that the accuracy of the predictions depends on the quality of the data and is independent to a certain extent on the sophistication of the numerical models. Data assimilation has become one of the more critical steps in the overall weather prediction business and consequently substantial improvements in the quality of the data would have transformational benefits. This paper describes the use of infrasound inversion technology, enabled through exascale computing, that could potentially achieve orders of magnitude improvement in data quality and therefore transform weather predictions with significant impact on many aspects of our society.

58 GEOSCIENCES↗
DOE OSTI2021

Simple, low-cost and accurate data-driven geophysical forecasting with learned kernels

Modelling geophysical processes as low-dimensional dynamical systems and regressing their vector field from data is a promising approach for learning emulators of such systems. We show that when the kernel of these emulators is also learned from data (using kernel flows, a variant of cross-validation), then the resulting data-driven models are not only faster than equation-based models but are easier to train than neural networks such as the long short-term memory neural network. In addition, they are also more accurate and predictive than the latter. When trained on geophysical observational data, for example the weekly averaged global sea-surface temperature, considerable gains are also observed by the proposed technique in comparison with classical partial differential equation-based models in terms of forecast computational cost and accuracy. When trained on publicly available re-analysis data for the daily temperature of the North American continent, we observe significant improvements over classical baselines such as climatology and persistence-based forecast techniques. Although our experiments concern specific examples, the proposed approach is general, and our results support the viability of kernel methods (with learned kernels) for interpretable and computationally efficient geophysical forecasting for a large diversity of processes.

58 GEOSCIENCES↗