Search NASA⌕ Search

SEARCH · Search NASA

Results for “forecast evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

A Two-Level Model Predictive Control-Based Approach for Building Energy Management including Photovoltaics, Energy Storage, Solar Forecasting and Building Loads

This paper uses a two-level model predictive control-based approach for the coordinated control and energy management of an integrated system that includes photovoltaic (PV) generation, energy storage, and building loads. Novel features of the proposed local controller include (1) the ability to simultaneously manage building loads and energy storage to achieve different operational objectives such as energy efficiency, economic cost efficiency, demand response and grid optimization through the design of specific power trajectory tracking performance functionals, (2) an energy trim function that minimizes the impact of solar forecasting errors on system performance, and (3) the design of a state of charge controller that uses day-ahead forecast of solar power and building loads to intialize energy storage at the start of each day. The local controller is tested in simulation using an exemplary system with PV generation, energy storage and dispatchable building loads. Two sample days with different PV forecasts and multiple case scenarios are considered, and the performance of the algorithm in managing the real and reactive net building load trajectories and the ramp rate of PV injections into the utility network are evaluated. The simulations are based on actual forecasted and measured PV data, and the results show that the local controller meets the tracking requirements for real and reactive power within the operating constraints of the building.

14 SOLAR ENERGY↗
DOE OSTI2022

Coupling localized Noah-MP-Crop model with the WRF model improved dynamic crop growth simulation across Northeast China

Croplands play a critical role in regulating the energy and moisture exchanges between the land surface and atmosphere. However, the interactions between cropland and climate are usually poorly represented due to a lack of detailed representation in crop types and field management. Here, we coupled the Noah-MP-Crop model with the state-of-the-art Weather Research and Forecasting (WRF) model to explore and evaluate the crop growth dynamics in response to climate variations across Northeast China. The default parameters of the crop model were not exactly suitable for the agricultural ecosystems in Northeast China. The detailed cropland distribution, and crop phenology parameters including growing degree days (GDD) and planting (harvesting) date were first created using multi-source remote sensing products and reanalysis data, and was then successfully used to simulate the growth and yield for corn and soybean and associated energy exchanges. We also optimized and calibrated other crop parameters using the time-series of the Moderate Resolution Imaging Spectroradiometer (MODIS) land surface products. The modified crop model substantially improved the simulation of crop growth, plant physiology, and biomass accumulation for both corn and soybean. Coupling the localized dynamic crop model into the WRF led to considerable decreases in the simulated mean-absolute-errors (MAEs) and biases of the leaf area index, evapotranspiration, and gross primary production compared with the MODIS observed values. Compared with the statistical yield from each province, the modified crop model underestimated the corn yield from 11.1% to 48.6%, whereas overestimated the soybean yield from 16.5% to 162.6%.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Computational Experiment Design for Operations Model Simulation

Computer simulations that demonstrate the value of novel approaches are crucial to developing more flexible and robust power systems operations with high penetrations of renewable energy at multiple geographic and temporal scales. However, optimization-based simulations that depend on forecast data often face challenges in evaluating performance, reproducing results, and testing under realistic simulation scenarios. In this paper, we develop scientific computing best-practices for the validation and reproduction of power systems operational models. We then employ two case studies to demonstrate the proposed validation and reproduction framework.

MATHEMATICS AND COMPUTING,POWER TRANSMISSION AND D↗
DOE OSTI2023

A review of risk and uncertainty assessment for geologic carbon storage

Carbon capture, utilization, and storage (CCUS) in geological formations play a key role in mitigating anthropogenic CO 2 emissions and achieving the aggressive goal of net-zero greenhouse gas emissions. Risk and uncertainty assessment is crucial for ensuring the safety and reliability of geologic carbon storage (GCS) by evaluating CO 2 migration in subsurface, forecasting potential leakage and induced seismicity risks, and optimizing operational and monitoring plans. In this review, the use and progress of risk assessment for GCS over the last few decades are examined. Here, we use the Southwest Regional Partnership on Carbon Sequestration (SWP), which is one of the seven regional partnerships supported by the United States Department of Energy (U.S. DOE), as an example of large-scale CCUS projects in North America. Additionally, future trends and requirements for risk assessment in GCS are discussed. The information provided in this review can help readers understand the significance of risk and uncertainty assessment and apply it effectively in large-scale GCS projects.

58 GEOSCIENCES↗
DOE OSTI2021

A Benchmark to Test Generalization Capabilities of Deep Learning Methods to Classify Severe Convective Storms in a Changing Climate

Abstract This is a test case study assessing the ability of deep learning methods to generalize to a future climate (end of 21st century) when trained to classify thunderstorms in model output representative of the present‐day climate. A convolutional neural network (CNN) was trained to classify strongly rotating thunderstorms from a current climate created using the Weather Research and Forecasting model at high‐resolution, then evaluated against thunderstorms from a future climate and found to perform with skill and comparatively in both climates. Despite training with labels derived from a threshold value of a severe thunderstorm diagnostic (updraft helicity), which was not used as an input attribute, the CNN learned physical characteristics of organized convection and environments that are not captured by the diagnostic heuristic. Physical features were not prescribed but rather learned from the data, such as the importance of dry air at mid‐levels for intense thunderstorm development when low‐level moisture is present (i.e., convective available potential energy). Explanation techniques also revealed that thunderstorms classified as strongly rotating are associated with learned rotation signatures. Results show that the creation of synthetic data with ground truth is a viable alternative to human‐labeled data and that a CNN is able to generalize a target using learned features that would be difficult to encode due to spatial complexity. Most importantly, results from this study show that deep learning is capable of generalizing to future climate extremes and can exhibit out‐of‐sample robustness with hyperparameter tuning in certain applications.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

The Contribution of Subtropical Moisture Within an Atmospheric River on Moisture Flux, Cloud Structure, and Precipitation Over the Salmon River Mountains of Idaho Using Moisture Tracers

The impact of an atmospheric river (AR) on the flux of subtropical moisture across Idaho's Salmon River Mountains and precipitation over the mountains is evaluated using the Weather, Research, and Forecasting model with water vapor tracers (WRF-WVT). The AR impacted Idaho between 17 and 19 January 2017 during the Seeded and Natural Orographic Wintertime Clouds: The Idaho Experiment (SNOWIE) campaign. WRF-WVT is configured to isolate the subtropical moisture contribution to the AR, the moisture flux, and precipitation. Subtropical water vapor advected by the AR into Idaho is tagged and tracked in three-dimensional space throughout the run. This allows the contribution of the subtropical moisture to the vertical distribution of water vapor and the precipitation to be directly calculated. The simulated cloud structure is compared with airborne radar data collected during two SNOWIE intensive operation periods. This study found that more than 70% of the moisture flux and more than 80% of the precipitation across the Idaho Mountains during SNOWIE IOP 4 could be attributed to subtropical moisture within the AR. Nearly all of the moisture flux in the upper cloud and 50% of the moisture in the lower cloud was attributable to the subtropical moisture. The subtropical moisture contribution within the AR to precipitation ranged from 35% in northern Idaho to more than 90% in southern Idaho. Across the entire period of impact of the AR, more than 60% of precipitation in Idaho was attributable to the subtropical moisture within the AR, with this percentage increasing toward the south across the state.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

NRAP Recommended Practices for Containment Assurance and Leakage Risk Quantification

This report presents recommendations for quantitatively assessing and managing leakage risk and effective containment assurance at geologic carbon storage (GCS) sites, amidst uncertainty in site characteristics and performance. These recommended practices are a product of the U.S. Department of Energy’s (DOE) Office of Fossil Energy and Carbon Management’s National Risk Assessment Partnership (NRAP)—a multi-year collaborative research effort of Lawrence Berkeley National Laboratory, Lawrence Livermore National Laboratory, Los Alamos National Laboratory, National Energy Technology Laboratory, and Pacific Northwest National Laboratory. These recommended practices are organized as a set of conceptual workflows that correspond with the stages of the GCS project lifecycle. They consider aspects of site characterization, quantification of potential leakage, assessing the spatial extent of potential environmental impacts, optimizing monitoring design for detection of potential leakage, evaluation of concordance between monitored and forecasted reservoir response, assessing conformance with expected/accepted performance, and determination of the timeline over which risk of impactful leakage can be expected to abate. These conceptual workflows are intended to aid stakeholders in making risk management decisions at GCS sites and to support development of sound, risk-based justifications for characterizing the subsurface, defining area of potential leakage impact, designing site monitoring plans, selecting leakage risk management/mitigation alternatives, and estimating site closure timelines.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Quantitative Risk Assessment (Topical Report)

Carbon capture, utilization, and storage (CCUS) in geological formations plays a key role in mitigating anthropogenic CO 2 emissions and achieving the aggressive goal of net-zero greenhouse gas emissions. Quantitative risk assessment is crucial for ensuring the safety and reliability of geologic carbon storage (GCS) by evaluating CO 2 migration in subsurface, forecasting potential leakage and induced seismicity risks, and optimizing operational and monitoring plans. We present the use, progress, and research trends of risk assessment from the Southwest Regional Partnership on Carbon Sequestration (SWP) as an example of large-scale CCUS projects in North America. The information provided in this report can help readers understand the significance of risk and uncertainty assessment and apply them effectively in large-scale CCUS projects.

42 ENGINEERING↗
DOE OSTI2025

Cloud-atmosphere impacts on the central Arctic surface energy budget (Final Report)

This project investigated how clouds and atmosphere-sea ice interactions shape the central Arctic surface energy budget using observations from the MOSAiC expedition and coordinated model experiments. Analyses focused on three themes. First, a new method applied to thermistor string data revealed that snow thermal conductivity is systematically higher than assumed in most models, strongly influencing conductive heat flux and sea ice growth, especially under clear skies. Second, a full annual cycle of surface energy budget observations, combined with detailed cloud microphysical retrievals, demonstrated how cloud regimes regulate radiative, turbulent, and conductive fluxes, driving seasonal contrasts in sea ice energy balance: in winter, radiative forcing elicits strong turbulent and conductive responses, while in summer excess energy is primarily partitioned into surface melt. Third, these observations were used to evaluate and improve regional and global forecast models. While some models captured key Arctic cloud-radiation states, most exhibited persistent deficiencies in representing liquid-containing clouds, snow-on-sea-ice processes, and flux responses to radiative forcing. Together, these results provide unprecedented benchmarks for understanding coupled Arctic processes, improve parameterizations of snow, sea ice, and cloud interactions, and highlight critical pathways by which atmospheric variability drives sea ice change in a rapidly evolving Arctic system. Lastly, the project was a resounding success, resulting in 38 peer-reviewed publications and dozens of presentations, while also supporting an early career scientist.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Predictions of the Nancy Grace Roman Space Telescope Galactic Exoplanet Survey. II. Free-floating Planet Detection Rates

The Nancy Grace Roman Space Telescope (Roman) will perform a Galactic Exoplanet Survey (RGES) to discover bound exoplanets with semimajor axes greater than 1 au using gravitational microlensing. Roman will even be sensitive to planetary-mass objects that are not gravitationally bound to any host star. Such free-floating planetary-mass objects (FFPs) will be detected as isolated microlensing events with timescales shorter than a few days. A measurement of the abundance and mass function of FFPs is a powerful diagnostic of the formation and evolution of planetary systems, as well as the physics of the formation of isolated objects via direct collapse. We show that Roman will be sensitive to FFP lenses that have masses from that of Mars (0.1 M {sub ⊕}) to gas giants (M ≳ 100 M {sub ⊕}) as isolated lensing events with timescales from a few hours to several tens of days, respectively. We investigate the impact of the detection criteria on the survey, especially in the presence of finite-source effects for low-mass lenses. The number of detections will depend on the abundance of such FFPs as a function of mass, which is at present poorly constrained. Assuming that FFPs follow the fiducial mass function of cold, bound planets adapted from Cassan et al., we estimate that Roman will detect ∼250 FFPs with masses down to that of Mars (including ∼60 with masses ≤ M {sub ⊕}). We also predict that Roman will improve the upper limits on FFP populations by at least an order of magnitude compared to currently existing constraints.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

How Can Probabilistic Solar Power Forecasts Be Used to Lower Costs and Improve Reliability in Power Spot Markets? A Review and Application to Flexiramp Requirements

Net load uncertainty in electricity spot markets is rapidly growing. There are five general approaches by which system operators and market participants can use probabilistic forecasts of wind, solar, and load to help manage this uncertainty. These include operator situation awareness, resource risk hedging, reserves procurement, definition of contingencies, and explicit stochastic optimization. We review these approaches, and then provide a case study in which a method for using probabilistic solar forecasts to define needs for reserves is developed and evaluated. The case study has three parts. First, we describe building blocks for enhancing the Watt-Sun solar forecasting system to produce probabilistic irradiance and power forecasts. Second, relationships between Watt-Sun forecasts for multiple sites in California and the system's need for flexible ramp capability (flexiramp) are defined by machine learning and statistical methods. Third, the performance of present methods to defining flexiramp requirements, which are not conditioned on weather and renewables forecasts, is compared with that of probabilistic solar forecast-based requirements, using a multi-timescale production costing model with an 1820-bus representation of the WECC power system. Significant potential savings in fuel and flexiramp procurement costs from using solar-informed reserve requirements are found.

14 SOLAR ENERGY↗
DOE OSTI2021

A Statistical Evaluation of WRF-LES Trace Gas Dispersion Using Project Prairie Grass Measurements

In recent years, new measurement systems have been deployed to monitor and quantify methane emissions from the natural gas sector. Large-eddy simulation (LES) has complemented measurement campaigns by serving as a controlled environment in which to study plume dynamics and sampling strategies. However, with few comparisons with controlled-release experiments, the accuracy of LES for modeling natural gas emissions is poorly characterized. In this paper, we evaluate LES from the Weather Research and Forecasting (WRF) Model against Project Prairie Grass campaign measurements and surface layer similarity theory. Using WRF-LES, we simulate continuous emissions from 30 near-surface trace gas sources in two stability regimes: strong convection and weak convection. We examine the impact of grid resolutions ranging from 6.25 to 52 m in the horizontal dimension on model results. We evaluate performance in a statistical framework, calculating fractional bias and conducting Welch’s t tests. WRF-LES accurately simulates observed surface concentrations at 100 m and beyond under strong convection; simulated concentrations pass t tests in this region irrespective of grid resolution. However, in weakly convective conditions with strong winds, WRF-LES substantially overpredicts concentrations—the magnitude of fractional bias often exceeds 30%, and all but one t test fails. The good performance of WRF-LES under strong convection correlates with agreement with local free convection theory and a minimal amount of parameterized turbulent kinetic energy. The poor performance under weak convection corresponds to misalignment with Monin–Obukhov similarity theory and a significant amount of parameterized turbulent kinetic energy.

17 WIND ENERGY↗
DOE OSTI2021

Forecasting Dynamic Line Rating with Spatial Variation Considerations

Dynamic line rating (DLR) is a technology that allows the ampacity of an electrical conductor to be calculated using real-time or forecasted weather conditions. Historically, the ampacity of a conductor has been determined using a static line rating method which assumes conservative weather assumptions. Therefore, not only can DLR give a more accurate measurement of the true ampacity of a conductor, but it can also increase its ampacity during weather conditions with greater thermal mitigations. The two primary cooling factors in the ampacity calculations are wind speed and direction. In complex terrain, wind speed and direction can have large variations over short distances. Therefore, accurately identifying the limiting span of a transmission line requires high spatial resolution of the wind along its path. One solution is to install dense weather stations along their path, though this can become costly over long distances. Therefore, researchers have investigated the use of Computation Fluid Dynamic (CFD) simulations to accurately compute the wind field along the path of a transmission line and use these results to identify the limiting section of the conductor. This work presents a case study that evaluates the coupling of CFD simulations and forecasted weather simulations using the High-Resolution Rapid Refresh (HRRR) model points over a 2-year span within a region in south eastern Idaho. The primary goal of the work is the evaluation of the number of HRRR model points used, i.e., weather stations, along the path of the line and the accuracy of the resulting DLR ampacity. This was done using 4, 10, 17, 26, and 35 HRRR model points along two transmission line paths. The results indicate that as the number of model points are increased, the DLR ampacity of the lines decrease, yet converge as more points are added and demonstrate little change with additional HRRR points. It is expected that these results can help transmission line operators identify the number of weather stations that must be installed when coupled with CFD simulations and DLR ampacity to ensure accurate ratings and safe operations.

17 WIND ENERGY↗
DOE OSTI2021

Forecasting Dynamic Line Rating with Spatial Variation Considerations

Dynamic line rating (DLR) is a technology that allows the ampacity of an electrical conductor to be calculated using real-time or forecasted weather conditions. Historically, the ampacity of a conductor has been determined using a static line rating method which assumes conservative weather assumptions. Therefore, not only can DLR give a more accurate measurement of the true ampacity of a conductor, but it can also increase its ampacity during weather conditions with greater thermal mitigations. The two primary cooling factors in the ampacity calculations are wind speed and direction. In complex terrain, wind speed and direction can have large variations over short distances. Therefore, accurately identifying the limiting span of a transmission line requires high spatial resolution of the wind along its path. One solution is to install dense weather stations along their path, though this can become costly over long distances. Therefore, researchers have investigated the use of Computation Fluid Dynamic (CFD) simulations to accurately compute the wind field along the path of a transmission line and use these results to identify the limiting section of the conductor. This work presents a case study that evaluates the coupling of CFD simulations and forecasted weather simulations using the High-Resolution Rapid Refresh (HRRR) model points over a 2-year span within a region in south eastern Idaho. The primary goal of the work is the evaluation of the number of HRRR model points used, i.e., weather stations, along the path of the line and the accuracy of the resulting ampacity. This was done using 4, 10, 17, 26, and 35 HRRR model points along two transmission line paths. The results indicate that as the number of model points are increased, the DLR ampacity of the lines decrease, yet converge as more points are added and demonstrate little change with additional HRRR points. It is expected that these results can help transmission line operators identify the number of weather stations that must be installed when coupled with CFD simulations and DLR ampacity to ensure accurate ratings and safe operations.

17 WIND ENERGY↗
DOE OSTI2025

Huge ensembles – Part 1: Design of ensemble weather forecasts using spherical Fourier neural operators

Abstract. Simulating low-likelihood high-impact extreme weather events in a warming world is a significant and challenging task for current ensemble forecasting systems. While these systems presently use up to 100 members, larger ensembles could enrich the sampling of internal variability. They may capture the long tails associated with climate hazards better than traditional ensemble sizes. Due to computational constraints, it is infeasible to generate huge ensembles (comprised of 1000–10 000 members) with traditional, physics-based numerical models. In this two-part paper, we replace traditional numerical simulations with machine learning (ML) to generate hindcasts of huge ensembles. In Part 1, we construct an ensemble weather forecasting system based on spherical Fourier neural operators (SFNOs), and we discuss important design decisions for constructing such an ensemble. The ensemble represents model uncertainty through perturbed-parameter techniques, and it represents initial condition uncertainty through bred vectors, which sample the fastest-growing modes of the forecast. Using the European Centre for Medium-Range Weather Forecasts Integrated Forecasting System (IFS) as a baseline, we develop an evaluation pipeline composed of mean, spectral, and extreme diagnostics. With large-scale, distributed SFNOs with 1.1 billion learned parameters, we achieve calibrated probabilistic forecasts. As the trajectories of the individual members diverge, the ML ensemble mean spectra degrade with lead time, consistent with physical expectations. However, the individual ensemble members' spectra stay constant with lead time. Therefore, these members simulate realistic weather states during the rollout, and the ML ensemble passes a crucial spectral test in the literature. The IFS and ML ensembles have similar extreme forecast indices, and we show that the ML extreme weather forecasts are reliable and discriminating. These diagnostics ensure that the ensemble can reliably simulate the time evolution of the atmosphere, including low-likelihood high-impact extremes. In Part 2, we generate a huge ensemble initialized each day in summer 2023, and we characterize the simulations of extremes.

Mahesh, Ankur↗
DOE OSTI2024

Evaluating Ensemble Predictions of South Asian Monsoon Low Pressure System Genesis

Abstract Synoptic-scale vortices known as monsoon low pressure systems (LPSs) frequently produce intense precipitation and hydrological disasters in South Asia, so accurately forecasting LPS genesis is crucial for improving disaster preparedness and response. However, the accuracy of LPS genesis forecasts by numerical weather prediction models has remained unknown. Here, we evaluate the performance of two global ensemble models—the U.S. Global Ensemble Forecast System (GEFS) and the Ensemble Prediction System of the European Centre for Medium-Range Weather Forecasts (ECMWF)—in predicting LPS genesis during the years 2021–22. The GEFS successfully predicted about half the observed LPS genesis events 1–2 days in advance; the ECMWF model captured an additional 10% of observed genesis events. Both models had a false alarm ratio (FAR) of around 50% for 1–2-day lead times. In both ensembles, the control run typically exhibited a higher probability of detection (POD) of observed events and a lower FAR compared to the perturbed ensemble members. However, a consensus forecast, in which genesis is predicted when at least 20% of ensemble members forecast LPS formation, had POD values surpassing those of the control run for all lead times. Moreover, probabilistic predictions of genesis over the Bay of Bengal, where most LPSs form, were skillful, with the fraction of ensemble members predicting LPS formation over a 5-day lead time approximating the observed frequency of genesis, without any adjustment or bias correction.

Suhas, D. L.↗
DOE OSTI2025

Demonstrating Hierarchical System Development With the Common Community Physics Package Single‐Column Model: A Case Study Over the Southern Great Plains

This study demonstrates a specific application of the hierarchical system development (HSD) approach to investigate, analyze, and attribute model issues within the Unified Forecast System (UFS), with a focus on process isolation. By evaluating a non‐precipitating, shallow cumulus case at the Atmospheric Radiation Measurement Southern Great Plains site in the UFS global forecast against the observation, the investigation identifies a warmer and deeper daytime convective planetary boundary layer (PBL) and misrepresented nocturnal PBL transition. Hypothesis testing, which employs the Common Community Physics Package (CCPP) single‐column model (SCM) and uses the same physics as the UFS global model, confirms that these issues are attributed to the model physics and initialization. Specifically, misrepresented PBL processes are linked to problematic surface condition and a lack of cloud formation, which may stem from deficiencies in PBL and cloud microphysics parameterizations and their interactions. The UFS initial condition contributes to an earlier, excessively collapsed daytime convective boundary layer and a lack of decoupling between the stable boundary layer and residual layer late in the afternoon. This work introduces an avenue for the community to engage with the application of HSD, along with the CCPP and CCPP SCM, to understand the interplay of model physics, disentangle the roles of model components, as well as facilitate model and forecast improvement.

54 ENVIRONMENTAL SCIENCES↗