Search NASA⌕ Search

SEARCH · Search NASA

Results for “model skill”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 253 records · Page 14

Continental-Scale Controls on Hyporheic Respiration Revealed by Knowledge-Guided Machine Learning

Hyporheic zone sediments regulate organic matter turnover and in-stream respiration, yet controls on sediment respiration remain poorly constrained across heterogeneous river networks, limiting prediction of stream metabolism and carbon processing at continental scales. Here, we integrate observations from ~90 river corridors across the United States in the WHONDRS consortium with a knowledge-guided machine learning (KGML) framework that couples thermodynamic rate theory with machine learning to identify dominant controls on hyporheic respiration. Diagnostic analyses show that organic matter concentration and thermodynamic favorability define an upper bound on respiration potential, whereas biological catalytic capacity and physical accessibility jointly govern realized respiration rates through interaction effects. To represent unmeasurable accessibility constraints, we use the mechanistic model as a scaffold for KGML, allowing machine learning to target residual structure not explained by process theory. This hybrid framework improves predictive skill relative to both the mechanistic model alone and fully data-driven models while preserving interpretability. These results indicate that variability in hyporheic respiration is largely mechanistically structured and demonstrate how integrating process theory with explainable AI enhances predictive performance while enabling scalable synthesis of river corridor observations.

Zheng, Jianqiu↗

Educational Consortium for Energy-related Data Science & Computation in Building Engineering Programs

The project spearheaded by Pennsylvania State University aims to address the growing need for integrating energy-focused computation and data science into building engineering education. As the demand for energy-efficient building designs and operations increases, the educational sector must adapt to equip future engineers with the necessary skills. This initiative responds to this need by developing a consortium that unites multiple institutions to enhance curriculum development, dataset curation, and resource sharing, thereby ensuring students are well-prepared for the evolving energy sector. The primary goal of the project is to establish a consortium that will develop and disseminate educational materials and training programs focused on energy-related data science and computation. Key accomplishments include the creation of a beta website for resource sharing, the development of training programs and standalone modules, and the curation of datasets accessible to the public. This effort will culminate in a curriculum that incorporates advanced modeling technologies and data science skills into building engineering programs.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Initial Results from the GEM Challenge on the Spacecraft Surface Charging Environment

Spacecraft surface charging during geomagnetically disturbed times is one of the most important causes of satellite anomalies. Predicting the surface charging environment is one prevalent task of the geospace environment models. Therefore, the Geospace Environment Modeling (GEM) Focus Group "Inner Magnetosphere Crossenergy/Population Interactions" initiated a communitywide challenge study to assess the capability of several inner magnetosphere ring current models in determining surface charging environment for the Van Allen Probes orbits during the 17 March 2013 storm event. The integrated electron flux between 10 and 50 keV is used as the metrics. Various skill scores are applied to quantitatively measure the modeling performance against observations. Results indicate that no model consistently perform the best in all of the skill scores or for both satellites. We find that from these simulations the ring current model with observational flux boundary condition and Weimer electric potential driver generally reproduces the most realistic flux level around the spacecraft. A simple and weaker VollandStern electric field is not capable of effectively transporting the same plasma at the boundary toward the Earth. On the other hand, if the ring current model solves the electric field selfconsistently and obtains similar strength and pattern in the equatorial plane as the Weimer model, the boundary condition plays another crucial role in determining the electron flux level in the inner region. When the boundary flux spectra based on magnetohydrodynamics (MHD) model/empirical model deviate from the shape or magnitude of the observed distribution function, the simulation produces poor skill scores along Van Allen Probes orbits.

Yu, Yiqun↗

Hemispherically Asymmetric Volcanic Forcing of Tropical Hydroclimate During the Last Millennium

Volcanic aerosols exert the most important natural radiative forcing of the last millennium. State-of-the-art paleoclimate simulations of this interval are typically forced with diverse spatial patterns of volcanic forcing, leading to different responses in tropical hydroclimate. Recently, theoretical considerations relating the intertropical convergence zone (ITCZ) position to the demands of global energy balance have emerged in the literature, allowing for a connection to be made between the paleoclimate simulations and recent developments in the understanding of ITCZ dynamics. These energetic considerations aid in explaining the well-known historical, paleoclimatic, and modeling evidence that the ITCZ migrates away from the hemisphere that is energetically deficient in response to asymmetric forcing. Here we use two separate general circulation model (GCM) suites of experiments for the last millennium to relate the ITCZ position to asymmetries in prescribed volcanic sulfate aerosols in the stratosphere and related asymmetric radiative forcing. We discuss the ITCZ shift in the context of atmospheric energetics and discuss the ramifications of transient ITCZ migrations for other sensitive indicators of changes in the tropical hydrologic cycle, including global streamflow. For the first time, we also offer insight into the large-scale fingerprint of water isotopologues in precipitation (delta sup 18 Op) in response to asymmetries in radiative forcing. The ITCZ shifts away from the hemisphere with greater volcanic forcing. Since the isotopic composition of precipitation in the ITCZ is relatively depleted compared to areas outside this zone, this meridional precipitation migration results in a large-scale enrichment (depletion) in the isotopic composition of tropical precipitation in regions the ITCZ moves away from (toward). Our results highlight the need for careful consideration of the spatial structure of volcanic forcing for interpreting volcanic signals in proxy records and therefore in evaluating the skill of Common Era climate model output.

Radiative forcing↗

Prediction Skill of the MJO Teleconnection Signals in the NASA GEOS Subseasonal Reforecasts

Tropical-extratropical teleconnections are considered key to advancing subseasonal prediction. The Madden Julian oscillation (MJO), characterized by large scale convective envelopes propagating along the tropical Indo-Pacific sector, is known to modulate midlatitude circulation and associated weather patterns. Although there is a general consensus on the MJO's influence on the midlatitude circulation, which is thought to be due to modulations of the North Atlantic Oscillation (NAO) and the Pacific North America (PNA) pattern, relatively less is known about the predictability of these teleconnection signals in dynamical forecast models. The composite evolution of the midlatitude circulation anomalies and associated wave train structure as the delayed response to tropical heating are reported in many studies that have examined reanalyses and long model simulations. However, it is yet to be determined whether they lend any beneficial subseasonal forecast skill, especially to weekly mean surface temperature and precipitation over North America. Investigating useful predictable signals from the MJO teleconnections is also complicated by the fact that the MJO is a moving heat source with an approximate periodicity of 30-60 days, and that the structure and amplitude of the midlatitude response can be sensitive to the longitudinal positioning of the heating anomaly as well as the propagation speed of the MJO. The objective of this study is to investigate the impact of MJO teleconnections on forecast accuracy at 2-3 week lead over North America, with an emphasis on the above-mentioned lesser known aspects of these teleconnections. To this end, we utilize a suite of subseasonal reforecasts performed with the latest NASA GEOS-5 seasonal-to-subseasonal (S2S) system. These reforecasts were performed as part of the NOAA SubX project, wherein the NASA GEOS-5 atmosphere-ocean coupled model was run at degree horizontal resolution, initialized every 5 days for the period 1999-2016. The GEOS-5 model shows skillful predictions of the MJO, with the correlation coefficient based on the real-time multivariate MJO (RMM) index staying at or above 0.5 up to forecast lead 26-36 days. The system is thus a useful tool for investigating MJO teleconnection processes.

Achuthavarier, Deepthi↗

Evaluating E3SM Global Storm‐Resolving Model Simulations of Deep Convection: Insights From DP‐SCREAM During TRACER

Global Storm-Resolving Models (GSRMs) are becoming increasingly vital for advancing climate modeling and improving the prediction of extreme weather events. Houston, a coastal region frequently affected by deep convective storms, offers an ideal setting to evaluate the ability of GSRMs to simulate deep convection. This study assesses the performance of the Doubly Periodic Simple Cloud-Resolving E3SM (Energy Exascale Earth System Model) Atmosphere Model (DP-SCREAM) using observations from the TRacking Aerosol Convection interactions ExpeRiment (TRACER) campaign. DP-SCREAM effectively reproduces the diurnal cycles of clouds and precipitation, demonstrating much greater skill than the E3SM single column model. The DP-SCREAM is demonstrated to be applicable to coastal regions, partially due to the forcing data sets already capturing the influence of breezes. DP-SCREAM also replicates biases persistent in the global version of SCREAM: the underrepresentation of boundary layer shallow clouds, a lack of mid-level congestus clouds, and the popcorn convection, characterized by small and disorganized convective cells generating the strongest precipitation. To investigate these issues, two sensitivity experiments were conducted: increasing the mixing length and scaling up the buoyancy flux within the Simplified Higher Order Closure scheme. Increasing the mixing length improved mid-level congestus representation and reduced unrealistic early morning fog occurrence. Enhancing buoyancy flux only marginally improved the bias of underproduced big convective cells. In conclusion, an additional resolution sensitivity test at 0.5 km grid spacing demonstrated that a refined horizontal resolution alone is insufficient to resolve these biases.

54 ENVIRONMENTAL SCIENCES↗

Linking the subseasonal variability of the East Asia winter monsoon and the Madden-Julian Oscillation through wave disturbances along the subtropical jet

Despite an urgent demand for reliable subseasonal-to-seasonal (S2S) predictions to guide disaster preparedness, our current climate models show limited S2S prediction skill, particularly for precipitation, due to an inadequate understanding of the key processes that drive regional S2S variability. Here we demonstrate that the leading subseasonal variability mode of precipitation over the East Asian Winter Monsoon (EAWM) region is not only closely tied to the activity of the Madden-Julian Oscillation (MJO), but also linked to precipitation and temperature extremes worldwide, influenced by a circumglobal Rossby wave-train along the subtropical westerly jet. Despite a close phase-lock relationship between the MJO and subseasonal EAWM precipitation, our findings indicate that the MJO itself may only play a minor role in the subseasonal EAWM variability. Given its significant impact on the S2S variability of global weather extremes, we call for coordinated community efforts to enhance the understanding and prediction of the circumglobal Rossby wave-train.

Atmospheric science↗

SetGo: Metadata Readiness for Scientific AI Datasets

Scientific datasets intended for AI use require both computational readiness for model training and metadata readiness for discovery, sharing, and reuse. The Readiness Engine for Data Integration (REDI) addresses computational readiness, but no corresponding tool evaluates whether a dataset’s metadata are sufficiently complete, governed, and standards-compliant for publication and agent-based consumption. Existing FAIR assessors operate only on published repository records, and no single system covers FAIR compliance, licensing, provenance, governance, reproducibility, and catalog readiness together. We present SetGo, an open-source Python toolkit that assesses and repairs metadata readiness across these six dimensions before a dataset is published or archived. Applied to four scientific corpora, SetGo surfaces deficiencies that general-purpose tools do not detect: ERA5 climate metadata scores 4% on ACDD 1.3 compliance; materials datasets fail OPTIMADE species-definition requirements; and PDB-derived proteomics data carries licensing terms incompatible with standard SPDX identifiers. Guided enrichment raises overall FAIR scores from 52–57% to 81–91%, and a single setgo publish command pushes to Hugging Face Hub, CKAN, or OpenMetadata with ML Commons Croissant 1.0 metadata sidecars. To support interactive and automated workflows, SetGo integrates with coding agents powered by large language models (LLMs) through a /setgo skill that enables natural-language execution of the full assess–enrich–publish loop, with user involvement limited to supplying missing metadata values.

Wilkinson, Sean [ORNL] (ORCID:0000000214437479)↗

5-minute Wind Power Data based on WFIP2 WRF Simulation

The second Wind Forecast Improvement Project (WFIP2) was a public-private partnership funded by the U.S. Department of Energy and NOAA, aimed at enhancing the forecast skill of numerical weather prediction models for turbine-height winds in regions with complex terrain. An 18-month Weather Research and Forecasting (WRF) model simulation was conducted over the Pacific Northwest, with model outputs validated against observational data collected during WFIP2. Simulated wind speeds were used to estimate wind power generation using reV (the Renewable Energy Potential Model developed by NREL) at ten wind project sites. Two sets of results were produced: one using wind speeds extracted from the model grid cell at the project centroid, and another using wind speeds from the actual turbine locations. For each dataset, power output was calculated using both actual turbine-specific power curves and nine generic power curves to convert wind speed into power.

17 WIND ENERGY↗

The effect of a gravity wave drag parameterization scheme on GLA fourth order GCM forecasts

Ten-day forecast experiments have been performed to determined whether the introduction of a simple orographic gravity wave drag scheme into a fourth-order GCM would reduce the climate drift of the fine resolution model and improve the model's medium range predictive skill. Error reduction due to the gravity waves is found in stratospheric predictions, where the improvement is confined mainly to the zonal mean component. Improvements are noted in the Northern Hemisphere climatology, where low level westerlies are weakened and shifted poleward, and in the Southern Hemisphere, where the roaring forties and fifties are better simulated.

Helfand, H. M.↗

Anticipatory Eye Movements in Interleaving Templates of Human Behavior

Performance modeling has been made easier by architectures which package psychological theory for reuse at useful levels of abstraction. CPM-GOMS uses templates of behavior to package at a task level (e.g., mouse move-click, typing) predictions of lower-level cognitive, perceptual, and motor resource use. CPM-GOMS also has a theory for interleaving resource use between templates. One example of interleaving is anticipatory eye movements. This paper describes the use of ACT-Stitch, a framework for translating CPM-GOMS templates and interleaving theory into ACT-R, to model anticipatory eye movements in skilled behavior. The anticipatory eye movements explain performance in a well-practiced perceptual/motor task, and the interleaving theory is supported with results from an eye-tracking experiment.

Matessa, Michael↗

Winter Precipitation in North America and the Pacific-North America Pattern in GEOS-S2Sv2 Seasonal Hindcast

Reliable prediction of precipitation remains one of the most pivotal and complex challenges in seasonal forecasting. Previous studies show that various large-scale climate modes, such as ENSO, PNA and NAO play significant role in winter precipitation variability over the Northern America. The influences are most pronounced in years of strong indices of such climate modes. This study evaluates model bias, predictability and forecast skills of monthly winter precipitation in GEOS5-S2S 2.0 retrospective forecast from 1981 to 2016, with emphasis on the forecast skill of precipitation over North America during the extreme events of ENSO, PNA and NAO by applying EOF and composite analysis.

Li, Zhao↗

How Can TOLNet Help to Better Understand Tropospheric Ozone? A Satellite Perspective

Potential sources of a priori ozone (O3) profiles for use in Tropospheric Emissions: Monitoring of Pollution (TEMPO) satellite tropospheric O3 retrievals are evaluated with observations from multiple Tropospheric Ozone Lidar Network (TOLNet) systems in North America. An O3 profile climatology (tropopause-based O3 climatology (TB-Clim), currently proposed for use in the TEMPO O3 retrieval algorithm) derived from ozonesonde observations and O3 profiles from three separate models (operational Goddard Earth Observing System (GEOS-5) Forward Processing (FP) product, reanalysis product from Modern-Era Retrospective analysis for Research and Applications version 2 (MERRA2), and the GEOS-Chem chemical transport model (CTM)) were: 1) evaluated with TOLNet measurements on various temporal scales (seasonally, daily, hourly) and 2) implemented as a priori information in theoretical TEMPO tropospheric O3 retrievals in order to determine how each a priori impacts the accuracy of retrieved tropospheric (0-10 km) and lowermost tropospheric (LMT, 0-2 km) O3 columns. We found that all sources of a priori O3 profiles evaluated in this study generally reproduced the vertical structure of summer-averaged observations. However, larger differences between the a priori profiles and lidar observations were observed when evaluating inter-daily and diurnal variability of tropospheric O3. The TB-Clim O3 profile climatology was unable to replicate observed inter-daily and diurnal variability of O3 while model products, in particular GEOS-Chem simulations, displayed more skill in reproducing these features. Due to the ability of models, primarily the CTM used in this study, on average to capture the inter-daily and diurnal variability of tropospheric and LMT O3 columns, using a priori profiles from CTM simulations resulted in TEMPO retrievals with the best statistical comparison with lidar observations. Furthermore, important from an air quality perspective, when high LMT O3 values were observed, using CTM a priori profiles resulted in TEMPO LMT O3 retrievals with the least bias. The application of time-specific (non-climatological) hourly/daily model predictions as the a priori profile in TEMPO O3 retrievals will be best suited when applying this data to study air quality or event-based processes as the standard retrieval algorithm will still need to use a climatology product. Follow-on studies to this work are currently being conducted to investigate the application of different CTM-predicted O3 climatology products in the standard TEMPO retrieval algorithm. Finally, similar methods to those used in this study can be easily applied by TEMPO data users to recalculate tropospheric O3 profiles provided from the standard retrieval using a different source of a priori.

Satellite↗

Seasonal Predictability of Cloud Droplet Number Concentration

Aerosol emissions modify the properties of clouds hence impacting climate. The aerosol indirect effect may have offset part of the global warming caused by anthropogenic greenhouse gas emissions during the industrial era. It however remains unclear whether the same effect is significant over time scales relevant for seasonal and weather climate prediction. Answering such a question has been difficult since most weather prediction systems lack a proper representation of the aerosol evolution and transport and their interaction with clouds. Even in advanced systems it is not clear to what extent cloud microphysical properties are predictable over subseasonal to seasonal time scales. Such an issue is addressed in this study. We use a set of 30 year, four ensemble member, 9 month lead hindcast simulations of the NASA GEOS seasonal prediction system (GEOS-S2S) to study the predictability of cloud droplet number concentration in warm stratocumulus clouds. The latest version GEOS-S2S system implements interactive aerosol as well as a two moment cloud microphysics scheme therefore it is suitable for studying the aerosol indirect effect on climate. Long term retrievals from the MODIS (Moderate Resolution Imaging Spectroradiometer) are used to validate the model predictions and assess its skill in predicting cloud droplet number concentration.

Barahona, Donifan↗

An Observation‐Driven Approach to Improve Vegetation Phenology in a Global Land Surface Model

An empirical model calibration approach is presented that aims to approximate missing biosphere processes in a global land surface model without the need for substantial model structural changes. The strategy is implemented here using the NASA Catchment‐CN land surface model and Moderate Resolution Imaging Spectroradiometer (MODIS) observations of the fraction of absorbed photosynthetically active radiation (FPAR). Existing plant functional types (PFTs) of the Catchment‐CN model are divided into three subtypes, based on the bias between the model‐simulated and MODIS‐observed FPAR. Separate sets of vegetation parameters for each subtype are then calibrated at a small number of grid cells with homogeneous, single‐PFT land cover, using MODIS FPAR reference observations from 2003 to 2009. The effectiveness of the empirical approach at improving the realism of modeled vegetation dynamics is investigated with two global model simulations for the period 2010–2016, one using the newly calibrated parameter values and the other using the original values. Globally, the calibrated parameters reduce the root mean square error (RMSE) of the modeled FPAR with respect to MODIS by 0.029 (∼10%) on average. In some regions, substantially larger RMSE reductions are achieved. RMSE reductions are primarily driven by model bias reductions, with neutral effects on the temporal correlation skill. While the empirical approach is suitable for achieving consistent model improvements, it is shown to be sensitive to the characteristics of the model error, specifically a dominance of the bias component in the case of Catchment‐CN. Ultimately, more fundamental model structural changes may be required to achieve better improvements.

J. Kolassa↗

Using Paleo-climate Comparisons to Constrain Future Projections in CMIP5

We present a description of the theoretical framework and best practice for using the paleo-climate model component of the Coupled Model Intercomparison Project (Phase 5) (CMIP5) to constrain future projections of climate using the same models. The constraints arise from measures of skill in hindcasting paleo-climate changes from the present over 3 periods: the Last Glacial Maximum (LGM) (21 thousand years before present, ka), the mid-Holocene (MH) (6 ka) and the Last Millennium (LM) (8501850 CE). The skill measures may be used to validate robust patterns of climate change across scenarios or to distinguish between models that have differing outcomes in future scenarios. We find that the multi-model ensemble of paleo-simulations is adequate for addressing at least some of these issues. For example, selected benchmarks for the LGM and MH are correlated to the rank of future projections of precipitationtemperature or sea ice extent to indicate that models that produce the best agreement with paleoclimate information give demonstrably different future results than the rest of the models. We also find that some comparisons, for instance associated with model variability, are strongly dependent on uncertain forcing timeseries, or show time dependent behaviour, making direct inferences for the future problematic. Overall, we demonstrate that there is a strong potential for the paleo-climate simulations to help inform the future projections and urge all the modeling groups to complete this subset of the CMIP5 runs.

paleoclimate↗

Evaluation of the 2022 West Nile virus forecasting challenge, USA

Abstract Background West Nile virus (WNV) is the most common cause of mosquito-borne disease in the continental USA, with an average of ~1200 severe, neuroinvasive cases reported annually from 2005 to 2021 (range 386–2873). Despite this burden, efforts to forecast WNV disease to inform public health measures to reduce disease incidence have had limited success. Here, we analyze forecasts submitted to the 2022 WNV Forecasting Challenge, a follow-up to the 2020 WNV Forecasting Challenge. Methods Forecasting teams submitted probabilistic forecasts of annual West Nile virus neuroinvasive disease (WNND) cases for each county in the continental USA for the 2022 WNV season. We assessed the skill of team-specific forecasts, baseline forecasts, and an ensemble created from team-specific forecasts. We then characterized the impact of model characteristics and county-specific contextual factors (e.g., population) on forecast skill. Results Ensemble forecasts for 2022 anticipated a season at or below median long-term WNND incidence for nearly all (> 99%) counties. More counties reported higher case numbers than anticipated by the ensemble forecast median, but national caseload (826) was well below the 10-year median (1386). Forecast skill was highest for the ensemble forecast, though the historical negative binomial baseline model and several team-submitted forecasts had similar forecast skill. Forecasts utilizing regression-based frameworks tended to have more skill than those that did not and models using climate, mosquito surveillance, demographic, or avian data had less skill than those that did not, potentially due to overfitting. County-contextual analysis showed strong relationships with the number of years that WNND had been reported and permutation entropy (historical variability). Evaluations based on weighted interval score and logarithmic scoring metrics produced similar results. Conclusions The relative success of the ensemble forecast, the best forecast for 2022, suggests potential gains in community ability to forecast WNV, an improvement from the 2020 Challenge. Similar to the previous challenge, however, our results indicate that skill was still limited with general underprediction despite a relative low incidence year. Potential opportunities for improvement include refining mechanistic approaches, integrating additional data sources, and considering different approaches for areas with and without previous cases. Graphical Abstract

54 ENVIRONMENTAL SCIENCES↗

Predictability of the Minimum Sea Ice Extent from Late Winter Fram Strait Ice Export: Model vs Observations

Late winter coastal divergence along the Eurasian coastline (referred to as the ice factory) – or the Fram Strait ice export, a proxy for coastal divergence in the ice factory – is a skillful predictor of the minimum sea ice extent (Williams et al., 2016; Brunette et al., 2018; Sesternikov and, 1979). Coastal divergence leads to the formation of coastal polynya where new ice grows but to a thickness that is not large enough to survive the following summer melt. This signal is then amplified by the ice albedo feedback and leads to more open water at the end of the summer melt season. In this thesis project, we will identify if this source of predictive skill in the seasonal forecast of the minimum sea ice extent is present in General Circulation Models, more precisely in the CESM2-LE, GISS-E2.1-G, GFDL FLOR-LE, CNRM-CM6-1 and the CanESM5. Since even small biases in the large-scale atmospheric circulation simulated by a model can short circuit this coupling between dominant modes of atmospheric variability (NAO and AO), coastal divergence along the Eurasian coastline, Fram Strait ice export and therefore seasonal forecasting skill of the model, failure to reproduce this coupling observed in the real Arctic will be used to identify biases in GCM. Preliminary results show that subtle changes in the large-scale atmospheric circulation leads to opposite statistical relationship between Fram Strait ice area export, coastal divergence along the Eurasian coastline, and seasonal predictability of the minimum sea ice extent both in the CESM2-LE and the GISS-E2.1-G models. Differences are linked with the partitioning between recirculation within the Beaufort Gyre, ice exported through Fram Strait and ridging north of the Canadian Arctic Archipelago.

Sea ice↗