Search NASA⌕ Search

SEARCH · Search NASA

Results for “multi-model evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

117 records · Page 7
All sourcesNASA NTRSDOE OSTI
NASA NTRS2022

Investigating Stable Oxygen and Carbon Isotopic Variability in Speleothem Records Over the Last Millennium Using Multiple Isotope-Enabled Climate Models

The incorporation of water isotopologues into the hydrology of general circulation models (GCMs) facilitates the comparison between modeled and measured proxy data in paleoclimate archives. However, the variability and drivers of measured and modeled water isotopologues, as well as the diversity of their representation in different models, are not well constrained. Improving our understanding of this variability in past and present climates will help to better constrain future climate change projections and decrease their range of uncertainty. Speleothems are a precisely datable terrestrial paleoclimate archives and provide well-preserved (semi-)continuous multivariate isotope time series in the lower latitudes and mid-latitudes and are therefore well suited to assess climate and isotope variability on decadal and longer timescales. However, the relationships of speleothem oxygen and carbon isotopes to climate variables are influenced by site-specific parameters, and their comparison to GCMs is not always straightforward. Here we compare speleothem oxygen and carbon isotopic signatures from the Speleothem Isotopes Synthesis and Analysis database version 2 (SISALv2) to the output of five different water-isotope-enabled GCMs (ECHAM5-wiso, GISS-E2-R, iCESM, iHadCM3, and isoGSM) over the last millennium (850–1850 CE). We systematically evaluate differences and commonalities between the standardized model simulation outputs. The goal is to distinguish climatic drivers of variability for modeled isotopes and compare them to those of measured isotopes. We find strong regional differences in the oxygen isotope signatures between models that can partly be attributed to differences in modeled surface temperature. At low latitudes, precipitation amount is the dominant driver for stable water isotope variability; however, at cave locations the agreement between modeled temperature variability is higher than for precipitation variability. While modeled isotopic signatures at cave locations exhibited extreme events coinciding with changes in volcanic and solar forcing, such fingerprints are not apparent in the speleothem isotopes. This may be attributed to the lower temporal resolution of speleothem records compared to the events that are to be detected. Using spectral analysis, we can show that all models underestimate decadal and longer variability compared to speleothems (albeit to varying extents). We found that no model excels in all analyzed comparisons, although some perform better than the others in either mean or variability. Therefore, we advise a multi-model approach whenever comparing proxy data to modeled data. Considering karst and cave internal processes, e.g., through isotope-enabled karst models, may alter the variability in speleothem isotopes and play an important role in determining the most appropriate model. By exploring new ways of analyzing the relationship between the oxygen and carbon isotopes, their variability, and co-variability across timescales, we provide methods that may serve as a baseline for future studies with different models using, e.g., different isotopes, different climate archives, or different time periods.

isotope-enabled simulations↗
NASA NTRS2022

Are Soybean Models Ready for Climate Change Food Impact Assessments?

An accurate estimation of crop yield under climate change scenarios is essential to quantify our ability to feed a growing population and develop agronomic adaptations to meet future food demand. A coordinated evaluation of yield simulations from process-based eco-physiological models for climate change impact assessment is still missing for soybean, the most widely grown grain legume and the main source of protein in our food chain. In this first soybean multi-model study, we used ten prominent models capable of simulating soybean yield under varying temperature and atmospheric CO 2 concentration [CO 2 ] to quantify the uncertainty in soybean yield simulations in response to these factors. Models were first parametrized with high quality measured data from five contrasting environments. We found considerable variability among models in simulated yield responses to increasing temperature and [CO 2 ]. For example, under a + 3 °C temperature rise in our coolest location in Argentina, some models simulated that yield would reduce as much as 24%, while others simulated yield increases up to 29%. In our warmest location in Brazil, the models simulated a yield reduction ranging from a 38% decrease under + 3 °C temperature rise to no effect on yield. Similarly, when increasing [CO 2 ] from 360 to 540 ppm, the models simulated a yield increase that ranged from 6% to 31%. Model calibration did not reduce variability across models but had an unexpected effect on modifying yield responses to temperature for some of the models. The high uncertainty in model responses indicates the limited applicability of individual models for climate change food projections. However, the ensemble mean of simulations across models was an effective tool to reduce the high uncertainty in soybean yield simulations associated with individual models and their parametrization. Ensemble, ensemble mean yield responses to temperature and [CO 2 ] were similar to those reported from the literature. Our study is the first demonstration of the benefits achieved from using an ensemble of grain legume models for climate change food projections, and highlights that further soybean model development with experiments under elevated [CO 2 ] and temperature is needed to reduce the uncertainty from the individual models.

Agricultural Model Inter-comparison and Improvemen↗
NASA NTRSDate not supplied

Regional and Model-Specific Response Types in A Global Gridded Crop Model Ensemble

Crop models are often employed to project crop yields under changing conditions such as global warming and associated management change for adaptation. Multi-model ensembles are promoted to enhance the robustness of projections, but questions remain on what causes often large differences between projections of individual models. Global Gridded Crop Models (GGCMs) are especially exposed to this question when applied for assessing climate change impacts, adaptation, environmental impacts of agricultural production, because their results are used in downstream analyses, such as in integrated assessment or economic modeling for projecting future land-use change. Even though global gridded crop models are often based on detailed field-scale models or have implemented similar modeling principles in other ecosystem models, global-scale models are subject to substantial uncertainties from both model structure and parametrization as well as from calibration and input data quality. AgMIP’s Global Gridded Crop Model Intercomparison (GGCMI) has thus set out to intercompare GGCMs in order to evaluate model performance, describe model uncertainties, identify inconsistencies within the ensemble and underlying reasons, and to ultimately improve models and modeling capacities. In phase 2 of the GGCMI activities, 12 modeling groups followed a modeling protocol that asked for up to 1404 31-year global simulations at 0.5 arc-degree spatial resolution to assess models’ sensitivities to changes in carbon dioxide (C; 4 different levels) temperature (T; 7 different offset levels), water supply (W; 9 levels), and nitrogen (N; 3 levels), the so-called CTWN experiment (Franke et al. 2020; http://dx.doi.org/10.5194/gmd-13-2315-2020). We here present analyses of model response types using impact response surfaces along the C, T, W, and N dimensions, respectively and collectively. Doing so, we can understand differences in simulated responses per driver rather than aggregated changes in yields. We find that models’ sensitivities to the individual driver dimensions are substantially different and often more different across models than across regions. A cluster analysis finds regional and model-specific patterns. There is some agreement across models with respect to the spatial patterns of response types but strong differences in the distribution of response type clusters across models suggests that models need to undergo further scrutiny. We suggest establishing standards in model process evaluation not only against historical dynamics but also against dedicated experiments across the CTWN dimensions.

crop models↗
NASA NTRS2023

Arctic Tropospheric Ozone: Assessment of Current Knowledge and Model Performance

As the third most important greenhouse gas (GHG) after carbon dioxide (CO2) and methane (CH4), tropospheric ozone (O3) is also an air pollutant causing damage to human health and ecosystems. This study brings together recent research on observations and modeling of tropospheric O3 in the Arctic, a rapidly warming and sensitive environment. At different locations in the Arctic, the observed surface O3 seasonal cycles are quite different. Coastal Arctic locations, for example, have a minimum in the springtime due to O3 depletion events resulting from surface bromine chemistry. In contrast, other Arctic locations have a maximum in the spring. The 12 state-of-the-art models used in this study lack the surface halogen chemistry needed to simulate coastal Arctic surface O3 depletion in the springtime; however, the multi-model median (MMM) has accurate seasonal cycles at non-coastal Arctic locations. There is a large amount of variability among models, which has been previously reported, and we show that there continues to be no convergence among models or improved accuracy in simulating tropospheric O3 and its precursor species. The MMM underestimates Arctic surface O3 by 5 % to 15 % depending on the location. The vertical distribution of tropospheric O3 is studied from recent ozonesonde measurements and the models. The models are highly variable, simulating free-tropospheric O3 within a range of ±50 % depending on the model and the altitude. The MMM performs best, within ±8 % for most locations and seasons. However, nearly all models overestimate O3 near the tropopause (∼300 hPa or ∼8 km), likely due to ongoing issues with underestimating the altitude of the tropopause and excessive downward transport of stratospheric O3 at high latitudes. For example, the MMM is biased high by about 20 % at Eureka. Observed and simulated O3 precursors (CO, NOx, and reservoir PAN) are evaluated throughout the troposphere. Models underestimate wintertime CO everywhere, likely due to a combination of underestimating CO emissions and possibly overestimating OH. Throughout the vertical profile (compared to aircraft measurements), the MMM underestimates both CO and NOx but overestimates PAN. Perhaps as a result of competing deficiencies, the MMM O3 matches the observed O3 reasonably well. Our findings suggest that despite model updates over the last decade, model results are as highly variable as ever and have not increased in accuracy for representing Arctic tropospheric O3.

Arctic↗
NASA NTRSDate not supplied

Inferred Sea Level Prediction in the NASA GMAO Seasonal Forecasting System

Reliable predictions of sea level anomalies on seasonal timescales with lead times of 1 to 9 months may have relevance to stakeholders – for example, in the advance deployment of resources for coastal flood mitigation. Routine prediction and analysis may also highlight physical processes associated with sea level change and modeling capabilities on seasonal and other timescales. These forecasts may represent interannual changes in the seasonal slope of the ocean surface, teleconnection effects such as the El Niño/Southern Oscillation phenomenon, and variations in seasonal hydrology including precipitation and coastal runoff. Coupled atmosphere/ocean models are routinely used in the seasonal prediction of temperature anomalies, precipitation anomalies, sea ice cover, and climate indices such as the Niño3.4 predictions under the North American Multi-Model Ensemble (NMME) protocol. Within the limits of their configuration, these complex Earth-system models have a potential for depicting regional changes in oceanic column properties, including the sea surface height. Seasonal prediction models generally have no representation of long-term mass contributions from melting land ice, or changes in vertical land motion; their output may be more specifically characterized as predictions of the ocean dynamic sea level. In practice however, the sea surface height prognostic variable is substantially compromised by the forecast model response to initial conditions. Imbalances between the initial, observed hydrologic cycle and the forecast model state produce abrupt adjustments in the model sea surface height. As a result, most seasonal prediction systems employ a constraint on the globally-averaged sea surface height that is applied at each time step. This essentially renders the prognostic sea surface height variable as unserviceable. Several approaches have previously been used to retrieve sea level information from seasonal forecasts beyond the use of the sea surface height variable. Here, we extend a method of relating other prognostic values, including ocean circulation and climate indices, to observed sea level variations. We use the merged altimetry record of the NASA MEaSUREs Gridded Sea Surface Height Anomalies data set and monthly revised local reference gauge observations from the National Oceanography Centre Permanent Service for Mean Sea Level (PSMSL) to evaluate derived prognostic variables from the NASA Global Modeling and Assimilation Office subseasonal-to-seasonal system version 2.1 (GMAO S2S v2.1). We focus on results for the midlatitudes with particular emphasis on US gauge locations. As shown in previous studies, prognostic ENSO-related indices in boreal winter are well correlated with gauge observations for the US west coast, but also for other locations in the southeastern US. Other forecast climate indices such as the North Atlantic Oscillation have relations to sea level that are limited both seasonally and spatially. As expected, surface atmospheric pressure (e.g., inverse barometer effect) is found to be particularly well correlated with observed sea level. We provide a characterization of forecast skill for seasonal sea level with this method.

Richard I Cullather↗
NASA NTRSDate not supplied

Dynamical Sensitivity in Response to a Wide Range of Abrupt CO 2 Forcings

An improved understanding of dynamical variability is important for projecting future changes in extratropical weather and the interaction between the extratropical troposphere and the tropics, stratosphere, cryosphere and ocean. Despite their simplicity, the abrupt 2x- and 4xCO 2 forcing simulations from the Coupled Model Intercomparison Project (CMIP) Phase 6 DECK experiments enable a mechanistic look into the forcing and feedback response characteristics of models that can be unambiguously attributed to an increase in carbon dioxide concentrations. Thus, while typically used to evaluate the climate sensitivity in models, with a primary focus on global surface temperature change, here we focus on several measures of extratropical variability, including projected changes in the storm tracks and in stratospheric polar vortex variability. Results are primarily based on simulations produced using low- and high-top versions of the NASA Goddard Institute for Space Studies Model (ModelE) but results are also presented from the larger CMIP6 multi-model ensemble. In addition to the 2x- and 4xCO 2 simulations, we also explore the linearity of the response of extratropical dynamical variability in ModelE to varying levels of CO 2 spanning the range 1/8-8xCO 2 . In particular, we show that the expansion of the jet streams varies nonlinearly with increasing CO 2 , especially in the Northern Hemisphere, and that this can be interpreted in terms of variations in ocean heat transport. The impact of composition feedbacks on changes in variability is also discussed.

dynamical variability↗
NASA NTRS2023

Observational Constraints Reduce Model Spread but Not Uncertainty in Global Wetland Methane Emission Estimates

The recent rise in atmospheric methane (CH 4 ) concentrations accelerates climate change and offsets mitigation efforts. Although wetlands are the largest natural CH 4 source, estimates of global wetland CH 4 emissions vary widely among approaches taken by bottom-up (BU) process-based biogeochemical models and top-down (TD) atmospheric inversion methods. Here, we integrate in situ measurements, multi-model ensembles, and a machine learning upscaling product into the International Land Model Benchmarking system to examine the relationship between wetland CH 4 emission estimates and model performance. We find that using better-performing models identified by observational constraints reduces the spread of wetland CH 4 emission estimates by 62% and 39% for BU- and TD-based approaches, respectively. However, global BU and TD CH 4 emission estimate discrepancies increased by about 15% (from 31 to 36 TgCH 4 year −1 ) when the top 20% models were used, although we consider this result moderately uncertain given the unevenly distributed global observations. Our analyses demonstrate that model performance ranking is subject to benchmark selection due to large inter-site variability, highlighting the importance of expanding coverage of benchmark sites to diverse environmental conditions. We encourage future development of wetland CH 4 models to move beyond static benchmarking and focus on evaluating site-specific and ecosystem-specific variabilities inferred from observations.

Kuang-Yu Chang↗
NASA NTRS2024

Evaluation of CMIP6 Model Simulations of PM 2.5 and its Components Over China

Earth system models (ESMs) participating in the latest Coupled Model Intercomparison Project Phase 6 (CMIP6) simulate various components of fine particulate matter (PM 2.5 ) as major climate forcers. Yet the model performance for PM 2.5 components remains little evaluated due in part to a lack of observational data. Here, we evaluate near-surface concentrations of PM 2.5 and its five main components over China as simulated by 14 CMIP6 models, including organic carbon (OC; available in 14 models), black carbon (BC; 14 models), sulfate (14 models), nitrate (4 models), and ammonium (5 models). For this purpose, we collect observational data between 2000 and 2014 from a satellite-based dataset for total PM2.5 and from 2469 measurement records in the literature for PM 2.5 components. Seven models output total PM 2.5 concentrations, and they all underestimate the observed total PM 2.5 over eastern China, with GFDL-ESM4 (−1.5 %) and MPI-ESM-1-2-HAM (−1.1 %) exhibiting the smallest biases averaged over the whole country. The other seven models, for which we recalculate total PM 2.5 from the available component output, underestimate the total PM 2.5 concentrations partly because of the missing model representations of nitrate and ammonium. Concentrations of the five individual components are underestimated in almost all models, except that sulfate is overestimated in MPI-ESM-1-2-HAM by 12.6 % and in MRI-ESM2-0 by 24.5 %. The underestimation is the largest for OC (by −71.2 % to −37.8 % across the 14 models) and the smallest for BC (−47.9 % to −12.1 %). The multi-model mean (MMM) reproduces the observed spatial pattern for OC (R = 0.51), sulfate (R = 0.57), nitrate (R = 0.70) and ammonium (R = 0.74) fairly well, yet the agreement is poorer for BC (R = 0.39). The varying performances of ESMs on total PM 2.5 and its components have important implications for the modeled magnitude and spatial pattern of aerosol radiative forcing.

CMIP6↗
DOE OSTI2023

Nuclear's Role in the U.S. Electricity System: A Multi-Model Inter-Comparison Analysis

A variety of US electric power sector capacity expansion models are used by decision makers and analysts to evaluate competition among generation, transmission, and storage technologies to meeting the demands of the system. CEMs use least-cost optimization to identify optimal portfolios of investments capable of satisfying all specified requirements. While CEMs are a useful tool to inform pathways to meet future needs, projections can differ significantly between tools for apparently similar scenario assumptions. Differences in model structure, scope, and input assumption contribute to this issue. This work compares model response with harmonization between four modeling teams on issues significant to the representation and development of nuclear energy.

capacity expansion model↗