SEARCH · Search NASA

Results for “multi-model evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

All sources NASA NTRS DOE OSTI

NASA NTRS2009

Advances in Land Data Assimilation at the NASA Goddard Space Flight Center

Research in land surface data assimilation has grown rapidly over the last decade. In this presentation we provide a brief overview of key research contributions by the NASA Goddard Space Flight Center (GSFC). The GSFC contributions to land assimilation primarily include the continued development and application of the Land Information System (US) and the ensemble Kalman filter (EnKF). In particular, we have developed a method to generate perturbation fields that are correlated in space, time, and across variables and that permit the flexible modeling of errors in land surface models and observations, along with an adaptive filtering approach that estimates observation and model error input parameters. A percentile-based scaling method that addresses soil moisture biases in model and observational estimates opened the path to the successful application of land data assimilation to satellite retrievals of surface soil moisture. Assimilation of AMSR-E surface soil moisture retrievals into the NASA Catchment model provided superior surface and root zone assimilation products (when validated against in situ measurements and compared to the model estimates or satellite observations alone). The multi-model capabilities of US were used to investigate the role of subsurface physics in the assimilation of surface soil moisture observations. Results indicate that the potential of surface soil moisture assimilation to improve root zone information is higher when the surface to root zone coupling is stronger. Building on this experience, GSFC leads the development of the Level 4 Surface and Root-Zone Soil Moisture (L4_SM) product for the planned NASA Soil-Moisture-Active-Passive (SMAP) mission. A key milestone was the design and execution of an Observing System Simulation Experiment that quantified the contribution of soil moisture retrievals to land data assimilation products as a function of retrieval and land model skill and yielded an estimate of the error budget for the SMAP L4_SM product. Terrestrial water storage observations from GRACE satellite system were also successfully assimilated into the NASA Catchment model and provided improved estimates of groundwater variability when compared to the model estimates alone. Moreover, satellite-based land surface temperature (LST) observations from the ISCCP archive were assimilated using a bias estimation module that was specifically designed for LST assimilation. As with soil moisture, LST assimilation provides modest yet statistically significant improvements when compared to the model or satellite observations alone. To achieve the improvement, however, the LST assimilation algorithm must be adapted to the specific formulation of LST in the land model. An improved method for the assimilation of snow cover observations was also developed. Finally, the coupling of LIS to the mesoscale Weather Research and Forecasting (WRF) model enabled investigations into how the sensitivity of land-atmosphere interactions to the specific choice of planetary boundary layer scheme and land surface model varies across surface moisture regimes, and how it can be quantified and evaluated against observations. The on-going development and integration of land assimilation modules into the Land Information System will enable the use of GSFC software with a variety of land models and make it accessible to the research community.

Reichle, Rolf↗

NASA NTRS2012

A HTAP Multi-Model Assessment of the Influence of Regional Anthropogenic Emission Reductions on Aerosol Direct Radiative Forcing and the Role of Intercontinental Transport

In this study, we assess changes of aerosol optical depth (AOD) and direct radiative forcing (DRF) in response to the reduction of anthropogenic emissions in four major pollution regions in the northern hemisphere by using results from 10 global chemical transport models in the framework of the Hemispheric Transport of Air Pollution (HTAP). The multi-model results show that on average, a 20% reduction of anthropogenic emissions in North America, Europe, East Asia and South Asia lowers the global mean AOD and DRF by about 9%, 4%, and 10% for sulfate, organic matter, and black carbon aerosol, respectively. The impacts of the regional emission reductions on AOD and DRF extend well beyond the source regions because of intercontinental transport. On an annual basis, intercontinental transport accounts for 10-30% of the overall AOD and DRF in a receptor region, with domestic emissions accounting for the remainder, depending on regions and species. While South Asia is most influenced by import of sulfate aerosol from Europe, North America is most influenced by import of black carbon from East Asia. Results show a large spread among models, highlighting the need to improve aerosol processes in models and evaluate and constrain models with observations.

Yu, Hongbin↗

NASA NTRS2013

Impacts of Climate Change on Surface Ozone and Intercontinental Ozone Pollution: A Multi-Model Study

The impact of climate change between 2000 and 2095 SRES A2 climates on surface ozone (O)3 and on O3 source-receptor (S-R) relationships is quantified using three coupled climate-chemistry models (CCMs). The CCMs exhibit considerable variability in the spatial extent and location of surface O3 increases that occur within parts of high NOx emission source regions (up to 6 ppbv in the annual average and up to 14 ppbv in the season of maximum O3). In these source regions, all three CCMs show a positive relationship between surface O3 change and temperature change. Sensitivity simulations show that a combination of three individual chemical processes-(i) enhanced PAN decomposition, (ii) higher water vapor concentrations, and (iii) enhanced isoprene emission-largely reproduces the global spatial pattern of annual-mean surface O3 response due to climate change (R2 = 0.52). Changes in climate are found to exert a stronger control on the annual-mean surface O3 response through changes in climate-sensitive O3 chemistry than through changes in transport as evaluated from idealized CO-like tracer concentrations. All three CCMs exhibit a similar spatial pattern of annual-mean surface O3 change to 20% regional O3 precursor emission reductions under future climate compared to the same emission reductions applied under present-day climate. The surface O3 response to emission reductions is larger over the source region and smaller downwind in the future than under present-day conditions. All three CCMs show areas within Europe where regional emission reductions larger than 20% are required to compensate climate change impacts on annual-mean surface O3.

Troposphere: constituent transport and chemistry↗

NASA NTRS2013

How Well Do Global Climate Models Simulate the Variability of Atlantic Tropical Cyclones Associated with ENSO?

The variability of Atlantic tropical cyclones (TCs) associated with El Nino-Southern Oscillation (ENSO) in model simulations is assessed and compared with observations. The model experiments are 28-yr simulations forced with the observed sea surface temperature from 1982 to 2009. The simulations were coordinated by the U.S. CLIVAR Hurricane Working Group and conducted with five global climate models (GCMs) with a total of 16 ensemble members. The model performance is evaluated based on both individual model ensemble means and multi-model ensemble mean. The latter has the highest anomaly correlation (0.86) for the interannual variability of TCs. Previous observational studies show a strong association between ENSO and Atlantic TC activity, as well as distinctions in the TC activities during eastern Pacific (EP) and central Pacific (CP) El Nino events. The analysis of track density and TC origin indicates that each model has different mean biases. Overall, the GCMs simulate the variability of Atlantic TCs well with weaker activity during EP El Nino and stronger activity during La Nina. For CP El Nino, there is a slight increase in the number of TCs as compared with EP El Nino. However, the spatial distribution of track density and TC origin is less consistent among the models. Particularly, there is no indication of increasing TC activity over the U.S. southeast coastal region as in observations. The difference between the models and observations is likely due to the bias of vertical wind shear in response to the shift of tropical heating associated with CP El Nino, as well as the model bias in the mean circulation.

Tropical Cyclones↗

NASA NTRS2013

Evaluation of Intercomparisons of Four Different Types of Model Simulating TWP-ICE

Four model intercomparisons were run and evaluated using the TWP-ICE field campaign, each involving different types of atmospheric model. Here we highlight what can be learnt from having single-column model (SCM), cloud-resolving model (CRM), global atmosphere model (GAM) and limited-area model (LAM) intercomparisons all based around the same field campaign. We also make recommendations for anyone planning further large multi-model intercomparisons to ensure they are of maximum value to the model development community. CRMs tended to match observations better than other model types, although there were exceptions such as outgoing long-wave radiation. All SCMs grew large temperature and moisture biases and performed worse than other model types for many diagnostics. The GAMs produced a delayed and significantly reduced peak in domain-average rain rate when compared to the observations. While it was shown that this was in part due to the analysis used to drive these models, the LAMs were also driven by this analysis and did not have the problem to the same extent. Based on differences between the models with parametrized convection (SCMs and GAMs) and those without (CRMs and LAMs), we speculate that that having explicit convection helps to constrain liquid water whereas the ice contents are controlled more by the representation of the microphysics.

convection↗

NASA NTRS2015

AgMIP Training in Multiple Crop Models and Tools

The Agricultural Model Intercomparison and Improvement Project (AgMIP) has the goal of using multiple crop models to evaluate climate impacts on agricultural production and food security in developed and developing countries. There are several major limitations that must be overcome to achieve this goal, including the need to train AgMIP regional research team (RRT) crop modelers to use models other than the ones they are currently familiar with, plus the need to harmonize and interconvert the disparate input file formats used for the various models. Two activities were followed to address these shortcomings among AgMIP RRTs to enable them to use multiple models to evaluate climate impacts on crop production and food security. We designed and conducted courses in which participants trained on two different sets of crop models, with emphasis on the model of least experience. In a second activity, the AgMIP IT group created templates for inputting data on soils, management, weather, and crops into AgMIP harmonized databases, and developed translation tools for converting the harmonized data into files that are ready for multiple crop model simulations. The strategies for creating and conducting the multi-model course and developing entry and translation tools are reviewed in this chapter.

farm crops↗

NASA NTRS2015

The Agricultural Model Intercomparison and Improvement Project: Phase I Activities by a Global Community of Science

The Agricultural Model Intercomparison and Improvement Project (AgMIP) was founded in 2010. Its mission is to improve substantially the characterization of world food security as affected by climate variability and change, and to enhance adaptation capacity in both developing and developed countries. The objectives of AgMIP are to: Incorporate state-of-the-art climate, crop/livestock, and agricultural economic model improvements into coordinated multi-model regional and global assessments of future climate impacts and adaptation and other key aspects of the food system. Utilize multiple models, scenarios, locations, crops/livestock, and participants to explore uncertainty and the impact of data and methodological choices. Collaborate with regional experts in agronomy, animal sciences, economics, and climate to build a strong basis for model applications, addressing key climate related questions and sustainable intensification farming systems. Improve scientific and adaptive capacity in modeling for major agricultural regions in the developing and developed world, with a focus on vulnerable regions. Improve agricultural data and enhance data-sharing based on their intercomparison and evaluation using best scientific practices. Develop modeling frameworks to identify and evaluate promising adaptation technologies and policies and to prioritize strategies.

farm crops↗

NASA NTRS2015

Multi-Model Ensemble Wake Vortex Prediction

Several multi-model ensemble methods are investigated for predicting wake vortex transport and decay. This study is a joint effort between National Aeronautics and Space Administration and Deutsches Zentrum fuer Luft- und Raumfahrt to develop a multi-model ensemble capability using their wake models. An overview of different multi-model ensemble methods and their feasibility for wake applications is presented. The methods include Reliability Ensemble Averaging, Bayesian Model Averaging, and Monte Carlo Simulations. The methodologies are evaluated using data from wake vortex field experiments.

Koerner, Stephan↗

NASA NTRS2015

The ENSO Effects on Tropical Clouds and Top-of-Atmosphere Cloud Radiative Effects in CMIP5 Models

The El Nino-Southern Oscillation (ENSO) effects on tropical clouds and top-of-atmosphere (TOA) cloud radiative effects (CREs) in Coupled Model Intercomparison Project Phase5 (CMIP5) models are evaluated using satellite-based observations and International Satellite Cloud Climatology Project satellite simulator output. Climatologically, most CMIP5 models produce considerably less total cloud amount with higher cloud top and notably larger reflectivity than observations in tropical Indo-Pacific (60 degrees East - 200 degrees East; 10 degrees South - 10 degrees North). During ENSO, most CMIP5 models considerably underestimate TOA CRE and cloud changes over western tropical Pacific. Over central tropical Pacific, while the multi-model mean resembles observations in TOA CRE and cloud amount anomalies, it notably overestimates cloud top pressure (CTP) decreases; there are also substantial inter-model variations. The relative effects of changes in cloud properties, temperature and humidity on TOA CRE anomalies during ENSO in the CMIP5 models are assessed using cloud radiative kernels. The CMIP5 models agree with observations in that their TOA shortwave CRE anomalies are primarily contributed by total cloud amount changes, and their TOA longwave CRE anomalies are mostly contributed by changes in both total cloud amount and CTP. The model biases in TOA CRE anomalies particularly the strong underestimations over western tropical Pacific are, however, mainly explained by model biases in CTP and cloud optical thickness (tau) changes. Despite the distinct model cloud biases particularly in tau regime, the TOA CRE anomalies from cloud amount changes are comparable between the CMIP5 models and observations, because of the strong compensations between model underestimation of TOA CRE anomalies from thin clouds and overestimation from medium and thick clouds.

Su, Wenying↗

NASA NTRS2015

Multi-Model Ensemble Wake Vortex Prediction

Koerner, Stephan↗

NASA NTRS2016

Multi-Wheat-Model Ensemble Responses to Interannual Climate Variability

We compare 27 wheat models' yield responses to interannual climate variability, analyzed at locations in Argentina, Australia, India, and The Netherlands as part of the Agricultural Model Intercomparison and Improvement Project (AgMIP) Wheat Pilot. Each model simulated 1981e2010 grain yield, and we evaluate results against the interannual variability of growing season temperature, precipitation, and solar radiation. The amount of information used for calibration has only a minor effect on most models' climate response, and even small multi-model ensembles prove beneficial. Wheat model clusters reveal common characteristics of yield response to climate; however models rarely share the same cluster at all four sites indicating substantial independence. Only a weak relationship (R2 0.24) was found between the models' sensitivities to interannual temperature variability and their response to long-termwarming, suggesting that additional processes differentiate climate change impacts from observed climate variability analogs and motivating continuing analysis and model development efforts.

uncertainty↗

NASA NTRS2016

Crop Model Improvement Reduces the Uncertainty of the Response to Temperature of Multi-Model Ensembles

To improve climate change impact estimates and to quantify their uncertainty, multi-model ensembles (MMEs) have been suggested. Model improvements can improve the accuracy of simulations and reduce the uncertainty of climate change impact assessments. Furthermore, they can reduce the number of models needed in a MME. Herein, 15 wheat growth models of a larger MME were improved through re-parameterization and/or incorporating or modifying heat stress effects on phenology, leaf growth and senescence, biomass growth, and grain number and size using detailed field experimental data from the USDA Hot Serial Cereal experiment (calibration data set). Simulation results from before and after model improvement were then evaluated with independent field experiments from a CIMMYT worldwide field trial network (evaluation data set). Model improvements decreased the variation (10th to 90th model ensemble percentile range) of grain yields simulated by the MME on average by 39% in the calibration data set and by 26% in the independent evaluation data set for crops grown in mean seasonal temperatures greater than 24 C. MME mean squared error in simulating grain yield decreased by 37%. A reduction in MME uncertainty range by 27% increased MME prediction skills by 47%. Results suggest that the mean level of variation observed in field experiments and used as a benchmark can be reached with half the number of models in the MME. Improving crop models is therefore important to increase the certainty of model-based impact assessments and allow more practical, i.e. smaller MMEs to be used effectively.

damage assessment↗

NASA NTRS2016

Lessons from Climate Modeling on the Design and Use of Ensembles for Crop Modeling

Working with ensembles of crop models is a recent but important development in crop modeling which promises to lead to better uncertainty estimates for model projections and predictions, better predictions using the ensemble mean or median, and closer collaboration within the modeling community. There are numerous open questions about the best way to create and analyze such ensembles. Much can be learned from the field of climate modeling, given its much longer experience with ensembles. We draw on that experience to identify questions and make propositions that should help make ensemble modeling with crop models more rigorous and informative. The propositions include defining criteria for acceptance of models in a crop MME, exploring criteria for evaluating the degree of relatedness of models in a MME, studying the effect of number of models in the ensemble, development of a statistical model of model sampling, creation of a repository for MME results, studies of possible differential weighting of models in an ensemble, creation of single model ensembles based on sampling from the uncertainty distribution of parameter values or inputs specifically oriented toward uncertainty estimation, the creation of super ensembles that sample more than one source of uncertainty, the analysis of super ensemble results to obtain information on total uncertainty and the separate contributions of different sources of uncertainty and finally further investigation of the use of the multi-model mean or median as a predictor.

Model ensembles↗

NASA NTRS2016

Uncertainty of Wheat Water Use: Simulated Patterns and Sensitivity to Temperature and CO2

Projected global warming and population growth will reduce future water availability for agriculture. Thus, it is essential to increase the efficiency in using water to ensure crop productivity. Quantifying crop water use (WU; i.e. actual evapotranspiration) is a critical step towards this goal. Here, sixteen wheat simulation models were used to quantify sources of model uncertainty and to estimate the relative changes and variability between models for simulated WU, water use efficiency (WUE, WU per unit of grain dry mass produced), transpiration efficiency (Teff, transpiration per kg of unit of grain yield dry mass produced), grain yield, crop transpiration and soil evaporation at increased temperatures and elevated atmospheric carbon dioxide concentrations ([CO2]). The greatest uncertainty in simulating water use, potential evapotranspiration, crop transpiration and soil evaporation was due to differences in how crop transpiration was modelled and accounted for 50 of the total variability among models. The simulation results for the sensitivity to temperature indicated that crop WU will decline with increasing temperature due to reduced growing seasons. The uncertainties in simulated crop WU, and in particularly due to uncertainties in simulating crop transpiration, were greater under conditions of increased temperatures and with high temperatures in combination with elevated atmospheric [CO2] concentrations. Hence the simulation of crop WU, and in particularly crop transpiration under higher temperature, needs to be improved and evaluated with field measurements before models can be used to simulate climate change impacts on future crop water demand.

Sensitivity↗

NASA NTRS2016

Regime-Based Evaluation of Cloudiness in CMIP5 Models

The concept of Cloud Regimes (CRs) is used to develop a framework for evaluating the cloudiness of 12 fifth Coupled Model Intercomparison Project (CMIP5) models. Reference CRs come from existing global International Satellite Cloud Climatology Project (ISCCP) weather states. The evaluation is made possible by the implementation in several CMIP5 models of the ISCCP simulator generating for each gridcell daily joint histograms of cloud optical thickness and cloud top pressure. Model performance is assessed with several metrics such as CR global cloud fraction (CF), CR relative frequency of occurrence (RFO), their product (long-term average total cloud amount [TCA]), cross-correlations of CR RFO maps, and a metric of resemblance between model and ISCCP CRs. In terms of CR global RFO, arguably the most fundamental metric, the models perform unsatisfactorily overall, except for CRs representing thick storm clouds. Because model CR CF is internally constrained by our method, RFO discrepancies yield also substantial TCA errors. Our findings support previous studies showing that CMIP5 models underestimate cloudiness. The multi-model mean performs well in matching observed RFO maps for many CRs, but is not the best for this or other metrics. When overall performance across all CRs is assessed, some models, despite their shortcomings, apparently outperform Moderate Resolution Imaging Spectroradiometer (MODIS) cloud observations evaluated against ISCCP as if they were another model output. Lastly, cloud simulation performance is contrasted with each model's equilibrium climate sensitivity (ECS) in order to gain insight on whether good cloud simulation pairs with particular values of this parameter.

cloud climatology↗

NASA NTRS2017

The Critical Role of the Routing Scheme in Simulating Peak River Discharge in Global Hydrological Models

Global hydrological models (GHMs) have been applied to assess global flood hazards, but their capacity to capture the timing and amplitude of peak river discharge which is crucial in flood simulations has traditionally not been the focus of examination. Here we evaluate to what degree the choice of river routing scheme affects simulations of peak discharge and may help to provide better agreement with observations. To this end we use runoff and discharge simulations of nine GHMs forced by observational climate data (1971-2010) within the ISIMIP2a (Inter-Sectoral Impact Model Intercomparison Project phase 2a) project. The runoff simulations were used as input for the global river routing model CaMa-Flood (Catchment-based Macro-scale Floodplain). The simulated daily discharge was compared to the discharge generated by each GHM using its native river routing scheme. For each GHM both versions of simulated discharge were compared to monthly and daily discharge observations from 1701 GRDC (Global Runoff Data Centre) stations as a benchmark. CaMa-Flood routing shows a general reduction of peak river discharge and a delay of about two to three weeks in its occurrence, likely induced by the buffering capacity of floodplain reservoirs. For a majority of river basins, discharge produced by CaMa-Flood resulted in a better agreement with observations. In particular, maximum daily discharge was adjusted, with a multi-model averaged reduction in bias over about two-thirds of the analysed basin area. The increase in agreement was obtained in both managed and near-natural basins. Overall, this study demonstrates the importance of routing scheme choice in peak discharge simulation, where CaMa-Flood routing accounts for floodplain storage and backwater effects that are not represented in most GHMs. Our study provides important hints that an explicit parameterisation of these processes may be essential in future impact studies.

peak river discharge↗

NASA NTRS2019

The Subseasonal Experiment (SubX): A Multi-Model Subseasonal Prediction Experiment

SubX is a multi-model subseasonal prediction experiment designed around operational requirements with the goal of improving subseasonal forecasts. Seven global models have produced seventeen years of retrospective (re-) forecasts and more than a year of weekly real-time forecasts. The re-forecasts and forecasts are archived at the Data Library of the International Research Institute for Climate and Society, Columbia University, providing a comprehensive database for research on subseasonal to seasonal predictability and predictions. The SubX models show skill for temperature and precipitation three weeks ahead of time in specific regions. The SubX multi-model ensemble mean is more skillful than any individual model overall. Skill in simulating the Madden-Julian Oscillation (MJO) and the North Atlantic Oscillation (NAO), two sources of subseasonal predictability, is also evaluated with skillful predictions of the MJO four weeks in advance and of the NAO 2 weeks in advance. SubX is also able to make useful contributions to operational forecast guidance at the Climate Prediction Center. Additionally, SubX provides information on the potential for extreme precipitation associated with tropical cyclones which can help emergency management and aid organizations to plan for disasters. (Capsule Summary) A research to operations project in service of developing better operational subseasonal forecasts.

Precipitation↗

NASA NTRS2019

Historical (1700–2012) Global Multi-Model Estimates of the Fire Emissions from the Fire Modeling Intercomparison Project (FireMIP)

Fire emissions are critical for carbon and nutrient cycles, climate, and air quality. Dynamic Global Vegetation Models (DGVMs) with interactive fire modeling provide important estimates for long-term and large-scale changes of fire emissions. Here we present the first multi-model estimates of global gridded historical fire emissions for 1700-2012, including carbon and 33 species of trace gases and aerosols. The dataset is based on simulations of nine DGVMs with different state-of-the-art global fire models that participated in the Fire Modeling Intercomparison Project (FireMIP), using the same and standardized protocols and forcing data, and the most up-to-date fire emission factor table from field and laboratory studies over various land cover types. We evaluate the simulations of present-day fire emissions by comparing them with satellite-based products. Evaluation results show that most DGVMs simulate present-day global fire emission totals within the range of satellite-based products, and can capture the high emissions over the tropical savannas, low emissions over the arid and sparsely vegetated regions, and the main features of seasonality. However, most of the models fail to simulate the interannual variability, partly due to a lack of modeling peat fires and tropical deforestation fires. Historically, all models show only a weak trend in global fire emissions before ~1850s, consistent with multi-source merged historical reconstructions. The long-term trends among DGVMs are quite different for the 20th century, with some models showing an increase and others a decrease in fire emissions, mainly as a result of the discrepancy in their simulated responses to human population density change and land-use and land-cover change (LULCC). Our study provides a basic dataset for developing regional and global multi-source merged historical reconstructions and merging methods, and analyzing historical changes of fire emissions and their uncertainties as well as their role in the Earth system. It also highlights the importance of accurately modeling the responses of fire emissions to LULCC and population density change in reducing uncertainties in historical reconstructions of fire emissions and providing more reliable future projections.

Li, Fang↗