Search NASA⌕ Search

SEARCH · Search NASA

Results for “multi-model ensemble methods”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

25 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Predictive performance of multi-model ensemble forecasts of COVID-19 across European nations

Background: Short-term forecasts of infectious disease burden can contribute to situational awareness and aid capacity planning. Based on best practice in other fields and recent insights in infectious disease epidemiology, one can maximise the predictive performance of such forecasts if multiple models are combined into an ensemble. Here, we report on the performance of ensembles in predicting COVID-19 cases and deaths across Europe between 08 March 2021 and 07 March 2022. Methods: We used open-source tools to develop a public European COVID-19 Forecast Hub. We invited groups globally to contribute weekly forecasts for COVID-19 cases and deaths reported by a standardised source for 32 countries over the next 1–4 weeks. Teams submitted forecasts from March 2021 using standardised quantiles of the predictive distribution. Each week we created an ensemble forecast, where each predictive quantile was calculated as the equally-weighted average (initially the mean and then from 26th July the median) of all individual models’ predictive quantiles. We measured the performance of each model using the relative Weighted Interval Score (WIS), comparing models’ forecast accuracy relative to all other models. We retrospectively explored alternative methods for ensemble forecasts, including weighted averages based on models’ past predictive performance. Results: Over 52 weeks, we collected forecasts from 48 unique models. We evaluated 29 models’ forecast scores in comparison to the ensemble model. We found a weekly ensemble had a consistently strong performance across countries over time. Across all horizons and locations, the ensemble performed better on relative WIS than 83% of participating models’ forecasts of incident cases (with a total N=886 predictions from 23 unique models), and 91% of participating models’ forecasts of deaths (N=763 predictions from 20 models). Across a 1–4 week time horizon, ensemble performance declined with longer forecast periods when forecasting cases, but remained stable over 4 weeks for incident death forecasts. In every forecast across 32 countries, the ensemble outperformed most contributing models when forecasting either cases or deaths, frequently outperforming all of its individual component models. Among several choices of ensemble methods we found that the most influential and best choice was to use a median average of models instead of using the mean, regardless of methods of weighting component forecast models. Conclusions: Our results support the use of combining forecasts from individual models into an ensemble in order to improve predictive performance across epidemiological targets and populations during infectious disease epidemics. Our findings further suggest that median ensemble methods yield better predictive performance more than ones based on means. Our findings also highlight that forecast consumers should place more weight on incident death forecasts than incident case forecasts at forecast horizons greater than 2 weeks. Funding: AA, BH, BL, LWa, MMa, PP, SV funded by National Institutes of Health (NIH) Grant 1R01GM109718, NSF BIG DATA Grant IIS-1633028, NSF Grant No.: OAC-1916805, NSF Expeditions in Computing Grant CCF-1918656, CCF-1917819, NSF RAPID CNS-2028004, NSF RAPID OAC-2027541, US Centers for Disease Control and Prevention 75D30119C05935, a grant from Google, University of Virginia Strategic Investment Fund award number SIF160, Defense Threat Reduction Agency (DTRA) under Contract No. HDTRA1-19-D-0007, and respectively Virginia Dept of Health Grant VDH-21-501-0141, VDH-21-501-0143, VDH-21-501-0147, VDH-21-501-0145, VDH-21-501-0146, VDH-21-501-0142, VDH-21-501-0148. AF, AMa, GL funded by SMIGE - Modelli statistici inferenziali per governare l'epidemia, FISR 2020-Covid-19 I Fase, FISR2020IP-00156, Codice Progetto: PRJ-0695. AM, BK, FD, FR, JK, JN, JZ, KN, MG, MR, MS, RB funded by Ministry of Science and Higher Education of Poland with grant 28/WFSN/2021 to the University of Warsaw. BRe, CPe, JLAz funded by Ministerio de Sanidad/ISCIII. BT, PG funded by PERISCOPE European H2020 project, contract number 101016233. CP, DL, EA, MC, SA funded by European Commission - Directorate-General for Communications Networks, Content and Technology through the contract LC-01485746, and Ministerio de Ciencia, Innovacion y Universidades and FEDER, with the project PGC2018-095456-B-I00. DE., MGu funded by Spanish Ministry of Health / REACT-UE (FEDER). DO, GF, IMi, LC funded by Laboratory Directed Research and Development program of Los Alamos National Laboratory (LANL) under project number 20200700ER. DS, ELR, GG, NGR, NW, YW funded by National Institutes of General Medical Sciences (R35GM119582; the content is solely the responsibility of the authors and does not necessarily represent the official views of NIGMS or the National Institutes of Health). FB, FP funded by InPresa, Lombardy Region, Italy. HG, KS funded by European Centre for Disease Prevention and Control. IV funded by Agencia de Qualitat i Avaluacio Sanitaries de Catalunya (AQuAS) through contract 2021-021OE. JDe, SMo, VP funded by Netzwerk Universitatsmedizin (NUM) project egePan (01KX2021). JPB, SH, TH funded by Federal Ministry of Education and Research (BMBF; grant 05M18SIA). KH, MSc, YKh funded by Project SaxoCOV, funded by the German Free State of Saxony. Presentation of data, model results and simulations also funded by the NFDI4Health Task Force COVID-19 ( https://www.nfdi4health.de/task-force-covid-19-2 ) within the framework of a DFG-project (LO-342/17-1). LP, VE funded by Mathematical and Statistical modelling project (MUNI/A/1615/2020), Online platform for real-time monitoring, analysis and management of epidemic situations (MUNI/11/02202001/2020); VE also supported by RECETOX research infrastructure (Ministry of Education, Youth and Sports of the Czech Republic: LM2018121), the CETOCOEN EXCELLENCE (CZ.02.1.01/0.0/0.0/17-043/0009632), RECETOX RI project (CZ.02.1.01/0.0/0.0/16-013/0001761). NIB funded by Health Protection Research Unit (grant code NIHR200908). SAb, SF funded by Wellcome Trust (210758/Z/18/Z).

60 APPLIED LIFE SCIENCES↗
NASA NTRS2009

Advances in Land Data Assimilation at the NASA Goddard Space Flight Center

Research in land surface data assimilation has grown rapidly over the last decade. In this presentation we provide a brief overview of key research contributions by the NASA Goddard Space Flight Center (GSFC). The GSFC contributions to land assimilation primarily include the continued development and application of the Land Information System (US) and the ensemble Kalman filter (EnKF). In particular, we have developed a method to generate perturbation fields that are correlated in space, time, and across variables and that permit the flexible modeling of errors in land surface models and observations, along with an adaptive filtering approach that estimates observation and model error input parameters. A percentile-based scaling method that addresses soil moisture biases in model and observational estimates opened the path to the successful application of land data assimilation to satellite retrievals of surface soil moisture. Assimilation of AMSR-E surface soil moisture retrievals into the NASA Catchment model provided superior surface and root zone assimilation products (when validated against in situ measurements and compared to the model estimates or satellite observations alone). The multi-model capabilities of US were used to investigate the role of subsurface physics in the assimilation of surface soil moisture observations. Results indicate that the potential of surface soil moisture assimilation to improve root zone information is higher when the surface to root zone coupling is stronger. Building on this experience, GSFC leads the development of the Level 4 Surface and Root-Zone Soil Moisture (L4_SM) product for the planned NASA Soil-Moisture-Active-Passive (SMAP) mission. A key milestone was the design and execution of an Observing System Simulation Experiment that quantified the contribution of soil moisture retrievals to land data assimilation products as a function of retrieval and land model skill and yielded an estimate of the error budget for the SMAP L4_SM product. Terrestrial water storage observations from GRACE satellite system were also successfully assimilated into the NASA Catchment model and provided improved estimates of groundwater variability when compared to the model estimates alone. Moreover, satellite-based land surface temperature (LST) observations from the ISCCP archive were assimilated using a bias estimation module that was specifically designed for LST assimilation. As with soil moisture, LST assimilation provides modest yet statistically significant improvements when compared to the model or satellite observations alone. To achieve the improvement, however, the LST assimilation algorithm must be adapted to the specific formulation of LST in the land model. An improved method for the assimilation of snow cover observations was also developed. Finally, the coupling of LIS to the mesoscale Weather Research and Forecasting (WRF) model enabled investigations into how the sensitivity of land-atmosphere interactions to the specific choice of planetary boundary layer scheme and land surface model varies across surface moisture regimes, and how it can be quantified and evaluated against observations. The on-going development and integration of land assimilation modules into the Land Information System will enable the use of GSFC software with a variety of land models and make it accessible to the research community.

Reichle, Rolf↗
NASA NTRS2014

A Practical Philosophy of Complex Climate Modelling

We give an overview of the practice of developing and using complex climate models, as seen from experiences in a major climate modelling center and through participation in the Coupled Model Intercomparison Project (CMIP).We discuss the construction and calibration of models; their evaluation, especially through use of out-of-sample tests; and their exploitation in multi-model ensembles to identify biases and make predictions. We stress that adequacy or utility of climate models is best assessed via their skill against more naive predictions. The framework we use for making inferences about reality using simulations is naturally Bayesian (in an informal sense), and has many points of contact with more familiar examples of scientific epistemology. While the use of complex simulations in science is a development that changes much in how science is done in practice, we argue that the concepts being applied fit very much into traditional practices of the scientific method, albeit those more often associated with laboratory work.

complex simulation↗
NASA NTRSDate not supplied

A Framework for Evaluating Climate Model Performance Metrics

The CMIP5 archive contains future climate projections from over 50 models provided by dozens of modeling centers from around the world. Individual model projections, however, are subject to biases created by structural model uncertainties. As a result, ensemble averaging of multiple models is often used to add value to model projections: consensus projections have been shown to consistently outperform individual models. Previous reports for the IPCC establish climate change projections based on an equal-weighted average of all model projections. However, certain models reproduce climate processes better than other models. Should models be weighted based on performance? Unequal ensemble averages have previously been constructed using a variety of mean state metrics. What metrics are most relevant for constraining future climate projections? This project develops a framework for systematically testing metrics in models to identify optimal metrics for unequal weighting multi-model ensembles. A unique aspect of this project is the construction and testing of climate process-based model evaluation metrics. A climate process-based metric is defined as a metric based on the relationship between two physically related climate variables?e.g., outgoing longwave radiation and surface temperature. Metrics are constructed using high-quality Earth radiation budget data from NASA's Clouds and Earth's Radiant Energy System (CERES) instrument and surface temperature data sets. It is found that regional values of tested quantities can vary significantly when comparing weighted and unweighted model ensembles. For example, one tested metric weights the ensemble by how well models reproduce the time-series probability distribution of the cloud forcing component of reflected shortwave radiation. The weighted ensemble for this metric indicates lower simulated precipitation (up to .7 mm/day) in tropical regions than the unweighted ensemble: since CMIP5 models have been shown to overproduce precipitation, this result could indicate that the metric is effective in identifying models which simulate more realistic precipitation. Ultimately, the goal of the framework is to identify performance metrics for advising better methods for ensemble averaging models and create better climate predictions.

Noel C Baker↗
NASA NTRS2021

Increases in Future AR Count and Size: Overview of 2 the ARTMIP Tier 2 CMIP5/6 Experiment

The Atmospheric River (AR) Tracking Method Intercomparison Project (ARTMIP) is a community effort to systematically assess how the uncertainties from AR detectors (ARDTs) impact our scientific understanding of ARs. This study describes the ARTMIP Tier 2 experimental design and initial results using the Coupled Model Intercomparison Project (CMIP) Phases 5 and 6 multi-model ensembles. We show that AR statistics from a given ARDT in CMIP5/6 historical simulations compare remarkably well with the MERRA-2 reanalysis. In CMIP5/6 future simulations, most ARDTs project a global increase in AR frequency, counts, and sizes, especially along the western coastlines of the Pacific and Atlantic oceans. We find that the choice of ARDT is the dominant contributor to the uncertainty in projected AR frequency when compared with model choice. These results imply that new projects investigating future changes in ARs should explicitly consider ARDT uncertainty as a core part of the experimental design. Plain Language Summary: Atmospheric rivers (ARs) are a type of weather pattern known to be important for moving water vapor from the warm, moist tropics to the cool, dry polar regions; when they reach midlatitudes in the winter time, they are commonly associated with heavy precipitation. Recent studies that assess the impacts of global climate change on ARs tend to agree that there will be more ARs in a warmer climate, and that ARs will tend to be more extreme. However, it has been increasingly recognized by the AR research community that these results may depend on the method used to identify ARs and the choice of climate model. This study reports results from a controlled experiment, involving an international research community, that aims to show how different AR identification methods and climate models might impact our scientific understanding of ARs in the future. This experiment shows that there will likely be more ARs in the future, and that ARs will generally have a larger spatial footprint. This experiment also shows that uncertainty in these results are large, with the uncertainty from AR identification methods outweighing that of climate models. Future efforts to better understand the physics of ARs may help us reduce this uncertainty.

T A OBrien↗
NASA NTRSDate not supplied

Inferred Sea Level Prediction in the NASA GMAO Seasonal Forecasting System

Reliable predictions of sea level anomalies on seasonal timescales with lead times of 1 to 9 months may have relevance to stakeholders – for example, in the advance deployment of resources for coastal flood mitigation. Routine prediction and analysis may also highlight physical processes associated with sea level change and modeling capabilities on seasonal and other timescales. These forecasts may represent interannual changes in the seasonal slope of the ocean surface, teleconnection effects such as the El Niño/Southern Oscillation phenomenon, and variations in seasonal hydrology including precipitation and coastal runoff. Coupled atmosphere/ocean models are routinely used in the seasonal prediction of temperature anomalies, precipitation anomalies, sea ice cover, and climate indices such as the Niño3.4 predictions under the North American Multi-Model Ensemble (NMME) protocol. Within the limits of their configuration, these complex Earth-system models have a potential for depicting regional changes in oceanic column properties, including the sea surface height. Seasonal prediction models generally have no representation of long-term mass contributions from melting land ice, or changes in vertical land motion; their output may be more specifically characterized as predictions of the ocean dynamic sea level. In practice however, the sea surface height prognostic variable is substantially compromised by the forecast model response to initial conditions. Imbalances between the initial, observed hydrologic cycle and the forecast model state produce abrupt adjustments in the model sea surface height. As a result, most seasonal prediction systems employ a constraint on the globally-averaged sea surface height that is applied at each time step. This essentially renders the prognostic sea surface height variable as unserviceable. Several approaches have previously been used to retrieve sea level information from seasonal forecasts beyond the use of the sea surface height variable. Here, we extend a method of relating other prognostic values, including ocean circulation and climate indices, to observed sea level variations. We use the merged altimetry record of the NASA MEaSUREs Gridded Sea Surface Height Anomalies data set and monthly revised local reference gauge observations from the National Oceanography Centre Permanent Service for Mean Sea Level (PSMSL) to evaluate derived prognostic variables from the NASA Global Modeling and Assimilation Office subseasonal-to-seasonal system version 2.1 (GMAO S2S v2.1). We focus on results for the midlatitudes with particular emphasis on US gauge locations. As shown in previous studies, prognostic ENSO-related indices in boreal winter are well correlated with gauge observations for the US west coast, but also for other locations in the southeastern US. Other forecast climate indices such as the North Atlantic Oscillation have relations to sea level that are limited both seasonally and spatially. As expected, surface atmospheric pressure (e.g., inverse barometer effect) is found to be particularly well correlated with observed sea level. We provide a characterization of forecast skill for seasonal sea level with this method.

Richard I Cullather↗
NASA NTRS2023

Heat Stress to Jeopardize Crop Production in the US Corn Belt Based on Downscaled CMIP5 Projections

CONTEXT Global food security faces increasing challenges from the changing climate. Changes of agricultural output from some of the most productive regions such as the US Corn Belt can largely affect the world's food market. Developing predictive understanding of the agricultural risk of climate change and potential mitigation strategies is critical for the global food security. OBJECTIVE The objective of this study is to assess the responses of maize and soybean yield to projected climate changes in the Corn Belt, identify the shifting environment stressors on crop yield, and tackle potential climate adaptation strategies. METHODS We drive a process-based model, the Decision Support System for Agrotechnology Transfer, with high-resolution statistically downscaled and bias-corrected historical and future climates from ten CMIP5 models in the MACA-2 database. RESULTS AND CONCLUSIONS The multi-model ensemble mean suggests a 12% decrease of maize yield by mid-century and 40% by late century, with a high degree of model consensus in the direction of changes; for individual models, the projected decrease of maize yield by late century ranges from <5% to over 80%, with the worst crop outcome corresponding to the most sensitive climate models. Soybean yield is projected to increase by midcentury with a high degree of model consensus, but such consensus is lost by late century as some projections shift to significant decreases. Crop yield in the Corn Belt is currently limited by water stress, but is projected to be increasingly limited by heat stress as well after the midcentury. The mounting heat stress will drive the most productive zone for maize to shift from central to northern part of the Corn Belt, but the projected increase in the northern states cannot fully compensate for the decrease in the south, causing the total production to decrease if agricultural practice stays the same. Earlier planting can alleviate only a small fraction of the heat-induced crop loss in a warmer climate. Climate change will (at least partially) offset the yield boost caused by agricultural technology and intensification. SIGNIFICANCE This study advances our predictive understanding of crop yield responses to climate change, and suggests that a multitude of strategies will be needed to address the climate change challenges for the U.S. agriculture.

Crop yield↗