DOE OSTI · 3452368
Rapid Evaluation Framework for the CMIP7 Assessment Fast Track
Abstract
As Earth system models (ESMs) grow in complexity and in volume of output data, there is an increasing need for rapid, comprehensive evaluation of their scientific performance. The upcoming Assessment Fast Track for the Seventh Phase of the Coupled Model Intercomparison Project (CMIP7) will require expeditious response for model analyses designed to inform and drive integrated Earth system assessments. To meet this challenge, the Rapid Evaluation Framework (REF), a community-driven platform for benchmarking and performance assessment of ESMs, was designed and developed. The initial implementation of the REF, constructed to meet the near-term needs of the CMIP7 Assessment Fast Track, builds upon four disparate community evaluation and benchmarking tools that are coupled together using the Coordinated Model Evaluation Capabilities (CMEC) framework. The REF runs within a containerized workflow for portability and reproducibility and is aimed at generating and organizing diagnostics covering a variety of model variables. The REF leverages well documented observational datasets to provide assessments of model fidelity across a collection of diagnostics. All diagnostics were identified and selected with community involvement and consultation. Operational integration with the Earth System Grid Federation (ESGF) will permit automated execution of the REF for selected diagnostics as soon as model output data are published on ESGF by the originating modeling centers. The REF is designed to be portable across a range of current computational platforms to facilitate use by modeling centers for assessing the evolution of model versions or gauging the relative performance of CMIP simulations before being published on ESGF. When integrated into production simulation workflows, results from the REF provide immediate quantitative feedback that allows model developers and scientists to quickly identify model biases and performance issues. After the REF is released to the community, its subsequent development and support will be prioritized by an international consortium of scientists and engineers, enabling a broader impact across Earth science disciplines. For instance, the REF will facilitate improvements to models and will enhance confidence in model projections through process-based selection of models based on their performance with respect to observations. Production of reproducible diagnostics and community-based assessments are key features of the REF. Furthermore, providing interoperability with existing evaluation packages assures that contributions from previous community efforts will be available for use in future model intercomparison projects.
Keep this discovery
Explore connections, maps & timelines
Hoffman, Forrest [ORNL] (ORCID:0000000158024134), Hassler, Birgit [German Aerospace Center (DLR) Oberpfaffenhofen], Swaminathan, Ranjini [University of Reading, UK], Lewis, Jared [Climate Resource], Andela, Bouwe [Netherlands eScience Center], Collier, Nathaniel [ORNL] (ORCID:0000000273673981), Hegedus, Dora [STFC Rutherford Appleton Laboratory, NCAS/BADC], Lee, Jiwoo [Lawrence Livermore National Laboratory (LLNL)], Pascoe, Charlotte [National Centre for Atmospheric Science (NCAS), Natural Environment Research Council, UK], Pfluger, Mika [Climate Resource], Ullrich, Paul [Lawrence Livermore National Laboratory (LLNL)], Xu, Min [ORNL] (ORCID:0000000334430300), Bock, Lisa [Deutsches Zentrum für Luft- und Raumfahrt e.V. (DLR)], Chun, Felicity [Australian National University (ANU)], Gier, Bettina [Deutsches Zentrum für Luft- und Raumfahrt e.V. (DLR)], Kelley, Douglas [UK Centre for Ecology & Hydrology], Lauer, Axel [German Aerospace Center (DLR) Oberpfaffenhofen], Lenhardt, Julien [Swedish Meteorological and Hydrological Institute], Schlund, Manuel [German Aerospace Center (DLR) Oberpfaffenhofen], Sreeush, Mohanan G. [Alfred Wegener Institute (AWI)], Weigel, Katja [University of Bremen, Germany], Blockley, Ed [UK Met Office, Exeter, UK], Beadling, Rebecca [Temple University], Beucher, Romain [Australian National University (ANU)], Dugassa, Demiso [Arba Minch University], Lembo, Valerio [Institute of Atmospheric Sciences and Climate], Lu, Jianhua [School of Atmospheric Sciences, Sun Yat-sen University & Southern Marine Science and Engineering Gua], Brands, Swen [Instituto de Física de Cantabria (CSIC-UC)], Tjiputra, Jerry F. [Uni Research, Climate, Bergen, Norway], Malinina, Elizaveta [Canadian Centre for Climate Modeling and Analysis], Mederios, Brian [National Center for Atmospheric Research (NCAR)], Soccimarro, Enrico [Euro-Mediterranean Center on Climate Change, Italy], Walton, Jeremy [Hadley Centre, UK Met Office], Kershaw, Philip [STFC Rutherford Appleton Laboratory, NCAS/BADC], Marquez, Andre [National Institute for Space Research (INPE), Sao Paulo, Brazil], Roberts, Malcolm [UK Met Office, Exeter, UK], O'Rourke, Eleanor [CMIP International Project Office, European Space Agency], Dingley, Elisabeth [CMIP International Project Office, European Space Agency], Turner, Briony [CMIP International Project Office, European Space Agency], Hewitt, Helene [UK Met Office, Exeter, UK], Dunne, John [NOAA Geophysical Fluid Dynamics Laboratory (GFDL), Princeton, NJ]. 2026-08-01. Rapid Evaluation Framework for the CMIP7 Assessment Fast Track. https://doi.org/10.5194/gmd-19-7415-2026
Cite the original work for its findings. Save a collection to share your selection of sources.