Search NASA⌕ Search

SEARCH · Search NASA

Results for “process-oriented metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

9 records
All sourcesNASA NTRSDOE OSTI
NASA NTRS2019

Ongoing Breakthroughs in Convective Parameterization

While the increase of computer power mobilizes a part of the community towards models with explicit convection or based on machine learning, we review the part of the literature dedicated to convective parameterization development for large-scale forecast and climate models. Recent findings: Many developments are underway to overcome endemic limitations of traditional convective parameterizations, either in unified or multi-object frameworks: scale-aware and stochastic approaches, new prognostic equations or representations of new components such as cold pools. Understanding their impact on the emergent properties of a model remains challenging, due to subsequent tuning of parameters and the limited understanding given by traditional metrics. Summary: Further effort still needs to be dedicated to the representation of the life cycle of convective systems, in particular their mesoscale organization and associated cloud cover. The development of more process-oriented metrics based on new observations is also needed to help quantify model improvement and better understand the mechanisms of climate change.

parameterizations for large-scale models↗
DOE OSTI2022

Exploratory Precipitation Metrics: Spatiotemporal Characteristics, Process-Oriented, and Phenomena-Based

Abstract Precipitation sustains life and supports human activities, making its prediction one of the most societally relevant challenges in weather and climate modeling. Limitations in modeling precipitation underscore the need for diagnostics and metrics to evaluate precipitation in simulations and predictions. While routine use of basic metrics is important for documenting model skill, more sophisticated diagnostics and metrics aimed at connecting model biases to their sources and revealing precipitation characteristics relevant to how model precipitation is used are critical for improving models and their uses. This paper illustrates examples of exploratory diagnostics and metrics including 1) spatiotemporal characteristics metrics such as diurnal variability, probability of extremes, duration of dry spells, spectral characteristics, and spatiotemporal coherence of precipitation; 2) process-oriented metrics based on the rainfall–moisture coupling and temperature–water vapor environments of precipitation; and 3) phenomena-based metrics focusing on precipitation associated with weather phenomena including low pressure systems, mesoscale convective systems, frontal systems, and atmospheric rivers. Together, these diagnostics and metrics delineate the multifaceted and multiscale nature of precipitation, its relations with the environments, and its generation mechanisms. The metrics are applied to historical simulations from phases 5 and 6 of the Coupled Model Intercomparison Project. Models exhibit diverse skill as measured by the suite of metrics, with very few models consistently ranked as top or bottom performers compared to other models in multiple metrics. Analysis of model skill across metrics and models suggests possible relationships among subsets of metrics, motivating the need for more systematic analysis to understand model biases for informing model development.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2019

Radiative Feedbacks Associated with the Madden-Julian Oscillation

Radiative kernels derived from CloudSat/CALIPSO measurements are used to diagnose radiative feedbacks induced by the Madden-Julian oscillation (MJO). Over the Indo-Pacific warm pool, positive cloud and water vapor feedbacks are coincident with the convective envelope of the MJO during its active phases, whereas the lapse rate feedback shows less obvious correlation to the propagating disturbance. During phase 2/3, when the convective envelope is over the Indian Ocean, water vapor exhibits a vertically coherent response, with the largest anomalies and strongest feedback in the middle troposphere. Though spatial structures of the feedbacks vary, the most prominent difference lies in the magnitude. Cloud changes induce the largest radiative perturbations associated with the MJO. We also find that for individual MJO events, cloud feedback and precipitation are highly correlated, suggesting that the strength of individual MJO events is largely dictated by the magnitude of cloud radiative heating of the atmosphere. In addition, stronger radiative heating due to water vapor and clouds helps the MJO survive the barrier effect of the Maritime Continent, leading to further eastward propagation. These results offer process-oriented metrics that could help to improve model simulations and predictions of the MJO in the future.

radiative feedbacks↗
DOE OSTI2020

Evaluation of Mesoscale Convective Systems in Climate Simulations: Methodological Development and Results from MPAS-CAM over the U.S.

In this study, a process-oriented approach is developed to evaluate warm-season mesoscale convective system (MCS) precipitation and their favorable large-scale meteorological patterns (FLSMPs) over the U.S. This approach features a novel observation-driven MCS-tracking algorithm using infrared brightness temperature and precipitation feature at 12, 25 and 50 km resolution and metrics to evaluate the model large-scale environment favorable for MCS initiation. The tracking algorithm successfully reproduces the observed MCS statistics from a reference 4-km radar MCS database. To demonstrate the utility of the new methodologies in evaluating MCS in climate simulations with mesoscale resolution, the process-oriented approach is applied to two climate simulations produced by the Variable-Resolution Model for Prediction Across Scales coupled to the Community Atmosphere Model physics, with refined horizontal grid spacing at 50 km and 25 km over North America. With the tracking algorithm applied to simulations and observations at equivalent resolutions, the simulated number of MCS and associated precipitation amount, frequency and intensity are found to be consistently underestimated in the Central U.S., particularly from May to August. The simulated MCS precipitation shows little diurnal variation and lasts too long, while MCS precipitation area is too large and intensity is too weak. The model is able to simulate four types of observed FLSMP associated with frontal systems and low-level jets (LLJ) in spring, but the frequencies are underestimated because of low-level dry bias and weaker LLJ. Precipitation simulated under different FLSMPs peak during daytime, in contrast to the observed nocturnal peak. Implications of these findings for future model development and diagnostics are discussed.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2026

ARM Data-Oriented Metrics and Diagnostics Package for Climate Model Evaluation

A Python-based metrics and diagnostics package is currently being developed by the U.S. Department of Energy (DOE) Atmospheric Radiation Measurement (ARM) Infrastructure Team at Lawrence Livermore National Laboratory (LLNL) to facilitate the use of long-term, high-frequency measurements from the ARM Facility in evaluating the regional climate simulation of clouds, radiation, and precipitation. This metrics and diagnostics package computes climatological means of targeted climate model simulation and generates tables and plots for comparing the model simulation with ARM observational data. The Coupled Model Intercomparison Project (CMIP) model data sets are also included in the package to enable model intercomparison as demonstrated in Zhang et al. (2017). The mean of the CMIP model can serve as a reference for individual models. Basic performance metrics are computed to measure the accuracy of mean state and variability of climate models. The evaluated physical quantities include cloud fraction, temperature, relative humidity, cloud liquid water path, total column water vapor, precipitation, sensible and latent heat fluxes, and radiative fluxes, with plan to extend to more fields, such as aerosol and microphysics properties. Process-oriented diagnostics focusing on individual cloud- and precipitation-related phenomena are also being developed for the evaluation and development of specific model physical parameterizations. The version 1.0 package is designed based on data collected at ARM’s Southern Great Plains (SGP) Research Facility, with the plan to extend to other ARM sites. The metrics and diagnostics package is currently built upon standard Python libraries and additional Python packages developed by DOE (such as CDMS and CDAT). The ARM metrics and diagnostic package is available publicly with the hope that it can serve as an easy entry point for climate modelers to compare their models with ARM data. In this report, we first present the input data, which constitutes the core content of the metrics and diagnostics package in section 2, and a user's guide documenting the workflow/structure of the version 1.0 codes, and including step-by-step instruction for running the package in section 3.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

ARM data-oriented metrics and diagnostics package for climate model evaluation (ARM-DIAGS-V3) version 3

A Python-based metrics and diagnostics package is currently being developed by the ARM Infrastructure Team at Lawrence Livermore National Laboratory to facilitate the use of long-term high frequency measurements from the ARM program in evaluating the regional climate simulation of clouds, radiation, precipitation, and aerosols. This metrics and diagnostics package computes climatological means of targeted climate model simulation and generates tables and plots for comparing the model simulation with ARM observational data. The CMIP model data sets are also included in the package to enable model inter-comparison as demonstrated in Zhang et al. (2018) and Zhang et al. (2020). The mean of the CMIP model can be served as a reference for individual models. Basic performance metrics are computed to measure the accuracy of mean state and variability of climate models. The evaluated physical quantities include cloud fraction, temperature, relative humidity, cloud liquid water path, total column water vapor, precipitation, sensible and latent heat fluxes, aerosol optical depth, and radiative fluxes, with plan to extend to more fields, such as the evaluation of model simulated aerosol physicochemical properties and cloud microphysics properties. Process-oriented diagnostics focusing on aerosol, cloud, and precipitation-related phenomena are also being developed for the evaluation and development of specific model physical parameterizations. In addition to the Southern Great Plains (SGP), North Slope of Alaska (NSA) and Tropical Western Pacific (TWP) atmospheric observatories in the ARMDIAGS version 2.0, the version 3.0 package have extended to the data collected at the ARM Eastern North Atlantic (ENA) atmospheric observatory and the Observation and Modeling of the Green Ocean Amazon (GOAMAZON) field campaign. The metrics and diagnostics package are currently built upon standard Python libraries and additional Python packages developed by DOE (CDAT). The ARM metrics and diagnostic package is available publicly with the hope that it can serve as an easy entry point for climate modelers to compare their models with ARM data. In this report, we first provide an overview of major metrics in section 2. The input data, which constitutes the core content of the metrics and diagnostics package, is summarized in section 3. A user's guide documenting the workflow/structure of the version 3.0 codes and including step-by-step instruction for running the package is described in section 4.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Process-Oriented Diagnostics: Principles, Practice, Community Development, and Common Standards

Process-oriented diagnostics (PODs) aim to provide feedback for model developers through model analysis based on physical hypotheses. However, the step from a diagnostic based on relationships among variables, even when hypothesis driven, to specific guidance for revising model formulation or parameterizations can be substantial. The POD may provide more information than a purely performance-based metric, but a gap between POD principles and providing actionable information for specific model revisions can remain. Furthermore, in coordinating diagnostics development, there is a trade-off between freedom for the developer, aiming to capture innovation, and near-term utility to the modeling center. Best practices that allow for the former, while conforming to specifications that aid the latter, are important for community diagnostics development that leads to tangible model improvements. Promising directions to close the gap between principles and practice include the interaction of PODs with perturbed physics experiments and with more quantitative process models as well as the inclusion of personnel from modeling centers in diagnostics development groups for immediate feedback during climate model revisions. Examples are provided, along with best-practice recommendations, based on practical experience from the NOAA Model Diagnostics Task Force (MDTF). Common standards for metrics and diagnostics that have arisen from a collaboration between the MDTF and the Department of Energy’s Coordinated Model Evaluation Capability are advocated as a means of uniting community diagnostics efforts.

58 GEOSCIENCES↗
DOE OSTI2023

ENSO-Induced Teleconnection: Process-Oriented Diagnostics to Assess Rossby Wave Sources and Ambient Flow Properties in Climate Models

Climate model fidelity in representing ENSO-induced teleconnection is assessed with process-oriented diagnostics that examine a chain of processes, from equatorial Pacific precipitation to the midlatitude circulation pattern over the Pacific–North American regions. Such processes are rarely addressed during model development. Using an upper-tropospheric divergent level, local vorticity gradient of the ambient zonal flow (∂ 2 U¯/∂y 2 ) and a restoring force for Rossby waves (β * ) are estimated, the equivalent barotropic vorticity equation is solved, and an anomalous Rossby wave source (RWS') quantified. The analysis is applied to AMIP5 and AMIP6 simulations. For a realistic circulation response representation, the hypothesis that models accurately represent the strength and location of RWS', and spatial variations in β * is tested. Compared to AMIP5, in AMIP6 there are clear improvements in representing RWS' and β * . To validate the hypothesis, the analysis identifies two metrics: spatially coherent RWS' in the subtropical North Pacific, and longitudes of negative β * over the western-central North Pacific. By projecting these metrics in two and three-dimensional views, improvements or degradations in model versions are apparent. If a model’s fidelity in representing ∂ 2 U¯/∂y 2 and RWS' are compromised, then radiated Rossby waves are reflected more equatorward, resulting in zonally elongated circulation anomalies over the central North Pacific. Thus, during climate model development, applying this analysis frequently will keep a regular check on the fidelity of the modeled response to anomalous El Niño convection in conjunction with changing model ambient flow dependencies. As a result, this analysis is intended to form a process-oriented diagnostics package, a community contribution to the NOAA Model Diagnostics Task Force.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Use-Inspired, Process-Oriented GCM Selection: Prioritizing Models for Regional Dynamical Downscaling

Dynamical downscaling is a crucial process for providing regional climate information for broad uses, using coarser-resolution global models to drive higher-resolution regional climate simulations. The pool of global climate models (GCMs) providing the fields needed for dynamical downscaling has increased from the previous generations of the Coupled Model Intercomparison Project (CMIP). However, with limited computational resources, the need for prioritizing the GCMs for subsequent downscaling studies remains. GCM selection for dynamical downscaling should focus on evaluating processes relevant for providing boundary conditions to the regional models and be inspired by regional uses such as the response of extremes to changes in the boundary conditions. This leads to the need for metrics representing processes of relevance to diverse stakeholders and subregions of a domain. Procedures to account for metric redundancy and the statistical distinguishability of GCM rankings are required. Further, procedures for selecting realizations from ensembles of top-performing GCM simulations can be used to span the range of climate change signals in multiple ways. As a result, distinct weighting of metrics and prioritization of particular realizations may depend on user needs. We provide high-level guidelines for such region-specific evaluations and address how CMIP7 might enable dynamical downscaling of a representative sample of high-quality models across representative shared socioeconomic pathways (SSPs).

54 ENVIRONMENTAL SCIENCES↗