Search NASA⌕ Search

SEARCH · Search NASA

Results for “model evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
NASA NTRS2022

Using Atmospheric Trace Gas Vertical Profiles to Evaluate Model Fluxes: A Case Study of Arctic-Cap Observations and Geos Simulations for the ABoVE Domain

Accurate estimates of carbon–climate feedbacks require an independent means for evaluating surface flux models at regional scales. The altitude-integrated enhancement (AIE) derived from the Arctic Carbon Atmospheric Profiles (Arctic-CAP) project demonstrates the utility of this bulk quantity for surface flux model evaluation. This bulk quantity leverages background mole fraction values from the middle free troposphere, is agnostic to uncertainties in boundary layer height, and can be derived from model estimates of mole fractions and vertical gradients. To demonstrate the utility of the bulk quantity, six airborne profiling surveys of atmospheric carbon dioxide (CO2), methane (CH4), and carbon monoxide (CO) throughout Alaska and northwestern Canada between April and November 2017 were completed as part of NASA’s Arctic–Boreal Vulnerability Experiment (ABoVE). The Arctic-CAP sampling strategy involved acquiring vertical profiles of CO2, CH4, and CO from the surface to 5 km altitude at 25 sites around the ABoVE domain on a 4- to 6-week time interval. All Arctic-CAP measurements were compared to a global simulation using the Goddard Earth Observing System (GEOS) modeling system. Comparisons of the AIE bulk quantity from aircraft observations and GEOS simulations of atmospheric CO2, CH4, and CO highlight the fidelity of the modeled surface fluxes. The model–data comparison over the ABoVE domain reveals that while current state-of-the-art models and flux estimates are able to capture broad-scale spatial and temporal patterns in near-surface CO2 and CH4 concentrations, more work is needed to resolve fine-scale flux features that are captured in CO observations

CO2↗
DOE OSTI2017

Process-level model evaluation: a snow and heat transfer metric

Land models require evaluation in order to understand results and guide future development. Examining functional relationships between model variables can provide insight into the ability of models to capture fundamental processes and aid in minimizing uncertainties or deficiencies in model forcing. This study quantifies the proficiency of land models to appropriately transfer heat from the soil through a snowpack to the atmosphere during the cooling season (Northern Hemisphere: October–March). Using the basic physics of heat diffusion, we investigate the relationship between seasonal amplitudes of soil versus air temperatures due to insulation from seasonal snow. Observations demonstrate the anticipated exponential relationship of attenuated soil temperature amplitude with increasing snow depth and indicate that the marginal influence of snow insulation diminishes beyond an effective snow depth of about 50 cm. A snow and heat transfer metric (SHTM) is developed to quantify model skill compared to observations. Land models within the CMIP5 experiment vary widely in SHTM scores, and deficiencies can often be traced to model structural weaknesses. The SHTM value for individual models is stable over 150 years of climate, 1850–2005, indicating that the metric is insensitive to climate forcing and can be used to evaluate each model's representation of the insulation process.

58 GEOSCIENCES↗
DOE OSTI2023

The winter central Arctic surface energy budget: A model evaluation using observations from the MOSAiC campaign

This study evaluates the simulation of wintertime (15 October, 2019, to 15 March, 2020) statistics of the central Arctic near-surface atmosphere and surface energy budget observed during the MOSAiC campaign with short-term forecasts from 7 state-of-the-art operational and experimental forecast systems. Five of these systems are fully coupled ocean-sea ice-atmosphere models. Forecast systems need to simultaneously simulate the impact of radiative effects, turbulence, and precipitation processes on the surface energy budget and near-surface atmospheric conditions in order to produce useful forecasts of the Arctic system. This study focuses on processes unique to the Arctic, such as, the representation of liquid-bearing clouds at cold temperatures and the representation of a persistent stable boundary layer. It is found that contemporary models still struggle to maintain liquid water in clouds at cold temperatures. Given the simple balance between net longwave radiation, sensible heat flux, and conductive ground flux in the wintertime Arctic surface energy balance, a bias in one of these components manifests as a compensating bias in other terms. This study highlights the different manifestations of model bias and the potential implications on other terms. Three general types of challenges are found within the models evaluated: representing the radiative impact of clouds, representing the interaction of atmospheric heat fluxes with sub-surface fluxes (i.e., snow and ice properties), and representing the relationship between stability and turbulent heat fluxes.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2019

Model Evaluation Report for Corrective Action Unit 97: Yucca Flat/Climax Mine, Nevada National Security Site, Nye County, Nevada (Revision 1)

This Model Evaluation Report (MER) has been prepared for Corrective Action Unit (CAU) 97, Yucca Flat/Climax Mine (YF/CM), Nevada National Security Site (NNSS), Nevada. The YF/CM CAU is located in the northeastern portion of the NNSS and comprises 720 corrective action sites. A total of 747 underground nuclear detonations took place within this CAU between 1957 and 1992 and resulted in the release of radionuclides (RNs) in the subsurface in the vicinity of the detonation cavities.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2008

Climate Model Evaluation using New Datasets from the Clouds and the Earth's Radiant Energy System (CERES)

There are some in the science community who believe that the response of the climate system to anthropogenic radiative forcing is unpredictable and we should therefore call off the quest . The key limitation in climate predictability is associated with cloud feedback. Narrowing the uncertainty in cloud feedback (and therefore climate sensitivity) requires optimal use of the best available observations to evaluate and improve climate model processes and constrain climate model simulations over longer time scales. The Clouds and the Earth s Radiant Energy System (CERES) is a satellite-based program that provides global cloud, aerosol and radiative flux observations for improving our understanding of cloud-aerosol-radiation feedbacks in the Earth s climate system. CERES is the successor to the Earth Radiation Budget Experiment (ERBE), which has widely been used to evaluate climate models both at short time scales (e.g., process studies) and at decadal time scales. A CERES instrument flew on the TRMM satellite and captured the dramatic 1998 El Nino, and four other CERES instruments are currently flying aboard the Terra and Aqua platforms. Plans are underway to fly the remaining copy of CERES on the upcoming NPP spacecraft (mid-2010 launch date). Every aspect of CERES represents a significant improvement over ERBE. While both CERES and ERBE measure broadband radiation, CERES calibration is a factor of 2 better than ERBE. In order to improve the characterization of clouds and aerosols within a CERES footprint, we use coincident higher-resolution imager observations (VIRS, MODIS or VIIRS) to provide a consistent cloud-aerosol-radiation dataset at climate accuracy. Improved radiative fluxes are obtained by using new CERES-derived Angular Distribution Models (ADMs) for converting measured radiances to fluxes. CERES radiative fluxes are a factor of 2 more accurate than ERBE overall, but the improvement by cloud type and at high latitudes can be as high as a factor of 5. Diurnal cycles are explicitly resolved by merging geostationary satellite observations with CERES and MODIS. Atmospheric state data are provided from a frozen version of the Global Modeling and Assimilation Office- Data Assimilation System at the NASA Goddard Space Flight Center. In addition to improving the accuracy of top-of-atmosphere (TOA) radiative fluxes, CERES also produces radiative fluxes at the surface and at several levels in the atmosphere using radiative transfer modeling, constrained at the TOA by CERES (ERBE was limited to the TOA). In all, CERES uses 11 instruments on 7 spacecraft all integrated to obtain climate accuracy in TOA to surface fluxes. This presentation will provide an overview of several new CERES datasets of interest to the climate community (including a new adjusted TOA flux dataset constrained by estimates of heat storage in the Earth system), show direct comparisons between CERES ad ERBE, and provide a detailed error analysis of CERES fluxes at various time and space scales. We discuss how observations can be used to reduce uncertainties in cloud feedback and climate sensitivity and strongly argue why we should NOT "call off the quest".

Loeb, Norman G.↗
NASA NTRS1974

Solid rocket booster performance evaluation model. Volume 1: Engineering description

The space shuttle solid rocket booster performance evaluation model (SRB-II) is made up of analytical and functional simulation techniques linked together so that a single pass through the model will predict the performance of the propulsion elements of a space shuttle solid rocket booster. The available options allow the user to predict static test performance, predict nominal and off nominal flight performance, and reconstruct actual flight and static test performance. Options selected by the user are dependent on the data available. These can include data derived from theoretical analysis, small scale motor test data, large motor test data and motor configuration data. The user has several options for output format that include print, cards, tape and plots. Output includes all major performance parameters (Isp, thrust, flowrate, mass accounting and operating pressures) as a function of time as well as calculated single point performance data. The engineering description of SRB-II discusses the engineering and programming fundamentals used, the function of each module, and the limitations of each module.

Source record↗
NASA NTRS2019

Taking Climate Model Evaluation to the Next Level

Earth system models are complex and represent a large number of processes, resulting in a persistent spread across climate projections for a given future scenario. Owing to different model performances against observations and the lack of independence among models, there is now evidence that giving equal weight to each available model projection is suboptimal. This Perspective discusses newly developed tools that facilitate a more rapid and comprehensive evaluation of model simulations with observations, process-based emergent constraints that are a promising way to focus evaluation on the observations most relevant to climate projections, and advanced methods for model weighting. These approaches are needed to distil the most credible information on regional climate changes, impacts, and risks for stakeholders and policy-makers.

Eyring, Veronika↗
NASA NTRS1974

Solid rocket booster performance evaluation model. Volume 2: Users manual

This users manual for the solid rocket booster performance evaluation model (SRB-II) contains descriptions of the model, the program options, the required program inputs, the program output format and the program error messages. SRB-II is written in FORTRAN and is operational on both the IBM 370/155 and the MSFC UNIVAC 1108 computers.

Source record↗
DOE OSTI2019

The Single Column Atmosphere Model Version 6 (SCAM6): Not a Scam but a Tool for Model Evaluation and Development

The Single Column Atmosphere Model (SCAM) is a single column model version of the Community Atmosphere Model (CAM). Here we describe the functionality and features of SCAM6, available as part of CAM6 in the Community Earth System Model, version 2 (CESM2). SCAM6 features a wide selection of standard cases, as well as the ability to easily configure a case specified by the user based on a particular point in a CAM 3-D simulation. This work illustrates how SCAM6 reproduces CAM6 results for physical parameterizations, mostly of moisture and clouds. We demonstrate how SCAM6 can be used for model development through different physics selections, as well as with parameter sweep experiments to highlight the sensitivity of cloud properties to the specification of the vapor deposition process in the cloud microphysics. Furthermore, we use SCAM6 to illustrate the sensitivity of CAM6 cloud radiative properties and precipitation to variable drop number (cloud microphysics properties). Finally, we illustrate how SCAM6 can be used to explore critical emergent processes such as cloud feedbacks and show that CAM6 cloud responses to surface warming in stratus and stratocumulus regimes are similar to those in CAM5. CAM6 has a larger response in the shallow cumulus regime than CAM5. CAM6 cloud feedbacks in the shallow cumulus regime are sensitive to turbulence parameters. SCAM6 is thus a valuable tool for model development, evaluation, and scientific analy sis and an important part of the model hierarchy in Community Earth System Model, version 2.

58 GEOSCIENCES↗
NASA NTRS1989

Model evaluation, recommendation and prioritizing of future work for the manipulator emulator testbed

The Manipulator Emulator Testbed (MET) is to provide a facility capable of hosting the simulation of various manipulator configurations to support concept studies, evaluation, and other engineering development activities. Specifically, the testbed is intended to support development of the Space Station Remote Manipulator System (SSRMS) and related systems. The objective of this study is to evaluate the math models developed for the MET simulation of a manipulator's rigid body dynamics and the servo systems for each of the driven manipulator joints. Specifically, the math models are examined with regard to their amenability to pipeline and parallel processing. Based on this evaluation and the project objectives, a set of prioritized recommendations are offered for future work.

Kelly, Frederick A.↗
NASA NTRSDate not supplied

Climate Information and Lessons Learned from the NASA Earth eXchange (NEX), Short-term Prediction Research and Transition Center (NASA SPoRT), and the NASA Regional Climate Model Evaluation System (RCMES)

Climate resilience can emerge from a combination of (1) climate projections that enable future-oriented planning and (2) contemporary observations that can be used for adaptation in the face of realized climate. NASA is uniquely positioned to provide global climate projections and satellite-based observations to users who must take a global perspective on climate risk (e.g., supply chain managers). Here, we will share the capabilities and experiences of three NASA projects. The first project, the NASA Earth eXchange (NEX) provides two downscaled climate projections, NEX-GDDP-CMIP6, a global 25 km resolution product, and NEX-DCP30-CMIP6, an 800 m resolution product covering the coterminous U.S. specifically designed for U.S. National Climate Assessment (NCA). NEX also offers land surface products that mirror the twice-daily products provided by the Moderate Resolution Imaging Spectroradiometer (MODIS) but are derived from operational geostationary weather satellite data collected at 5-10 minutes intervals. The second project, the Short-term Prediction Research and Transition Center (SPoRT), was established in 2002 to transition unique NASA satellite products and capabilities to stakeholders to improve decision-making that benefits society. SPoRT provides products and capabilities derived from NASA observations and models to end users in government, academia, and the private sector. SPoRT translates NASA Earth science into actionable solutions reaching over 400 stakeholders across 80 organizations. Finally, the Regional Climate Model Evaluation System (RCMES) is designed to facilitate regional-scale evaluations of climate models by providing standardized access to a vast and comprehensive set of observations, such as satellite, in-situ, and reanalyses, and modeling resources such as CMIP & CORDEX. These three projects form part of the core capabilities at the NASA Centers deliver climate information under the new NASA Earth Science to Action (ES2A) strategy.

NEX↗
DOE OSTI2020

PCMDI - An Earth System Model Evaluation Project Lawrence Livermore National Laboratory Annual Report

This annual report describes the activities of PCMDI – An Earth System Model Evaluation Project for the period November 1, 2019 – June 30, 2020. PCMDI is a project at Lawrence Livermore National Laboratory that is funded by the Climate and Environmental Sciences Division (CESD) of the Department of Energy. PCMDI is funded by two efforts within CESD: The Regional and Global Model Analysis program area (Dr. Renu Joseph, program manager) and the Atmospheric Systems Research program (Dr. Shaima Nasiri and Jeffrey Stehr, program managers). As instructed by Drs. Joseph, Nasiri, and Stehr, we provide an annual report addressing sections 4 (Progress and Plans) and 5 (Staffing and Budget Summary) of the SFA Program Management and Performance Reporting Criteria.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2018

On the Forward Modeling of Radar Doppler Spectrum Width From LES: Implications for Model Evaluation

Large-eddy simulations of an observed single-layer Arctic mixed-phase cloud are analyzed to study the value of forward modeling of profiling millimeter wave cloud radar Doppler spectral width for model evaluation. Individual broadening terms and their uncertainties are quantified for the observed spectral width and compared to modeled broadening terms. Modeled turbulent broadening is narrower than the observed values when the turbulent kinetic energy dissipation rate from the subgrid scale model is used in the forward model. The total dissipation rates, estimated with the subgrid scale dissipation rates and the numerical dissipation rates, agree much better with both the retrieved dissipation rates and those inferred from the power spectra of the simulated vertical air velocity. The comparison of the microphysical broadening provides another evaluative measure of the ice properties in the simulation. To accurately retrieve dissipation rates as well as each broadening term from the observations, we suggest a few modifications to previously presented techniques. First, we show that the inertial subrange spectrum filtered with the radar sampling volume is a better underlying model than the unfiltered -5/3 law for the retrieval of the dissipation rate from the power spectra of the mean Doppler velocity. Second, we demonstrate that it is important to filter out turbulence and remove the layer-mean reflectivity-weighted mean fall speed from the observed mean Doppler velocity to avoid overestimation of shear broadening. Finally, we provide a method to quantify the uncertainty in the retrieved dissipation rates, which eventually propagates to the uncertainty in the microphysical broadening.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS1994

Data assimilation and model evaluation experiment datasets

The Institute for Naval Oceanography, in cooperation with Naval Research Laboratories and universities, executed the Data Assimilation and Model Evaluation Experiment (DAMEE) for the Gulf Stream region during fiscal years 1991-1993. Enormous effort has gone into the preparation of several high-quality and consistent datasets for model initialization and verification. This paper describes the preparation process, the temporal and spatial scopes, the contents, the structure, etc., of these datasets. The goal of DAMEE and the need of data for the four phases of experiment are briefly stated. The preparation of DAMEE datasets consisted of a series of processes: (1) collection of observational data; (2) analysis and interpretation; (3) interpolation using the Optimum Thermal Interpolation System package; (4) quality control and re-analysis; and (5) data archiving and software documentation. The data products from these processes included a time series of 3D fields of temperature and salinity, 2D fields of surface dynamic height and mixed-layer depth, analysis of the Gulf Stream and rings system, and bathythermograph profiles. To date, these are the most detailed and high-quality data for mesoscale ocean modeling, data assimilation, and forecasting research. Feedback from ocean modeling groups who tested this data was incorporated into its refinement. Suggestions for DAMEE data usages include (1) ocean modeling and data assimilation studies, (2) diagnosis and theoretical studies, and (3) comparisons with locally detailed observations.

Lai, Chung-Cheng A.↗
DOE OSTI2022

ARM data-oriented metrics and diagnostics package for climate model evaluation (ARM-DIAGS-V3) version 3

A Python-based metrics and diagnostics package is currently being developed by the ARM Infrastructure Team at Lawrence Livermore National Laboratory to facilitate the use of long-term high frequency measurements from the ARM program in evaluating the regional climate simulation of clouds, radiation, precipitation, and aerosols. This metrics and diagnostics package computes climatological means of targeted climate model simulation and generates tables and plots for comparing the model simulation with ARM observational data. The CMIP model data sets are also included in the package to enable model inter-comparison as demonstrated in Zhang et al. (2018) and Zhang et al. (2020). The mean of the CMIP model can be served as a reference for individual models. Basic performance metrics are computed to measure the accuracy of mean state and variability of climate models. The evaluated physical quantities include cloud fraction, temperature, relative humidity, cloud liquid water path, total column water vapor, precipitation, sensible and latent heat fluxes, aerosol optical depth, and radiative fluxes, with plan to extend to more fields, such as the evaluation of model simulated aerosol physicochemical properties and cloud microphysics properties. Process-oriented diagnostics focusing on aerosol, cloud, and precipitation-related phenomena are also being developed for the evaluation and development of specific model physical parameterizations. In addition to the Southern Great Plains (SGP), North Slope of Alaska (NSA) and Tropical Western Pacific (TWP) atmospheric observatories in the ARMDIAGS version 2.0, the version 3.0 package have extended to the data collected at the ARM Eastern North Atlantic (ENA) atmospheric observatory and the Observation and Modeling of the Green Ocean Amazon (GOAMAZON) field campaign. The metrics and diagnostics package are currently built upon standard Python libraries and additional Python packages developed by DOE (CDAT). The ARM metrics and diagnostic package is available publicly with the hope that it can serve as an easy entry point for climate modelers to compare their models with ARM data. In this report, we first provide an overview of major metrics in section 2. The input data, which constitutes the core content of the metrics and diagnostics package, is summarized in section 3. A user's guide documenting the workflow/structure of the version 3.0 codes and including step-by-step instruction for running the package is described in section 4.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2018

Model Evaluation Report for LNG Dispersion on Fire Dynamics Simulator Version 6.5.3

This report is an assessment of the computational fluid dynamics (CFD) code Fire Dynamics Simulator (FDS), version 6.5.3, using the Model Evaluation Protocol (MEP) for Liquefied Natural Gas (LNG). The MEP consists of model validation and a scientific assessment that encompasses model information and model verification activities. Model validation entails comparison against various field and wind-tunnel trials for LNG and other dense gases with and without obstructions. The results indicate that FDS is generally over-predictive for most of the field scale dense gas releases and cases involving obstructions without requiring a safety factor of 2, however, since there are uncertainties in extrapolating to accident scales a safety factor of 2 (1/2 LFL criteria) may be appropriate. The results of this scientific assessment and prior verification efforts demonstrate the soundness of the numerical techniques and robustness of FDS. The validation results indicate that FDS is suitable for modeling dense gas dispersion with and without obstructions.

03 NATURAL GAS↗
NASA NTRS2017

Aerosols at the Poles: An Aerocom Phase II Multi-Model Evaluation

Atmospheric aerosols from anthropogenic and natural sources reach the polar regions through long-range transport and affect the local radiation balance. Such transport is, however, poorly constrained in present-day global climate models, and few multi-model evaluations of polar anthropogenic aerosol radiative forcing exist. Here we compare the aerosol optical depth (AOD) at 550 nm from simulations with 16 global aerosol models from the AeroCom Phase II model intercomparison project with available observations at both poles. We show that the annual mean multi-model median is representative of the observations in Arctic, but that the intermodel spread is large. We also document the geographical distribution and seasonal cycle of the AOD for the individual aerosol species: black carbon (BC) from fossil fuel and biomass burning, sulfate, organic aerosols (OAs), dust, and sea-salt. For a subset of models that represent nitrate and secondary organic aerosols (SOAs), we document the role of these aerosols at high latitudes. The seasonal dependence of natural and anthropogenic aerosols differs with natural aerosols peaking in winter (seasalt) and spring (dust), whereas AOD from anthropogenic aerosols peaks in late spring and summer. The models produce a median annual mean AOD of 0.07 in the Arctic (defined here as north of 60 degrees N). The models also predict a noteworthy aerosol transport to the Antarctic (south of 70 degrees S) with a resulting AOD varying between 0.01 and 0.02. The models have estimated the shortwave anthropogenic radiative forcing contributions to the direct aerosol effect (DAE) associated with BC and OA from fossil fuel and biofuel (FF), sulfate, SOAs, nitrate, and biomass burning from BC and OA emissions combined. The Arctic modelled annual mean DAE is slightly negative (-0.12 W m(exp. -2), dominated by a positive BC FF DAE in spring and a negative sulfate DAE in summer. The Antarctic DAE is governed by BC FF. We perform sensitivity experiments with one of the AeroCom models (GISS modelE) to investigate how regional emissions of BC and sulfate and the lifetime of BC influence the Arctic and Antarctic AOD. A doubling of emissions in eastern Asia results in a 33 percent increase in Arctic AOD of BC. A doubling of the BC lifetime results in a 39 percent increase in Arctic AOD of BC. However, these radical changes still fall within the AeroCom model range.

Sand, Maria↗