Search NASA⌕ Search

SEARCH · Search NASA

Results for “model evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6
All sourcesNASA NTRSDOE OSTI
DOE OSTI2016

A simple physiologically based pharmacokinetic model evaluating the effect of anti-nicotine antibodies on nicotine disposition in the brains of rats and humans

Physiologically based pharmacokinetic (PBPK) modeling was applied to investigate the effects of anti-nicotine antibodies on nicotine disposition in the brains of rats and humans. Successful construction of both rat and human models was achieved by fitting model outputs to published nicotine concentration time course data in the blood and in the brain. Key parameters presumed to have the most effect on the ability of these antibodies to prevent nicotine from entering the brain were selected for investigation using the human model. These parameters, which included antibody affinity for nicotine, antibody cross-reactivity with cotinine, and antibody concentration, were broken down into different, clinically-derived in silico treatment levels and fed into the human PBPK model. Model predictions suggested that all three parameters, in addition to smoking status, have a sizable impact on anti-nicotine antibodies' ability to prevent nicotine from entering the brain and that the antibodies elicited by current human vaccines do not have sufficient binding characteristics to reduce brain nicotine concentrations. If the antibody binding characteristics achieved in animal studies can similarly be achieved in human studies, however, nicotine vaccine efficacy in terms of brain nicotine concentration reduction is predicted to meet threshold values for alleviating nicotine dependence. - Highlights: • Modelling of nicotine disposition in the presence of anti-nicotine antibodies • Key vaccine efficacy factors are evaluated in silico in rats and in humans. • Model predicts insufficient antibody binding in past human nicotine vaccines. • Improving immunogenicity and antibody specificity may lead to vaccine success.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS1987

Chemical kinetics and photochemical data for use in stratospheric modeling evaluation Number 8

This is the eighth in a series of evaluated sets of rate constants and photochemical cross sections compiled by the NASA Panel for Data Evaluation. The primary application of the data is in the modeling of stratospheric processes, with particular emphasis on the ozone layer and its possible perturbation by anthropogenic and natural phenomena. Copies of this evaluation are available from the Jet Propulsion Laboratory, Documentation Section, 111-116B, California Institute of Technology, Pasadena, California, 91109.

Demore, W. B.↗
DOE OSTI2018

Process-Based Model Evaluation Using Surface Energy Budget Observations in Central Greenland

Energy exchange at the Greenland ice sheet surface governs surface temperature variability, a factor critical for representing surface melt. Physical processes link driving forces to subsequent surface energy budget responses, including radiative, turbulent, and ground heat fluxes, and ultimately control surface temperature evolution. A reanalysis product (ERA-Interim, ERA-I), operational model (Climate Forecast System version 2, CFSv2), and climate model (Community Earth System Model, CESM) are evaluated using a comprehensive set of surface energy budget observations and process-based relationships obtained at Summit, Greenland. Simulated downwelling longwave radiation is underestimated, which is linked to a deficiency of liquid-bearing clouds. Lower than observed surface albedo, especially in ERA-I, compensates for summer deficiencies in downwelling longwave radiation. In winter, such deficiencies are compensated by an overestimation of the sensible heat flux. Process-based relationships convey that all three models underestimate the response of surface temperature to changes in radiative forcing, primarily due to an overactive ground heat flux response in ERA-I, turbulent heat fluxes in CFSv2, and sensible heat flux in CESM. Cross comparison of three distinct models indicates that the ground heat flux response for ERA-I, CFSv2, and CESM is too high, too low, and comparatively accurate, respectively, signifying the benefit of using an advanced representation of snow properties. Relatively small biases in CESM surface albedo suggest that advances in the representation of cloud microphysics result in more realistic radiative forcing. These results provide insight into model strengths and deficiencies, indicating the importance of representing physical processes when portraying cloud impacts on surface temperature variability.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Large language model evaluation for high–performance computing software development

We apply AI-assisted large language model (LLM) capabilities of GPT-3 targeting high-performance computing (HPC) kernels for (i) code generation, and (ii) auto-parallelization of serial code in C ++, Fortran, Python and Julia. Our scope includes the following fundamental numerical kernels: AXPY, GEMV, GEMM, SpMV, Jacobi Stencil, and CG, and language/programming models: (1) C++ (e.g., OpenMP [including offload], OpenACC, Kokkos, SyCL, CUDA, and HIP), (2) Fortran (e.g., OpenMP [including offload] and OpenACC), (3) Python (e.g., numpy, Numba, cuPy, and pyCUDA), and (4) Julia (e.g., Threads, CUDA.jl, AMDGPU.jl, and KernelAbstractions.jl). Kernel implementations are generated using GitHub Copilot capabilities powered by the GPT-based OpenAI Codex available in Visual Studio Code given simple + + prompt variants. To quantify and compare the generated results, we propose a proficiency metric around the initial 10 suggestions given for each prompt. For auto-parallelization, we use ChatGPT interactively giving simple prompts as in a dialogue with another human including simple “prompt engineering” follow ups. Results suggest that correct outputs for C++ correlate with the adoption and maturity of programming models. For example, OpenMP and CUDA score really high, whereas HIP is still lacking. We found that prompts from either a targeted language such as Fortran or the more general-purpose Python can benefit from adding language keywords, while Julia prompts perform acceptably well for its Threads and CUDA.jl programming models. Finally, we expect to provide an initial quantifiable point of reference for code generation in each programming model using a state-of-the-art LLM. Overall, understanding the convergence of LLMs, AI, and HPC is crucial due to its rapidly evolving nature and how it is redefining human-computer interactions.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2015

A-Train Based Observational Metrics for Model Evaluation in Extratropical Cyclones

Extratropical cyclones contribute most of the precipitation in the midlatitudes, i.e. up to 70 during winter in the northern hemisphere, and can generate flooding, extreme winds, blizzards and have large socio-economic impacts. As such, it is important that general circulation models (GCMs) accurately represent these systems so their evolution in a warming climate can be understood. However, there are still uncertainties on whether warming will increase their frequency of occurrence, their intensity and how much rain or snow they bring. Part of the issue is that models have trouble representing their strength, but models also have biases in the amount of clouds and precipitation they produce. This is caused by potential issues in various aspects of the models: convection, boundary layer, and cloud scheme to only mention a few. In order to pinpoint which aspects of the models need improvement for a better representation of extratropical cyclone precipitation and cloudiness, we will present A-train based observational metrics: cyclone-centered, warm and cold frontal composites of cloud amount and type, precipitation rate and frequency of occurrence. Using the same method to extract similar fields from the model, we will present an evaluation of the GISS-ModelE2 and the IPSL-LMDZ-5B models, based on their AR5 and more recent versions. The AR5 version of the GISS model underestimates cloud cover in extratropical cyclones while the IPSL AR5 version overestimates it. In addition, we will show how the observed CloudSat-CALIPSO cloud vertical distribution across cold fronts changes with moisture amount and cyclone strength, and test if the two models successfully represent these changes. We will also show how CloudSat-CALIPSO derived cloud type (i.e. convective vs. stratiform) evolves across warm fronts as cyclones age, and again how this is represented in the models. Our third process-based analysis concerns cumulus clouds in the post-cold frontal region and how their amount relates to the stability of the boundary layer. This test uses Aqua cloud and vertical atmospheric profiles and when applied to the model output can help assess the accuracy of the convection, boundary layer and cloud scheme.

Convection clouds↗
DOE OSTI2021

A Machine Learning-based Reliability Evaluation Model for Integrated Power-Gas Systems

This article proposes a hybrid machine learning method for the reliability evaluation of integrated power-gas systems (IPGS) under the uncertain component failure probability distributions. The Random Forest (RF) method is designed to select important features to solve the insufficient quantity of data and the curse of dimensionality problems. The Extreme Gradient Boosting (XGBoost) regression algorithm is developed to quantify the relationship between the uncertain parameters and reliability metrics. Moreover, a ten-fold cross-validation method is employed to further improve the accuracy of the regression model. Simulation results on three test systems show that the proposed method can achieve high accuracy for the reliability evaluation.

24 POWER TRANSMISSION AND DISTRIBUTION↗
NASA NTRSDate not supplied

Model Evaluation and Intercomparison Using Data Collected by the Langley Mobile Ozone Lidar in Hampton, Virginia

Throughout the year 2022, the Langley Mobile Ozone Lidar (LMOL) has been frequently collecting ozone and aerosol measurements in the lower troposphere. The lidar is located in the parking lot behind the Atmospheric Sciences building at NASA Langley in Hampton, Virginia, approximately 37.095 N, -76.389 W. The data was collected during a range of several distinct atmospheric conditions, including stratospheric intrusions, surface frontal passages, and long-range transported wildfire smoke plumes. We use this data to evaluate and intercompare the forecast accuracy of two atmospheric chemistry models: the GEOS Composition Forecasting model (GEOS-CF) and the Weather Research and Forecasting model with Chemistry (WRF-Chem). Both models make daily three-dimensional forecasts of many trace gases and species for the contiguous United States. Over a range of ten distinct collection events, each spanning 36 hours to nearly seven days, the forecast models predict tropospheric ozone at NASA Langley with reasonable accuracy. The models best predict the timing and shape of the observed stratospheric intrusions in the middle troposphere but often vary in the magnitude of the ozone mixing ratios. For other atmospheric conditions, there is more variability in the model accuracy. Here, we summarize the accuracy of the models for all events and investigate reasons for differences between the models and the lidar.

Daniel B. Phoenix↗
NASA NTRS1992

VPPA weld model evaluation

NASA uses the Variable Polarity Plasma Arc Welding (VPPAW) process extensively for fabrication of Space Shuttle External Tanks. This welding process has been in use at NASA since the late 1970's but the physics of the process have never been satisfactorily modeled and understood. In an attempt to advance the level of understanding of VPPAW, Dr. Arthur C. Nunes, Jr., (NASA) has developed a mathematical model of the process. The work described in this report evaluated and used two versions (level-0 and level-1) of Dr. Nunes' model, and a model derived by the University of Alabama at Huntsville (UAH) from Dr. Nunes' level-1 model. Two series of VPPAW experiments were done, using over 400 different combinations of welding parameters. Observations were made of VPPAW process behavior as a function of specific welding parameter changes. Data from these weld experiments was used to evaluate and suggest improvements to Dr. Nunes' model. Experimental data and correlations with the model were used to develop a multi-variable control algorithm for use with a future VPPAW controller. This algorithm is designed to control weld widths (both on the crown and root of the weld) based upon the weld parameters, base metal properties, and real-time observation of the crown width. The algorithm exhibited accuracy comparable to that of the weld width measurements for both aluminum and mild steel welds.

Mccutcheon, Kimble D.↗
NASA NTRS2023

Earth-System-Model Evaluation of Cloud and Precipitation Occurrence for Supercooled and Warm Clouds Over the Southern Ocean's Macquarie Island

Over the remote Southern Ocean (SO), cloud feedbacks contribute substantially to Earth system model (ESM) radiative biases. The evolution of low Southern Ocean clouds (cloud-top heights < ∼ 3 km) is strongly modulated by precipitation and/or evaporation, which act as the primary sink of cloud condensate. Constraining precipitation processes in ESMs requires robust observations suitable for process-level evaluations. A year-long subset (April 2016–March 2017) of ground-based profiling instrumentation deployed during the Macquarie Island Cloud and Radiation Experiment (MICRE) field campaign (54.5∘ S, 158.9∘ E) combines a 95 GHz (W-band) Doppler cloud radar, two lidar ceilometers, and balloon-borne soundings to quantify the occurrence frequency of precipitation from the liquid-phase cloud base. Liquid-based clouds at Macquarie Island precipitate ∼ 70 % of the time, with deeper and colder clouds precipitating more frequently and at a higher intensity compared to thinner and warmer clouds. Supercooled cloud layers precipitate more readily than layers with cloud-top temperatures > 0 ∘C, regardless of the geometric thickness of the layer, and also evaporate more frequently. We further demonstrate an approach to employ these observational constraints for evaluation of a 9-year GISS-ModelE3 ESM simulation. Model output is processed through the Earth Model Column Collaboratory (EMC2) radar and lidar instrument simulator with the same instrument specifications as those deployed during MICRE, therefore accounting for instrument sensitivities and ensuring a coherent comparison. Relative to MICRE observations, the ESM produces a smaller cloud occurrence frequency, smaller precipitation occurrence frequency, and greater sub-cloud evaporation. The lower precipitation occurrence frequency by the ESM relative to MICRE contrasts with numerous studies that suggest a ubiquitous bias by ESMs to precipitate too frequently over the SO when compared with satellite-based observations, likely owing to sensitivity limitations of spaceborne instrumentation and different sampling methodologies for ground- versus space-based observations. Despite these deficiencies, the ESM reproduces the observed tendency for deeper and colder clouds to precipitate more frequently and at a higher intensity. The ESM also reproduces specific cloud regimes, including near-surface clouds that account for ∼ 25 % of liquid-based clouds during MICRE and optically thin, non-precipitating clouds that account for ∼ 27 % of clouds with bases higher than 250 m. We suggest that the demonstrated framework, which merges observations with appropriately constrained model output, is a valuable approach to evaluate processes responsible for cloud radiative feedbacks in ESMs.

cloud feedbacks↗
DOE OSTI2023

Earth-system-model evaluation of cloud and precipitation occurrence for supercooled and warm clouds over the Southern Ocean's Macquarie Island

Abstract. Over the remote Southern Ocean (SO), cloud feedbacks contribute substantially to Earth system model (ESM) radiative biases. The evolution of low Southern Ocean clouds (cloud-top heights < ∼ 3 km) is strongly modulated by precipitation and/or evaporation, which act as the primary sink of cloud condensate. Constraining precipitation processes in ESMs requires robust observations suitable for process-level evaluations. A year-long subset (April 2016–March 2017) of ground-based profiling instrumentation deployed during the Macquarie Island Cloud and Radiation Experiment (MICRE) field campaign (54.5∘ S, 158.9∘ E) combines a 95 GHz (W-band) Doppler cloud radar, two lidar ceilometers, and balloon-borne soundings to quantify the occurrence frequency of precipitation from the liquid-phase cloud base. Liquid-based clouds at Macquarie Island precipitate ∼ 70 % of the time, with deeper and colder clouds precipitating more frequently and at a higher intensity compared to thinner and warmer clouds. Supercooled cloud layers precipitate more readily than layers with cloud-top temperatures > 0 ∘C, regardless of the geometric thickness of the layer, and also evaporate more frequently. We further demonstrate an approach to employ these observational constraints for evaluation of a 9-year GISS-ModelE3 ESM simulation. Model output is processed through the Earth Model Column Collaboratory (EMC2) radar and lidar instrument simulator with the same instrument specifications as those deployed during MICRE, therefore accounting for instrument sensitivities and ensuring a coherent comparison. Relative to MICRE observations, the ESM produces a smaller cloud occurrence frequency, smaller precipitation occurrence frequency, and greater sub-cloud evaporation. The lower precipitation occurrence frequency by the ESM relative to MICRE contrasts with numerous studies that suggest a ubiquitous bias by ESMs to precipitate too frequently over the SO when compared with satellite-based observations, likely owing to sensitivity limitations of spaceborne instrumentation and different sampling methodologies for ground- versus space-based observations. Despite these deficiencies, the ESM reproduces the observed tendency for deeper and colder clouds to precipitate more frequently and at a higher intensity. The ESM also reproduces specific cloud regimes, including near-surface clouds that account for ∼ 25 % of liquid-based clouds during MICRE and optically thin, non-precipitating clouds that account for ∼ 27 % of clouds with bases higher than 250 m. We suggest that the demonstrated framework, which merges observations with appropriately constrained model output, is a valuable approach to evaluate processes responsible for cloud radiative feedbacks in ESMs.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

The Global Atmosphere‐aerosol Model ICON‐A‐HAM2.3–Initial Model Evaluation and Effects of Radiation Balance Tuning on Aerosol Optical Thickness

Abstract The Hamburg Aerosol Module version 2.3 (HAM2.3) from the ECHAM6.3‐HAM2.3 global atmosphere‐aerosol model is coupled to the recently developed icosahedral nonhydrostatic ICON‐A (icon‐aes‐1.3.00) global atmosphere model to yield the new ICON‐A‐HAM2.3 atmosphere‐aerosol model. The ICON‐A and ECHAM6.3 host models use different dynamical cores, parameterizations of vertical mixing due to sub‐grid scale turbulence, and parameter settings for radiation balance tuning. Here, we study the role of the different host models for simulated aerosol optical thickness (AOT) and evaluate impacts of using HAM2.3 and the ECHAM6‐HAM2.3 two‐moment cloud microphysics scheme on several meteorological variables. Sensitivity runs show that a positive AOT bias over the subtropical oceans is remedied in ICON‐A‐HAM2.3 because of a different default setting of a parameter in the moist convection parameterization of the host models. The global mean AOT is biased low compared to MODIS satellite instrument retrievals in ICON‐A‐HAM2.3 and ECHAM6.3‐HAM2.3, but the bias is larger in ICON‐A‐HAM2.3 because negative AOT biases over the Amazon, the African rain forest, and the northern Indian Ocean are no longer compensated by high biases over the sub‐tropical oceans. ICON‐A‐HAM2.3 shows a moderate improvement with respect to AOT observations at AERONET sites. A multivariable bias score combining biases of several meteorological variables into a single number is larger in ICON‐A‐HAM2.3 compared to standard ICON‐A and standard ECHAM6.3. In the tropics, this multivariable bias is of similar magnitude in ICON‐A‐HAM2.3 and in ECHAM6.3‐HAM2.3. In the extra‐tropics, a smaller multivariable bias is found for ICON‐A‐HAM2.3 than for ECHAM6.3‐HAM2.3.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS1994

Model evaluation of the radiative and temperature effects of the ozone content changes in the global atmosphere of 1980's

Radiative and temperature effects of the observed ozone and greenhouse gas atmospheric content changes in 1980 - 1990 are evaluated using the two-dimensional energy balance radiative-convective model of the zonally and annually averaged troposphere and stratosphere. Calculated radiative flux changes for standard conditions quantitatively agree with their estimates in WMO/UNEP 1991 review. Model estimates indicate rather small influence of ozone depletion in the lower stratosphere on the greenhouse tropospheric warming rate, being more significant in the non-tropical Southern Hemisphere. The calculated cooling of the lower stratosphere is close to the observed temperature trends there in the last decade.

Karol, Igor L.↗
NASA NTRS2012

Assimilation of GRACE Terrestrial Water Storage into a Land Surface Model: Evaluation 1 and Potential Value for Drought Monitoring in Western and Central Europe

A land surface model s ability to simulate states (e.g., soil moisture) and fluxes (e.g., runoff) is limited by uncertainties in meteorological forcing and parameter inputs as well as inadequacies in model physics. In this study, anomalies of terrestrial water storage (TWS) observed by the Gravity Recovery and Climate Experiment (GRACE) satellite mission were assimilated into the NASA Catchment land surface model in western and central Europe for a 7-year period, using a previously developed ensemble Kalman smoother. GRACE data assimilation led to improved runoff correlations with gauge data in 17 out of 18 hydrological basins, even in basins smaller than the effective resolution of GRACE. Improvements in root zone soil moisture were less conclusive, partly due to the shortness of the in situ data record. In addition to improving temporal correlations, GRACE data assimilation also reduced increasing trends in simulated monthly TWS and runoff associated with increasing rates of precipitation. GRACE assimilated root zone soil moisture and TWS fields exhibited significant changes in their dryness rankings relative to those without data assimilation, suggesting that GRACE data assimilation could have a substantial impact on drought monitoring. Signals of drought in GRACE TWS correlated well with MODIS Normalized Difference Vegetation Index (NDVI) data in most areas. Although they detected the same droughts during warm seasons, drought signatures in GRACE derived TWS exhibited greater persistence than those in NDVI throughout all seasons, in part due to limitations associated with the seasonality of vegetation.

Li, Bailing↗
DOE OSTI2024

gcm_eval (Global Climate Model Evaluation) [SWR-24-37]

The interplay between energy, climate, and weather is becoming more complex as our changing climate continues to affect the weather we experience which in turn drives changes in the ever increasing share of renewable energy generation and energy demand. Energy system analyses commonly rely on meteorological inputs to estimate renewable energy generation and energy demand; however, these inputs rarely represent the estimated impacts of future climate change. Climate models and publicly available climate change datasets can be used for this purpose, but the selection of inputs from the myriad of available models and datasets is a nuanced and often subjective process. There is no single perfect climate model or dataset for all applications. In this repository, we include software and the corresponding assessments of various global climate models (GCMs) from the Coupled Model Intercomparison Project (CMIP6), evaluating their skills with respect to the historical climate and comparing of their future projections of climate change. We focus on variables that directly affect the energy system including the representation of extreme values that can drive grid resilience events. The objective of this repository is not to recommend the best climate model and dataset for a given analysis, but instead to provide a reference to facilitate the selection of climate models and datasets in subsequent work.

Buster, Grant↗
DOE OSTI2016

On polarimetric radar signatures of deep convection for model evaluation: columns of specific differential phase observed during MC3E

The representation of deep convection in general circulation models is in part informed by cloud-resolving models (CRMs) that function at higher spatial and temporal resolution; however, recent studies have shown that CRMs often fail at capturing the details of deep convection updrafts. With the goal of providing constraint on CRM simulation of deep convection updrafts, ground-based remote sensing observations are analyzed and statistically correlated for four deep convection events observed during the Midlatitude Continental Convective Clouds Experiment (MC3E). Since positive values of specific differential phase observed above the melting level are associated with deep convection updraft cells, so-called columns are analyzed using two scanning polarimetric radars in Oklahoma: the National Weather Service Vance WSR-88D (KVNX) and the Department of Energy C-band Scanning Atmospheric Radiation Measurement (ARM) Precipitation Radar (C-SAPR). KVNX and C-SAPR volumes and columns are then statistically correlated with vertical winds retrieved via multi-Doppler wind analysis, lightning flash activity derived from the Oklahoma Lightning Mapping Array, and KVNX differential reflectivity . Results indicate strong correlations of volume above the melting level with updraft mass flux, lightning flash activity, and intense rainfall. Analysis of columns reveals signatures of changing updraft properties from one storm event to another as well as during event evolution. Comparison of to shows commonalities in information content of each, as well as potential problems with associated with observational artifacts.

Atm↗
DOE OSTI2020

Clouds and Radiation Processes in Regional Climate Models Evaluated Using Observations Over the Ice‐free Arctic Ocean

Abstract The presence of clouds in the Arctic regulates the surface energy budget (SEB) over the sea‐ice surface and the ice‐free ocean. Following several previous field campaigns, the cloud‐radiation relationship, including cloud vertical structure and phase, has been elucidated; however, modeling of this relationship has matured slowly. In recognition of the recent decline in the Arctic sea‐ice extent, representation of the cloud system in numerical models should consider the effects of areas covered by sea ice and ice‐free areas. Using an in situ stationary meteorological observation data set obtained over the ice‐free Arctic Ocean by the Japanese Research Vessel Mirai (September 2014), coordinated evaluation of six regional climate models (RCMs) with nine model runs was performed by focusing on clouds and the SEB. The most remarkable findings were as follows: (1) reduced occurrence of unstable stratification with low‐level cloud water in all models in comparison to the observations, (2) significant differences in cloud water representations between single‐ and double‐moment cloud schemes, (3) extensive differences in partitioning of hydrometeors including solid/liquid precipitation, and (4) pronounced lower‐tropospheric air temperature biases. These issues are considered as the main sources of SEB uncertainty over ice‐free areas of the Arctic Ocean. The results from a coupled RCM imply that the SEB is constrained by both the atmosphere and the ocean (and sea ice) with considerable feedback. Coordinated improvement of both stand‐alone atmospheric and coupled RCMs would promote a more comprehensive and improved understanding of the Arctic air‐ice‐sea coupled system.

Inoue, Jun↗