Search NASA⌕ Search

SEARCH · Search NASA

Results for “model evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 667 records · Page 37

Basin Scale Estimates of Evapotranspiration Using GRACE and other Observations

Evapotranspiration is integral to studies of the Earth system, yet it is difficult to measure on regional scales. One estimation technique is a terrestrial water budget, i.e., total precipitation minus the sum of evapotranspiration and net runoff equals the change in water storage. Gravity Recovery and Climate Experiment (GRACE) satellite gravity observations are now enabling closure of this equation by providing the terrestrial water storage change. Equations are presented here for estimating evapotranspiration using observation based information, taking into account the unique nature of GRACE observations. GRACE water storage changes are first substantiated by comparing with results from a land surface model and a combined atmospheric-terrestrial water budget approach. Evapotranspiration is then estimated for 14 time periods over the Mississippi River basin and compared with output from three modeling systems. The GRACE estimates generally lay in the middle of the models and may provide skill in evaluating modeled evapotranspiration.

Rodell, M.↗

The Ocean Carbon States Database: A Proof-of-Concept Application of Cluster Analysis in the Ocean Carbon Cycle

In this paper, we present a database of the basic regimes of the carbon cycle in the ocean, the 'ocean carbon states', as obtained using a data mining/pattern recognition technique in observation-based as well as model data. The goal of this study is to establish a new data analysis methodology, test it and assess its utility in providing more insights into the regional and temporal variability of the marine carbon cycle. This is important as advanced data mining techniques are becoming widely used in climate and Earth sciences and in particular in studies of the global carbon cycle, where the interaction of physical and biogeochemical drivers confounds our ability to accurately describe, understand, and predict CO2 concentrations and their changes in the major planetary carbon reservoirs. In this proof-of-concept study, we focus on using well-understood data that are based on observations, as well as model results from the NASA Goddard Institute for Space Studies (GISS) climate model. Our analysis shows that ocean carbon states are associated with the subtropical-subpolar gyre during the colder months of the year and the tropics during the warmer season in the North Atlantic basin. Conversely, in the Southern Ocean, the ocean carbon states can be associated with the subtropical and Antarctic convergence zones in the warmer season and the coastal Antarctic divergence zone in the colder season. With respect to model evaluation, we find that the GISS model reproduces the cold and warm season regimes more skillfully in the North Atlantic than in the Southern Ocean and matches the observed seasonality better than the spatial distribution of the regimes. Finally, the ocean carbon states provide useful information in the model error attribution. Model air-sea CO2 flux biases in the North Atlantic stem from wind speed and salinity biases in the subpolar region and nutrient and wind speed biases in the subtropics and tropics. Nutrient biases are shown to be most important in the Southern Ocean flux bias.

carbon cycle↗

Summary of Historical Production for Nevada Binary Facilities

The analysis described was initiated to validate inputs used in the US Department of Energy’s (DOE) economic modeling tool GETEM (Geothermal Electricity Technology Evaluation Model) by using publically available data to identify production trends at operating geothermal binary facilities in the state of Nevada. Data required for this analysis was obtained from the Nevada Bureau of Mines and Geology (NBMG), whom received the original operator reports from the Nevada Division of Minerals (NDOM). The data from the NBMG was inputted into Excel files that have been uploaded to the DOE’s National Geothermal Data System (NGDS). Once data was available in an Excel format, production trends for individual wells and facilities could be established for the periods data was available (thru 2009). Additionally, this analysis identified relationships existing between production (temperature and flow rates), power production and plant conversion efficiencies. The data trends showed that temperature declines have a significant impact on power production, and that in some instances operators increased production flow rate to offset power declines. The production trends with time that were identified are being used to update GETEM’s default inputs.

15 GEOTHERMAL ENERGY↗

Full-Column Greenhouse Gas Sampling Field Campaign Report

The AirCore is a balloon-borne sampler patented and developed at the National Oceanic and Atmospheric Administration (NOAA) Earth System Research Laboratory (ESRL) Global Monitoring Division (GMD) (Tans 2009) that has the capability to provide full-column measurements of greenhouse gases and other trace gas species (Karion et al. 2010). It consists of a long, coated stainless steel tube that is filled with a known trace gas mixture prior to balloon launch (Figure 1). The AirCore is opened at one end prior to balloon launch and, as the balloon ascends, it evacuates to ambient pressure. At 30 km, the balloon is cut away and the AirCore descends via parachute, now capturing an atmospheric air sample or “core” of air from 30 km to the surface due to the pressure gradient between the evacuated AirCore and the environment. Upon landing, the AirCore is closed, preserving the air sample collected in-flight. Because diffusion acts slowly on the AirCore sample in the four hours that it typically takes to launch, retrieve, and analyze the sample, it is possible to capture more than 100 discrete samples in the atmospheric profile collected by the AirCore. Currently, the AirCore is analyzed for carbon dioxide (CO2), methane (CH4), and carbon monoxide (CO) at NOAA/ESRL GMD, but the sample collected can be measured on a variety of other analyzers to obtain full-column profiles of other trace gas species. Unlike routine aircraft measurements, the AirCore provides a low-cost alternative to atmospheric profiling and captures >95% of the atmospheric column. It has a variety of applications, including to full-column greenhouse and trace gas sampling and monitoring, to satellite and remote-sensing instrumentation evaluation, and to tropospheric and stratospheric model evaluation.

Greenhouse gases, Southern Great Plains, balloon-b↗

Evaluating atmospheric correction models for retrieving surface temperatures from the AVHRR over a tallgrass prairie

The effects of atmospheric attenuation on surface radiative temperatures obtained by the AVHRR over a tallgrass prairie area in the Flint Hills of Kansas are examined. Six atmospheric correction models developed primarily for sea-surface temperature studies are used to test their utility for retrieval of radiative temperatures over the land surface. An uncertainty of + or - 3.0 C was found for the AVHRR data, and used to evaluate the performance of a given model. When the difference between in situ and AVHRR surface temperatures was smaller than the uncertainty, the model was judged to be adequate. Among the six models evaluated, only the NOAA split-window model consistently adjusted the AVHRR surface temperatures within + or - 3.0 C of the in situ measurements.

Cooper, D. I.↗

Efficient surrogate modeling methods for large-scale Earth system models based on machine-learning techniques

Abstract. Improving predictive understanding of Earth system variability and change requires data–model integration. Efficient data–model integration for complex models requires surrogate modeling to reduce model evaluation time. However, building a surrogate of a large-scale Earth system model (ESM) with many output variables is computationally intensive because it involves a large number of expensive ESM simulations. In this effort, we propose an efficient surrogate method capable of using a few ESM runs to build an accurate and fast-to-evaluate surrogate system of model outputs over large spatial and temporal domains. We first use singular value decomposition to reduce the output dimensions and then use Bayesian optimization techniques to generate an accurate neural network surrogate model based on limited ESM simulation samples. Our machine-learning-based surrogate methods can build and evaluate a large surrogate system of many variables quickly. Thus, whenever the quantities of interest change, such as a different objective function, a new site, and a longer simulation time, we can simply extract the information of interest from the surrogate system without rebuilding new surrogates, which significantly reduces computational efforts. We apply the proposed method to a regional ecosystem model to approximate the relationship between eight model parameters and 42 660 carbon flux outputs. Results indicate that using only 20 model simulations, we can build an accurate surrogate system of the 42 660 variables, wherein the consistency between the surrogate prediction and actual model simulation is 0.93 and the mean squared error is 0.02. This highly accurate and fast-to-evaluate surrogate system will greatly enhance the computational efficiency of data–model integration to improve predictions and advance our understanding of the Earth system.

58 GEOSCIENCES↗

Evaluation of Turbulence-Model Performance as Applied to Jet-Noise Prediction

The accurate prediction of jet noise is possible only if the jet flow field can be predicted accurately. Predictions for the mean velocity and turbulence quantities in the jet flowfield are typically the product of a Reynolds-averaged Navier-Stokes solver coupled with a turbulence model. To evaluate the effectiveness of solvers and turbulence models in predicting those quantities most important to jet noise prediction, two CFD codes and several turbulence models were applied to a jet configuration over a range of jet temperatures for which experimental data is available.

Woodruff, S. L.↗

Unified constitutive materials model development and evaluation for high-temperature structural analysis applications

Unified constitutive material models were developed for structural analyses of aircraft gas turbine engine hot section components with particular application to an isotropic material used for combustor liners. Differential forms of models independently developed were considered in this study. These models combine the interactions of time-dependent (creep) and time-independent (plasticity) inelastic behavior of a material. Predicted stress-strain responses from these models were evaluated against cyclic isothermal and nonisothermal test results for uniaxial specimens of a nickel-base superalloy. The unified models were implemented in a nonlinear structural analysis code. Two unique NASA Lewis test facilities were used in the evaluation of the models for complex geometry specimens and evaluation of advanced temperature and high-temperature strain measurement instrumentation. Predicted nonlinear structural responses from one of the models for a flat plate and a segment of a conventional combustor liner are presented.

Thompson, R. L.↗

Evaluation of help model replacement codes

This work evaluates the computer codes that are proposed to be used to predict percolation of water through the closure-cap and into the waste containment zone at the Department of Energy closure sites. This work compares the currently used water-balance code (HELP) with newly developed computer codes that use unsaturated flow (Richards’ equation). It provides a literature review of the HELP model and the proposed codes, which result in two recommended codes for further evaluation: HYDRUS-2D3D and VADOSE/W. This further evaluation involved performing actual simulations on a simple model and comparing the results of those simulations to those obtained with the HELP code and the field data. From the results of this work, we conclude that the new codes perform nearly the same, although moving forward, we recommend HYDRUS-2D3D.

97 MATHEMATICS AND COMPUTING↗

A new test statistic for climate models that includes field and spatial dependencies using Gaussian Markov random fields

Abstract. A new test statistic for climate model evaluation has been developed that potentially mitigates some of the limitations that exist for observing and representing field and space dependencies of climate phenomena. Traditionally such dependencies have been ignored when climate models have been evaluated against observational data, which makes it difficult to assess whether any given model is simulating observed climate for the right reasons. The new statistic uses Gaussian Markov random fields for estimating field and space dependencies within a first-order grid point neighborhood structure. We illustrate the ability of Gaussian Markov random fields to represent empirical estimates of field and space covariances using "witch hat" graphs. We further use the new statistic to evaluate the tropical response of a climate model (CAM3.1) to changes in two parameters important to its representation of cloud and precipitation physics. Overall, the inclusion of dependency information did not alter significantly the recognition of those regions of parameter space that best approximated observations. However, there were some qualitative differences in the shape of the response surface that suggest how such a measure could affect estimates of model uncertainty.

54 ENVIRONMENTAL SCIENCES↗

A riming‐dependent parameterization of scattering by snowflakes using the self‐similar Rayleigh–Gans approximation

Abstract Riming is a key process of precipitation formation in ice‐containing clouds, but quantifying riming from observations is challenging, limiting our ability to evaluate the riming process in numerical weather models. One challenge for radar observations is that riming changes both the physical properties (mass, area cross‐section) and scattering properties of ice particles. These changes need to be implemented consistently as a function of riming in radar forward operators, which are required for retrievals and model evaluation in observation space. In this study, mass–size, cross‐section area–size, and backscattering cross‐section relations are developed as a function of the normalized rime mass for aggregates composed of various monomer types (columns, dendrites, needles, plates, and rosettes). The proposed framework allows us to simulate scattering properties of aggregated ice particles consistently as a function of riming in retrievals and radar forward operators. The parameterizations are developed from a large data set of simulated rimed aggregates of different sizes and monomer crystal types. The backscattering cross‐section parameterization (the “riming‐dependent parameterization”) is evaluated for radar frequencies of 35.6 and 94.0 GHz and is based on the Self‐Similar Rayleigh–Gans approximation (SSRGA), which is increasingly used to calculate microwave scattering of ice crystals and snowflakes. Compared with parameterizations from the literature that do not consider riming, the riming‐dependent parameterization leads to significantly smaller biases in terms of backscattering cross‐section. When using the particle masses and scattering properties of the individual particles simulated by the aggregation and riming model as a reference, the bias of our parameterization is below 1 dB when integrating over an exponential particle size distribution with sizes from 0.1–10 mm.

54 ENVIRONMENTAL SCIENCES↗

Collaborative Project: The problem of bias in defining uncertainty in computationally enabled strategies for data-driven climate model development

The objective of the project is to develop strategies for better representing scientific sensibilities within statistical measures of model skill that then can be used within a Bayesian statistical framework for data-driven climate model development and improved measures of model scientific uncertainty. One of the thorny issues in model evaluation is quantifying the effect of biases on climate projections. While any bias is not desirable, only those biases that affect feedbacks affect scatter in climate projections. The effort at the University of Texas is to analyze previously calculated ensembles of CAM3.1 with perturbed parameters to discover how biases affect projections of global warming. The hypothesis is that compensating errors in the control model can be identified by their effect on a combination of processes and that developing metrics that are sensitive to dependencies among state variables would provide a way to select version of climate models that may reduce scatter in climate projections. Gabriel Huerta was responsible for leading the effort to develop statistical methods for evaluating these field dependencies. The UT effort will incorporate these developments into MECS, which is a set of python scripts being developed at the University of Texas for managing the workflow associated with data-driven climate model development over HPC resources. We regularly consult with Peter Gleckler (LLNL), Mark Taylor (SNL), and Rich Neale (NCAR) to provide additional insight into issues concerning CAM5 model development and multi-model evaluation of the AR5 archive. This grant was originally assigned to Indiana University where both the PI (Huerta) and the Postdoc (Nosedal) worked on the project from March 1, 2012-March 31, 2013.

97 MATHEMATICS AND COMPUTING↗

A comparison of surrogate constitutive models for viscoplastic creep simulation of HT-9 steel

Mechanistic microstructure-informed constitutive models for the mechanical response of polycrystals are a cornerstone of computational materials science. However, as these models become increasingly more complex – often involving coupled differential equations describing the effect of specific deformation modes – their associated computational costs can become prohibitive, particularly in optimization or uncertainty quantification tasks that require numerous model evaluations. To address this challenge, surrogate constitutive models that balance accuracy and computational efficiency are highly desirable. Data-driven surrogate models, that learn the constitutive relation directly from data, have emerged as a promising solution. In this work, we develop two local surrogate models for the viscoplastic response of a steel: a piecewise response surface method and a mixture of experts model. These surrogates are designed to adapt to complex material behavior, which may vary with material parameters or operating conditions. The surrogate constitutive models are applied to creep simulations of HT-9 steel, an alloy of considerable interest to the nuclear energy sector due to its high tolerance to radiation damage, using training data generated from viscoplastic self-consistent (VPSC) simulations. In conclusion, we define a set of test metrics to numerically assess the accuracy of our surrogate models for predicting viscoplastic material behavior, and show that the mixture of experts model outperforms the piecewise response surface method in terms of accuracy.

36 MATERIALS SCIENCE↗

Constraining the Equation of State of Dense Neutron-rich Matter (Final Report)

Using a new Time Projection Chamber (TPC), called the SpiRIT (SAMURAI pion Reconstruction Ion Tracker) constructed and funded by DOE office of science (award # DE-SC0004835), we performed two experiments with the SAMURAI spectrometer at RIKEN, Japan to study the equation of state of neutron rich matter. As a result of the project, the SpiRIT collaboration, an international collaboration consisting of groups from US, Japan, Korea, Poland, China and Germany, has been formed to pursue the science opportunities provided by the SpiRIT TPC. After the experiments, we developed the software to analyze the SpiRIT experiments and extract constraints of symmetry energy at supra-saturation densities. As a result 270 TB of data have been obtained. Over 10 technical papers on the TPC and 4 science papers on the nuclear equation of state have been published. A total of six (2 US, 1 Japanese and 1 Korean) PhD students have graduated based on their research on the EOS research using the SpiRIT data. In addition, the data also stimulated theorists on transport models to form the “Transport Model Evaluation Project” collaboration to improve the transport model to compare calculations with the data.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Constraining the Equation of State of Dense Neutron-rich Matter (Final Technical Report)

Using a new Time Projection Chamber (TPC), called the SpiRIT (SAMURAI pion Reconstruction Ion Tracker) constructed and funded by DOE office of science (award # DE-SC0004835), we performed two experiments with the SAMURAI spectrometer at RIKEN, Japan to study the equation of state of neutron rich matter. As a result of the project, the SpiRIT collaboration, an international collaboration consisting of groups from US, Japan, Korea, Poland, China and Germany, has been formed to pursue the science opportunities provided by the SpiRIT TPC. After the experiments, we developed the software to analyze the SpiRIT experiments and extract constraints of symmetry energy at supra-saturation densities. As a result 270 TB of data have been obtained. Over 10 technical papers on the TPC and 4 science papers on the nuclear equation of state have been published. A total of six (2 US, 1 Japanese and 1 Korean) PhD students have graduated based on their research on the EOS research using the SpiRIT data. In addition, the data also stimulated theorists on transport models to form the “Transport Model Evaluation Project” collaboration to improve the transport model to compare calculations with the data.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Representation of the Community Earth System Model (CESM1) CAM4-chem within the Chemistry-Climate Model Initiative (CCMI)

The Community Earth System Model (CESM1) CAM4-chem has been used to perform the Chemistry Climate Model Initiative (CCMI) reference and sensitivity simulations. In this model, the Community Atmospheric Model version 4 (CAM4) is fully coupled to tropospheric and stratospheric chemistry. Details and specifics of each configuration, including new developments and improvements are described. CESM1 CAM4-chem is a low-top model that reaches up to approximately 40 km and uses a horizontal resolution of 1.9° latitude and 2.5° longitude. For the specified dynamics experiments, the model is nudged to Modern-Era Retrospective Analysis for Research and Applications (MERRA) reanalysis. We summarize the performance of the three reference simulations suggested by CCMI, with a focus on the last 15 years of the simulation when most observations are available. Comparisons with selected data sets are employed to demonstrate the general performance of the model. We highlight new data sets that are suited for multi-model evaluation studies. Most important improvements of the model are the treatment of stratospheric aerosols and the corresponding adjustments for radiation and optics, the updated chemistry scheme including improved polar chemistry and stratospheric dynamics and improved dry deposition rates. These updates lead to a very good representation of tropospheric ozone within 20 % of values from available observations for most regions. In particular, the trend and magnitude of surface ozone is much improved compared to earlier versions of the model. Furthermore, stratospheric column ozone of the Southern Hemisphere in winter and spring is reasonably well represented. In conclusion, all experiments still underestimate CO most significantly in Northern Hemisphere spring and show a significant underestimation of hydrocarbons based on surface observations.

54 ENVIRONMENTAL SCIENCES↗