Search NASA⌕ Search

SEARCH · Search NASA

Results for “model evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 811 records · Page 45

Evaluating alternative ebullition models for predicting peatland methane emission and its pathways via data–model fusion

Abstract. Understanding the dynamics of peatland methane (CH4) emissions and quantifying sources of uncertainty in estimating peatland CH4 emissions are critical for mitigating climate change. The relative contributions of CH4 emission pathways through ebullition, plant-mediated transport, and diffusion, together with their different transport rates and vulnerability to oxidation, determine the quantity of CH4 to be oxidized before leaving the soil. Notwithstanding their importance, the relative contributions of the emission pathways are highly uncertain. In particular, the ebullition process is more uncertain and can lead to large uncertainties in modeled CH4 emissions. To improve model simulations of CH4 emission and its pathways, we evaluated two model structures: (1) the ebullition bubble growth volume threshold approach (EBG) and (2) the modified ebullition concentration threshold approach (ECT) using CH4 flux and concentration data collected in a peatland in northern Minnesota, USA. When model parameters were constrained using observed CH4 fluxes, the CH4 emissions simulated by the EBG approach (RMSE = 0.53) had a better agreement with observations than the ECT approach (RMSE = 0.61). Further, the EBG approach simulated a smaller contribution from ebullition but more frequent ebullition events than the ECT approach. The EBG approach yielded greatly improved simulations of pore water CH4 concentrations, especially in the deep soil layers, compared to the ECT approach. When constraining the EBG model with both CH4 flux and concentration data in model–data fusion, uncertainty of the modeled CH4 concentration profiles was reduced by 78 % to 86 % in comparison to constraints based on CH4 flux data alone. The improved model capability was attributed to the well-constrained parameters regulating the CH4 production and emission pathways. Our results suggest that the EBG modeling approach better characterizes CH4 emission and underlying mechanisms. Moreover, to achieve the best model results both CH4 flux and concentration data are required to constrain model parameterization.

59 BASIC BIOLOGICAL SCIENCES↗

Influence of Precipitation Forcing Uncertainty on Hydrological Simulations with the NASA South Asia Land Data Assimilation System

Accurate meteorological estimates are critical for process-based hydrological simulationand prediction. This presents a significant challenge in mountainous Asia where in situmeteorological stations are limited and major river basins cross international borders. In thiscontext, remotely sensed and model-derived meteorological estimates are often necessary inputsfor distributed hydrological analysis. However, these datasets are difficult to evaluate on accountof limited access to ground data. In this case, the implications of uncertainty associated withprecipitation forcing for hydrological simulations is explored by driving the South Asia Land DataAssimilation System (South Asia LDAS) using a range of meteorological forcing products.MERRA2, GDAS, and CHIRPS produce a wide range of estimates for rainfall, which causes awidespread simulated streamflow and evapotranspiration. A combination of satellite-derived andlimited in situ data are applied to evaluate model simulations and, by extension, to constrain theestimates of precipitation. The results show that available gridded precipitation estimates based onin situ data may systematically underestimate precipitation in mountainous regions and thatperformance of gridded satellite-derived or modeled precipitation estimates varies systematicallyacross the region. Since no station-based data or product including station data is satisfactoryeverywhere, our results suggest that the evaluation of the hydrological simulation of streamflowand ET can be used as an indirect evaluation of precipitation forcing based on ground-basedproducts or in-situ data. South Asia LDAS produces reasonable evapotranspiration and streamflowwhen forced with appropriate meteorological forcing and the choice of meteorological forcingshould be made based on the geographical location as well as on the purpose of the simulations.

South Asia land data assimilation system (South As↗

A progress update on selected AREVA NP advanced BWR methodologies

This article presents an overview of select advances in AREVA NP methods concerning 3 topics: Analytical fuel rod model, Transient and accident simulation, and ATWSi post dryout treatment. Concerning the first topic, the analytical fuel rod model relies heavily on first-principles closed-form solutions instead of ever finer nodalization and higher dimensions of standard numerical methods such as finite differences or finite elements. This analytical fuel rod model takes full account of the complications that made numerical analysis necessary, e.g. variation of pellet thermal conductivity with temperature and burnup, and the radial dependence of fission power deposition in pellets. The model also calculates thermo-mechanical parameters such as clad stress and strain, fission gas release, pellet sintering and swelling, and pellet-clad gap size and thermal resistance. The model is suited for innovative fuel designs such as doped pellets. Concerning the second topic, the BWR evaluation model, AURORA-B, is an advanced transient and accident simulator including SRELAP5, a two fluid thermal hydraulic code comprehensively reviewed by international safety authorities for BWR and PWR applications; RODEX4, a best estimate fuel performance code used to evaluate the local thermal-mechanical behavior of fuel rods during postulated transients and accidents; and MB2-K, a neutron kinetics extension of the steady state core simulator MICROBURN-B2. The analytical domains will include transients as well as control rod drop and loss of coolant accidents. Extension of RODEX4 have been made to accommodate analysis of chromium-doped fuel. Concerning the third issue,The new AREVA post-dryout models can accurately predict dryout/re-wet followed by an ultimate failure to re-wet under oscillatory ATWSi (Anticipated Transient Without SCRAM with core instability) conditions. These models, while not a boiling curve based method, can qualitatively reproduce the characteristics of the boiling curve.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Precipitable water vapor over oceans from the Maritime Aerosol Network: Evaluation of global models and satellite products under clear sky conditions

We present results from an evaluation of precipitable water vapor (W) over remote oceanic areas as derived from global reanalysis models and from satellites against observations from the Maritime Aerosol Network (MAN) for cloudless skies during the period of 2004–2017. They cover polar, mid latitude and tropical oceanic regions and represent a first effort to use MAN observations for such evaluation. The global reanalysis model products evaluated in this study are from the Modern-Era Retrospective analysis for Research and Applications Version 2 (MERRA-2), the European Centre for Medium-Range Weather Forecasts (ECMWF) Interim Reanalysis (ERA I), and the Climate Forecast System Reanalysis (CFSR) model. The satellite products evaluated are from the Moderate Resolution Imaging Spectroradiometer (MODIS), the Polarization and Directionality of the Earth's Reflectances (POLDER), the Global Ozone Monitoring Experiment (GOME-2), the Scanning Imaging Absorption Spectrometer for Atmospheric Chartography (SCIAMACHY), and the Atmospheric Infra-red Sounder (AIRS). Satellite retrievals of W are based on the attenuation of solar reflected light by water vapor absorption bands, except those from AIRS that rely on brightness temperature measurements. A very good agreement is observed between the model estimates and MAN, with mean differences of ~5% and standard deviations of ~15%. These results are within the uncertainties associated with the models and the measurements, indicating the skill of the reanalysis models to estimate W over oceans under clear sky conditions. Mean differences of W between the satellite and MAN products are ~11, 6.7, 12, −7, and 3% for MODIS, POLDER, GOME-2, SCIAMACHY and AIRS respectively, while their standard deviations are 31, 29, 28, 20 and 17%. These differences reveal the need to address inconsistencies among different satellite sensors and ground-based measurements to reduce the uncertainties associated with the retrievals.

Daniel Pérez-Ramírez↗

Toward integrating high-fidelity CFD approaches in the thermal-hydraulic analysis of turbulent dry cask systems

Nuclear power plants have been supplying resilient and reliable electricity for decades, contributing to energy independence of the U.S.. However, nuclear waste management remains one of the most significant challenges in the industry. The safety of dry cask storage systems relies heavily on their thermal-hydraulic performance. Computational Fluid Dynamics (CFD) simulations are often used to demonstrate this performance and ensure that the system design meets safety standards. This study presents reduced numerical models for various types of dry cask systems. These numerical models can produce efficient and fast results based on the employed modeling strategies. Additionally, the study uses a novel approach to high-fidelity simulations to evaluate modeling assumptions in dry cask modeling. Large Eddy Simulations (LES) are used for this purpose, particularly in regions where fluid velocity is relatively high and the turbulence characteristics become important. Furthermore, the results of these high-fidelity simulations will enhance the interpretation of outcomes produced from a lower-fidelity CFD model.

CFD↗

Scalable posterior approximations for large-scale Bayesian inverse problems via likelihood-informed parameter and state reduction

Two prime bottlenecks to the solution of large-scale Bayesian inverse problems are the scaling of posterior sampling algorithms to high-dimensional parameter spaces and the computational cost of forward model evaluations. However, incomplete or noisy data, the state variation and parameter dependence of the forward model, and correlations in the prior collectively provide useful structure that can be exploited for dimension reduction in this setting—both in the parameter space of the inverse problem and in the state space of the forward model. To this end, we show how to jointly construct low-dimensional subspaces of the parameter space and the state space in order to accelerate the Bayesian solution of the inverse problem. As a byproduct of state dimension reduction, we also show how to identify low-dimensional subspaces of the data in problems with high-dimensional observations. These subspaces enable approximation of the posterior as a product of two factors: (i) a projection of the posterior onto a low-dimensional parameter subspace, wherein the original likelihood is replaced by an approximation involving a reduced model; and (ii) the marginal prior distribution on the high-dimensional complement of the parameter subspace. We introduce and compare several strategies for constructing these subspaces using only a limited number of forward and adjoint model simulations. The resulting posterior approximations can rapidly be characterized using standard sampling techniques, e.g., Markov chain Monte Carlo. Two numerical examples demonstrate the accuracy and efficiency of our approach: inversion of an integral equation in atmospheric remote sensing, where the data dimension is very high; and the inference of a heterogeneous transmissivity field in a groundwater system, which involves a partial differential equation forward model with high dimensional state and parameters.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Data, scripts, and figures associated with a manuscript studying impact of climate and topography on post-fire vegetation recovery.

This data package is associated with the publication “Impact of Topography and Climate on Post-fire Vegetation Recovery Across Different Burn Severity and Land Cover Types through Machine Learning” submitted to Remote Sensing of Environment (Zahura et al. 2023). In this research, a machine learning algorithm, random forest (RF), was utilized to examine the impact of climate and topography on post-fire vegetation recovery. We used enhanced vegetation index (EVI) to examine varying burn severity and land cover types. The data package includes the input files for RF model training, outputs from model predictions and analysis, and python scripts to run the model, analyze the results to understand model performance and interpretability, and plot manuscript figures. This data package contains three folders (Data, Scripts, and Figures), a file-level metadata (FLMD) csv, and a data dictionary (dd) csv. Please see Postfire_recovery_flmd.csv for a list of all files contained in this data package and descriptions for each. The data dictionary (Postfire_recovery_dd.csv) describes the csv column headers. The “Data” folder provides all the inputs and outputs to train the RF model, evaluate performance, and interpret predictions. The “Scripts” folder contains python scripts and jupyter notebooks for model training and result analysis. The “Figures” folder includes the figures used in the manuscript in “.png” and “.jpg” format.

54 ENVIRONMENTAL SCIENCES↗

Residential Mechanical Precooling

This research conducted by the Alliance for Residential Building Innovation team evaluated mechanical air conditioner pre-cooling strategies in homes throughout the United States. EnergyPlus modeling evaluated two homes with different performance characteristics in seven climates. Results are applicable to new construction homes and most existing homes built in the last 10 years, as well as fairly efficient retrofitted homes.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Arctic Cloud Microphysical Processes. Final report

The primary goal of our team’s effort under this grant was to quantify Arctic lower-troposphere processes by tying data from the large suite of NSA instruments to modeling efforts indirectly through data assimilation and directly through remote sensing and cloud modeling efforts. We maintained a distinct focus on how the ARM measurements may be used evaluate model simulations and/or physical processes. We will separate this report into two distinct sections: our data assimilation effort and the use of NSA instruments to improve our understanding and modeling of model physics to better characterize the critical surface energy budget in polar regimes.

58 GEOSCIENCES↗

A first large-scale flood inundation forecasting model

At present continental to global scale flood forecasting focusses on predicting at a point discharge, with little attention to the detail and accuracy of local scale inundation predictions. Yet, inundation is actually the variable of interest and all flood impacts are inherently local in nature. This paper proposes a first large scale flood inundation ensemble forecasting model that uses best available data and modeling approaches in data scarce areas and at continental scales. The model was built for the Lower Zambezi River in southeast Africa to demonstrate current flood inundation forecasting capabilities in large data-scarce regions. The inundation model domain has a surface area of approximately 170k km2. ECMWF meteorological data were used to force the VIC (Variable Infiltration Capacity) macro-scale hydrological model which simulated and routed daily flows to the input boundary locations of the 2-D hydrodynamic model. Efficient hydrodynamic modeling over large areas still requires model grid resolutions that are typically larger than the width of many river channels that play a key a role in flood wave propagation. We therefore employed a novel sub-grid channel scheme to describe the river network in detail whilst at the same time representing the floodplain at an appropriate and efficient scale. The modeling system was first calibrated using water levels on the main channel from the ICESat (Ice, Cloud, and land Elevation Satellite) laser altimeter and then applied to predict the February 2007 Mozambique floods. Model evaluation showed that simulated flood edge cells were within a distance of about 1 km (one model resolution) compared to an observed flood edge of the event. Our study highlights that physically plausible parameter values and satisfactory performance can be achieved at spatial scales ranging from tens to several hundreds of thousands of km2 and at model grid resolutions up to several km2. However, initial model test runs in forecast mode revealed that it is crucial to account for basin-wide hydrological response time when assessing lead time performances notwithstanding structural limitations in the hydrological model and possibly large inaccuracies in precipitation data.

hydrodynamic, hydrology, forecast, flood inundatio↗

Resolving Mesoscale Convective Systems: Grid Spacing Sensitivity in the Tropics and Midlatitudes

Abstract Mesoscale convective systems (MCSs) are a critical global water cycle component and drive extreme precipitation events in tropical and midlatitude regions. However, simulating deep convection remains challenging for modern numerical weather and climate models due to the complex interactions of processes from microscales to synoptic scales. Recent models with kilometer‐scale horizontal grid spacings offer notable improvements in simulating deep convection compared to coarser‐resolution models. Still, deficiencies in representing key physical processes, such as entrainment, lead to systematic biases. Additionally, evaluating model outputs using process‐oriented observational data remain difficult. This study presents an ensemble of MCS simulations with spanning the deep convective gray zone ( from 12 km to 125 m) in the Southern Great Plains of the U.S. and the Amazon Basin. Comparing these simulations with Atmospheric Radiation Measurement (ARM) wind profiler observations, we find greater sensitivity in the Amazon Basin compared to the Great Plains. Convective drafts converge structurally at sub‐kilometer scales, but some deficiencies remain. In both regions, simulated up and downdrafts are too deep and extreme downdrafts are not strong enough. Furthermore, Amazonian updrafts are too strong. Overall, we observe higher sensitivity in the tropics, including an artificial buildup in vertical kinetic energy at scales of , suggesting a need for 250 m in this region. Nevertheless, bulk convergence—agreement of storm‐average statistics—is achievable with kilometer‐scale simulations within a 10% error margin with 1 km providing a good balance between accuracy and computational cost.

54 ENVIRONMENTAL SCIENCES↗

Optimization and supervised machine learning methods for fitting numerical physics models without derivatives

Here, we address the calibration of a computationally expensive nuclear physics model for which derivative information with respect to the fit parameters is not readily available. Of particular interest is the performance of optimization-based training algorithms when dozens, rather than millions or more, of training data are available and when the expense of the model places limitations on the number of concurrent model evaluations that can be performed. As a case study, we consider the Fayans energy density functional model, which has characteristics similar to many model fitting and calibration problems in nuclear physics. We analyze hyperparameter tuning considerations and variability associated with stochastic optimization algorithms and illustrate considerations for tuning in different computational settings.

97 MATHEMATICS AND COMPUTING↗

An evaluation of recent internal field models

The paper reviews the current status of internal field models and evaluates several recently published models by comparing their predictions with annual means of the magnetic field measured at 140 magnetic observatories from 1973 to 1977. Three of the four models studied, viz. AWC/75, IGS/75, and Pogo 8/71, were nearly equal in their ability to predict the magnitude and direction of the current field. The fourth model, IGRF 1975, was significantly poorer in its ability to predict the current field. All models seemed to be able to extrapolate predictions quite well several years outside the data range used to construct the models.

Mead, G. D.↗

Models for Facilitating Government-Funded Activities in the Post-ISS LEO Ecosystem

NASA is preparing for the retirement of the ISS and transition of LEO activities to one or more Commercial LEO Destinations (CLDs) by 2030. This transition necessitates new models for connecting NASA and other government-funded users of the LEO environment to platforms and opportunities. This paper describes for consideration six models for facilitating government-funded activities in the post-ISS LEO ecosystem. These six models are illustrative and represent a wide trade space of potential options, each relying on unique mechanisms for facilitating activities on one or more commercial LEO platforms or vehicles. We assessed each model across three possible future scenarios varying in number and diversity of LEO activities and commercial offerings, and across five stakeholder-driven model evaluation criteria. We present the highlights of the analysis, including ways to modify and strengthen each model. The Government Research Broker model performs best across all future scenarios, followed by Innovation Campus, Anchor Tenant, and Fee for Service. While Matchmaker and Institute Network exhibit positive aspects, these models perform most favorably in future scenarios with well-established communities and markets. While each model has strengths and weaknesses, no single model in its current form performs well across all criteria in all three future scenarios. NASA leadership can adjust models as desired to align closer to their priorities using combinations of unique model mechanisms. A model that meets leadership priorities is likely a combination of features from multiple models.

Erica Rodgers↗

Models for Facilitating Government-Funded Activities in the Post-ISS LEO Ecosystem

NASA is preparing for the retirement of the ISS and transition of LEO activities to one or more Commercial LEO Destinations (CLDs) by 2030. This transition necessitates new models for connecting NASA and other government-funded users of the LEO environment to platforms and opportunities. This paper describes for consideration six models for facilitating government-funded activities in the post-ISS LEO ecosystem. These six models are illustrative and represent a wide trade space of potential options, each relying on unique mechanisms for facilitating activities on one or more commercial LEO platforms or vehicles. We assessed each model across three possible future scenarios varying in number and diversity of LEO activities and commercial offerings, and across five stakeholder-driven model evaluation criteria. We present the highlights of the analysis, including ways to modify and strengthen each model. The Government Research Broker model performs best across all future scenarios, followed by Innovation Campus, Anchor Tenant, and Fee for Service. While Matchmaker and Institute Network exhibit positive aspects, these models perform most favorably in future scenarios with well-established communities and markets. While each model has strengths and weaknesses, no single model in its current form performs well across all criteria in all three future scenarios. NASA leadership can adjust models as desired to align closer to their priorities using combinations of unique model mechanisms. A model that meets leadership priorities is likely a combination of features from multiple models.

Erica Rodgers↗

Models for Facilitating Government-Funded Activities in the Post-ISS LEO Ecosystem

NASA is preparing for the retirement of the ISS and transition of LEO activities to one or more Commercial LEO Destinations (CLDs) by 2030. This transition necessitates new models for connecting NASA and other government-funded users of the LEO environment to platforms and opportunities. This paper describes for consideration six models for facilitating government-funded activities in the post-ISS LEO ecosystem. These six models are illustrative and represent a wide trade space of potential options, each relying on unique mechanisms for facilitating activities on one or more commercial LEO platforms or vehicles. We assessed each model across three possible future scenarios varying in number and diversity of LEO activities and commercial offerings, and across five stakeholder-driven model evaluation criteria. We present the highlights of the analysis, including ways to modify and strengthen each model. The Government Research Broker model performs best across all future scenarios, followed by Innovation Campus, Anchor Tenant, and Fee for Service. While Matchmaker and Institute Network exhibit positive aspects, these models perform most favorably in future scenarios with well-established communities and markets. While each model has strengths and weaknesses, no single model in its current form performs well across all criteria in all three future scenarios. NASA leadership can adjust models as desired to align closer to their priorities using combinations of unique model mechanisms. A model that meets leadership priorities is likely a combination of features from multiple models.

Erica Rodgers↗

Workshop on Current Issues in Predictive Approaches to Intelligence and Security Analytics: Fostering the Creation of Decision Advantage through Model Integration and Evaluation

The increasing asymmetric nature of threats to the security, health and sustainable growth of our society requires that anticipatory reasoning become an everyday activity. Currently, the use of anticipatory reasoning is hindered by the lack of systematic methods for combining knowledge- and evidence-based models, integrating modeling algorithms, and assessing model validity, accuracy and utility. The workshop addresses these gaps with the intent of fostering the creation of a community of interest on model integration and evaluation that may serve as an aggregation point for existing efforts and a launch pad for new approaches.

97 MATHEMATICS AND COMPUTING↗

Representing Anthropogenic Dust From Agricultural Sources in E3SMv1: Implementation, Evaluation, and Assessment of the Radiative Forcing

Dust emissions related to anthropogenic activities (i.e., anthropogenic dust) is not represented in most global climate models and its radiative impact remains unassessed. In this study, we develop a new and physically based method to parameterize anthropogenic dust emission from agricultural sources (i.e., agricultural dust, or AD) based on the DOE's Energy Exascale Earth System Model version 1 (E3SMv1). This method relates AD emission to the crop land use fraction in the E3SMv1 land component. Major AD sources simulated by our parameterization include those over Central America, the Sahel, North India, and North China. The annual averaged AD emission with diameter less than 10 μm is 567 Tg yr −1 in present-day (year 2000), which contributes to 13.3% of total dust emission. Model evaluation against satellite and ground-based observations shows that the new parameterization can represent AD emissions and global dust cycle reasonably well. We find that the total dust emission increases by 13.1% (495 Tg yr −1 ) from 1850 to 2000 due to the cropland land use fraction changes without considering the impact from climate changes. This induces a net dust direct effective radiative forcing of −0.041 W m −2 at top of the atmosphere. This dust-induced cooling exceeds 10% of the total anthropogenic aerosol direct effective radiative forcing from 1750 to 2014 estimated by the Intergovernmental Panel on Climate Change Sixth Assessment Report. Our findings indicate an important role of anthropogenic dust in global climate change, which should be included in future climate change assessments.

agricultural dust↗