Search NASA⌕ Search

SEARCH · Search NASA

Results for “model evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 253 records · Page 14
All sourcesNASA NTRSDOE OSTI
NASA NTRS1980

Feasibility of quasi-random band model in evaluating atmospheric radiance

The use of the quasi-random band model in evaluating upwelling atmospheric radiation is investigated. The spectral transmittance and total band adsorptance are evaluated for selected molecular bands by using the line by line model, quasi-random band model, exponential sum fit method, and empirical correlations, and these are compared with the available experimental results. The atmospheric transmittance and upwelling radiance were calculated by using the line by line and quasi random band models and were compared with the results of an existing program called LOWTRAN. The results obtained by the exponential sum fit and empirical relations were not in good agreement with experimental results and their use cannot be justified for atmospheric studies. The line by line model was found to be the best model for atmospheric applications, but it is not practical because of high computational costs. The results of the quasi random band model compare well with the line by line and experimental results. The use of the quasi random band model is recommended for evaluation of the atmospheric radiation.

Tiwari, S. N.↗
DOE OSTI2020

Status, Challenges, and Plans for Consistent Modeling and Evaluation of Fission Data: Nubar, PFNS, and FPY [Slides]

This presentation discusses the importance of consistent evaluations for fission observables. Currently, prompt fission quantities (e.g., neutron multiplicity, neutron energy spectrum, and fission product yields) are evaluated independently of one another, either with separate models or data-only/non-model evaluations (few – if any – shared model inputs) and inconsistencies can arise in evaluated data (e.g. NSE 190, 258 (2018)). They have found that consistent, model-based evaluations lead to more physical constraints on free parameters in the model, more robust predictions for other observables and other isotopes, that there are correlations between the uncertainties on different observables, and that uncertainties can be propagated to unmeasured quantities. They are working on developing models to the point where fission observables are calculated at the quality needed for an evaluation. They are striving to consistently calculate fission observables, such as $\bar{v}$, PFNS, and FPY, to a quality suitable for an evaluation. Current evaluations for fission observables use separate models for each observable (with minimal shared input) or are purely data driven. LANL models, such as the Monte Carlo CGMF and deterministic BeoH, have different strengths; the similar fission fragment initial conditions and decay models leads to consistency between the two codes (FPY can be connected to $\bar{v}$ and PFNS, even when calculated by different codes). Work is underway by Lovell to optimize the models and Neudecker is working to perform detailed uncertainty quantification. The PFNS, in particular, presents a significant challenge.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
NASA NTRS2021

Technical Note: AQMEII4 Activity 1: Evaluation of Wet and Dry Deposition Schemes as an Integral Part of Regional-scale Air Quality Models

We present in this technical note the research protocol for phase 4 of the Air Quality Model Evaluation International Initiative (AQMEII4). This research initiative is divided into two activities, collectively having three goals: (i) to define the current state of the science with respect to representations of wet and especially dry deposition in regional models, (ii) to quantify the extent to which different dry deposition parameterizations influence retrospective air pollutant concentration and flux predictions, and (iii) to identify, through the use of a common set of detailed diagnostics, sensitivity simulations, model evaluation, and reduction of input uncertainty, the specific causes for the current range of these predictions. Activity 1 is dedicated to the diagnostic evaluation of wet and dry deposition processes in regional air quality models (described in this paper), and Activity 2 to the evaluation of dry deposition point models against ozone flux measurements at multiple towers with multiyear observations (to be described in future submissions as part of the special issue on AQMEII4). The scope of this paper is to present the scientific protocols for Activity 1, as well as to summarize the technical information associated with the different dry deposition approaches used by the participating research groups of AQMEII4. In addition to describing all common aspects and data used for this multi-model evaluation activity, most importantly, we present the strategy devised to allow a common process-level comparison of dry deposition obtained from models using sometimes very different dry deposition schemes. The strategy is based on adding detailed diagnostics to the algorithms used in the dry deposition modules of existing regional air quality models, in particular archiving diagnostics specific to land use–land cover (LULC) and creating standardized LULC categories to facilitate cross-comparison of LULC-specific dry deposition parameters and processes, as well as archiving effective conductance and effective flux as means for comparing the relative influence of different pathways towards the net or total dry deposition. This new approach, along with an analysis of precipitation and wet deposition fields, will provide an unprecedented process-oriented comparison of deposition in regional air quality models. Examples of how specific dry deposition schemes used in participating models have been reduced to the common set of comparable diagnostics defined for AQMEII4 are also presented.

Air Quality Model Evaluation International Initiat↗
DOE OSTI2014

WE-E-18A-09: Application of a Channelized Hotelling Observer Model to Evaluate Angiographic Imaging Equipment

Purpose: To develop and apply an observer model to objectively evaluate and compare the performance of different angiographic imaging equipment and acquisition variables. Methods: Image Acquisition— Iodine-based phantoms were created with target diameters: 0.5, 1, 2, and 4 mm. The phantoms were imaged using both planes of a bi-plane angiography system with detector pixel dimensions 0.1542 mm{sup 2} and 0.1842 mm{sup 2}, respectively. All four phantoms were imaged with magnification factors 1.5, 1.25 and 1 and with the large and small focal spots. Phantom position and the dose per frame (0.12– 0.24 μ Gy/frame) were varied for a single phantom size, magnification and focal spot. Observer Model— For each experimental condition, 1200 signal-present and signal-absent images were acquired and a detectability index (d') was calculated with a Gabor-channelized Hotelling observer (CHO) model. Detectability indices were evaluated as a function of dose, phantom size, and magnification. The model was then applied to compare d' of the two imaging planes and focal spots. Uncertainty in d' was estimated by bootstrapping the data and by examining the shift-variance of systems. Results: Detectability indices varied linearly with magnification and the square root of dose. For the 2 and 4 mm phantoms, d' varied linearly with diameter. For the 0.5 and 1 mm phantoms, d' expectedly deviated from this linear relationship due to substantial detector and focal spot blurring of the phantoms. The small focal spot yielded up to 50% greater d' values than the large focal spot. For the two detectors, differences in d' did not exceed the estimated ∼7% error. Conclusions: The detectability indices predictably scaled with dose, diameter, magnification, and focal spot size and serve to validate the model. Results demonstrate statistically similar target detectability for both investigated detectors, despite differences in pixel dimensions. This CHO model provides a framework to evaluate angiographic system performance.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2016

Managing complexity in simulations of land surface and near-surface processes

Increasing computing power and the growing role of simulation in Earth systems science have led to an increase in the number and complexity of processes in modern simulators. We present a multiphysics framework that specifies interfaces for coupled processes and automates weak and strong coupling strategies to manage this complexity. Process management is enabled by viewing the system of equations as a tree, where individual equations are associated with leaf nodes and coupling strategies with internal nodes. A dynamically generated dependency graph connects a variable to its dependencies, streamlining and automating model evaluation, easing model development, and ensuring models are modular and flexible. Additionally, the dependency graph is used to ensure that data requirements are consistent between all processes in a given simulation. Here we discuss the design and implementation of these concepts within the Arcos framework, and demonstrate their use for verification testing and hypothesis evaluation in numerical experiments.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Technical note: AQMEII4 Activity 1: evaluation of wet and dry deposition schemes as an integral part of regional-scale air quality models

We present in this technical note the research protocol for phase 4 of the Air Quality Model Evaluation International Initiative (AQMEII4). This research initiative is divided into two activities, collectively having three goals: (i) to define the current state of the science with respect to representations of wet and especially dry deposition in regional models, (ii) to quantify the extent to which different dry deposition parameterizations influence retrospective air pollutant concentration and flux predictions, and (iii) to identify, through the use of a common set of detailed diagnostics, sensitivity simulations, model evaluation, and reduction of input uncertainty, the specific causes for the current range of these predictions. Activity 1 is dedicated to the diagnostic evaluation of wet and dry deposition processes in regional air quality models (described in this paper), and Activity 2 to the evaluation of dry deposition point models against ozone flux measurements at multiple towers with multiyear observations (to be described in future submissions as part of the special issue on AQMEII4). The scope of this paper is to present the scientific protocols for Activity 1, as well as to summarize the technical information associated with the different dry deposition approaches used by the participating research groups of AQMEII4. In addition to describing all common aspects and data used for this multi-model evaluation activity, most importantly, we present the strategy devised to allow a common process-level comparison of dry deposition obtained from models using sometimes very different dry deposition schemes. The strategy is based on adding detailed diagnostics to the algorithms used in the dry deposition modules of existing regional air quality models, in particular archiving diagnostics specific to land use–land cover (LULC) and creating standardized LULC categories to facilitate cross-comparison of LULC-specific dry deposition parameters and processes, as well as archiving effective conductance and effective flux as means for comparing the relative influence of different pathways towards the net or total dry deposition. This new approach, along with an analysis of precipitation and wet deposition fields, will provide an unprecedented process-oriented comparison of deposition in regional air quality models. Examples of how specific dry deposition schemes used in participating models have been reduced to the common set of comparable diagnostics defined for AQMEII4 are also presented.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS1982

Review of Hydrologic Models for Evaluating Use of Remote Sensing Capabilities

Hydrologic models most commonly used by federal agencies for hydrologic forecasting are reviewed. Six catchment models and one snow accumulation and ablation model are reviewed. Information on the structure, parameters, states, and required inputs is presented in schematic diagrams and in tables. The primary and secondary roles of parameters and state variables with respect to their function in the models are identified. The information will be used to evaluate the usefulness of remote sensing capabilities in the operational use of hydrologic models.

Peck, E. L.↗
DOE OSTI2015

Multi-scale predictions of massive conifer mortality due to chronic temperature rise

Global temperature rise and extremes accompanying drought threaten forests and their associated climatic feedbacks. Our ability to accurately simulate drought-induced forest impacts remains highly uncertain in part owing to our failure to integrate physiological measurements, regional-scale models, and dynamic global vegetation models (DGVMs). Here we show consistent predictions of widespread mortality of needleleaf evergreen trees (NET) within Southwest USA by 2100 using state-of-the-art models evaluated against empirical data sets. Experimentally, dominant Southwest USA NET species died when they fell below predawn water potential (Ψ pd ) thresholds (April–August mean) beyond which photosynthesis, hydraulic and stomatal conductance, and carbohydrate availability approached zero. The evaluated regional models accurately predicted NET Ψ pd , and 91% of predictions (10 out of 11) exceeded mortality thresholds within the twenty-first century due to temperature rise. The independent DGVMs predicted ≥50% loss of Northern Hemisphere NET by 2100, consistent with the NET findings for Southwest USA. Notably, the global models underestimated future mortality within Southwest USA, highlighting that predictions of future mortality within global models may be underestimates. Taken together, the validated regional predictions and the global simulations predict widespread conifer loss in coming decades under projected global warming.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

Accelerating Scientific Simulations with Bi-Fidelity Weighted Transfer Learning

High-fidelity modeling is an essential design tool for many engineering applications. However, for complex systems, computational cost can be a limiting factor. Analyzing parameter sensitivity, uncertainty quantification, and design optimization require many model evaluations. Surrogate models are often used to develop the relationship between model parameters and quantities of interest. However, in the case of complex systems, surrogate models require several degrees of freedom and, thus, a large number of data points to determine the correct dependencies. For many applications, this may be prohibitively expensive. The reduction of computational requirements can be achieved by leveraging low-fidelity models. Low-fidelity models represent the system at a coarser resolution with the advantage of computational efficiency. Therefore, a bi-fidelity modeling paradigm, which augments the accuracy of a low-fidelity model in a computationally efficient manner by invoking limited runs of a high-fidelity model, can be leveraged to sufficiently balance the accuracy and computational requirements. In this work, a bi-fidelity weighted transfer learning method using neural networks was applied to a computational fluid dynamics heat transfer modeling problem. The transfer learning advantage was investigated as a function of hyperparameters. Our main finding is that the use of a bi-fidelity modeling paradigm achieves accuracy close to that of a high-fidelity Gaussian process model while significantly reducing computational cost. The bi-fidelity model achieves comparable performance with 90 high-fidelity samples-that is, 60% less than the samples needed to achieve similar accuracy without the use of bi-fidelity modeling,

Borowiec, Katarzyna↗
DOE OSTI2025

Comprehensive Neural Posterior Estimation for Galaxy-Galaxy Strong Lensing

We present a deep learning model based on neural posterior estimation (NPE) for comprehensive extraction of astrophysical parameters from galaxy-scale strong gravitational lenses. The unprecedentedly large amount of galaxy-scale strong lenses expected in future cosmological surveys (${\cal O}(10^5)$) promises to enable valuable statistical constraints in various studies ranging from galaxy formation to the nature of dark matter, but it also poses a significant challenge for traditional modelling pipelines. To this end, our automated model includes several new, state-of-the-art features and approaches leveraging the framework of simulation-based inference (SBI). We infer a total of 20 parameters describing the mass and light profiles of both lens and source galaxies, using simulated raw multi-band data modelled under noise and observing conditions expected by the Legacy Survey of Space and Time (LSST), with its summary statistics generated by a residual network. We examine the efficacy of multi-band data in extracting nearly 20 model parameters simultaneous from strong lensing images including lens light. Finally, We perform a comprehensive set of diagnostics for SBI models, evaluating the model's prediction accuracy, stability, and uncertainty quantification.

Zhao, Roy J. [Chicago U., KICP]↗
DOE OSTI2023

Flash Diffusivity Measurement of Semi-Porous Insulation Material Intended for Radioisotope Thermoelectric Generators

This research involves the analysis of flash-heating data pursuant to find the thermal diffusivity of a semi-porous insulation material (Min-K) that is being considered for use in a radioisotope thermoelectric generator (RTG). An RTG uses radioactive nuclear fuel to produce electricity through a temperature difference imposed on a bimetallic thermocouple. Insulation is required to protect sensitive equipment from high temperatures and to conserve heat in the fuel. Using flash diffusivity temperature data, various simulations were fitted in order to find the most appropriate heat transfer model for the experiments. Four models were allowed to compete and the standard deviation of the residuals were compared for each model in evaluating model performance. The residuals are simply the difference in temperatures between the measurements and the mathematical models at each measurement point.

McMasters, Robert L.↗
DOE OSTI2023

Arctic Mixed-Phase Cloud Base Ice Precipitation Properties During the M-PACE Field Campaign

Cloud-climate feedbacks are still the greatest source of uncertainty in current climate projections. Arctic clouds, which are predominantly stratiform and supercooled, often long-lived, and nearly-continuously precipitate ice particles, contribute roughly 10% of the uncertainty attributed to the global cloud feedback. This Arctic cloud uncertainty is driven by incomplete observational and theoretical knowledge required to estimate and explain the state and active processes occurring in those clouds. A focus on ice precipitation properties at Arctic cloud base rather than the surface deconfounds the product of cloud condensate sink processes from the influence of the atmospheric thermodynamic state below cloud base, rendering cloud-base properties a more appealing target for inference and evaluation of model simulations. This dataset provides a set of 25 samples from the M-PACE field campaign, all of which were retrieved using the synthesis of ARM radar and lidar measurements. The retrieved ice precipitation variables in this dataset include, among others, the ice number concentration, water content, PSD parameters, precipitation rate, mass-weighted fall velocity, vertical air motion, and effective radius, all of which are highly valuable for model evaluation and a general understanding of polar cloud sink processes. Each variable sample includes its mean value and associated uncertainty. Additional variables based on ARM measurements (liquid layer statistics, etc.) are included in this dataset as well. The retrieval algorithm and analysis of this dataset are described in Silber (JGR, 2023, https://doi.org/10.1029/2022JD038202).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Arctic Mixed-Phase Cloud Base Ice Precipitation Properties Over the NSA Site

Cloud-climate feedbacks are still the greatest source of uncertainty in current climate projections. Arctic clouds, which are predominantly stratiform and supercooled, often long-lived, and nearly continuously precipitate ice particles, contribute roughly 10% of the uncertainty attributed to the global cloud feedback. This arctic cloud uncertainty is driven by incomplete observational and theoretical knowledge required to estimate and explain the state and active processes occurring in those clouds. A focus on ice precipitation properties at arctic cloud base rather than the surface deconfounds the product of cloud condensate sink processes from the influence of the atmospheric thermodynamic state below cloud base, rendering cloud-base properties a more appealing target for inference and evaluation of model simulations. This data set provides more than 1800 samples of cloud-base ice precipitation properties over Utqiagvik, North Slope of Alaska, all of which were retrieved using the synthesis of ARM radar and lidar measurements. The retrieved ice precipitation variables in this data set include, among others, the ice number concentration, water content, PSD parameters, precipitation rate, mass-weighted fall velocity, vertical air motion, and effective radius, all of which are highly valuable for model evaluation and a general understanding of polar cloud sink processes. Each variable sample includes its mean value and associated uncertainty. Additional variables based on ARM measurements (liquid layer statistics, etc.) are included in this data set. The retrieval algorithm and analysis of this data set are described in Silber (JGR, 2023, https://doi.org/10.1029/2022JD038202).

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2015

An Empirical Jet-Surface Interaction Noise Model with Temperature and Nozzle Aspect Ratio Effects

An empirical model for jet-surface interaction (JSI) noise produced by a round jet near a flat plate is described and the resulting model evaluated. The model covers unheated and hot jet conditions (1 less than or equal to jet total temperature ratio less than or equal to 2.7) in the subsonic range (0.5 less than or equal to M(sub a) less than or equal to 0.9), surface lengths 0.6 less than or equal to (axial distance from jet exit to surface trailing edge (inches)/nozzle exit diameter) less than or equal to 10, and surface standoff distances (0 less than or equal to (radial distance from jet lipline to surface (inches)/axial distance from jet exit to surface trailing edge (inches)) less than or equal to 1) using only second-order polynomials to provide predictable behavior. The JSI noise model is combined with an existing jet mixing noise model to produce exhaust noise predictions. Fit quality metrics and comparisons to between the predicted and experimental data indicate that the model is suitable for many system level studies. A first-order correction to the JSI source model that accounts for the effect of nozzle aspect ratio is also explored. This correction is based on changes to the potential core length and frequency scaling associated with rectangular nozzles up to 8:1 aspect ratio. However, more work is needed to refine these findings into a formal model.

Noise↗
NASA NTRS2022

Constraining Global Coronal Models with Multiple Independent Observables

Global coronal models seek to produce an accurate physical representation of the Sun's atmosphere that can be used, for example, to drive space-weather models. Assessing their accuracy is a complex task, and there are multiple observational pathways to provide constraints and tune model parameters. Here, we combine several such independent constraints, defining a model-agnostic framework for standardized comparison. We require models to predict the distribution of coronal holes at the photosphere, and neutral line topology at the model's outer boundary. We compare these predictions to extreme-ultraviolet (EUV) observations of coronal hole locations, white-light Carrington maps of the streamer belt, and the magnetic sector structure measured in situ by Parker Solar Probe and 1 au spacecraft. We study these metrics for potential field source surface (PFSS) models as a function of source surface height and magnetogram choice, as well as comparing to the more physical Wang–Sheeley–Arge (WSA) and the Magnetohydrodynamic Algorithm outside a Sphere (MAS) models. We find that simultaneous optimization of PFSS models to all three metrics is not currently possible, implying a trade-off between the quality of representation of coronal holes and streamer belt topology. WSA and MAS results show the additional physics that they include address this by flattening the streamer belt while maintaining coronal hole sizes, with MAS also improving coronal hole representation relative to WSA. We conclude that this framework is highly useful for inter- and intra-model comparisons. Integral to the framework is the standardization of observables required of each model, evaluating different model aspects.

S. T. Badman↗
DOE OSTI2017

Utilization of Model Predictive Control to Balance Power Absorption Against Load Accumulation: Preprint

Wave energy converter (WEC) control strategies have been primarily focused on maximizing power absorption. The use of model predictive control strategies allows for a finite-horizon, multiterm objective function to be solved. This work utilizes a multiterm objective function to maximize power absorption while minimizing the structural loads on the WEC system. Furthermore, a Kalman filter and autoregressive model were used to estimate and forecast the wave exciting force and predict the future dynamics of the WEC. The WEC's power-take-off time-averaged power and structural loads under a perfect forecast assumption in irregular waves were compared against results obtained from the Kalman filter and autoregressive model to evaluate model predictive control performance.

16 TIDAL AND WAVE POWER↗