Search NASA⌕ Search

SEARCH · Search NASA

Results for “model diagnostics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Jacobian-based model diagnostics and application to equation oriented modeling of a carbon capture system

It can be difficult to identify the specific variables or equations responsible for convergence issues in large mathematical programming models. The Institute for the Design of Advanced Energy Systems Integrated Platform (IDAES-IP) contains a tool to identify poorly scaled constraints and variables by searching for rows and columns of the Jacobian matrix with small L2-norms. A singular value decomposition is then performed to identify degenerate sets of equations and remaining scaling issues. Here, this work presents a flowsheet developed for post-combustion carbon capture using a monoethanolamine (MEA) solvent system as a case study. This work takes the reader through the entire process of model diagnostics and reformulation, from a basic introduction to the mathematics behind these model diagnostics to the reformulations necessary to make the model numerically robust, including a significantly modified enhancement factor model.

IDAES↗
DOE OSTI2021

Integrated assessment model diagnostics: key indicators and model evolution

Integrated assessment models (IAMs) form a prime tool in informing climate mitigation strategies. Diagnostic indicators that allow to compare these models can help to describe and explain differences in model projections. This also increases transparency and comparability. Earlier, the IAM community has developed an approach to diagnose models (Kriegler et al., 2015). Here we build on this, by proposing a selected set of well-defined indicators as a community standard, similar to metrics used for other modeling communities such as climate models. These indicators are the relative abatement index (RAI), emission reduction type index (ERT), inertia timescale (IT), fossil fuel reduction (FFR), transformation index (TI) and cost per abatement value (CAV). We apply the approach to 17 IAMs, including both older version as well as their latest versions, as applied in the IPCC 6th Assessment Report (AR6). The study shows that the approach can be easily applied and allows for comparison of model versions in time. The indicators and their trends can often be explained in terms of model characteristics and changes. We show that together, the set of six indicators can provide an useful indication of the main traits of the model and can roughly indicate the general model behavior. The results also show that there is often a considerable spread across the models. Interestingly, the diagnostic values often change for different model versions, but there does not seem to be a distinct trend across the different models.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Model Diagnostics for Equation Oriented Models: Roadblocks and the Path Forward

This poster was presented at the Foundations of Computer Aided Process Design (FOCAPD) 2024 conference with a paper describing efforts to develop a unified workflow and toolbox for diagnosing issues in equation-oriented models as part of the IDAES modeling framework. The poster presents the work currently underway within the IDAES project to develop and integrate cutting edge model analysis techniques into a common toolbox and workflow for model developers to use.

Lee, Andrew↗
DOE OSTI2024

Model Diagnostics for Equation Oriented Models: Roadblocks and the Path Forward

Conference paper for FOCAPD24 describing efforts to develop a unified workflow and toolbox for diagnosing issues in equation-oriented models as part of the IDAES modeling framework. The paper presents the work currently underway within the IDAES project to develop and integrate cutting edge model analysis techniques into a common toolbox and workflow for model developers to use.

Lee, Andrew↗
DOE OSTI2024

Jacobian-based Model Diagnostics and Application to Equation Oriented Modeling of a Carbon Capture System

Equation-oriented (EO) modeling has the potential to enable the effective design and optimization of the operation of advanced energy systems. However, advanced modeling of energy systems results in a large number of variables and non-linear equations, and it can be difficult to search through these to identify the culprit(s) responsible for convergence issues. The Institute for the Design of Advanced Energy Systems Integrated Platform (IDAES-IP) contains a tool to identify poorly scaled constraints and variables by searching for rows and columns of the Jacobian matrix with small L2-norms so they can be rescaled. A further singular value decomposition can be per-formed to identify degenerate sets of equations and remaining scaling issues. This work presents an EO model of a flowsheet developed for post-combustion carbon capture using a monoethanolamine (MEA) solvent system as a case study. The IDAES diagnostics tools were successfully applied to this flowsheet to identify problems to improve model robustness and enable the optimization of process design and operating conditions of a carbon capture system.

Allan, Douglas↗
DOE OSTI2021

A Bayesian Calibration Framework with Embedded Model Error for Model Diagnostics

We study the utility and performance of a Bayesian model error embedding construction in the context of molecular dynamics modeling of metallic alloys, where we embed model error terms in existing interatomic potential model parameters. To alleviate the computational burden of this approach, we propose a framework combining likelihood approximation and Gaussian process surrogates. Here we leverage sparse Gaussian process techniques to construct a hierarchy of increasingly accurate but more expensive surrogate models. This hierarchy is then exploited by multilevel Markov chain Monte Carlo methods to efficiently sample from the target posterior distribution. We illustrate the utility of this approach by calibrating an interatomic potential model for a family of gold-copper alloys. In particular, this case study highlights effective means for dealing with computational challenges with Bayesian model error embedding in large-scale physical models, and the utility of embedded model error for model diagnostics.

Bayesian inference↗
DOE OSTI2023

Rainbows and Climate Change: A tutorial on climate model diagnostics and parameterization

Earth System Models (ESMs) must represent processes below the grid scale of a model using representations (parameterizations) of physical and chemical processes. As a tutorial exercise to understand diagnostics and parameterization, this work presents a representation of rainbows for an ESM: the Community Earth System Model version 2 (CESM2). Using the 'state' of the model, basic physical laws, and some assumptions, we generate a representation of this unique optical phenomena as a diagnostic output. Rainbow occurrence and it's possible changes are related to cloud occurrence and rain formation which are critical uncertainties for climate change prediction. The work highlights issues which are typical of many diagnostics parameterizations such as assumptions, uncertain parameters and the difficulty of evaluation against uncertain observations. Results agree qualitatively with limited available global 'observations' of Rainbows. Rainbows are seen in expected locations in the sub-tropics over the ocean where broken clouds and frequent precipitation occurs. The diurnal peak is in the morning over ocean and in the evening over land. The representation of rainbows is found to be quantitatively sensitive to the assumed amount of cloudiness and the amount of stratiform rain. Rainbows are projected to have decreased, mostly in the Northern Hemisphere, due to aerosol pollution effects increasing cloud coverage since 1850. In the future, continued climate change is projected to decrease cloud cover, associated with a positive cloud feedback. As a result the rainbow diagnostic projects that rainbows will increase in the future, with the largest changes at mid-latitudes. The diagnostic may be useful for assessing cloud parameterizations, and is an exercise in how to build and test parameterizations of atmospheric phenomena.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Advancing Diagnostic Model Evaluation to Better Understand Water Shortage Mechanisms in Institutionally Complex River Basins

Abstract Water resources systems models enable valuable inferences on consequential system stressors by representing both the geophysical processes determining the movement of water and the human elements distributing it to its various competing uses. This study contributes a diagnostic evaluation framework that pairs exploratory modeling with global sensitivity analysis to enhance our ability to make inferences on water scarcity vulnerabilities in institutionally complex river basins. Diagnostic evaluation of models representing institutionally complex river basins with many stakeholders poses significant challenges. First, it needs to exploit a large and diverse suite of simulations to capture important human‐natural system interactions as well as institutionally aware behavioral mechanisms. Second, it needs to have performance metrics that are consequential and draw on decision‐relevant model outputs that adequately capture the multisector concerns that emerge from diverse basin stakeholders. We demonstrate the proposed model diagnostic framework by evaluating how potential interactions between changing hydrologic conditions and human demands influence the frequencies and durations of water shortages of varying magnitudes experienced by hundreds of users in a subbasin of the Colorado River. We show that the dominant factors shaping these effects vary both across users and, for an individual user, across percentiles of shortage magnitude. These differences hold even for users sharing diversion locations, demand levels or water right seniority. Our findings underline the importance of detailed institutional representation for such basins, as institutions strongly shape how dominant factors of stakeholder vulnerabilities propagate through the complex network of users.

Hadjimichael, Antonia↗
DOE OSTI2020

Convective Transition Statistics over Tropical Oceans for Climate Model Diagnostics: GCM Evaluation

To assess deep convective parameterizations in a variety of GCMs and examine the fast-time-scale convective transition, a set of statistics characterizing the pickup of precipitation as a function of column water vapor (CWV), PDFs and joint PDFs of CWV and precipitation, and the dependence of the moisture–precipitation relation on tropospheric temperature is evaluated using the hourly output of two versions of the GFDL Atmospheric Model, version 4 (AM4), NCAR CAM5 and superparameterized CAM (SPCAM). The 6-hourly output from the MJO Task Force (MJOTF)/GEWEX Atmospheric System Study (GASS) project is also analyzed. Contrasting statistics produced from individual models that primarily differ in representations of moist convection suggest that convective transition statistics can substantially distinguish differences in convective representation and its interaction with the large-scale flow, while models that differ only in spatial–temporal resolution, microphysics, or ocean–atmosphere coupling result in similar statistics. Most of the models simulate some version of the observed sharp increase in precipitation as CWV exceeds a critical value, as well as that convective onset occurs at higher CWV but at lower column RH as temperature increases. While some models quantitatively capture these observed features and associated probability distributions, considerable intermodel spread and departures from observations in various aspects of the precipitation–CWV relationship are noted. For instance, in many of the models, the transition from the low-CWV, nonprecipitating regime to the moist regime for CWV around and above critical is less abrupt than in observations. Additionally, some models overproduce drizzle at low CWV, and some require CWV higher than observed for strong precipitation. For many of the models, it is particularly challenging to simulate the probability distributions of CWV at high temperature.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Combining diagnostics, modeling, and control systems for automated alignment of the TES beamline

X-ray beamlines are essential components of all synchrotron light sources. Practical operations involve frequent variation in beamline component positions and orientation, particularly when photon beam parameters shift due to experimental needs, or due to variations in the incoming photon beam. The alignment process can be time consuming and takes away from valuable beam time for experimental data collection. We describe progress in the automation of certain alignment tasks on the tender-energy X-ray spectroscopy (TES) beamline at the National Synchrotron Light Source II (NSLS-II). The beamline is controlled using the BlueSky software in which high level experimental plans guide the beamline components during an experiment. Numerous software packages exist for beamline modeling, and they may be tied to the beamline control system using a package we are continuing to develop called Sirepo-Bluesky. The photon beam distribution may be measured with fluorescent screens, and a relation between beam and machine state can be found by varying the mirror and aperture settings over a multi-dimensional range. We describe the results of such parameter varying measurements and how we are combining Sirepo-Bluesky with machine learning methods and reduced models to automate mirror alignment on the TES beamline.

36 MATERIALS SCIENCE↗
DOE OSTI2020

A Probabilistic Model-Based Diagnostic Framework for Nuclear Engineering Systems

A fault diagnostic framework was investigated in this study for applications in thermal–hydraulic systems of nuclear power plants. The proposed framework consists of quantitative model-based diagnosis, statistical change detection and probabilistic reasoning. The use of physics-based diagnostic models provides high detection sensitivity and allows noise and measurement uncertainty to be incorporated robustly. Performance-related parametric models for each component are constructed based on first principles. Numerical model residuals are generated using the concept of analytical redundancy. Statistical change detection methods are employed to detect non-zero residuals in the presence of uncertainty. The diagnosis task is performed using Bayesian inference to detect and localize possible faults. Application to a single-phase heat exchanger for demonstration showed that the proposed probabilistic framework can provide improved results in comparison with traditional approaches while remaining less sensitive to false alarms in the presence of measurement and modeling uncertainty.

Bayesian network↗
DOE OSTI2024

From minimum-viable-products to full models: a step-wise development of diagnostic forward models in support of design, analysis and modelling on the ST40 tokamak

Like most magnetic confined fusion experiments, the ST40 tokamak started off with a small subset of diagnostics and gradually increased the diagnostic set to include more complex and comprehensive systems. To make the most of each operational phase, forward models of various diagnostics are used and developed to aid design, provide consistency-checks during commissioning, test analysis methods, and build workflows to constrain high-level parameters to inform interpretation, theory and modelling. For new models and new analysis workflows, minimum-viable-products are released early, and their complexity is increased in a step-wise manner, facilitating the support of all programme phases on multiple parallel applications, while enabling learning opportunities and feedback loops. In this contribution we review the philosophy, scope and architecture of the framework under development. We discuss the details of some forward models, with examples on how they are used to aid diagnostic design, to investigate analysis methodologies through synthetic data, and how they are embedded in experimental analysis workflows. We compare previously published experimental results with new, more advanced analysis workflows employing more recent, detailed models and new diagnostic data, providing confirmation of the published material from the 2021–22 experimental campaign.

integrated data analysis↗