Search NASA⌕ Search

SEARCH · Search NASA

Results for “model credibility”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Physical Review Journal Club: Structural uncertainty assessment for fire-engulfed objects in crosswind: Establishing credibility for a multiphysics wall-modeled large-eddy simulation paradigm

This is an invited presentation for APS's monthly Journal Club where important papers that have recently been accepted for publication are highlighted in a seminar-like setting. The presentation will be "live" (May 1, 2025) and streamed across the world (including an archived recording).

Domino, Stefan Paul [Sandia National Laboratories ↗
DOE OSTI2025

Adopting Code Verification Methodology Based on Model Form

Code verification is an essential part of credibility analysis for computational models. It assesses whether the mathematical model is implemented correctly into the code and whether the numerical methods behave consistently, and is done before solution verification and validation. Robust guidance for code verification exists in the literature. However, there is no known, concise guide for selecting the approach based on the model form that also presents an overview of the common elements. This document was written to address this gap as an accessible reference for beginning a code-verification effort.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Adapting Code Verification Methodology to Model Form

Code verification is an essential part of credibility analysis for computational models. It assesses whether the mathematical model is implemented correctly into the code and whether the numerical methods behave consistently, and is done before solution verification and validation. Robust guidance for code verification exists in the literature. However, there is no known, concise guide for selecting the approach based on the model form that also presents an overview of the common elements. This document was written to address this gap as an accessible reference for beginning a code-verification effort.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

White paper on Verification and Validation for Cyber Emulation Models

All disciplines that use models to predict the behavior of real-world systems need to determine the accuracy of the models’ results. Techniques for verification, validation, and uncertainty quantification (VVUQ) focus on improving the credibility of computational models and assessing their predictive capability. VVUQ emphasizes rigorous evaluation of models and how they are applied to improve understanding of model limitations and quantify the accuracy of model predictions.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Photometric redshift uncertainties in weak gravitational lensing shear analysis: models and marginalization

ABSTRACT Recovering credible cosmological parameter constraints in a weak lensing shear analysis requires an accurate model that can be used to marginalize over nuisance parameters describing potential sources of systematic uncertainty, such as the uncertainties on the sample redshift distribution n(z). Due to the challenge of running Markov chain Monte Carlo (MCMC) in the high-dimensional parameter spaces in which the n(z) uncertainties may be parametrized, it is common practice to simplify the n(z) parametrization or combine MCMC chains that each have a fixed n(z) resampled from the n(z) uncertainties. In this work, we propose a statistically principled Bayesian resampling approach for marginalizing over the n(z) uncertainty using multiple MCMC chains. We self-consistently compare the new method to existing ones from the literature in the context of a forecasted cosmic shear analysis for the HSC three-year shape catalogue, and find that these methods recover statistically consistent error bars for the cosmological parameter constraints for predicted HSC three-year analysis, implying that using the most computationally efficient of the approaches is appropriate. However, we find that for data sets with the constraining power of the full HSC survey data set (and, by implication, those upcoming surveys with even tighter constraints), the choice of method for marginalizing over n(z) uncertainty among the several methods from the literature may modify the 1σ uncertainties on Ωm–S8 constraints by ∼4 per cent, and a careful model selection is needed to ensure credible parameter intervals.

Zhang, Tianqing (ORCID:000000025596198X)↗
DOE OSTI2021

Multi-fidelity thermal modeling of laser powder bed additive manufacturing

Laser powder bed fusion (LPBF) Additive manufacturing (AM) has attracted interest as an agile method of building production metal parts to reduce design-build-test cycle times for systems. However, predicting part performance is difficult due to inherent process variabilities. This makes qualification challenging. Computational process models have attempted to address some of these challenges, including mesoscale, full physics models and reduced fidelity conduction models. The goal of this work is credible multi-fidelity modeling of the LPBF process by investigating methods for estimating the error between models of two different fidelities. Two methods of error estimation are investigated, adjoint-based error estimation and Bayesian calibration. Adjoint-based error estimation is found to effectively bounding the error between the two models, but with very conservative bounds, making predictions highly uncertain. Bayesian parameter calibration applied to conduction model heat source parameters is found to effectively bound the observed error between the models for melt pool morphology quantities of interest. However, the calibrations do not effectively bound the error in heat distribution.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Manufactured solutions for an electromagnetic slot model

The accurate modeling of electromagnetic penetration is an important topic in computational electromagnetics. Electromagnetic penetration occurs through intentional or inadvertent openings in an otherwise closed electromagnetic scatterer, which prevent the contents from being fully shielded from external fields. To efficiently model electromagnetic penetration, aperture or slot models can be used with surface integral equations to solve Maxwell's equations. A necessary step towards establishing the credibility of these models is to assess the correctness of the implementation of the underlying numerical methods through code verification. Surface integral equations and slot models yield multiple interacting sources of numerical error and other challenges, which render traditional code-verification approaches ineffective. In this paper, we provide approaches to separately measure the numerical errors arising from these different error sources for the method-of-moments implementation of the electric-field integral equation with a slot model. Finally, we demonstrate the effectiveness of these approaches for a variety of cases.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2020

Differential credibility assessment for statistical downscaling

Climate science is increasingly using (i) ensembles of climate projections from multiple models derived using different assumptions and/or scenarios and (ii) process-oriented diagnostics of model fidelity. Efforts to assign differential credibility to projections and/or models are also rapidly advancing. A framework to quantify and depict the credibility of statistically downscaled model output is presented and demonstrated. Here, the approach employs transfer functions in the form of robust and resilient generalized linear models applied to downscale daily minimum and maximum temperature anomalies at 10 locations using predictors drawn from ERA-Interim reanalysis and two global climate models (GCM; GFDL-ESM2M and MPI-ESM-LR). The downscaled time series are used to derive several impact relevant CLIMDEX temperature indices that are assigned credibility based on (1) the reproduction of relevant large-scale predictors by the GCMs (i.e. fraction of regression beta-weights derived from predictors that are well-reproduced) and (2) the degree of variance in the observations reproduced in the downscaled series following application of a new variance inflation technique. Credibility of the downscaled predictands varies across locations, between the two GCM and is generally higher for minimum temperature than maximum temperature. The differential credibility assessment framework demonstrated here is easy to use and flexible. It can be applied as is to inform decision makers regarding projection confidence, and/or extended to include other components of the transfer functions, and/or used to weight members of a statistically downscaled ensemble.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Understanding and Explicitly Measuring Linguistic and Stylistic Properties of Deception via Generation and Translation

Massive digital disinformation is one of the main risks of modern society. Hundreds of models and linguistic analyses have been done to compare and contrast misleading and credible content online. However, most models do not remove the confounding factor of a topic or narrative when training, so the resulting models learn a clear topical separation for misleading versus credible content. We study the feasibility of using two strategies to disentangle the topic bias from the models to understand and explicitly measure linguistic and stylistic properties of content from misleading versus credible content. First, we develop conditional generative models to create news content that is characteristic of different credibility levels. We perform multi-dimensional evaluation of model performance on mimicking both the style and linguistic differences that distinguish news of different credibility using machine translation metrics and classification models. We show that even though generative models are able to imitate both the style and language of the original content, additional conditioning on both the news category and the topic leads to reduced performance. In a second approach, we perform deception style ``transfer" by translating deceptive content into the style of credible content and vice versa. Extending earlier studies, we demonstrate that, when conditioned on a topic, deceptive content is shorter, less readable, more biased, and more subjective than credible content, and transferring the style from deceptive to credible content is more challenging than the opposite direction.

Saldanha, Emily G.↗
DOE OSTI2022

Bay Area Regional Energy: Network Integrated Commercial Retrofits (BRICR) Project. Final Report

The BRICR project applied large-scale building energy modeling concepts with the aim of reducing the cost of energy efficiency targeting, design, and project development, and measurement of energy savings for energy efficiency programs implemented by local governments that serve small and medium commercial buildings (SMB). The project leveraged the services and resources of existing local government energy programs serving disadvantaged and hard-to-reach SMB customers. In contrast to programs run by utilities, local government programs generally do not have direct access to energy billing records for an entire class of customers in a geographic area, which prior research demonstrated useful for large-scale building energy model baseline development and calibration. , However, local governments are rich in public records that offer important clues about physical attributes and uses that, along with behavior, determine energy use. Relying only on public records, BRICR demonstrated development of credible baseline energy models for 3,792 office, retail, and hotel buildings. Publicly disclosed annual energy use data from a local energy benchmarking program and anonymized data from the Building Performance Database, the nation’s largest dataset about energy-related characteristics of buildings, were utilized to validate and calibrate energy models via an innovative method comparing distributions of energy intensity by fuel type for portfolios of buildings of similar size, vintage, and use. Portfolio calibration does not provide certainty that an energy model fits an individual building; the method is useful when billing data is not accessible – a common situation for researchers, energy service providers and ESCOs, local governments, and any party other than a utility. A software component was developed, the BRICR gem, which automates simulation when relevant data is added or edited by the user to a file saved in the standardized BuildingSync XML schema for energy audit data. The component was demonstrated as a simplified means to generate a mass of energy models corresponding to public records containing basic attributes such as building scale, location, use, year built, and aspect ratio in combination with building energy code prototype data corresponding to use and vintage. The component was also demonstrated as a simplified means to automate energy simulation when attributes are revised; the intention was to enable iterative improvement of the baseline model and energy savings estimates for common energy conservation measures as users revise relevant attributes based on their observations. In the context of institutional change and uncertainty for the participating local government energy programs, 13 whole building retrofits were completed. Impacts were measured by applying the CalTRACK2.0 methods to standardize measurement of normalized metered energy consumption. The GRIDMeter methods of stratified sampling and individual load shape analysis were applied to adjust for impacts of the effect of COVID-19 on retrofitted buildings in the context of all local buildings of similar size and use. Excluding impacts of the pandemic, retrofitted buildings demonstrated between 1.6% and 25.1% reduction in energy use. The project contributed use cases and feedback that helped inform evolution of the software tools and data formats that were combined for the first time in the BRICR project.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2020

Characteristics of Future Warmer Base States in CESM2

Abstract Simulations of 21st century climate with Community Earth System Model version 2 (CESM2) using the standard atmosphere (CAM6), denoted CESM2(CAM6), and the latest generation of the Whole Atmosphere Community Climate Model (WACCM6), denoted CESM2(WACCM6), are presented, and a survey of general results is described. The equilibrium climate sensitivity (ECS) of CESM2(CAM6) is 5.3°C, and CESM2(WACCM6) is 4.8°C, while the transient climate response (TCR) is 2.1°C in CESM2(CAM6) and 2.0°C in CESM2(WACCM6). Thus, these two CESM2 model versions have higher values of ECS than the previous generation of model, the CESM (CAM5) (hereafter CESM1), that had an ECS of 4.1°C, though the CESM2 versions have lower values of TCR compared to the CESM1 with a somewhat higher value of 2.3°C. All model versions produce credible simulations of the time evolution of historical global surface temperature. The higher ECS values for the CESM2 versions are reflected in higher values of global surface temperature increase by 2,100 in CESM2(CAM6) and CESM2(WACCM6) compared to CESM1 between comparable emission scenarios for the high forcing scenario. Future warming among CESM2 model versions and scenarios diverges around 2050. The larger values of TCR and ECS in CESM2(CAM6) compared to CESM1 are manifested by greater warming in the tropics. Associated with a higher climate sensitivity, for CESM2(CAM6) the first instance of an ice‐free Arctic in September occurs for all scenarios and ensemble members in the 2030–2050 time frame, but about a decade later in CESM2(WACCM6), occurring around 2040–2060.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Explainable Artificial Intelligence Recommendation System by Leveraging the Semantics of Adverse Childhood Experiences: Proof-of-Concept Prototype Development

The study of adverse childhood experiences and their consequences has emerged over the past 20 years. Although the conclusions from these studies are available, the same is not true of the data. Accordingly, it is a complex problem to build a training set and develop machine-learning models from these studies. Classic machine learning and artificial intelligence techniques cannot provide a full scientific understanding of the inner workings of the underlying models. This raises credibility issues due to the lack of transparency and generalizability. Explainable artificial intelligence is an emerging approach for promoting credibility, accountability, and trust in mission-critical areas such as medicine by combining machine-learning approaches with explanatory techniques that explicitly show what the decision criteria are and why (or how) they have been established. Hence, thinking about how machine learning could benefit from knowledge graphs that combine “common sense” knowledge as well as semantic reasoning and causality models is a potential solution to this problem. In this study, we aimed to leverage explainable artificial intelligence, and propose a proof-of-concept prototype for a knowledge-driven evidence-based recommendation system to improve mental health surveillance. We used concepts from an ontology that we have developed to build and train a question-answering agent using the Google DialogFlow engine. In addition to the question-answering agent, the initial prototype includes knowledge graph generation and recommendation components that leverage third-party graph technology. To showcase the framework functionalities, we here present a prototype design and demonstrate the main features through four use case scenarios motivated by an initiative currently implemented at a children’s hospital in Memphis, Tennessee. Ongoing development of the prototype requires implementing an optimization algorithm of the recommendations, incorporating a privacy layer through a personal health library, and conducting a clinical trial to assess both usability and usefulness of the implementation. This semantic-driven explainable artificial intelligence prototype can enhance health care practitioners’ ability to provide explanations for the decisions they make.

adverse childhood experiences↗
DOE OSTI2020

A ``Tail" of Two Cities: Fatality-based Modeling of COVID-19 Evolution in New York City and Cook County, IL

I describe $SIR$ modeling of the COVID-19 pandemic in two U.S. urban environments, New York City (NYC) and Cook County, IL, from onset through the month of June, 2020. Since testing was not widespread early in the pandemic in the U.S., I do not use data on confirmed cases and rely solely on public fatality data to estimate model parameters. Fits to the first 20 days of data determine a degenerate combination of the basic reproduction number, $R_0$, and the mean time to removal from the infectious population, $\gamma^{-1}$, with $\gamma (R_0-1) = 0.25 (0.21)$ inverse days for NYC (Cook County). Equivalently, the initial doubling time was $t_d = 2.8 (3.4)$ days for NYC (Cook). The early fatality data suggest that both locations had infections in early February. I model the mitigation measures implemented in mid-March in both locations (distancing, quarantine, isolation, etc) via a time-dependent reproduction number $R_t$ that declines monotonically from $R_0$ to a sma ller asymptotic value, with a parameterized functional form. The timing (mid-March) and duration (several days) of the transitions in $R_t$ appear well determined by the data. However, the fatality data determine only a degenerate combination of the parameters $R_0$, the percentage reduction in social contact due to mitigation measures, $X$, and the infection fatality rate (IFR), $f$. With flat priors, based on simulations the NYC model parameters have 95.45\% credible intervals of $R_0 = 3.0-5.4$, $X= 80-99.9\%$ and $f=2-6\%$, with $5-13\%$ of the population asymptotically infected. A strong external prior indicating a lower value of $f$ or of $\gamma^{-1}$ would imply lower values of $R_0$ and $X$ and higher percentage infection of the population. For Cook County, the evolution was qualitatively different: after mitigation measures were implemented, the daily fatality counts reached a plateau for about a month before tailing off. This is consistent with an $SIR$ model that exhibits ``critical slowing-down", in which $R_t$ plateaus ! at a value just above unity. For Cook County, the 95.45\% credible intervals for the model parameters are much broader and shifted downward, $R_0=1.4-4.7$, $X=26-54\%$, and $f=0.1-0.6\%$ with $15-88\%$ of the population asymptotically infected. Despite the apparently lower efficacy of its social contact reduction measures, Cook County has had significantly fewer fatalities per population than NYC, $D_\infty/N=100$ vs. 270 per 100,000. In the model, this is attributed to the lower inferred IFR for Cook; an external prior pointing to similar values of the IFR for the two locations would instead chalk up the difference in $D/N$ to differences in the relative growth rate of the disease. I derive a model-dependent threshold, $X^{crit}_e$, for `safe' re-opening, that is, for easing of contact reduction that would not trigger a second wave; for NYC, the models predict that increasing social contact by more than 20\% from post-mitigation levels will lead to renewed spread, while for Cook County the threshold value is very uncertain, given the parameter degeneracies. The timing of 2nd-wave growth will depend on the amplitude of contact increase relative to $X^{crit}_e$ and on the asymptotic growth rate, and the impact in terms of fatalities will depend on the parameter $f$.

Frieman, Joshua↗
DOE OSTI2026

Polywell Revisited

The Polywell fusion concept, originally proposed by Robert W. Bussard in 1985, has been investigated for over four decades as a potential solution for achieving net fusion energy in a compact and economically viable reactor. It combines two distinct approaches: high-beta magnetic cusp confinement of electrons using polyhedral coil configurations and electrostatic ion confinement via a potential well formed by injected electron beams. While the hybrid nature of the Polywell system offers advantages in plasma stability and engineering simplicity, previous efforts have been limited by persistent challenges in achieving sufficient plasma confinement required to generate a net energy gain. Here, in this study, we examine previous works and identify limitations of several Polywell embodiments that have historically impeded progress. We present an updated Polywell physics model incorporating experimental findings and recent first-principles particle-in-cell simulations. This updated model outlines a credible path toward overcoming confinement losses and achieving net energy gain using deuterium-tritium (D-T) fuels. Our findings provide a renewed scientific basis for the continued development of the Polywell fusion concept as a practical and scalable approach to fusion energy.

Electrostatic potential well↗
DOE OSTI2025

Robust Solution Verification Experiments on Nonuniform Meshes

The activities of verification, validation, and uncertainty quantification (VVUQ) provide a comprehensive means to assess the credibility of computational models. Within VVUQ, solution verification assesses numerical errors and evaluates whether the simulation is sufficiently accurate for its intended applications. As computational modeling gains traction in the development of complex, high-consequence systems, the need for robust solution verification intensifies, particularly because experimental data for these systems are often limited. This work examines improvements in the robustness of Richardson extrapolation (RE), a method commonly used in solution verification to study the discretization error of computational models using a power law. Nonuniform mesh refinement is discussed alongside other pollutants that affect the robustness of the power law model. Maximum likelihood estimation (MLE) is proposed as a robust strategy to address the uncertainty generated by nonuniform mesh refinement. An exploratory computational fluid dynamics (CFD) study of a 2D planar Poiseuille flow is conducted to determine if nonuniform mesh noise can be modeled with this MLE approach for more robust RE.

Weinmeister, Justin [ORNL] (ORCID:0000000160090237↗
DOE OSTI2023

Test and Evaluation of Systems with Embedded Machine Learning Components

As Machine Learning (ML) continues to advance, it is being integrated into more systems. Often, the ML component represents a significant portion of the system that reduces the burden on the end user or significantly improves task performance. However, the ML component represents an unknown complex phenomenon that is learned from collected data without the need to be explicitly programmed. Despite the improvement in task performance, the models are often black boxes. Evaluating the credibility and the vulnerabilities of ML models poses a gap in current test and evaluation practice. For high consequence applications, the lack of testing and evaluation procedures represents a significant source of uncertainty and risk. To help reduce that risk, here we present considerations to evaluate systems embedded with an ML component within a red-teaming inspired methodology. We focus on (1) cyber vulnerabilities to an ML model, (2) evaluating performance gaps, and (3) adversarial ML vulnerabilities.

97 MATHEMATICS AND COMPUTING↗