SEARCH · Search NASA

Results for “model credibility”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13

All sources NASA NTRS DOE OSTI

DOE OSTI2022

Operation-adversarial scenario generation

This paper proposes a modified conditional generative adversarial network (cGAN) model to generate net load scenarios for power systems that are statistically credible, conditioned by given labels (e.g., seasons), and, at the same time, “stressful” to the system operations and dispatch decisions. The measure of stress used in this paper is based on the operating cost increases due to net load changes. The proposed operation-adversarial cGAN (OA-cGAN) internalizes a DC optimal power flow model and seeks to maximize the operating cost and achieve a worst-case data generation. The training and testing stages employed in the proposed OA-cGAN use historical day-ahead net load forecast errors and has been implemented for the realistic NYISO 11-zone system. In conclusion, our numerical experiments demonstrate that the generated operation-adversarial forecast errors lead to more cost-effective and reliable dispatch decisions.

42 ENGINEERING↗

NASA NTRS2019

Taking Climate Model Evaluation to the Next Level

Earth system models are complex and represent a large number of processes, resulting in a persistent spread across climate projections for a given future scenario. Owing to different model performances against observations and the lack of independence among models, there is now evidence that giving equal weight to each available model projection is suboptimal. This Perspective discusses newly developed tools that facilitate a more rapid and comprehensive evaluation of model simulations with observations, process-based emergent constraints that are a promising way to focus evaluation on the observations most relevant to climate projections, and advanced methods for model weighting. These approaches are needed to distil the most credible information on regional climate changes, impacts, and risks for stakeholders and policy-makers.

Eyring, Veronika↗

DOE OSTI2023

The suitability of differentiable, physics-informed machine learning hydrologic models for ungauged regions and climate change impact assessment

As a genre of physics-informed machine learning, differentiable process-based hydrologic models (abbreviated as δ or delta models) with regionalized deep-network-based parameterization pipelines were recently shown to provide daily streamflow prediction performance closely approaching that of state-of-the-art long short-term memory (LSTM) deep networks. Meanwhile, δ models provide a full suite of diagnostic physical variables and guaranteed mass conservation. Here, we ran experiments to test (1) their ability to extrapolate to regions far from streamflow gauges and (2) their ability to make credible predictions of long-term (decadal-scale) change trends. We evaluated the models based on daily hydrograph metrics (Nash–Sutcliffe model efficiency coefficient, etc.) and predicted decadal streamflow trends. For prediction in ungauged basins (PUB; randomly sampled ungauged basins representing spatial interpolation), δ models either approached or surpassed the performance of LSTM in daily hydrograph metrics, depending on the meteorological forcing data used. They presented a comparable trend performance to LSTM for annual mean flow and high flow but worse trends for low flow. For prediction in ungauged regions (PUR; regional holdout test representing spatial extrapolation in a highly data-sparse scenario), δ models surpassed LSTM in daily hydrograph metrics, and their advantages in mean and high flow trends became prominent. In addition, an untrained variable, evapotranspiration, retained good seasonality even for extrapolated cases. The δ models' deep-network-based parameterization pipeline produced parameter fields that maintain remarkably stable spatial patterns even in highly data-scarce scenarios, which explains their robustness. Combined with their interpretability and ability to assimilate multi-source observations, the δ models are strong candidates for regional and global-scale hydrologic simulations and climate change impact assessment.

54 ENVIRONMENTAL SCIENCES↗

NASA NTRS1998

Credible Software and Simulation Uncertainty

The utility of software primarily depends on its reliability and performance; whereas, its significance depends solely on its credibility for intended use. The credibility of simulations confirms the credibility of software. The level of veracity and the level of validity of simulations determine the degree of credibility of simulations. The process of assessing this credibility in fields such as computational mechanics (CM) differs from that followed by the Defense Modeling and Simulation Office in operations research. Verification and validation (V&V) of CM simulations is not the same as V&V of CM software. Uncertainty is the measure of simulation credibility. Designers who use software are concerned with management of simulation uncertainty. Terminology and concepts are presented with a few examples from computational fluid dynamics.

Mehta, Unmeel B.↗

DOE OSTI2021

Task 12 Sustainability - Methodological Guidelines on Net Energy Analysis of Photovoltaic Electricity (2nd Edition)

Net Energy Analysis (NEA) is a structured, comprehensive method of quantifying the extent to which a given energy source is able to provide a net energy gain (i.e., an energy surplus) to the end user, after accounting for all the energy losses occurring along the chain of processes that are required to exploit it (i.e., for its extraction, processing and transformation into a usable energy carrier, and delivery to the end user), as well as for all the additional energy 'investments' that are required in order to carry out the same chain of processes. However, this general framework leaves the individual practitioner with a range of choices that can affect the results and thus, the conclusions of a NEA study. The current IEA PVPS guidelines were developed to provide guidance on assuring consistency, balance, and quality to enhance the credibility and reliability of the results from photovoltaic (PV) NEAs. The guidelines represent a consensus among the authors - PV NEA experts in North America and Europe - for assumptions made on PV performance, process inputs and outputs, methods of analysis, and reporting of the results. Guidance is given on photovoltaic-specific parameters used as inputs in NEA and on choices and assumptions in inventory data analysis and on implementation of modelling approaches. A consistent approach towards system modelling, the functional unit, the system boundaries and allocation aspects enhance the credibility of PV electricity NEA studies and enables balanced NEA-based comparisons. Specifically, "apples-to-oranges" comparisons of different energy carriers (e.g., fuels vs. electricity) are not methodologically sound and are to be avoided in all cases; also, any comparison across renewable and non-renewable electricity generation technologies must clearly point out the intrinsically short-term nature of the NEA viewpoint, which does not capture the long-term sustainability implications of renewable vs. non-renewable primary energy harvesting and use: non-renewable primary energy resources are depleted and finally exhausted (irrespective of the size of the EROI), while renewable primary energy resources are not. This document provides an in-depth discussion of a common metric of NEA, namely the energy return on investment (EROI), and how this is to be interpreted vis-a-vis the deceptively similar-sounding metrics in the field of Life Cycle Assessment (LCA): cumulative energy demand (CED) and non-renewable cumulative energy demand (nr-CED) per unit output. Specifically, a number of key differences are highlighted between these metrics as applied to electricity production systems, which are listed in Table S-1.

14 SOLAR ENERGY↗

NASA NTRS1978

A method motion simulator design based on modeling characteristics of the human operator

A design criteria is obtained to compare two simulators and evaluate their equivalence or credibility. In the subsequent analysis the comparison of two simulators can be considered as the same problem as the comparison of a real world situation and a simulation's representation of this real world situation. The design criteria developed involves modeling of the human operator and defining simple parameters to describe his behavior in the simulator and in the real world situation. In the process of obtaining human operator parameters to define characteristics to evaluate simulators, measures are also obtained on these human operator characteristics which can be used to describe the human as an information processor and controller. First, a study is conducted on the simulator design problem in such a manner that this modeling approach can be used to develop a criteria for the comparison of two simulators.

Repperger, D. W.↗

NASA NTRS2011

Sleep Disruption Medical Intervention Forecasting (SDMIF) Module for the Integrated Medical Model

The NASA Integrated Medical Model (IMM) assesses the risk, including likelihood and impact of occurrence, of all credible in-flight medical conditions. Fatigue due to sleep disruption is a condition that could lead to operational errors, potentially resulting in loss of mission or crew. Pharmacological consumables are mitigation strategies used to manage the risks associated with sleep deficits. The likelihood of medical intervention due to sleep disruption was estimated with a well validated sleep model and a Monte Carlo computer simulation in an effort to optimize the quantity of consumables. METHODS: The key components of the model are the mission parameter program, the calculation of sleep intensity and the diagnosis and decision module. The mission parameter program was used to create simulated daily sleep/wake schedules for an ISS increment. The hypothetical schedules included critical events such as dockings and extravehicular activities and included actual sleep time and sleep quality. The schedules were used as inputs to the Sleep, Activity, Fatigue and Task Effectiveness (SAFTE) Model (IBR Inc., Baltimore MD), which calculated sleep intensity. Sleep data from an ISS study was used to relate calculated sleep intensity to the probability of sleep medication use, using a generalized linear model for binomial regression. A human yes/no decision process using a binomial random number was also factored into sleep medication use probability. RESULTS: These probability calculations were repeated 5000 times resulting in an estimate of the most likely amount of sleep aids used during an ISS mission and a 95% confidence interval. CONCLUSIONS: These results were transferred to the parent IMM for further weighting and integration with other medical conditions, to help inform operational decisions. This model is a potential planning tool for ensuring adequate sleep during sleep disrupted periods of a mission.

Lewandowski, Beth↗

DOE OSTI2023

Comparing Automated Posterior Estimation Techniques for Modeling Strong Lenses In Ground-based Survey Data

Current and future ground-based cosmological surveys, such as the Dark Energy Survey (DES), and the Vera Rubin Observatory Legacy Survey of Space and Time (LSST), are predicted to discover thousands to tens of thousands of strong gravitational lenses. The large number of strong lenses discoverable in future surveys will make strong lensing a highly competitive and complementary cosmic probe. However, conventional lens modeling techniques are unable to scale up to the sheer number of lenses that will be discovered through upcoming surveys. Therefore, the use of automated lens analysis techniques is necessary. We demonstrate that machine learning methods can be used to automate the inference of informative model posteriors of strong lensing systems in ground-based surveys with credible uncertainty estimation. We present two Simulation-Based Inference (SBI) approaches for lens parameter estimation of galaxy-galaxy lenses. We demonstrate applications of Neural Posteriors Estima tors (NPEs) and Bayesian Neural Network (BNNs) to automate the inference of a 12-parameter lensing system for DES-like ground-based imaging data. We apply a suite of diagnostics (e.g., posterior coverage and SBC) to validate the performance of our methods. We find that NPEs outperform the BNN, producing posterior distributions that are for the most part both more accurate and more precise; in particular, several source-light model parameters are systematically biased in the BNN implementation.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

NASA NTRS1998

Executive Summary: Special Section on Credible Computational Fluid Dynamics Simulations

This summary presents the motivation for the Special Section on the credibility of computational fluid dynamics (CFD) simulations, its objective, its background and context, its content, and its major conclusions. Verification and validation (V&V) are the processes for establishing the credibility of CFD simulations. Validation assesses whether correct things are performed and verification assesses whether they are performed correctly. Various aspects of V&V are discussed. Progress is made in verification of simulation models. Considerable effort is still needed for developing a systematic validation method that can assess the credibility of simulated reality.

Mehta, Unmeel B.↗

DOE OSTI2022

SSR COST – COmposite Structural Technologies for advanced reactor deployment (Final Scientific/Technical Report)

Moltex is designing a Generation 4 or Advanced nuclear reactor, the molten salt fast spectrum SSR-W. The design intent is to reduce the cost and complexity of the nuclear systems by relying on inherent safety characteristics, instead of engineered systems. One such characteristic as a natural tendency for power to decrease as temperature increases. Another characteristic that improves natural safety is the fact that the working fluids (molten salts) are at atmospheric pressure at operating temperatures. This significantly reduces the need for pressure boundary components and systems. Moltex is highly motivated to develop simple and economic reactors to reduce the carbon intensity and improve energy security of global electricity production. The COST project investigated the opportunities for reducing construction cost of advanced reactors with the adoption of Rigid Elastic Composite Civil works (RECC). The construction design and methods used in currently operational light water power plants would generally be suboptimal for the SSR-W since there is no requirement for civil structures to withstand high pressure accidents such as hydrogen combustion or steam explosion. In parallel to downselecting the most promising RECC-type technology, accident scenarios representing the largest credible challenge to the SSR-W civil and construction design were modelled. The capability of an SSR-W reactor constructed with and without the selected construction technology was evaluated and the consequential licensing challenges identified. A construction schedule and cost estimate of the SSR-W with and without the selected RECC construction technology was developed. Once the above data was collected, then Moltex was able to review the safety performance, license-ability and cost/schedule benefit of applying the preferred construction technology. The CAPEX/NPV benefit was quantified, and the licensing risks and opportunities were documented. Finally, the CAPEX/NPV benefit of the advanced construction technology was modest due to the suitability of traditional stick-built construction for the SSR-W design. However, previously unidentified benefits of the advanced construction technology were revealed during the studies that could credibly de-risk certain aspects of the design to be licensed. Since licensing (duration and delays) are very significant in new build project costs, the cost benefits may be significant. Further studies on the risk reduction of the construction technology are foreseen.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

DOE OSTI2025

Deep learning model for fast, science-based forecasting of fluid migration along faults in geologic carbon storage scenarios

Effective long-term geologic storage depends on robust site selection and credible, science-based forecasting of subsurface behavior to ensure storage integrity. For this work, we develop a deep learning–based reduced-order model (ROM) to quantify potential carbon dioxide (CO₂) and brine migration through geological faults. The ROM combines a Transformer model for binary classification and a Stacked Ensemble for regression, trained on a comprehensive dataset generated from 1400 physics-based reservoir simulations. Key geologic and operational parameters—including fault geometry, reservoir structure, and injection conditions—were systematically varied to capture a wide range of fluid migration scenarios. The ROM accurately predicts the onset of migration, cumulative migration volumes of both CO₂ and brine, and associated migration rates, as compared to an independent set of validation simulations, while significantly reducing computational cost compared to traditional simulation methods. Model performance was evaluated across diverse fault configurations, revealing that shallow reservoir geometry and fault angle are among the most influential factors governing migration behavior. Sensitivity analysis using SHapley Additive exPlanations (SHAP) provided interpretability, revealing distinct patterns in how geological and operational features drive transient versus cumulative migration outcomes. The ROM’s ability to rapidly simulate fault migration scenarios enables efficient sensitivity analyses, scenario evaluations, and decision support for site selection and monitoring design. This approach enhances the safety, scalability, and long-term operational performance of geologic carbon storage (GCS) systems by providing a robust, interpretable tool for predicting subsurface fluid migration and assessing fault-related migration potential.

42 ENGINEERING↗

DOE OSTI2026

Comparing ICME simulations with scaled laboratory experiment

In stellar physics and astrophysics, numerical simulations and laboratory experiments are often compared to observational data to support their representation of the real world. However, there is also merit in comparing numerical simulations to properly scaled experiments, especially when the experiment and the simulation are both emulating the solar phenomena. Confirming the credibility of scaled experiments and their scaling with well-validated models is important to expand our knowledge of the associated physical phenomena. This is significant because experiments and simulations can be performed frequently, whereas observations may be limited by location, field of view, and missing data. In this work, we use the Alfvén Wave Solar atmosphere Model, a well-validated magnetohydrodynamic model, to simulate an interplanetary coronal mass ejection (ICME) and compare it to an experiment which provides a scaled analog to a physical ICME. The experiment was performed on the Big Red Ball facility and scaled using dimensionless parameters such as plasma β and magnetosonic Mach number to reproduce the main structure of an ICME. We compare the model-simulated temperature, density, and magnetic field to those from the experiment, as well as the scaling parameters used in the experiment, to those calculated from the simulation. This comparison is performed to further justify the scaling arguments made by the experiment. Additionally, the comparison would lead to the development of stronger scaling arguments for future experiments.

Bryant, K. [University of Michigan, Ann Arbor, MI ↗

DOE OSTI2021

Differential Credibility of Climate Modes in CMIP6

This work quantitatively evaluates the fidelity with which the northern annular mode (NAM), southern annular mode (SAM), Pacific–North American pattern (PNA), El Niño–Southern Oscillation (ENSO), Pacific decadal oscillation (PDO), Atlantic multidecadal oscillation (AMO), and the first-order mode interactions are represented in Earth system model (ESM) output from the CMIP6 archive. Several skill metrics are used as part of a differential credibility assessment (DCA) of both spatial and temporal characteristics of the modes across ESMs, ESM families, and specific ESM realizations relative to ERA5. The spatial patterns and probability distributions are generally well represented but skill scores that measure the degree to which the frequencies of maximum variance are captured are consistently lower for most ESMs and climate modes. Substantial variability in skill scores manifests across realizations from individual ESMs for the PNA and oceanic modes. Further, the ESMs consistently overestimate the strength of the NAM–PNA first-order interaction and underestimate the NAM–AMO connection. Furthermore, these results suggest that the choice of ESM and ESM realizations will continue to play a critical role in determining climate projections at the global and regional scale at least in the near term.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2023

A scalable framework for quantifying field-level agricultural carbon outcomes

Agriculture contributes nearly a quarter of global greenhouse gas (GHG) emissions, which is motivating interest in adopting certain farming practices that have the potential to reduce GHG emissions or sequester carbon in soil. The related GHG emission (including N 2 O and CH 4 ) and changes in soil carbon stock are defined here as “agricultural carbon outcomes”. Accurate quantification of agricultural carbon outcomes is the basis for achieving emission reductions for agriculture, but existing approaches for measuring carbon outcomes (including direct measurements, emission factors, and process-based modeling) fall short of achieving the required accuracy and scalability necessary to support credible, verifiable, and cost-effective measurement and improvement of these carbon outcomes. Here we propose a foundational and scalable framework to quantify field-level carbon outcomes for farmland, which is based on the holistic carbon balance of the agroecosystem: Agroecosystem Carbon Outcomes = Environment (E) × Management (M) × Crop (C). Following a comprehensive review of the scientific challenges associated with existing approaches, as well as their tradeoffs between cost and accuracy, we propose that the most viable path for the quantification of field-level carbon outcomes in agricultural land is through an effective integration of various approaches (e.g. diverse observations, sensor/in-situ data, and modeling), defined as the “System-of-Systems” solution. Such a “System-of-Systems” solution should simultaneously comprise the following components: (1) scalable collection of ground truth data and cross-scale sensing of environment variables (E), management practices (M), and crop conditions (C) at the local field level; (2) advanced modeling with necessary processes to support the quantification of carbon outcomes; (3) systematic Model-Data Fusion (MDF), i.e. robust and efficient methods to integrate sensing data and models at each local farmland level; (4) high computation efficiency and artificial intelligence (AI) to scale to millions of individual fields with low cost; and (5) robust and multi-tier validation systems and infrastructures to ensure solution fidelity and true scalability, i.e. the ability of a solution to perform robustly with accepted accuracy on all targeted fields. In this regard, we provide here the detailed scientific rationale, current progress, and future research and development (R&D) priorities to achieve different components of the “System-of-Systems” solution, thus accomplishing the Environment×Management×Crop framework to quantify field-level agricultural carbon outcomes.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2023

Hybrid Attack Graph Generation with Graph Convolutional Deep-Q Learning

Critical infrastructures such as power grids have become increasingly complex, connected, and vulnerable to adverse scenarios, including cyber and physical attacks and faults. Effective risk mitigation for such cyber-physical energy systems (CPES), requires preemptive knowledge of likely adversarial attack scenarios. Hybrid Attack Graph (HAG) is a structured way to represent an adversarial scenario as an attack sequence using a threat model. However, the scarcity of documented attack sequences hinders analysts and CPES planners’ ability to identify credible attack scenarios for a given CPES. We propose a data-driven Graph Convolutional Deep-Q Network (GCDQ) to address this data challenge through generating HAGs. By leveraging limited real-world observations from the MITRE ATT&CK knowledge base, our GCDQ model synthesizes realistic graphs with the targeted attribute of minimum detectability via reinforcement learning. This generative model is the first step in creating a tool to substantially boost the attack sequence dataset and enhance the performance of CPS defense-related tasks by providing insights into likely attack sequences with given attributes.

deep learning, artificial intelligence↗

DOE OSTI2022

Verification and Validation Activities for the Multi-Fidelity Toolkit

The Multi-Fidelity Toolkit (MFTK) is a simulation tool being developed at Sandia National Laboratories for aerodynamic predictions of compressible flows over a range of physics fidelities and computational speeds. These models include the Reynolds-Averaged-Navier-Stokes (RANS) equations, the Euler equations, and modified Newtonian aerodynamics (MNA) equations, and they can be invoked independently or coupled with hierarchical Kriging to interpolate between high-fidelity simulations using lower-fidelity data. However, as with any new simulation capability, verification and validation are necessary to gather credibility evidence. This work describes formal code- and solution-verification activities as well as model validation with uncertainty considerations. Code verification is performed on the MNA model by comparing with an analytical solution for flat-plate and inclined-plate geometries. Solution-verification activities include grid-refinement studies of HIFiRE-1 wind tunnel measurements, which are used for validation, for all model fidelities. A thorough treatment of the validation comparison with prediction error and validation uncertainty is also presented.

42 ENGINEERING↗

NASA NTRS1992

Analysis of gamma-ray burst spectra with cyclotron lines

Motivated by the recent developments in the cyclotron resonance upscattering of soft photons or CUSP model of Gamma Ray Burst (GBR) continuum spectra, we revisit a select database of GRBs with credible cyclotron absorption features. We measure the break energy of the continuum, the slope below the break and deduce the soft photon energy or the electron beam Lorentz factor cutoff. We study the correlation (or lack of) between various parameters in the context of the CUSP model. One surprise result is that there appears to be marginal correlation between the break energy and the spectral index below the break.

Kargatis, Vincent↗

DOE OSTI2022

Search for the decay $B^0$ → $\phiμ^+μ^–$

A search for the decay $B^0$ → $\phiμ^+μ^–$ is performed using proton-proton collisions at centre-of-mass energies of 7, 8, and 13 TeV collected by the LHCb experiment and corresponding to an integrated luminosity of 9 fb –1 . No evidence for the $B^0$ → $\phiμ^+μ^–$ decay is found and an upper limit on the branching fraction, excluding the $\phi$ and charmonium regions in the dimuon spectrum, of 4.4 × 10 –3 at a 90% credibility level, relative to that of the $B^0_s$ → $\phiμ^+μ^–$ branching fraction and assuming a phase-space model, the absolute branching fraction of the $B^0$ → $\phiμ^+μ^–$ in the full q 2 range is determined to be less than 3.2 x 10 –9 at a 90% credibility level.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗