Search NASA⌕ Search

SEARCH · Search NASA

Results for “model credibility”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11
All sourcesNASA NTRSDOE OSTI
NASA NTRS2015

Understanding Air Transportation Market Dynamics Using a Search Algorithm for Calibrating Travel Demand and Price

This paper presents a search algorithm based framework to calibrate origin-destination (O-D) market specific airline ticket demands and prices for the Air Transportation System (ATS). This framework is used for calibrating an agent based model of the air ticket buy-sell process - Airline Evolutionary Simulation (Airline EVOS) -that has fidelity of detail that accounts for airline and consumer behaviors and the interdependencies they share between themselves and the NAS. More specificially, this algorithm simultaneous calibrates demand and airfares for each O-D market, to within specified threshold of a pre-specified target value. The proposed algorithm is illustrated with market data targets provided by the Transportation System Analysis Model (TSAM) and Airline Origin and Destination Survey (DB1B). Although we specify these models and datasources for this calibration exercise, the methods described in this paper are applicable to calibrating any low-level model of the ATS to some other demand forecast model-based data. We argue that using a calibration algorithm such as the one we present here to synchronize ATS models with specialized forecast demand models, is a powerful tool for establishing credible baseline conditions in experiments analyzing the effects of proposed policy changes to the ATS.

Kumar, Vivek↗
DOE OSTI2024

Improved Statistical Analysis for the Neutrinoless Double-Beta Decay Matrix Element of 136Xe

Neutrinoless double beta decay nuclear matrix element (M0ν) for 136Xe was recently analyzed using a statistical approach (Phys. Rev. C 107, 045501 (2023)). In the analysis, three initial shell model effective Hamiltonians were randomly altered, and their results for 23 measured observables were used to infer credibility for the M0ν nuclear matrix element (NME) based on a Bayesian Model Averaging approach. In that analysis, a reasonable Gamow-Teller quenching factor of 0.7 was assumed for each starting effective Hamiltonian. Given that the result of the statistical analysis was sensible to this choice, we are here improving that analysis by assuming that the Gamow-Teller quenching factor is also randomly chosen within reasonabe limits for all three starting Hamiltonians. The outcomes are slightly higher expectation values and uncertainties for the M0ν NME.

Astronomy & Astrophysics↗
NASA NTRS2015

Representative Agricultural Pathways and Scenarios for Regional Integrated Assessment of Climate Change Impacts, Vulnerability, and Adaptation: Chapter - 5

The global change research community has recognized that new pathway and scenario concepts are needed to implement impact and vulnerability assessment where precise prediction is not possible, and also that these scenarios need to be logically consistent across local, regional, and global scales. For global climate models, representative concentration pathways (RCPs) have been developed that provide a range of time-series of atmospheric greenhouse-gas concentrations into the future. For impact and vulnerability assessment, new socio-economic pathway and scenario concepts have also been developed, with leadership from the Integrated Assessment Modeling Consortium (IAMC).This chapter presents concepts and methods for development of regional representative agricultural pathways (RAOs) and scenarios that can be used for agricultural model intercomparison, improvement, and impact assessment in a manner consistent with the new global pathways and scenarios. The development of agriculture-specific pathways and scenarios is motivated by the need for a protocol-based approach to climate impact, vulnerability, and adaptation assessment. Until now, the various global and regional models used for agricultural-impact assessment have been implemented with individualized scenarios using various data and model structures, often without transparent documentation, public availability, and consistency across disciplines. These practices have reduced the credibility of assessments, and also hampered the advancement of the science through model intercomparison, improvement, and synthesis of model results across studies. The recognition of the need for better coordination among the agricultural modeling community, including the development of standard reference scenarios with adequate agriculture-specific detail led to the creation of the Agricultural Model Intercomparison and Improvement Project (AgMIP) in 2010. The development of RAPs is one of the cross-cutting themes in AgMIP's work plan, and has been the subject of ongoing work by AgMIP since its creation.

agriculture↗
NASA NTRS2015

The Quick Fire Emissions Dataset (QFED): Documentation of Versions 2.1, 2.2 and 2.4: Technical Report Series on Global Modeling and Data Assimilation - Volume 38

Biomass burning is an important source of particulates and trace gases and a major element of the terrestrial carbon cycle. Well constrained emissions from vegetation fires are needed to model direct and indirect effects of biomass burning aerosols, to model homogeneous and heterogeneous chemistry in the atmosphere, and to perform credible Earth system analysis, and climate and air pollution studies. To improve the performance of NASA Goddard Earth Observing System Model (GEOS) in the areas of atmospheric constituent modeling with a focus on biomass burning we developed the Quick Fire Emissions Dataset (QFED). The QFED emissions are based on the fire radiative power (top-down) approach and draw on the cloud correction method developed in the Global Fire Assimilation System (GFAS). Location and fire radiative power of fires are obtained from the Moderate Resolution Imaging Spectroradiometer (MODIS) Level 2 fire products (MOD14 and MYD14) and the MODIS Geolocation products (MOD03 and MYD03). QFED strengths are high spatial and temporal resolutions and near-real time availability. Daily mean emissions are available at 0.3125 times 0.25 degrees and in recent versions also at 0.1 times 0.1 degrees. QFED provides emissions of black carbon, organic carbon, sulfur dioxide, carbon monoxide, carbon dioxide, PM2.5, ammonia, nitrogen oxides, methyl ethyl ketone, propylene, ethane, propane, n- and i-butane, acetaldehyde, formaldehyde, acetone and methane. Two QFED product systems are maintained by the NASA Global Modeling and Assimilation Office (GMAO): one that produces near real-time daily emissions used operationally in the GEOS-5 Data Assimilation System, and one that produces an extended historical dataset with daily emissions from March 2000 to the present. The historical dataset also provides monthly mean and monthly climatological emissions.

Koster, Randal D.↗
DOE OSTI2024

Nuclear Data Sensitivity/Uncertainty Studies of Tantalum-Reflected Systems

Tantalum (Ta) is a refractory metal with high melting point and high corrosion resistance. This makes it useful as a mold material for plutonium casting operations. Unfortunately, there are few criticality benchmarks with high nuclear data sensitivities. Given that, it is desirable to design new experiments to provide additional validation data for nuclear criticality safety, which is the objective of the Thales project. Multiple application models were used to represent process conditions (either normal conditions or credible upset conditions) both at Los Alamos National Laboratory (LANL) and Savannah River Site (SRS). A total of 40 application models were obtained. Given this, it was useful to down-select the number of needed application models. This work documents the down-selection based on several nuclear data sensitivity/uncertainty metrics. In addition to showing the result of the specific down-selection used for the Thales project, this work discusses potential ways to represent a large number of application models with a smaller number of proposed experiment designs.

42 ENGINEERING↗
DOE OSTI2022

Quantifying modeling uncertainties when combining multiple gravitational-wave detections from binary neutron star sources

Here, with the increasing sensitivity of gravitational-wave detectors, we expect to observe multiple binary neutron-star systems through gravitational waves in the near future. The combined analysis of these gravitational-wave signals offers the possibility to constrain the neutron-star radius and the equation of state of dense nuclear matter with unprecedented accuracy. However, it is crucial to ensure that uncertainties inherent in the gravitational-wave models will not lead to systematic biases when information from multiple detections is combined. To quantify waveform systematics, we perform an extensive simulation campaign of binary neutron-star sources and analyze them with a set of four different waveform models. For our analysis with 38 simulations, we find that statistical uncertainties in the neutron-star radius decrease to ± 250 m (2% at 90% credible interval) but that systematic differences between currently employed waveform models can be twice as large. Hence, it will be essential to ensure that systematic biases will not become dominant in inferences of the neutron-star equation of state when capitalizing on future developments.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
NASA NTRS2008

Conceptual Design Methods and the Application of a Tradespace Modeling Tool for Deep Space Missions

Concept studies for deep space missions are typically time-consuming and costly, given the variety of missions and uniqueness of each design. Yet, in an increasingly cost-constrained environment, it is critical to identify the most scientifically valuable and cost-effective designs early in the design process. Modeling is an integral part in helping to identify the most desirable design option. While some spacecraft design models currently exist for Earth-orbiting spacecraft, there has been less success with deep space missions. Instead, these missions require a modified design and modeling approach to enable the same construction of a comprehensive, yet credible, mission tradespace. This paper presents an approach for efficiently constructing such a mission tradespace. In addition to a proposed design and modeling approach, three case study missions are presented including a solar orbiter, a Europa orbiter, and a near-Earth asteroid (NEA) sample return mission.

space vehicles↗
DOE OSTI2021

Credibility Framework (CF) v.1.0

Credibility Framework provides tools such as PIRT (Phenomena Identification and Ranking Table) and PCMM (Predictive Capability Maturity Model) configurable by non-programmers through simple Excel spreadsheets for tailoring the credibility process to meet the needs of the analysis provider organizations, customers and regulatory agencies. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525. SAND2020-10230 M

Clay, RobertL.↗
NASA NTRSDate not supplied

Applying NASA-STD-7009 Standard for Models and Simulations to Surrogate and Other Statistical Models

NASA-STD-7009A, Standard for Models and Simulations, contains a worthy and insightfully-crafted credibility assessment. That set of metrics is intended to accumulate and communicate information that can be critical for users and their customers in evaluating findings due to a model or simulation. Was the model designed for the present use? Is it validated in the domain it’s being applied to? And so on. The standard was originally focused on physics-based models, but it can be applied more generally. It can – and should – be applied to high-impact statistical models, including surrogate models based on larger models and sims and regression models based directly on data. A key issue with many of such models is their relative simplicity compared to most M&S despite their use in high-risk engineering decisions. This report will make a case for using this Standard for a wider class of models, and how it might actually add value in excess of its cost for not only stakeholders, but modelers themselves. A rough example showing compliance for a statistical model is included.

K L Johnson↗
DOE OSTI2021

Determining the jet transport coefficient $\hat{q}$ from inclusive hadron suppression measurements using Bayesian parameter estimation

We report a new determination of $\hat{q}$, the jet transport coefficient of the quark-gluon plasma. We use the JETSCAPE framework, which incorporates a novel multistage theoretical approach to in-medium jet evolution and Bayesian inference for parameter extraction. The calculations, based on the Matter and Lbt jet quenching models, are compared to experimental measurements of inclusive hadron suppression in Au + Au collisions at the BNL Relativistic Heavy Ion Collider (RHIC) and Pb + Pb collisions at the CERN Large Hadron Collider (LHC). The correlation of experimental systematic uncertainties is accounted for in the parameter extraction. The functional dependence of $\hat{q}$ on jet energy or virtuality and medium temperature is based on a perturbative picture of in-medium scattering, with components reflecting the different regimes of applicability of Matter and Lbt. In the multistage approach, the switch between Matter and Lbt is governed by a virtuality scale Q 0 . Comparison of the posterior model predictions to the RHIC and LHC hadron suppression data shows reasonable agreement, with moderate tension in limited regions of phase space. The distribution of $\hat{q}/T^3$ extracted from the posterior distributions exhibits weak dependence on jet momentum and medium temperature T, with 90% credible region (CR) depending on the specific choice of model configuration. The choice of Matter + Lbt, with switching at virtuality Q 0 , has 90% CR of $2<\hat{q}/T^3<4$ for p T jet > 40 GeV=c. The value of Q 0 , determined here for the first time, is in the range 2.0-2.7 GeV.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
NASA NTRS2014

Using Ensemble Short-Term Initialized Coupled NASA GEOS5 Climate Model Integrations to Study Convective Bias Growth

The representation of convective processes, particularly deep convection in the tropics, remains a persistent problem in climate models. In fact structural biases in the distribution of tropical rainfall in the CMIP5 models is hardly different than that of the CMIP3 versions. Given that regional climate change at higher latitudes is sensitive to the configuration of tropical forcing, this persistent bias is a major issue for the credibility of climate change projections. In this study we use model output from integrations of the NASA Global Earth Observing System Five (GEOS5) climate modeling system to study the evolution of biases in the location and intensity of convective processes. We take advantage of a series of hindcast experiments done in support of the US North American Multi-Model Ensemble (NMME) initiative. For these experiments a nine-month forecast using a coupled model configuration is made approximately every five days over the past 30 years. Each forecast is started with an updated analysis of the ocean, atmosphere and land states. For a given calendar month we have approximately 180 forecasts with daily means of various quantities. These forecasts can be averaged to essentially remove "weather scales" and highlight systematic errors as they evolve. Our primary question is to ask how the spatial structure of daily mean precipitation over the tropics evolves from the initial state and what physical processes are involved. Errors in parameterized convection, various water and energy fluxes and the divergent circulation are found to set up on fast time scales (order five days) compared to errors in the ocean, although SST changes can be non-negligible over that time. For the month of June the difference between forecast day five versus day zero precipitation looks quite similar to the difference between the June precipitation climatology and that from the Global Precipitation Climatology Project (GPCP). We focus much of our analysis on the influence of SST gradients, associated PBL baroclinicity enabled by turbulent mixing, the ensuing PBL moisture convergence, and how changes in these processes relate to convective precipitation bias growth over this short period.

Cohen, Charlie↗
NASA NTRS1995

Calculations of Diffuser Flows with an Anisotropic K-Epsilon Model

A newly developed anisotropic K-epsilon model is applied to calculate three axisymmetric diffuser flows with or without separation. The new model uses a quadratic stress-strain relation and satisfies the realizability conditions, i.e., it ensures both the positivity of the turbulent normal stresses and the Schwarz' inequality between any fluctuating velocities. Calculations are carried out with a finite-element method. A second-order accurate, bounded convection scheme and sufficiently fine grids are used to ensure numerical credibility of the solutions. The standard K-epsilon model is also used in order to highlight the performance of the new model. Comparison with the experimental data shows that the anisotropic K-epsilon model performs consistently better than does the standard K-epsilon model in all of the three test cases.

Zhu, J.↗
DOE OSTI2026

Frequentist cosmological constraints from full-shape clustering measurements in DESI DR1

We present a frequentist analysis of clustering measurements from Data Release 1 of the Dark Energy Spectroscopic Instrument (DESI) using the standard profile likelihood method. While Bayesian inferences for effective field theory models of galaxy clustering can be highly sensitive to prior choices for extended cosmological models, frequentist inferences are not susceptible to such effects. We compare frequentist and Bayesian constraints for the parameter set {σ 8 , H 0 , Ω m , w 0 , w a } using the full-shape power spectrum multipoles, post-reconstruction baryon acoustic oscillation (BAO) measurements, and external datasets from the CMB and type Ia supernovae measurements. The frequentist confidence intervals are significantly shifted relative to the Bayesian credible intervals for the w 0 w a CDM model, unless supernovae data are included. When DESI full-shape and BAO data are fit jointly, we obtain the following 1σ frequentist confidence intervals for ΛCDM (w 0 w a CDM): σ 8 = 0.863 +0.048 -0.040 , H 0 = 68.96 +0.81 -0.80 km s -1 Mpc -1 , Ω m = 0.3034 ± 0.0110 (σ 8 = 0.782 +0.060 -0.036 , H 0 = 63.7 +4.2 -2.0 km s -1 Mpc -1 , Ω m = 0.378 +0.024 -0.047 , w 0 = -0.16 +0.10 -0.50 , w a = -3.0 +1.7 ), corresponding to 0.8σ, 0.3σ, 0.7σ (2.1σ, 4.1σ, 6.5σ, 6.3σ, 6.6σ) shifts between the maximum likelihood estimate and the Bayesian posterior mean for ΛCDM (w 0 w a CDM) respectively.

Bayesian reasoning↗
DOE OSTI2023

Predictive Capability Maturity Model Demonstration for Cylindrical Cavity Coupling Using Gemma in the Next Generation Workflow

The predictive capability maturity model (PCMM) uses the expert elicitation process to generate credibility evidence for a particular analysis. To ensure Gemma has the capability to efficiently produce this credibility evidence, next generation workflows (NGW) are created for the solution verification, calibration/validation, and input uncertainty quantification portions of the PCMM assessment. These workflows are then used on the Higgins cylinder problem, which is representative of applications involving external-to-internal electromagnetic field coupling through a slot. The uncertainties calculated using these workflows are then used to calculate the validation comparison error and the validation uncertainty for the model following the American Society of Mechanical Engineers (ASME) verification and validation (V&V) 20 standard. These workflows will enable analysts to iterate each element of PCMM more efficiently than if completed without using a NGW workflow. An example of this iterative process is shown in Section 7.2.

42 ENGINEERING↗
DOE OSTI2025

Evaluating an Earth system model from a water manager perspective

Abstract. The large spatial scale of global Earth system models (ESMs) is often cited as an obstacle to using the output by water resource managers in localized decisions. Recent advances in computing have improved the fidelity of hydrological responses in ESMs through increased connectivity between model components. However, the models are seldom evaluated for their ability to reproduce metrics that are important for and resonate with practitioners or that allow practitioners to situate higher-resolution model outputs within a cascade of uncertainty stemming from different models and scenarios. We draw on the combined experience of the author team and water manager workshop participants to identify salient water management metrics and evaluate whether they are credibly reproduced over the conterminous USA by the Community Earth System Model v2 (CESM2) Large Ensemble. We find that, while the exact values may not match the observations, aspects such as interannual variability can be reproduced by CESM2 for the mean wet day precipitation and length of dry spells. CESM2 also captures the proportion of total annual precipitation that derives from the heaviest rain days in watersheds that are not snow-dominated. Aggregating the 7 d mean daily runoff to two-digit Hydrological Unit Code (HUC2) watersheds also shows that rain-dominated regions capture the timing and interannual variability of annual maximum and minimum flows. We conclude that there is potential for far greater use of large-ensemble ESMs, such as CESM2, in long-range water management decisions to supplement high-resolution regional projections.

Tye, Mari R. (ORCID:0000000324911020)↗
NASA NTRS1979

Evaluation Applied to Reliability Analysis of Reconfigurable, Highly Reliable, Fault-Tolerant, Computing Systems for Avionics

Emulation techniques are proposed as a solution to a difficulty arising in the analysis of the reliability of highly reliable computer systems for future commercial aircraft. The difficulty, viz., the lack of credible precision in reliability estimates obtained by analytical modeling techniques are established. The difficulty is shown to be an unavoidable consequence of: (1) a high reliability requirement so demanding as to make system evaluation by use testing infeasible, (2) a complex system design technique, fault tolerance, (3) system reliability dominated by errors due to flaws in the system definition, and (4) elaborate analytical modeling techniques whose precision outputs are quite sensitive to errors of approximation in their input data. The technique of emulation is described, indicating how its input is a simple description of the logical structure of a system and its output is the consequent behavior. The use of emulation techniques is discussed for pseudo-testing systems to evaluate bounds on the parameter values needed for the analytical techniques.

Migneault, G. E.↗
NASA NTRS1979

Emulation applied to reliability analysis of reconfigurable, highly reliable, fault-tolerant computing systems

Emulation techniques applied to the analysis of the reliability of highly reliable computer systems for future commercial aircraft are described. The lack of credible precision in reliability estimates obtained by analytical modeling techniques is first established. The difficulty is shown to be an unavoidable consequence of: (1) a high reliability requirement so demanding as to make system evaluation by use testing infeasible; (2) a complex system design technique, fault tolerance; (3) system reliability dominated by errors due to flaws in the system definition; and (4) elaborate analytical modeling techniques whose precision outputs are quite sensitive to errors of approximation in their input data. Next, the technique of emulation is described, indicating how its input is a simple description of the logical structure of a system and its output is the consequent behavior. Use of emulation techniques is discussed for pseudo-testing systems to evaluate bounds on the parameter values needed for the analytical techniques. Finally an illustrative example is presented to demonstrate from actual use the promise of the proposed application of emulation.

Migneault, G. E.↗
NASA NTRS1990

Knowledge representation to support reasoning based on multiple models

Model Based Reasoning is a powerful tool used to design and analyze systems, which are often composed of numerous interactive, interrelated subsystems. Models of the subsystems are written independently and may be used together while they are still under development. Thus the models are not static. They evolve as information becomes obsolete, as improved artifact descriptions are developed, and as system capabilities change. Researchers are using three methods to support knowledge/data base growth, to track the model evolution, and to handle knowledge from diverse domains. First, the representation methodology is based on having pools, or types, of knowledge from which each model is constructed. In addition information is explicit. This includes the interactions between components, the description of the artifact structure, and the constraints and limitations of the models. The third principle we have followed is the separation of the data and knowledge from the inferencing and equation solving mechanisms. This methodology is used in two distinct knowledge-based systems: one for the design of space systems and another for the synthesis of VLSI circuits. It has facilitated the growth and evolution of our models, made accountability of results explicit, and provided credibility for the user community. These capabilities have been implemented and are being used in actual design projects.

Gillam, April↗