Search NASA⌕ Search

SEARCH · Search NASA

Results for “Model discrepancy”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 505 records · Page 28

A COMPARISON OF NEAR-INFRARED PHOTOMETRY AND SPECTRA FOR Y DWARFS WITH A NEW GENERATION OF COOL CLOUDY MODELS

We present YJHK photometry, or a subset, for the six Y dwarfs discovered in Wide-field Infrared Survey Explorer (WISE) data by Cushing et al. The data were obtained using the Near-Infrared Imager on the Gemini North telescope; YJHK were obtained for WISEP J041022.71+150248.5, WISEP J173835.52+273258.9, and WISEPC J205628.90+145953.3; YJH for WISEPC J140518.40+553421.5 and WISEP J154151.65225025.2; and YJK for WISEP J182831.08+265037.8. We also present a far-red spectrum obtained using GMOS-North for WISEPC J205628.90+145953.3. We compare the data to Morley et al. models, which include cloud decks of sulfide and chloride condensates. We find that the models with these previously neglected clouds can reproduce the energy distributions of T9 to Y0 dwarfs quite well, other than near 5 {mu}m where the models are too bright. This is thought to be because the models do not include departures from chemical equilibrium caused by vertical mixing, which would enhance the abundance of CO and CO{sub 2}, decreasing the flux at 5 {mu}m. Vertical mixing also decreases the abundance of NH{sub 3}, which would otherwise have strong absorption features at 1.03 {mu}m and 1.52 {mu}m that are not seen in the Y0 WISEPC J205628.90+145953.3. We find that the five Y0 to Y0.5 dwarfs have 300 {approx}< T {sub eff} K {approx}< 450, 4.0 {approx}< log g {approx}< 4.5, and f {sub sed} Almost-Equal-To 3. These temperatures and gravities imply a mass range of 5-15 M {sub Jupiter} and ages around 5 Gyr. We suggest that WISEP J182831.08+265037.8 is a binary system, as this better explains its luminosity and color. We find that the data can be made consistent with observed trends, and generally consistent with the models, if the system is composed of a T {sub eff} Almost-Equal-To 325 K and log g {approx}< 4.5 primary, and a T {sub eff} Almost-Equal-To 300 K and log g {approx}> 4.0 secondary, corresponding to masses of 10 and 7 M {sub Jupiter} and an age around 2 Gyr. If our deconvolution is correct, then the T {sub eff} Almost-Equal-To 300 K cloud-free model fluxes at K and W2 are too faint by 0.5-1.0 mag. We will address this discrepancy in our next generation of models, which will incorporate water clouds and mixing.

79 ASTRONOMY AND ASTROPHYSICS↗

Alignment-dependent ionization of hydrogen molecules in intense laser fields

Recent experiments showed that the alignment-dependent ionization probabilities of hydrogen molecules vary as a function of laser intensity, and the anisotropy deviates from the prediction of the molecular tunneling ionization model (MO-ADK). To investigate the physical origins of the deviation, we systematically studied the anisotropy of hydrogen molecules in intense laser fields for three wavelengths, 400 nm (multiphoton ionization), 1850 nm (tunneling ionization), and 800 nm (in between), by solving the time-dependent Schroedinger equation with a model potential. The calculated ratio of the ionization probabilities for laser fields parallel and perpendicular to the molecular axis are in reasonable agreement with experiment. Furthermore, by analyzing the molecular wave function and the model potential, we found that the discrepancies between experiment and the MO-ADK prediction originated from the inaccurate coefficients used in the model and that the assumption of isotropy of the effective potential in the tunneling region is invalid.

74 ATOMIC AND MOLECULAR PHYSICS↗

Bayesian calibration and uncertainty quantification of a rate-dependent cohesive zone model for polymer interfaces

In this work we present a rate-dependent cohesive zone model for the fracture of polymeric interfaces and performs a Bayesian calibration, an uncertainty quantification, and a sensitivity analysis for the model. The proposed cohesive zone model accounts for both reversible elastic and irreversible rate-dependent separation sliding deformation at the interface. The viscous dissipation due to the irreversible opening at the interface is modeled using elastic-viscoplastic kinematics that incorporates the effects of strain rate. Inverse calibration of parameters for such complex models through trial and error is challenging due to the large number of parameters of the model. Moreover, the calibrated parameter values are often non-unique and uncertain when the available experimental data is limited. To tackle this challenge, we employ a Bayesian calibration approach to identify parameters from experimental data, the resulting parameters significantly enhance the accuracy of the model. To quantify the uncertainty associated with the inverse parameter estimation, a modular Bayesian approach is employed to calibrate the unknown model parameters, accounting for the parameter uncertainty of the cohesive zone model. The advantages of the Bayesian calibration over a deterministic parameter fit are demonstrated. Further, to quantify the model uncertainties, such as incorrect assumptions or missing physics, a discrepancy function is introduced, which significantly improves the model’s prediction. Finally, the total uncertainty of the model is quantified in a predictive setting. A sensitivity analysis is performed to assess how changes in the input variables of the model affect the peak load, facilitating the identification of a concise set of highly influential parameters. The present approach can be used for calibration and uncertainty quantification for other complex computational mechanics models. It should also facilitate the designing of interface materials under uncertainty.

42 ENGINEERING↗

Benchmarking Simulated Precipitation Variability Amplitude across Time Scales

Here, objective performance metrics that measure precipitation variability across time scales from subdaily to interannual are presented and applied to Historical simulations of Coupled Model Intercomparison Project phase 5 and 6 (CMIP5 and CMIP6) models. Three satellite-based precipitation estimates (IMERG, TRMM, and CMORPH) are used as reference data. We apply two independent methods to estimate temporal variability of precipitation and compare the consistency in their results. The first method is derived from power spectra analysis of 3-hourly precipitation, measuring forced variability by solar insolation (diurnal and annual cycles) and internal variability at different time scales (subdaily, synoptic, subseasonal, seasonal, and interannual). The second method is based on time averaging and facilitates estimating the seasonality of subdaily variability. Supporting the robustness of our metric, we find a near equivalence between the results obtained from the two methods when examining simulated-to-observed ratios over large domains (global, tropics, extratropics, land, or ocean). Additionally, we demonstrate that our model evaluation is not very sensitive to the discrepancies between observations. Our results reveal that CMIP5 and CMIP6 models in general overestimate the forced variability while they underestimate the internal variability, especially in the tropical ocean and higher-frequency variability. The underestimation of subdaily variability is consistent across different seasons. The internal variability is overall improved in CMIP6, but remains underestimated, and there is little evidence of improvement in forced variability. Increased horizontal resolution results in some improvement of internal variability at subdaily and synoptic time scales, but not at longer time scales.

54 ENVIRONMENTAL SCIENCES↗

The Influence of Cloud Microphysics and Radiation on the Response of Water Vapor and Clouds to Climate Change

Uncertainties in representing the atmospheric water cycle are major obstacles to the accurate prediction of future climate. This project focused on addressing some of these uncertainties by implementing new physics for convection and radiation into the NCAR Community Atmosphere Model (CAM). To better understand and eventually better represent these processes in this major national climate model, we modified CAM3.5 to use the convection and cloud schemes developed by the Massachusetts Institute of Technology (MIT) and the RRTMG rapid radiation code for global climate models developed by Atmospheric and Environmental Research, Inc. (AER). The impact of the new physics on the CAM3.5 simulation of convection on diurnal and intra-seasonal scales, on intra-seasonal oscillations and on the distribution of water vapor has been investigated. In addition, the MIT and AER physics packages have been incorporated and tested in combination within the Weather Research and Forecasting (WRF) regional forecast model for the purpose of evaluating and improving convective and radiative processes on time scales appropriate to weather simulations. It has been found that the application of the AER radiation and MIT convection produces significant improvements in the modeled diurnal cycle of convection, especially over land, in the NCAR climate model. However, both the standard CAM3.5 and the modified CAM3.5 with the new physics are unable to capture the proper spectrum and propagating characteristics of dynamical intra-seasonal oscillations such as the Madden-Julian Oscillation. In addition, it has been shown that the new physics methods modify, but do not substantially improve, the distribution of upper tropospheric water vapor in CAM as established through the comparison of modeled and observed satellite radiances. This suggests that continuing regional discrepancies in water vapor amounts in the climate model may not be solely related to convective or radiative processes. The major results of this project have been described in more detail in a journal article titled The Impacts of AER Radiation and MIT Convection on the Water Cycle Simulated by CAM3.5 that will be submitted for publication during Fall 2010.

54 ENVIRONMENTAL SCIENCES↗

Measurement of the anomalous spin precession frequency $\omega_a$ in the Muon $g-2$ experiment at Fermilab

The muon anomaly, $a_\mu=(g_{\mu}-2)/2$, is a low-energy observable which can be both measured and computed to high precision, making it a sensitive test of the Standard Model (SM) and a probe for new physics. The current discrepancy between the experimental value and the Standard Model calculation from the Muon $g-2$ Theory Initiative is $a_{\mu}^{exp}-a_{\mu}^{SM}=(251\pm59)\cdot10^{-11}$, with a significance of $4.2\,\sigma$.The Fermilab E989 experiment aims, with the full statistical power, to improve by a factor of four the precision of the measurement.In April 2021 the collaboration published the first measurement, based on the first year of data taking.This paper will present the status of the measurement of the anomalous muon spin precession frequency, $\omega_a$, performed on the datasets collected during Run-2 and Run-3 (2019 and 2020 campaigns), with a preliminary projection of the systematic uncertainties.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Solar Neutrinos: Status and Prospects

We describe the current status of solar neutrino measurements and of the theory—both neutrino physics and solar astrophysics—employed in interpreting measurements. Important recent developments include Super-Kamiokande's determination of the ν−e elastic scattering rate for 8 B neutrinos to 3%; the latest Sudbury Neutrino Observatory (SNO) global analysis in which the inclusion of low-energy data from SNO I and II significantly narrowed the range of allowed values for the neutrino mixing angle θ 12 ; Borexino results for both the 7 Be and proton-electron-proton (pep) neutrino fluxes, the first direct measurements constraining the rate of proton-proton (pp) I and pp II burning in the Sun; global reanalyses of solar neutrino data that take into account new reactor results on θ 13 ; a new decadal evaluation of the nuclear physics of the pp chain and CNO cycle defining best values and uncertainties in the nuclear microphysics input to solar models; recognition of an emerging discrepancy between two tests of solar metallicity, helioseismological mappings of the sound speed in the solar interior, and analyses of the metal photoabsorption lines based on our best current description of the Sun's photosphere; a new round of standard solar model calculations optimized to agree either with helioseismology or with the new photospheric analysis; and, motivated by the solar abundance problem, the development of nonstandard, accreting solar models, in order to investigate possible consequences of the metal segregation that occurred in the proto-solar disk. Here, we review this progress and describe how new experiments such as SNO+ could help us further exploit neutrinos as a unique probe of stellar interiors.

helioseismology↗

Promoting CO 2 Release from CO 3 2- -Containing Solvents during Water Electrolysis for Direct Air Capture

The pH swings from water electrolysis are leveraged to condition OH – -based facile CO 2 capture solvents using an electrochemical flow cell for direct air capture (DAC). Besides demonstrating the DAC using a membrane contactor, promoting CO 2 release from a CO 3 2– solution at the anode is specifically studied by adjusting the volumetric flow rate, anode chamber volume, residence time, and K 2 CO 3 concentration. Through case-by-case comparisons coupled with modeled results, increasing current, reducing volumetric flow rate, and/or reducing CO 3 2– concentration are the effective methods to promote CO 2 release from a CO 3 2– -containing solvent, whereas enlarging the anode chamber volume poses a minor effect. Moreover, the discrepancies between the experimental and modeled results may be caused by H + crossover rather than K + transport through the Nafion membrane during water electrolysis based upon the total alkalinity measurements for the K 2 CO 3 solutions gleaned from the anode. Here, it is believed that such results will provide guidance to design and operate an electrochemical flow cell for electrochemistry-assisted DAC and point source CO 2 capture.

20 FOSSIL-FUELED POWER PLANTS↗

Assessment and validation of NEAMS tools for high-fidelity multiphysics transient modeling of microreactors: Application of NEAMS codes to perform multiphysics modeling analyses of micro-reactor concepts

The NEAMS Multiphysics Applications team aims at providing assessment of code useability and functionality for microreactor design and analyses, together with demonstration of their capabilities to properly capture the steady-state and time-dependent behavior of different microreactor concepts. In FY-24, significant progress was achieved in improving multi-physics models of several microreactors systems: HP-MR, GC-MR and KRUSTY. These efforts focused on solving more complex multiphysics problems enabled by enhanced tools capability, verifying and validating results obtained, providing feedback to developers for suggested improvements, and sharing these models to facilitate user training. A series of new multiphysics transients were completed on the HP-MR (using Griffin/BISON/Sockeye) with core startup transient, control drum inadvertent rotation accident, and hydrogen leakage from hydride moderator (also including SWIFT). On the GC-MR, a new full-core model was developed and analyzed through a series of new multiphysics (Griffin/BISON/SAM) transients to simulate moderator leakage (also including SWIFT), flow blockage and coolant depressurization. Additional and updated TRISO failure analyses were completed on the HP-MR unit-cell and GC-MR assembly models leveraging improved TRISO modeling capabilities. The amount of SiC failure following accidental transients at end-of-life was null. However, GC-MR assembly TRISO analysis highlighted Pd penetration rate can be problematic and may require design changes on the studied microreactor concept. The neutronics discrepancies observed on the KRUSTY model in previous years were resolved using hybrid set of Monte Carlo/Deterministic cross-sections. The multiphysics (Griffin neutronics / BISON thermal-mechanics) 15₵ insertion transient simulation displayed good agreement when comparing with experimental data. Initial modeling of the 30 ₵ reactivity insertion also displays promising results. Such close agreement provides important validation data that can be leveraged by the NEAMS program and by microreactor vendors to support licensing of their technology. Finally, important experience was gathered with the NEAMS tools leading to several user feedback shared with tools developers, especially with regards to MOOSE mesh generator and Griffin. This project led to many publications demonstrating modeling capabilities, and to three models shared on the Virtual Test Bed.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Nuclear Effects in Neutrino Interactions at Low Momentum Transfer

This is a study to identify predicted effects of the carbon nucleus environment on neutrino - nucleus interactions with low momentum transfer. A large sample of neutrino interaction data collected by the MINERvA experiment is analyzed to show the distribution of charged hadron energy in a region with low momentum transfer. These distributions reveal a major discrepancy between the data and a popular interaction model with only the simplest Fermi gas nuclear effects. Detailed analysis of systematic uncertainties due to energy scale and resolution can account for only a little of the discrepancy. Two additional nuclear model effects, a suppression/screening effect (RPA), and the addition of a meson exchange current process (MEC), are shown to improve the description of the data.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

SOUND-SPEED INVERSION OF THE SUN USING A NONLOCAL STATISTICAL CONVECTION THEORY

Helioseismic inversions reveal a major discrepancy in sound speed between the Sun and the standard solar model just below the base of the solar convection zone. We demonstrate that this discrepancy is caused by the inherent shortcomings of the local mixing-length theory adopted in the standard solar model. Using a self-consistent nonlocal convection theory, we construct an envelope model of the Sun for sound-speed inversion. Our solar model has a very smooth transition from the convective envelope to the radiative interior, and the convective energy flux changes sign crossing the boundaries of the convection zone. It shows evident improvement over the standard solar model, with a significant reduction in the discrepancy in sound speed between the Sun and local convection models.

79 ASTRONOMY AND ASTROPHYSICS↗

A comparison between ShapeFit compression and Full-Modelling method with PyBird for DESI 2024 and beyond

DESI aims to provide one of the tightest constraints on cosmological parameters by analysing the clustering of more than thirty million galaxies. However, obtaining such constraints requires special care in validating the methodology and efforts to reduce the computational time required through data compression and emulation techniques. In this work, we perform a rigorous validation of the PyBird power spectrum modelling code with both a traditional emulated Full-Modelling approach and the model-independent ShapeFit compression approach. By using cubic box simulations that accurately reproduce the clustering and precision of the DESI survey, we find that the cosmological constraints from ShapeFit and Full-Modelling are consistent with each other at the ∼ 0.5σ level for the ΛCDM model. Both ShapeFit and Full-Modelling are also consistent with the true ΛCDM simulation cosmology down to a scale of k max = 0.20 hMpc -1 even after including the hexadecapole. For extended models such as the wCDM and the oCDM models, we find that including the hexadecapole can significantly improve the constraints and reduce the modelling errors with the same k max . While their discrepancies between the constraints from ShapeFit and Full-Modelling are more significant than ΛCDM, they remain consistent within 0.7σ. Lastly, we also show that the constraints on cosmological parameters with the correlation function evaluated from PyBird down to s min = 30h -1 Mpc are unbiased and consistent with the constraints from the power spectrum.

79 ASTRONOMY AND ASTROPHYSICS↗

Generalized Bayesian Framework for Evaluation of Integral Benchmark Experiments

A recently published generalized Bayesian optimization framework has provided a way to retract any or all of the three common assumptions underlying the conventional Generalized Linear Least Squares (GLLS) optimization method based on the concepts introduced in Ref. [2]. These assumptions are: 1. Perfection: The model used for data evaluation and the prior probability distribution function (PDF) of generalized data are perfect. 2. Normality: The prior and posterior PDF are normal. 3. Linearity: The model is linear. In this work we outline how the framework in [1] could be directly adopted for improved evaluation of nuclear criticality integral benchmark experiments (IBEs) by: 1. Removing the first assumption alone by utilizing the concept of imperfections introduced in [1] to enable evaluation in the presence of discrepancies between the data and model or of missing covariance information by a GLLS method that will be seen as a generalization of the conventional GLLS method employed by the TSURFER code, and by 2. Removing the remaining two assumptions by implementing a Markov Chain Monte Carlo method for computation of the posterior PDF in the SAMPLER code, where TSURFER and SAMPLER are the uncertainty quantification (UQ) codes for IBEs in the SCALE code system based on the GLLS and the stochastic method, respectively. The graphic in Figure 1 categorizes the methods discussed in terms of the assumptions that they employ to determine posterior PDFs.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Generalized Bayesian Framework for Evaluation of Integral Benchmark Experiments

A recently published generalized Bayesian optimization framework has provided a way to retract any or all of the three common assumptions underlying the conventional Generalized Linear Least Squares (GLLS) optimization method based on the concepts introduced in reference two. These assumptions are: 1. Perfection: The model used for data evaluation and the prior probability distribution function (PDF) of generalized* data are perfect; 2. Normality: The prior and posterior PDF are normal; and 3. Linearity: The model is linear. In this work we outline how the framework in 1 could be directly adopted for improved evaluation of nuclear criticality integral benchmark experiments (IBEs) by: 1. Removing the first assumption alone by utilizing the concept of imperfections introduced in 1 to enable evaluation in the presence of discrepancies between the data and model or of missing covariance information by a GLLS method that will be seen as a generalization of the conventional GLLS method employed by the TSURFER code, and by 2. Removing the remaining two assumptions by implement- ing a Markov Chain Monte Carlo method for computation of the posterior PDF in the SAMPLER code, where TSURFER and SAMPLER are the uncertainty quantification (UQ) codes for IBEs in the SCALE code system based on the GLLS, and the stochastic method, respectively.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

PERIODIC RADIO AND H{alpha} EMISSION FROM THE L DWARF BINARY 2MASSW J0746425+200032: EXPLORING THE MAGNETIC FIELD TOPOLOGY AND RADIUS OF AN L DWARF

We present an 8.5 hr simultaneous radio, X-ray, UV, and optical observation of the L dwarf binary 2MASSW J0746425+200032. We detect strong radio emission, dominated by short-duration periodic pulses at 4.86 GHz with P = 124.32 {+-} 0.11 min. The stability of the pulse profiles and arrival times demonstrates that they are due to the rotational modulation of a B {approx} 1.7 kG magnetic field. A quiescent nonvariable component is also detected, likely due to emission from a uniform large-scale field. The H{alpha} emission exhibits identical periodicity, but unlike the radio pulses it varies sinusoidally and is offset by exactly 1/4 of a phase. The sinusoidal variations require chromospheric emission from a large-scale field structure, with the radio pulses likely emanating from the magnetic poles. While both light curves can be explained by a rotating misaligned magnetic field, the 1/4 phase lag rules out a symmetric dipole topology since it would result in a phase lag of 1/2 (poloidal field) or zero (toroidal field). We therefore conclude that either (1) the field is dominated by a quadrupole configuration, which can naturally explain the 1/4 phase lag; or (2) the H{alpha} and/or radio emission regions are not trivially aligned with the field. Regardless of the field topology, we use the measured period along with the known rotation velocity (v sin i {approx} 27 km s{sup -1}), and the binary orbital inclination (i {approx} 142 deg.), to derive a radius for the primary star of 0.078 {+-} 0.010 R {sub sun}. This is the first measurement of the radius of an L dwarf, and along with a mass of 0.085 {+-} 0.010 M {sub sun} it provides a constraint on the mass-radius relation below 0.1 M {sub sun}. We find that the radius is about 30% smaller than expected from theoretical models, even for an age of a few Gyr. The origin of this discrepancy is either a breakdown of the models at the bottom of the main sequence, or a significant misalignment between the rotational and orbital axes.

99 GENERAL AND MISCELLANEOUS↗

Investigating the effect of oxy-fuel combustion and light coal volatiles interaction: A mass spectrometric study

Given the multi-physical nature of coal combustion, the development and validation of detailed chemical models reproducing coal volatiles combustion under oxy-fuel conditions is a crucial step towards the advancement of predictive full-scale simulations. During the devolatilization process, a large variety of gases is released and undergoes secondary pyrolysis and oxidation reactions. Therefore, the ability to capture their interactions is a prerequisite for each chemical model used in its detailed or reduced form to simulate these processes. In this work, a high-resolution time-of-flight molecular-beam mass spectrometer was employed to enable fast and simultaneous detection of stable and unstable species in counterflow flames of typical light volatiles. Following an approach of increasing complexity, carbon dioxide and methane were progressively added to an argon diluted acetylene base flame. For the three flames investigated here, results showed a significant increase in the concentration of C 2 and C 3 hydrocarbons and oxygenated compounds caused by methane addition to the acetylene flame. By hindering the production of the butadienyl radical, the addition of methane induces the reduction of benzene which triggers the decrease of aromatic species. Conversely, CO 2 addition did not have significant effects on intermediates. To guide and interpret the measurements, numerical simulations with two existing chemical models were performed and the results were found to be consistent with the experimental data for small hydrocarbons. Some discrepancies were found between the two model predictions and between simulations and experiments for C 4 and C 5 species. In addition, numerical simulations were found to overestimate the role of the methyl radical in aromatics formation.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

YAHFC: A Code Framework to Model Nuclear Reactions and Estimate Correlated Uncertainties

Reaction modeling is a key ingredient in designing experiments and interpreting their results, and is an essential component in the process of evaluating nuclear data and assembling nuclear data libraries used in nuclear technology applications. Typically, experimental data are available only for a handful of reaction channels and theory models are used to fill in the gaps. In addition, theory is often called upon as the arbitrator between discrepant data. Most importantly, theory and modeling are required for an accurate determination of uncertainties in the evaluated data and the correlations between the multiple channels. A fast, accurate, and flexible modeling capability has been developed at LLNL with the code system YAHFC (Yet Another Hauser-Feshbach Code). YAHFC is a Monte Carlo, Hauser-Feshbach code framework, making full use of dynamic memory allocation, derived types, and parallel computing. YAHFC can generate events to simulate experiments and is guiding experiments designed to measure inelastic neutron scattering from actinide targets. YAHFC is also being used to analyze decays from surrogate experiments, thereby enabling the inference of reaction cross sections inaccessible by direct measurement. Finally, by modeling nuclear reactions with constraints from experimental data, YAHFC can deliver complete nuclear data libraries, with evaluated uncertainties, using the modernized Generalized Nuclear Data Structure (GNDS).

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Verification, Validation, and Calibration Through a Causal Lens

While typical validation and verification approaches focus on identifying the associations between data elements using statistical and machine learning methods, the novel methods in this paper focus instead on identifying causal relationships between data elements. Statistical and machine-learning-based approaches are strictly data-driven, meaning that they provide quantitative comparison measures between data sets without explicitly considering the hypotheses behind them. This can lead to the erroneous conclusion that, if two data sets are close enough, the models that generated them are similar. In addition, when experimental and simulated data differ to an extent that fails to meet the acceptance criteria, calibration techniques are used to tweak simulation model parameters to reduce the gap between the two types of data. This produces the false expectation that a simulation model will match reality. The methods presented in this paper move away from these strictly data-driven methods for validation and calibration toward more robust, model-driven methods based on causal inference. Causal inference aims to identify the possible mechanisms that might have generated data. Thus, this analysis targets the prediction of the effects when one (or more) of the identified mechanisms are altered. There are many approaches to identify, quantify, and illustrate causal relationships. For the scope of this paper, directed graphs are employed as causal models. If the directed graph lacks cycles, it is known as a directed acyclic graph. A node in such a graph represents an observed data element while a directed edge connecting two nodes represents a causal relationship between two variables. The developed causal methods are designed to extract causal models from simulation models and experimental data. Causal models capture the causal relationships between data elements (e.g., simulated and experimental data). In this context, validation and verification are performed by comparing causal models. The proposed approach does not only inform system analysts on how a simulation model matches real-world data, but also identifies elements of the simulation model that should be revised when discrepancies between simulation and experimental data are observed. Through these causal methods, analysts can identify the portion of the model equation(s) that are behind an edge connecting two variables. Hence, once the structural differences between causal models have been determined, model calibration can occur by changing only those model parameters that impact the identified causal relationships.

97 MATHEMATICS AND COMPUTING↗