SEARCH · Search NASA
Results for “model validation”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Model validation and error attribution for a drifting qubit
Qubit performance is often reported in terms of a variety of single-value metrics, each providing a facet of the underlying noise mechanism limiting performance. However, the value of these metrics may drift over long timescales, and reporting a single number for qubit performance fails to account for the low-frequency noise processes that give rise to this drift. Here, in this work, we demonstrate how we can use the distribution of these values to validate or invalidate candidate noise models. We focus on the case of randomized benchmarking (RB), where typically a single error rate is reported but this error rate can drift over time when multiple passes of RB are performed. We show that using a statistical test as simple as the Kolmogorov-Smirnov statistic on the distribution of RB error rates can be used to rule out noise models, assuming the experiment is performed over a long enough time interval to capture relevant low frequency noise. With confidence in a noise model, we show how care must be exercised when performing error attribution using the distribution of drifting RB error rate.
CFD Jet Mixing Model Validation against Zero-Boil-Off Tank (ZBOT) Microgravity Experiment
The Zero-Boil-Off Tank (ZBOT) experiments are a series of small scale two-phase fluids physics experiments conducted aboard the International Space Station (ISS) in 2017. A goal of these experiments is to study various fundamental fluid flow, heat and mass transport, and phase change phenomena that control storage tank pressurization and pressure control in microgravity. In this paper, results of a state-of-the-art two-phase CFD storage tank model are presented and validated against jet mixing data provided by microgravity ZBOT experiments. Jet-ullage interaction was studied in the experiment for different jet speeds representing different flow regimes, and tank fill levels. Effects of various physical parameters, such as, jet angle, gravitational acceleration and surface tension value were investigated. Computational parameters, such as turbulence and surface tension models and interface capturing scheme, affecting jet-ullage interaction were also studied. Computational results were compared to experimental results for model validation.
CFD Jet Mixing Model Validation Against Zero-Boil-Off Tank (ZBOT) Microgravity Experiment
The Zero-Boil-Off Tank (ZBOT) experiments are a series of small scale two-phase fluids physics experiments conducted aboard the International Space Station (ISS) in 2017. A goal of these experiments is to study various fundamental fluid flow, heat and mass transport, and phase change phenomena that control storage tank pressurization and pressure control in microgravity. In this paper, results of a state-of-the-art two-phase CFD (Computational Fluid Dynamics) storage tank model are presented and validated against jet mixing data provided by microgravity ZBOT experiments. Jet-ullage interaction was studied in the experiment for different jet speeds representing different flow regimes, and tank fill levels. Effects of various physical parameters, such as, jet angle, gravitational acceleration and surface tension value were investigated. Computational parameters, such as turbulence and surface tension models and interface capturing scheme, affecting jet-ullage interaction were also studied. Computational results were compared to experimental results for model validation.
Revisiting Turbulence Model Validation for High-Mach Number Axisymmetric Compression Corner Flows
Two axisymmetric shock-wave/boundary-layer interaction (SWBLI) cases are used to benchmark one- and two-equation Reynolds-averaged Navier-Stokes (RANS) turbulence models. This validation exercise was executed in the philosophy of the NASA Turbulence Modeling Resource and the AIAA Turbulence Model Benchmarking Working Group. Both SWBLI cases are from the experiments of Kussoy and Horstman for axisymmetric compression corner geometries with SWBLI inducing flares of 20 and 30 degrees, respectively. The freestream Mach number was approximately 7. The RANS closures examined are the Spalart-Allmaras one-equation model and the Menter family of kappa − omega two equation models including the Baseline and Shear Stress Transport formulations. The Wind-US and CFL3D RANS solvers are employed to simulate the SWBLI cases. Comparisons of RANS solutions to experimental data are made for a boundary layer survey plane just upstream of the SWBLI region. In the SWBLI region, comparisons of surface pressure and heat transfer are made. The effects of inflow modeling strategy, grid resolution, grid orthogonality, turbulent Prandtl number, and code-to-code variations are also addressed.
Radiative transfer model validations during the First ISLSCP Field Experiment
Two simple radiative transfer models, the 5S model based on Tanre et al. (1985, 1986) and the wide-band model of Morcrette (1984) are validated by comparing their outputs with results obtained during the First ISLSCP Field Experiment on concomitant radiosonde, aerosol turbidity, and radiation measurements and sky photographs. Results showed that the 5S model overestimates the short-wave irradiance by 13.2 W/sq m, whereas the Morcrette model underestimated the long-wave irradiance by 7.4 W/sq m.
NASA Engineering and Safety Center Technical Bulletin No. 14-03: COPV Mechanical Model Validation
Global and local deformation measurements should be incorporated into the composite overwrapped pressure vessel (COPV) design and analysis process to allow correlation of these measurements with finite element analysis (FEA) models. This correlation improves understanding of liner, liner/overwrap interface, and composite deformation response in COPVs. The improved accuracy reduces error in subsequent analyses, such as fracture, fatigue, and stress rupture that are critical for COPV qualification.
Noninvasive assessment of mitral inertness: clinical results with numerical model validation
Inertial forces (Mdv/dt) are a significant component of transmitral flow, but cannot be measured with Doppler echo. We validated a method of estimating Mdv/dt. Ten patients had a dual sensor transmitral (TM) catheter placed during cardiac surgery. Doppler and 2D echo was performed while acquiring LA and LV pressures. Mdv/dt was determined from the Bernoulli equation using Doppler velocities and TM gradients. Results were compared with numerical modeling. TM gradients (range: 1.04-14.24 mmHg) consisted of 74.0 +/- 11.0% inertial forcers (range: 0.6-12.9 mmHg). Multivariate analysis predicted Mdv/dt = -4.171(S/D (RATIO)) + 0.063(LAvolume-max) + 5. Using this equation, a strong relationship was obtained for the clinical dataset (y=0.98x - 0.045, r=0.90) and the results of numerical modeling (y=0.96x - 0.16, r=0.84). TM gradients are mainly inertial and, as validated by modeling, can be estimated with echocardiography.
PCM-based uncertainty reduction in support of model validation for depletion calculations
Not Available
SPHINX: An SEP Model Validation Infrastructure developed through Community Challenges and the SEP Scoreboards
Solar Energetic Particle (SEP) events are interesting from a scientific perspective as they are the product of a broad set of physical processes from the corona out through the extent of the heliosphere, and provide insight into processes of particle acceleration and transport that are widely applicable in astrophysics. From the operations perspective, SEP events pose a radiation hazard for aviation, electronics in space, and human space exploration, in particular for missions outside of the Earth’s protective magnetosphere including to the Moon and Mars (Whitman et al 2022). For these reasons, SEP modelers have developed a rich and diverse set of models with a wide variety of aims. Some models probe the basic physics at the heart of particle acceleration and transport. Others produce fast statistical forecasts or employ disruptive new techniques like Machine Learning with the goal to assist end users in making operational decisions. To enable a consistent and quantitative understanding of SEP model performance, a generalized, automated validation infrastructure, called SPHINX, is being developed at NASA SRAG in close collaboration with NASA CCMC, NASA M2M, NOAA SWPC, and BIRA-IASB. This infrastructure has been built up through a multi-year community challenge. Starting in 2018 at the SHINE workshop, an effort was launched through SHINE, ISWAT, and ESWW to encourage quantitative, comprehensive, and consistent validation of SEP models. This effort has defined a set of challenge SEP events with the aim of generating quantitative comparisons between forecasts and observations and a set of challenge “non-events” to assess false alarms. In 2023, these challenge lists have been extended to statistically significant numbers with a prescribed set of rules for producing forecasts and supported through the dedicated SEPVAL working meetings. The participation of the research community has allowed the infrastructure to validate all the types of outputs being produced by SEP models. In parallel, the SPHINX code is being applied to real time forecasts submitted to the SEP Scoreboards, ensuring that the validation infrastructure can interpret forecasts produced in an operational scenario and provide metrics meaningful for operations. Upon completion, SPHINX and its interactive user interface, SPHINX-Web, will be made available for public use.
The executive disruption model of tinnitus distress: Model validation in two independent datasets using factor score regression
This study presents the executive disruption model (EDM) of tinnitus distress and subsequently validates it statistically using two independent datasets (the Construction Dataset: n = 96 and the Validation Dataset: n = 200). The conceptual EDM was first operationalised as a structural causal model (construction phase). Then multiple regression was used to examine the effect of executive functioning on tinnitus-related distress (validation phase), adjusting for the additional contributions of hearing threshold and psychological distress. For both datasets, executive functioning negatively predicted tinnitus distress score by a similar amount (the Construction Dataset: β = −3.50, p = 0.13 and the Validation Dataset: β = −3.71, p = 0.02). Theoretical implications and applications of the EDM are subsequently discussed; these include the predictive nature of executive functioning in the development of distressing tinnitus, and the clinical utility of the EDM.
Model Validation and Demonstration of a Hydrogen Fuel Cell Parcel Delivery Truck
With an emphasis on reducing fossil fuel consumption and emissions, major delivery service corporations have begun deploying and operating Class 6 battery electric delivery vans. Fuel cell hybrid delivery vans offer a zero emissions solution with extended range and route planning flexibility. This paper follows a prior publication [1] on the vehicle modeling and design and presents demonstration data and a validation of the prior modeling and design effort. Here, the validation exercise shows that the modeling tools predict performance and energy efficiency within 5% for most of the routes, while the demonstration data shows that in service operation fuel economy was over 14 km/kg of hydrogen gas. In addition, results from a range test showed the vehicle design and onboard hydrogen storage was able to surpass the design target range of 200 km.
A Hardware Model Validation Tool for Use in Complex Space Systems
One of the many technological hurdles that must be overcome in future missions is the challenge of validating as-built systems against the models used for design. We propose a technique composed of intelligent parameter exploration in concert with automated failure analysis as a scalable method for the validation of complex space systems. The technique is impervious to discontinuities and linear dependencies in the data, and can handle dimensionalities consisting of hundreds of variables over tens of thousands of experiments.
NASA Engineering and Safety Center Technical Bulletin No. 14-02: Aerodynamic Reaction Control System (RCS) Orientation and Jet Interaction (JI) Model Validation
Careful consideration should be given to placement and orientation of RCS jet thrusters on hypersonic entry vehicles in order to minimize adverse RCS JI. Modern state-of-the-art computational fluid dynamics (CFD) tools should be able to predict the RCS effectiveness and JI accurately when coupled with verification and validation studies, including the use of appropriately designed experimental wind tunnel testing.
Velocity Discontinuity Propagation Model Validation Using an Approximate Point Source in Motion
The classic geometric acoustic solution for propagation through a mean flow velocity discontinuity is evaluated experimentally using an approximate point source in motion. The geometric approximation of the pressure field in both the frequency and time domains is first revisited for arbitrary subsonic flow and source motion along the flow axis. The derivation, computed via the method of stationary phase, shows the expected Doppler behavior of the radiated acoustic field due to the source motion acting in conjunction with the convective amplification effect for a stationary source in flow. The model is validated with a minimally intrusive, approximate point source of heat in the Quiet Flow Facility at the NASA Langley Research Center. Within the limitations of the experiment in this open-jet wind tunnel, data generally show agreement with the model across a range of flow speeds and microphone positions for a broad range of frequencies. Where disagreement between measurement and theory is noted, possible causes are discussed.
An all-pressure fluid-drop model: validation and species system dependency
A fundamental and very general model of fluid drop behavior at all pressures is presented and is shown to reproduce microgravity experimental data for heptane drops in nitrogen.