Search NASA⌕ Search

SEARCH · Search NASA

Results for “Validation Studies”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
NASA NTRS2019

Blind Study Validating Parametric Costing Tools PRICE True Planning and SEER-H for NASA Science Missions

Two of the primary parametric costing tools used to estimate the development and production cost of future spacecraft hardware are PRICE True Planning - Space Missions Catalog by PRICE Systems, and SEER-H by Galorath. These are standard tools used by NASA and industry to estimate the cost of new aerospace hardware. However, no independent verification of the accuracy of these tools is publicly available. Both PRICE Systems and Galorath have completed internal validation studies of their parametric cost estimating tools; however, they only provided the results of the studies and did not detail the exact methods used to perform the validation. In the present study, cost estimators used PRICE True Planning and SEER-H to estimate the cost of twelve different past NASA science missions. The estimators were prevented from knowing the actual cost of the missions in an effort to minimize cognitive biases. In the present study, SEER had an average error of 23%, median error of -0.3%, with a standard deviation of 43%. PRICE had an average error of 52%, median error of 50%, and standard deviation of 45%. There were several factors independent of PRICE and SEER which may have affected the accuracy of the results in the present study including: uncertainty in the technical data used for the estimates, the methods used to estimate uncertainty in spacecraft component mass and numbers of prototypes, and the experience of the estimators.

Friz, Paul D.↗
NASA NTRS2018

Blind Study Validating Parametric Costing Tools PRICE TruePlanning and SEER-H for NASA Science Missions

Two of the primary parametric costing tools used to estimate the development and production cost of future spacecraft hardware are PRICE TruePlanning - Space Missions Catalog by PRICE Systems, and SEER-H by Galorath. These are standard tools used by NASA and industry to estimate the cost of new aerospace hardware. However, no independent verification of the accuracy of these tools is publicly available. Both PRICE Systems and Galorath have completed internal validation studies of their parametric cost estimating tools; however, they only provided the results of the studies and did not detail the exact methods used to perform the validation. In the present study, cost estimators used PRICE TruePlanning and SEER-H to estimate the cost of twelve different past NASA science missions. The estimators were prevented from knowing the actual cost of the missions in an effort to minimize cognitive biases. In the present study, SEER had an average error of 23%, median error of -0.3%, with a standard deviation of 43%. PRICE had an average error of 52%, median error of 50%, and standard deviation of 45%. There were several factors independent of PRICE and SEER which may have affected the accuracy of the results in the present study including: uncertainty in the technical data used for the estimates, the methods used to estimate uncertainty in spacecraft component mass and numbers of prototypes, and the experience of the estimators.

Paul D Friz↗
DOE OSTI2024

Nonsteady Load Responses of Wind Turbines to Atmospheric and Mountain-Generated Turbulence Eddies, With Impacts on the Main Bearing: A Validation Study

Previous computational and field experiments identify three characteristic time scales in the aerodynamic responses of utility-scale wind turbine loads to atmospheric boundary layer (ABL) turbulence: a 30-90 second time scale for the passage of high/low speed "streaks" through the rotor plane, the blade and rotor rotation time scales (approximately 1 to 5 seconds), and a sub-second time scale created by blade rotation through gradients within eddy coherent structure. In the current study we compare aerodynamic load responses from daytime ABL turbulence quantified with large-eddy simulation and a actuator line model of the NREL 5 MW wind turbine with analysis of field data from the NREL/GE 1.5 MW wind turbine 5 kilometers east of the Rocky Mountain Front Range in Colorado. In addition, we contrast the responses to the passage of the mountain-generated eddies embedded within the westerly winds with the ABL eddies embedded within northerly/southerly winds. These analyses are in context with the nonsteady forcing of the main bearing by the aerodynamic generation of nontorque bending moments on the main shaft. Potentially relevant to main bearing failure mechanisms, both computational and field data show that the magnitudes of turbulence-generated nontorque bending moments, that we show generate nonsteady force on the main bearing, are of order, and often larger than, torque (which underlies power). However, the temporal variations in these two responses are uncorrelated, implying that the aerodynamic mechanisms that drive power and main bearing response are fundamentally different. We find this to be the case in the field with both mountain-generated eddies (westerly winds) and ABL-generated eddies (northerly/southerly winds). Whereas the time and length scales are comparable, the mountain eddies were somewhat more energetic than the northerly/southerly ABL eddies. Interestingly, however, the fluctuations in nontorque bending moment that force the main bearing were found to be stronger when forced by the ABL eddies than the mountain eddies. The field studies validate the key results from the computational study and show even stronger response in the nontorque bending moment than in the computer simulations. In all cases, the torque and nontorque bending moments are temporally uncorrelated, torque and power are driven by time variations in rotor-averaged horizontal wind velocity and nontorque bending moments are driven by time changes in the degree of nonuniformity in the distribution of velocity over the rotor plane. Thus the results generalize the mechanisms underlying nonsteady aerodynamic forcing to classes of turbulence eddy types with strength of order or stronger than ABL eddies with transverse scale of order the wind turbine rotor. These include atmospheric turbulence eddies, topography-generated turbulence eddies and, by extension, impacts of turbine-wake-scale turbulence eddies on downstream wind turbine rotors.

17 WIND ENERGY↗
DOE OSTI2023

Validation study of sodium pool fire modeling efforts in $\mathrm{MELCOR}$ and $\mathrm{SPHINCS}$ codes

Discharge of sodium coolant into containment from a sodium-cooled fast reactor vessel can occur in the event of a pipe leak or break. In this situation, some of the liquid sodium droplets discharged from the coolant system will react with oxygen in the air before reaching the containment. This phase of the event is normally termed the sodium spray fire phase. Unreacted sodium droplets pool on the containment floor where continued reaction with containment atmospheric oxygen occurs. This phase of the event is normally termed the sodium pool fire phase. Both phases of these sodium-oxygen reactions (or fires) are important to model because of the heat addition and aerosol generation that occur. Any fission products trapped in the sodium coolant may also be released during this progression of events, which if released from containment could pose a health risk to workers and the public. The paper describes progress of an international collaborative research in the area of the sodium fire modeling in the sodium-cooled fast reactors between the United States and Japan under the framework of the Civil Nuclear Energy Research and Development Working Group. In this collaboration between Sandia National Laboratories and Japan Atomic Energy Agency, the validation basis for and modeling capabilities of sodium spray and pool fires in MELCOR of Sandia National Laboratories and SPHINCS of Japan Atomic Energy Agency are being enhanced. Here this study documents MELCOR and SPHINCS sodium pool fire model validation exercises against the JAEA’s sodium pool fire experiments, F7-1 and F7-2. The proposed enhancement of the sodium pool fire models in MELCOR through addition of thermal hydraulic and sodium spreading models that enable a better representation of experimental results is also described.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2021

Modeling heat transport processes in enhanced geothermal systems: Validation study from EGS Collab Experiment 1

Heat recovery from enhanced geothermal systems (EGS) is a complex process involving heat transport in both fracture networks and rock formations. A comprehensive understanding of and the ability to model the underlying heat transport mechanisms is important for the success of EGS but remains challenging in practice due to the generally insufficient characterization of EGS reservoirs. In the present study, we analyze an extensively monitored intermediate-scale EGS field experiment performed in a well-characterized testbed. The high-resolution, high-quality measurements from the field experiment enable the development of a high-fidelity model incorporating a well-constrained fracture network. Based on the field experiment, we investigate the complex heat transport processes in an EGS-relevant environment and validate the capability of a numerical approach in simulating these inherently coupled heat transport processes. A series of numerical simulations were performed to study the effects of different heat transport mechanisms, including thermal convection with fracture flow, thermal conduction in rock formations, and the Joule-Thomson effect. The agreement of thermal responses between field measurements and simulation results indicates that our numerical approach can appropriately model the heat transport processes pertaining to heat recovery from EGS reservoirs.

Wu, Hui↗
DOE OSTI2024

High-Fidelity Numerical Wave Tank Verification & Validation Study: Wave Generation Through Paddle Motion: Preprint

This paper presents a numerical benchmark study of wave propagation due to a paddle motion using different high-fidelity numerical models, which are capable of replicating the nearly actual physical wave tank testing. A full time series of the measured wave generation paddle motion which was used to generate wave propagation in the physical wave tank will be utilized in each of the models contributed by IEA OES Task 10's participants, which includes both computational fluid dynamics (CFD) and smooth hydrodynamic particle (SPH). The high-fidelity simulations of the physical wave testcase will allow for the evaluation of the initial transient effects from wave ramp-up and its evolution in the wave tank over time for two representative regular waves with varying levels of nonlinearity. A couple of interesting metrics like the predicted wave surface elevation at select wave probes, wave period, and phase-shift in time will be assessed to evaluate the relative accuracy of numerical models versus experimental data within specified time intervals. These models will serve as a guide for modelers in the wave energy community and provide a base case to allow further and more detailed numerical modeling of the fixed Kramer Sphere Cases under wave excitation force wave tank testing.

HYDRO ENERGY,TIDAL AND WAVE POWER↗
DOE OSTI2024

SCEC/USGS Community Stress-Drop Validation Study: How Spectral Fitting Approaches Influence Measured Source Parameters

Spectral source parameters used to estimate an earthquake’s stress drop (⁠Δσ⁠) can vary significantly across measurement approaches. The Statewide California Earthquake Center/U.S. Geological Survey Community Stress‐Drop Validation Study was initiated to compare source parameter estimates, focusing initially on a dataset from the 2019 Ridgecrest earthquake sequence. As part of that validation effort, here we focus on one potential source of uncertainty: whether spectral fitting approaches alone, applied to a common set of spectra from the 2019 Ridgecrest sequence result in different source parameter estimates. By using a common set of benchmark spectra analyzed across a consistent frequency band of 1–40 Hz, we eliminate many sources of variability. A subgroup of validation study participants volunteered to estimate the low‐frequency displacement (⁠Ω 0 ⁠) and corner frequency (⁠ƒ c ⁠) by fitting a smooth function to benchmark displacement spectra. Participants used linear‐ or log‐sampled spectra, assumed a Brune or Boatwright spectral model, and applied different misfit criteria. We compare 17 approaches used to estimate ⁠Ω 0 ⁠, ƒ c ⁠, and Δσ for 54 earthquake spectra. Our results reveal that 35% of events have Δσ estimates within a factor of two, whereas others exhibit variations exceeding an order of magnitude. The variability in and can largely be attributed to whether a spectrum is consistent with the smooth function of an idealized simple crack model. The trade‐off between Ω 0 and ƒ c may be more pronounced when using linearly sampled spectra, as higher frequency spectral bumps control the fits. As expected, methods that assumed a Boatwright model tended to have lower Ω 0 and somewhat higher ƒ c compared to those assuming a Brune model, although resulting Δσ estimates are similar. Finally, when compared to the overall validation study results, the fitting approach alone may account for between 5% and 90% (25% on average) of the total variability in spectral Δσ⁠.

58 GEOSCIENCES↗
DOE OSTI2023

The Importance of Addressing Disagreements Between Nominal and Effective Treatments During Bat Mortality Minimization Validation Studies

As the wind energy industry grows, so too does our need for effective and low-cost bat mortality minimization solutions. Despite knowledge gaps in our understanding of what drives bats to collide with spinning turbines, minimization solutions have shown success during validation studies. Curtailment has been consistently effective, reducing bat mortality from 33%-79%, depending on the curtailment scenario and species present. Several validation studies have demonstrated deterrent solutions, specifically ultrasonic deterrents, to be effective, but overall, more variable compared to curtailment solutions. Further, a study combining curtailment with ultrasonic deterrents produced encouraging results, finding that adding ultrasonic deterrents to turbines that were designated to curtail, significantly reduced mortality rates compared to curtailment only turbines. Despite positive results, the strength of inference achieved with validation studies is limited by methodological constraints associated with appropriately assigning fatalities to treatments. Because of the temporal separation between when a bat collides with a turbine and when it is discovered during a ground-based carcass survey, we must assign carcasses to a treatment that ran during the prior night(s). This process may introduce errors at multiple stages. First, for studies that rotate treatments among turbines, mortality surveyors must be confident that carcasses are 'fresh' such that mortalities are correctly assigned to the treatment from the previous night(s). Second, we must reconcile any misalignments between how we assign treatments and how we implement treatments (e.g., turbines assigned with a deterrent treatment may be implemented as another treatment when deterrent devices are not operating as designed). Finally, for curtailment solutions, it is critical to recognize that measured effects are a function of not just the treatment as implemented but the proportion of the night the treatment is realized (e.g., if wind speeds are greater than the curtailment treatment cut-in speed for the entire night, we cannot expect there to me any differences in mortality reduction relative to control turbines). Using a dataset collected between June and October 2017, that rotated 3 treatments (Deterrent Only, 5 m/s Curtail only, and 5 m/s Curtail & Deterrent) and 1 control condition across 16 turbines each night, we explored the importance and implications of accounting for potential errors in assigning fatalities to treatments. We present preliminary results comparing the mortality associated with error-corrected treatments and control conditions highlighting how the measured effect of a treatment (mortality) greatly depends on site specific implementation.

bats↗
NASA NTRSDate not supplied

Blind Validation Study of PRICE TruePlanning and SEER-H

Two of the primary parametric costing tools used to estimate the development and production cost of future spacecraft hardware are PRICE TruePlanning by PRICE Systems, and System Estimation and Evaluation of Resources-Hardware (SEER-H) by Galorath. These are standard tools used by NASA and industry to estimate the cost of new aerospace hardware. This study is an independent verification of the accuracy of these tools and was originally published in 2018. The purpose of this presentation is to circulate the findings of that study among the NASA cost community. Both PRICE Systems and Galorath have completed internal validation studies of their parametric cost estimating tools; however, they only provided the results of the studies and did not detail the exact methods used to perform the validation. In the present study, cost estimators used PRICE TruePlanning and SEER-H to estimate the cost of twelve different past NASA science missions. The estimators were prevented from knowing the actual cost of the missions in an effort to minimize cognitive biases. In the present study, SEER-H had an average error of 23%, median error of -0.3%, with a standard deviation of 43%. PRICE had an average error of 52%, median error of 50%, and standard deviation of 45%. Nine of the twelve mission's actual costs fell within the 80% confidence interval of SEER's probabilistic estimates; however, none of the missions fell within PRICE’s confidence intervals. There were several factors independent of PRICE and SEER-H which may have affected the accuracy of the results in the present study including: uncertainty in the technical data used for the estimates, the methods used to estimate uncertainty in spacecraft component mass and numbers of prototypes, and the experience of the estimators.

SEER↗
NASA NTRS2015

Status of Shock Layer Radiation Validation Studies

This paper presents a status update for the shock layer radiation validation studies conducted at NASA. A review of the present capability for the simulation and validation of shock layer radiation is presented as well as providing an overview of the data obtained from the Electric Arc Shock Tube (EAST). The paper will include details covering updated convective and radiative heating correlations, provide an overview of the development of new kinetics for Mars entry and detail some recent work calculating after-body radiation. Furthermore, the paper will highlight conditions where there is high confidence in the validation of EAST data (e.g. Earth entry for speeds greater than approximately 10 kms and for many Mars entry conditions) and where further experimental data would be highly beneficial (e.g. lower speed Earth entry around 7.5 to 10 kms and higher speed CO2 entries relevant to Venus). Nominal test conditions for both Earth and Mars are provided for future potential facility-to-facility comparisons.

Entry systems modeling↗
DOE OSTI2024

Validation Study of Experimental Insulating and Air-Sealing Technology for Enclosed Roof Cavities

Validation Study of Experimental Insulating and Air-Sealing Technology for Enclosed Roof Cavities, explores a new minimally invasive system known as Pinhole Insulation ® for retroft insulation of cathedral ceilings, flat roofs, dormer roofs, and other enclosed roof cavities. It also explores the validation of the associated equipment, software, and materials used in this application. The Pinhole Insulation process essentially consists of drilling a ¼” hole in the ceiling surface and then injecting closed cell polyurethane foam such that the foam is deposited on the interior surface of the exterior roof sheathing. In a typical cathedral ceiling with R19 batts, the R value will be improved by at least 60% and the cavity will become completely air sealed. The system is quick, minimally invasive and at least an order of magnitude less expensive than alternatives such as removing the ceiling surface and spraying closed cell foam.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
NASA NTRS1989

A validation study of four Navier-Stokes codes for high-speed flows

A code validation study has been conducted for four different codes for solving the compressible Navier-Stokes equations. Computations for a series of nominally two-dimensional high-speed laminar separated flows were compared with detailed experimental shock-tunnel results. The shock wave-boundary layer interactions considered were induced by a compression ramp in one case and by an externally-generated incident shock in the second case. In general, good agreement was reached between the grid-refined calculations and experiment for the incipient- and small-separation conditions. For the most highly separated flow, three-dimensional calculations which included the finite-span effects of the experiment were required in order to obtain agreement with the data. The finite-span effects were important in determining the extent of separation as well as the time required to establish the steady-flow interaction. The results presented provide a resolution of discrepancies with the experimental data encountered in several recent computational studies.

Rudy, David H.↗
DOE OSTI2025

Overview of the SCEC/USGS Community Stress Drop Validation Study Using the 2019 Ridgecrest Earthquake Sequence

We present initial findings from the ongoing Community Stress Drop Validation Study to compare spectral stress-drop estimates for earthquakes in the 2019 Ridgecrest, California, sequence. This study uses a unified dataset to independently estimate earthquake source parameters through various methods. Stress drop, which denotes the change in average shear stress along a fault during earthquake rupture, is a critical parameter in earthquake science, impacting ground motion, rupture simulation, and source physics. Spectral stress drop is commonly derived by fitting the amplitude-spectrum shape, but estimates can vary substantially across studies for individual earthquakes. Sponsored jointly by the U.S. Geological Survey and the Statewide (previously, Southern) California Earthquake Center our community study aims to elucidate sources of variability and uncertainty in earthquake spectral stress-drop estimates through quantitative comparison of submitted results from independent analyses. The dataset includes nearly 13,000 earthquakes ranging from M 1 to 7 during a two-week period of the 2019 Ridgecrest sequence, recorded within a 1° radius. Here, in this article, we report on 56 unique submissions received from 20 different groups, detailing spectral corner frequencies (or source durations), moment magnitudes, and estimated spectral stress drops. Methods employed encompass spectral ratio analysis, spectral decomposition and inversion, finite-fault modeling, ground-motion-based approaches, and combined methods. Initial analysis reveals significant scatter across submitted spectral stress drops spanning over six orders of magnitude. However, we can identify between-method trends and offsets within the data to mitigate this variability. Averaging submissions for a prioritized subset of 56 events shows reduced variability of spectral stress drop, indicating overall consistency in recovered spectral stress-drop values.

58 GEOSCIENCES↗
NASA NTRS2001

A semi-automatic method for left ventricle volume estimate: an in vivo validation study

This study aims to the validation of the left ventricular (LV) volume estimates obtained by processing volumetric data utilizing a segmentation model based on level set technique. The validation has been performed by comparing real-time volumetric echo data (RT3DE) and magnetic resonance (MRI) data. A validation protocol has been defined. The validation protocol was applied to twenty-four estimates (range 61-467 ml) obtained from normal and pathologic subjects, which underwent both RT3DE and MRI. A statistical analysis was performed on each estimate and on clinical parameters as stroke volume (SV) and ejection fraction (EF). Assuming MRI estimates (x) as a reference, an excellent correlation was found with volume measured by utilizing the segmentation procedure (y) (y=0.89x + 13.78, r=0.98). The mean error on SV was 8 ml and the mean error on EF was 2%. This study demonstrated that the segmentation technique is reliably applicable on human hearts in clinical practice.

NASA Discipline Cardiopulmonary↗
NASA NTRS1993

Code Validation Study for Base Flows

New and old rocket launch concepts recommend the clustering of motors for improved lift capability. The flowfield of the base region of the rocket is very complex and can contain high temperature plume gases. These hot gases can cause catastrophic problems if not adequately designed for. To assess the base region characteristics, advanced computational fluid dynamics (CFD) is being used. As a precursor to these calculations the CFD code requires validation on base flows. The primary objective of this code validation study was to establish a high level of confidence in predicting base flows with the USA CFD code. USA has been extensively validated for fundamental flows and other applications. However, base heating flows have a number of unique characteristics so it was necessary to extend the existing validation for this class of problems. In preparation for the planned NLS 1.5 Stage base heating analysis, six case sets were studied to extend the USA code validation data base. This presentation gives a cursive review of three of these cases. The cases presented include a 2D axi-symmetric study, a 3D real nozzle study, and a 3D multi-species study. The results of all the studies show good general agreement with data with no adjustments to the base numerical algorithms or physical models in the code. The study proved the capability of the USA code for modeling base flows within the accuracy of available data.

Ascoli, Edward P.↗
NASA NTRS2002

Tree Canopy Characterization for EO-1 Reflective and Thermal Infrared Validation Studies: Rochester, New York

The tree canopy characterization presented herein provided ground and tree canopy data for different types of tree canopies in support of EO-1 reflective and thermal infrared validation studies. These characterization efforts during August and September of 2001 included stem and trunk location surveys, tree structure geometry measurements, meteorology, and leaf area index (LAI) measurements. Measurements were also collected on thermal and reflective spectral properties of leaves, tree bark, leaf litter, soil, and grass. The data presented in this report were used to generate synthetic reflective and thermal infrared scenes and images that were used for the EO-1 Validation Program. The data also were used to evaluate whether the EO-1 ALI reflective channels can be combined with the Landsat-7 ETM+ thermal infrared channel to estimate canopy temperature, and also test the effects of separating the thermal and reflective measurements in time resulting from satellite formation flying.

Ballard, Jerrell R., Jr.↗