Search NASA⌕ Search

SEARCH · Search NASA

Results for “Validation Studies”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
NASA NTRS2019

Blind Study Validating Parametric Costing Tools PRICE True Planning and SEER-H for NASA Science Missions

Two of the primary parametric costing tools used to estimate the development and production cost of future spacecraft hardware are PRICE True Planning - Space Missions Catalog by PRICE Systems, and SEER-H by Galorath. These are standard tools used by NASA and industry to estimate the cost of new aerospace hardware. However, no independent verification of the accuracy of these tools is publicly available. Both PRICE Systems and Galorath have completed internal validation studies of their parametric cost estimating tools; however, they only provided the results of the studies and did not detail the exact methods used to perform the validation. In the present study, cost estimators used PRICE True Planning and SEER-H to estimate the cost of twelve different past NASA science missions. The estimators were prevented from knowing the actual cost of the missions in an effort to minimize cognitive biases. In the present study, SEER had an average error of 23%, median error of -0.3%, with a standard deviation of 43%. PRICE had an average error of 52%, median error of 50%, and standard deviation of 45%. There were several factors independent of PRICE and SEER which may have affected the accuracy of the results in the present study including: uncertainty in the technical data used for the estimates, the methods used to estimate uncertainty in spacecraft component mass and numbers of prototypes, and the experience of the estimators.

Friz, Paul D.↗
NASA NTRS2018

Blind Study Validating Parametric Costing Tools PRICE TruePlanning and SEER-H for NASA Science Missions

Two of the primary parametric costing tools used to estimate the development and production cost of future spacecraft hardware are PRICE TruePlanning - Space Missions Catalog by PRICE Systems, and SEER-H by Galorath. These are standard tools used by NASA and industry to estimate the cost of new aerospace hardware. However, no independent verification of the accuracy of these tools is publicly available. Both PRICE Systems and Galorath have completed internal validation studies of their parametric cost estimating tools; however, they only provided the results of the studies and did not detail the exact methods used to perform the validation. In the present study, cost estimators used PRICE TruePlanning and SEER-H to estimate the cost of twelve different past NASA science missions. The estimators were prevented from knowing the actual cost of the missions in an effort to minimize cognitive biases. In the present study, SEER had an average error of 23%, median error of -0.3%, with a standard deviation of 43%. PRICE had an average error of 52%, median error of 50%, and standard deviation of 45%. There were several factors independent of PRICE and SEER which may have affected the accuracy of the results in the present study including: uncertainty in the technical data used for the estimates, the methods used to estimate uncertainty in spacecraft component mass and numbers of prototypes, and the experience of the estimators.

Paul D Friz↗
DOE OSTI2024

High-Fidelity Numerical Wave Tank Verification & Validation Study: Wave Generation Through Paddle Motion: Preprint

This paper presents a numerical benchmark study of wave propagation due to a paddle motion using different high-fidelity numerical models, which are capable of replicating the nearly actual physical wave tank testing. A full time series of the measured wave generation paddle motion which was used to generate wave propagation in the physical wave tank will be utilized in each of the models contributed by IEA OES Task 10's participants, which includes both computational fluid dynamics (CFD) and smooth hydrodynamic particle (SPH). The high-fidelity simulations of the physical wave testcase will allow for the evaluation of the initial transient effects from wave ramp-up and its evolution in the wave tank over time for two representative regular waves with varying levels of nonlinearity. A couple of interesting metrics like the predicted wave surface elevation at select wave probes, wave period, and phase-shift in time will be assessed to evaluate the relative accuracy of numerical models versus experimental data within specified time intervals. These models will serve as a guide for modelers in the wave energy community and provide a base case to allow further and more detailed numerical modeling of the fixed Kramer Sphere Cases under wave excitation force wave tank testing.

HYDRO ENERGY,TIDAL AND WAVE POWER↗
DOE OSTI2024

SCEC/USGS Community Stress-Drop Validation Study: How Spectral Fitting Approaches Influence Measured Source Parameters

Spectral source parameters used to estimate an earthquake’s stress drop (⁠Δσ⁠) can vary significantly across measurement approaches. The Statewide California Earthquake Center/U.S. Geological Survey Community Stress‐Drop Validation Study was initiated to compare source parameter estimates, focusing initially on a dataset from the 2019 Ridgecrest earthquake sequence. As part of that validation effort, here we focus on one potential source of uncertainty: whether spectral fitting approaches alone, applied to a common set of spectra from the 2019 Ridgecrest sequence result in different source parameter estimates. By using a common set of benchmark spectra analyzed across a consistent frequency band of 1–40 Hz, we eliminate many sources of variability. A subgroup of validation study participants volunteered to estimate the low‐frequency displacement (⁠Ω 0 ⁠) and corner frequency (⁠ƒ c ⁠) by fitting a smooth function to benchmark displacement spectra. Participants used linear‐ or log‐sampled spectra, assumed a Brune or Boatwright spectral model, and applied different misfit criteria. We compare 17 approaches used to estimate ⁠Ω 0 ⁠, ƒ c ⁠, and Δσ for 54 earthquake spectra. Our results reveal that 35% of events have Δσ estimates within a factor of two, whereas others exhibit variations exceeding an order of magnitude. The variability in and can largely be attributed to whether a spectrum is consistent with the smooth function of an idealized simple crack model. The trade‐off between Ω 0 and ƒ c may be more pronounced when using linearly sampled spectra, as higher frequency spectral bumps control the fits. As expected, methods that assumed a Boatwright model tended to have lower Ω 0 and somewhat higher ƒ c compared to those assuming a Brune model, although resulting Δσ estimates are similar. Finally, when compared to the overall validation study results, the fitting approach alone may account for between 5% and 90% (25% on average) of the total variability in spectral Δσ⁠.

58 GEOSCIENCES↗
NASA NTRSDate not supplied

Blind Validation Study of PRICE TruePlanning and SEER-H

Two of the primary parametric costing tools used to estimate the development and production cost of future spacecraft hardware are PRICE TruePlanning by PRICE Systems, and System Estimation and Evaluation of Resources-Hardware (SEER-H) by Galorath. These are standard tools used by NASA and industry to estimate the cost of new aerospace hardware. This study is an independent verification of the accuracy of these tools and was originally published in 2018. The purpose of this presentation is to circulate the findings of that study among the NASA cost community. Both PRICE Systems and Galorath have completed internal validation studies of their parametric cost estimating tools; however, they only provided the results of the studies and did not detail the exact methods used to perform the validation. In the present study, cost estimators used PRICE TruePlanning and SEER-H to estimate the cost of twelve different past NASA science missions. The estimators were prevented from knowing the actual cost of the missions in an effort to minimize cognitive biases. In the present study, SEER-H had an average error of 23%, median error of -0.3%, with a standard deviation of 43%. PRICE had an average error of 52%, median error of 50%, and standard deviation of 45%. Nine of the twelve mission's actual costs fell within the 80% confidence interval of SEER's probabilistic estimates; however, none of the missions fell within PRICE’s confidence intervals. There were several factors independent of PRICE and SEER-H which may have affected the accuracy of the results in the present study including: uncertainty in the technical data used for the estimates, the methods used to estimate uncertainty in spacecraft component mass and numbers of prototypes, and the experience of the estimators.

SEER↗
NASA NTRS2015

Status of Shock Layer Radiation Validation Studies

This paper presents a status update for the shock layer radiation validation studies conducted at NASA. A review of the present capability for the simulation and validation of shock layer radiation is presented as well as providing an overview of the data obtained from the Electric Arc Shock Tube (EAST). The paper will include details covering updated convective and radiative heating correlations, provide an overview of the development of new kinetics for Mars entry and detail some recent work calculating after-body radiation. Furthermore, the paper will highlight conditions where there is high confidence in the validation of EAST data (e.g. Earth entry for speeds greater than approximately 10 kms and for many Mars entry conditions) and where further experimental data would be highly beneficial (e.g. lower speed Earth entry around 7.5 to 10 kms and higher speed CO2 entries relevant to Venus). Nominal test conditions for both Earth and Mars are provided for future potential facility-to-facility comparisons.

Entry systems modeling↗
NASA NTRS1989

A validation study of four Navier-Stokes codes for high-speed flows

A code validation study has been conducted for four different codes for solving the compressible Navier-Stokes equations. Computations for a series of nominally two-dimensional high-speed laminar separated flows were compared with detailed experimental shock-tunnel results. The shock wave-boundary layer interactions considered were induced by a compression ramp in one case and by an externally-generated incident shock in the second case. In general, good agreement was reached between the grid-refined calculations and experiment for the incipient- and small-separation conditions. For the most highly separated flow, three-dimensional calculations which included the finite-span effects of the experiment were required in order to obtain agreement with the data. The finite-span effects were important in determining the extent of separation as well as the time required to establish the steady-flow interaction. The results presented provide a resolution of discrepancies with the experimental data encountered in several recent computational studies.

Rudy, David H.↗
DOE OSTI2025

Overview of the SCEC/USGS Community Stress Drop Validation Study Using the 2019 Ridgecrest Earthquake Sequence

We present initial findings from the ongoing Community Stress Drop Validation Study to compare spectral stress-drop estimates for earthquakes in the 2019 Ridgecrest, California, sequence. This study uses a unified dataset to independently estimate earthquake source parameters through various methods. Stress drop, which denotes the change in average shear stress along a fault during earthquake rupture, is a critical parameter in earthquake science, impacting ground motion, rupture simulation, and source physics. Spectral stress drop is commonly derived by fitting the amplitude-spectrum shape, but estimates can vary substantially across studies for individual earthquakes. Sponsored jointly by the U.S. Geological Survey and the Statewide (previously, Southern) California Earthquake Center our community study aims to elucidate sources of variability and uncertainty in earthquake spectral stress-drop estimates through quantitative comparison of submitted results from independent analyses. The dataset includes nearly 13,000 earthquakes ranging from M 1 to 7 during a two-week period of the 2019 Ridgecrest sequence, recorded within a 1° radius. Here, in this article, we report on 56 unique submissions received from 20 different groups, detailing spectral corner frequencies (or source durations), moment magnitudes, and estimated spectral stress drops. Methods employed encompass spectral ratio analysis, spectral decomposition and inversion, finite-fault modeling, ground-motion-based approaches, and combined methods. Initial analysis reveals significant scatter across submitted spectral stress drops spanning over six orders of magnitude. However, we can identify between-method trends and offsets within the data to mitigate this variability. Averaging submissions for a prioritized subset of 56 events shows reduced variability of spectral stress drop, indicating overall consistency in recovered spectral stress-drop values.

58 GEOSCIENCES↗
NASA NTRS2001

A semi-automatic method for left ventricle volume estimate: an in vivo validation study

This study aims to the validation of the left ventricular (LV) volume estimates obtained by processing volumetric data utilizing a segmentation model based on level set technique. The validation has been performed by comparing real-time volumetric echo data (RT3DE) and magnetic resonance (MRI) data. A validation protocol has been defined. The validation protocol was applied to twenty-four estimates (range 61-467 ml) obtained from normal and pathologic subjects, which underwent both RT3DE and MRI. A statistical analysis was performed on each estimate and on clinical parameters as stroke volume (SV) and ejection fraction (EF). Assuming MRI estimates (x) as a reference, an excellent correlation was found with volume measured by utilizing the segmentation procedure (y) (y=0.89x + 13.78, r=0.98). The mean error on SV was 8 ml and the mean error on EF was 2%. This study demonstrated that the segmentation technique is reliably applicable on human hearts in clinical practice.

NASA Discipline Cardiopulmonary↗
NASA NTRS1993

Code Validation Study for Base Flows

New and old rocket launch concepts recommend the clustering of motors for improved lift capability. The flowfield of the base region of the rocket is very complex and can contain high temperature plume gases. These hot gases can cause catastrophic problems if not adequately designed for. To assess the base region characteristics, advanced computational fluid dynamics (CFD) is being used. As a precursor to these calculations the CFD code requires validation on base flows. The primary objective of this code validation study was to establish a high level of confidence in predicting base flows with the USA CFD code. USA has been extensively validated for fundamental flows and other applications. However, base heating flows have a number of unique characteristics so it was necessary to extend the existing validation for this class of problems. In preparation for the planned NLS 1.5 Stage base heating analysis, six case sets were studied to extend the USA code validation data base. This presentation gives a cursive review of three of these cases. The cases presented include a 2D axi-symmetric study, a 3D real nozzle study, and a 3D multi-species study. The results of all the studies show good general agreement with data with no adjustments to the base numerical algorithms or physical models in the code. The study proved the capability of the USA code for modeling base flows within the accuracy of available data.

Ascoli, Edward P.↗
NASA NTRS2002

Tree Canopy Characterization for EO-1 Reflective and Thermal Infrared Validation Studies: Rochester, New York

The tree canopy characterization presented herein provided ground and tree canopy data for different types of tree canopies in support of EO-1 reflective and thermal infrared validation studies. These characterization efforts during August and September of 2001 included stem and trunk location surveys, tree structure geometry measurements, meteorology, and leaf area index (LAI) measurements. Measurements were also collected on thermal and reflective spectral properties of leaves, tree bark, leaf litter, soil, and grass. The data presented in this report were used to generate synthetic reflective and thermal infrared scenes and images that were used for the EO-1 Validation Program. The data also were used to evaluate whether the EO-1 ALI reflective channels can be combined with the Landsat-7 ETM+ thermal infrared channel to estimate canopy temperature, and also test the effects of separating the thermal and reflective measurements in time resulting from satellite formation flying.

Ballard, Jerrell R., Jr.↗
DOE OSTI2025

Alaska's Rural Building Stock: a Validation Study Using ResStock and Field Data

The availability of accurate national data on demographics, building stock, and energy use is vital for modeling residential buildings and evaluating decarbonization strategies. However, rural and Indigenous populations, including those in rural Alaska, are typically underrepresented in these datasets. These communities face unique challenges due to their remote locations, severe weather conditions, and limited access to resources, resulting in high energy burden. This report examines how rural Alaskan communities are underrepresented in the ResStock housing model and highlights the need for improved data to address their unique housing and energy challenges. Thus, this report examines the representation of rural Alaskan communities within the national housing stock model, ResStock. A validation study was conducted, considering ResStock, Field Data and Aerial and 3D-view data collection (A3DDC) datasets. The validation process started by using the down selecting approach on the ResStock building stock dataset. For the purpose of this study, only the rural Alaska Boroughs and Census areas located in ASHRAE IECC Climate Zone 8 were considered to ensure a more accurate and fair comparison with the field data, which was collected in rural areas located in climate zone 8, specifically within the Nome Census area. While ResStock may accurately represent several characteristics of the building stock for rural Alaska, some differences between modeled, field data, and aerial and 3D-view data collection datasets were identified. The following building characteristics have a high impact on modeled energy consumption and demonstrated large differences: Revisit heating setpoints and consider a substantially higher setpoint distribution, it could potentially address "missing loads" if this is the case. Develop and include Toyo heating in future modeling for ResStock and EnergyPlus. Remove natural gas as a water heater fuel type outside of North Slope County. Foundation type updated to have more crawlspaces rather than basements. Infiltration rates need reexamination for a larger distribution toward higher infiltration rates. Include more vinyl and less brick in exterior wall type and revise wall color for greater proportion of light rather than dark color. Roof material revised from majority shingles to majority metal. Update number of occupants to higher number of occupant count. Building orientation represents a higher proportion of south facing buildings rather than relatively equal. The findings suggest that updating ResStock's probability logic could better represent rural Alaskan buildings. ResStock can be utilized to identify the best upgrades or energy efficiency and energy efficiency improvements, helping community leaders in making more informed decisions.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
NASA NTRS2019

Blind Validation Study of Parametric Cost Estimation Tool SEER-H for NASA Space Missions

One of the primary parametric cost modeling tools used by NASA and the aerospace industry to estimate the development and production cost of future spacecraft hardware is SEER-H by Galorath. To date, no independent validation of this tool for cost estimation of space missions has been reported in the literature. In the present validation study, cost estimators used SEER-H to estimate the cost of twelve different past NASA science missions. The estimators were prevented from knowing the actual cost of the missions in an effort to minimize cognitive biases. The point estimates of SEER-H had an average error of 23%, median error of -0.3%, and a standard deviation of 43%. Nine of the twelve mission's actual costs fell within the 80% confidence interval of SEER's probabilistic estimates. Several factors independent of SEER that may have affected the accuracy of the results have been identified and are discussed; these include: uncertainty in the technical data used for the estimates, the methods used to estimate uncertainty in spacecraft component mass and numbers of prototypes, and the experience of the estimators.

SEER-H↗
NASA NTRS1974

Simulators' validation study: Problem solution logic

A study was conducted to validate the ground based simulators used for aircraft environment in ride-quality research. The logic to the approach for solving this problem is developed. The overall problem solution flow chart is presented. The factors which could influence the human response to the environment on board the aircraft are analyzed. The mathematical models used in the study are explained. The steps which were followed in conducting the validation tests are outlined.

Schoultz, M. B.↗
NASA NTRS2017

Progress Towards a Microgravity CFD Validation Study Using the ISS SPHERES-SLOSH Experiment

Understanding, predicting, and controlling fluid slosh dynamics is critical to safety and improving performance of space missions when a significant percentage of the spacecrafts mass is a liquid. Computational fluid dynamics simulations can be used to predict the dynamics of slosh, but these programs require extensive validation. Many CFD programs have been validated by slosh experiments using various fluids in earth gravity, but prior to the ISS SPHERES-Slosh experiment1, little experimental data for long-duration, zero-gravity slosh existed. This paper presents the current status of an ongoing CFD validation study using the ISS SPHERES-Slosh experimental data.

CFD↗
NASA NTRS2017

Progress Towards a Microgravity CFD Validation Study Using the ISS SPHERES-SLOSH Experiment

Understanding, predicting, and controlling fluid slosh dynamics is critical to safety and improving performance of space missions when a significant percentage of the spacecrafts mass is a liquid. Computational fluid dynamics simulations can be used to predict the dynamics of slosh, but these programs require extensive validation. Many CFD programs have been validated by slosh experiments using various fluids in earth gravity, but prior to the ISS SPHERES-Slosh experiment, little experimental data for long-duration, zero-gravity slosh existed. This paper presents the current status of an ongoing CFD validation study using the ISS SPHERES-Slosh experimental data.

microgravity↗
NASA NTRS2017

Progress Towards a Microgravity CFD Validation Study Using the ISS SPHERES-SLOSH Experiment

Understanding, predicting, and controlling fluid slosh dynamics is critical to safety and improving performance of space missions when a significant percentage of the spacecrafts mass is a liquid. Computational fluid dynamics simulations can be used to predict the dynamics of slosh, but these programs require extensive validation. Many CFD programs have been validated by slosh experiments using various fluids in earth gravity, but prior to the ISS SPHERES-Slosh experiment1, little experimental data for long-duration, zero-gravity slosh existed. This paper presents the current status of an ongoing CFD validation study using the ISS SPHERES-Slosh experimental data.

Storey, Jedediah M.↗