Search NASA⌕ Search

SEARCH · Search NASA

Results for “internal validity”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Successful International Validation Test Shipment of the PIP-II HB650 Cryomodule Transportation System

The PIP-II Project will receive fully assembled cryomodules from CEA and STFC-UKRI as in-kind contributions. Damage to these cryomodules during transport is understood to be a significant risk to the project, so an extensive testing and validation program has been executed to mitigate this risk. The centerpiece of this effort was the successful shipment, from FNAL to STFC-UKRI and back, of a prototype HB650 cryomodule with cold testing before and after shipment to verify no functionality changes from shipment. Building on an escalating test transport program, the prototype cryomodule was shipped to the UK and back using realistic logistics, handling, instrumentation, and planning. The process of executing this shipment, lessons learned, and plan moving forward will be presented here.

43 PARTICLE ACCELERATORS↗

Emulator-Based Bayesian Calibration of the CISNET Colorectal Cancer Models

Purpose To calibrate Cancer Intervention and Surveillance Modeling Network (CISNET)'s SimCRC, MISCAN-Colon, and CRC-SPIN simulation models of the natural history colorectal cancer (CRC) with an emulator-based Bayesian algorithm and internally validate the model-predicted outcomes to calibration targets.Methods We used Latin hypercube sampling to sample up to 50,000 parameter sets for each CISNET-CRC model and generated the corresponding outputs. We trained multilayer perceptron artificial neural networks (ANNs) as emulators using the input and output samples for each CISNET-CRC model. We selected ANN structures with corresponding hyperparameters (i.e., number of hidden layers, nodes, activation functions, epochs, and optimizer) that minimize the predicted mean square error on the validation sample. We implemented the ANN emulators in a probabilistic programming language and calibrated the input parameters with Hamiltonian Monte Carlo-based algorithms to obtain the joint posterior distributions of the CISNET-CRC models' parameters. We internally validated each calibrated emulator by comparing the model-predicted posterior outputs against the calibration targets.Results The optimal ANN for SimCRC had 4 hidden layers and 360 hidden nodes, MISCAN-Colon had 4 hidden layers and 114 hidden nodes, and CRC-SPIN had 1 hidden layer and 140 hidden nodes. The total time for training and calibrating the emulators was 7.3, 4.0, and 0.66 h for SimCRC, MISCAN-Colon, and CRC-SPIN, respectively. The mean of the model-predicted outputs fell within the 95% confidence intervals of the calibration targets in 98 of 110 for SimCRC, 65 of 93 for MISCAN, and 31 of 41 targets for CRC-SPIN.Conclusions Using ANN emulators is a practical solution to reduce the computational burden and complexity for Bayesian calibration of individual-level simulation models used for policy analysis, such as the CISNET CRC models. In this work, we present a step-by-step guide to constructing emulators for calibrating 3 realistic CRC individual-level models using a Bayesian approach.

artificial neural networks↗

Validation of International Space Station Electrical Performance Model via On-orbit Telemetry

The first U.S. power module on International Space Station (ISS) was activated in December 2000. Comprised of solar arrays, nickel-hydrogen (NiH2) batteries, and a direct current power management and distribution (PMAD) system, the electric power system (EPS) supplies power to housekeeping and user electrical loads. Modeling EPS performance is needed for several reasons, but primarily to assess near-term planned and off-nominal operations and because the EPS configuration changes over the life of the ISS. The System Power Analysis for Capability Evaluation (SPACE) computer code is used to assess the ISS EPS performance. This paper describes the process of validating the SPACE EPS model via ISS on-orbit telemetry. To accomplish this goal, telemetry was first used to correct assumptions and component models in SPACE. Then on-orbit data was directly input to SPACE to facilitate comparing model predictions to telemetry. It will be shown that SPACE accurately predicts on-orbit component and system performance. For example, battery state-of-charge was predicted to within 0.6 percentage points over a 0 to 100 percent scale and solar array current was predicted to within a root mean square (RMS) error of 5.1 Amps out of a typical maximum of 220 Amps. First, SPACE model predictions are compared to telemetry for the ISS EPS components: solar arrays, NiH2 batteries, and the PMAD system. Second, SPACE predictions for the overall performance of the ISS EPS are compared to telemetry and again demonstrate model accuracy.

Jannette, Anthony G.↗

CFD validation experiments for internal flows

Computational Fluid Dynamics (CFD) validation experiments at NASA Lewis Research Center are described. The material presented summarizes the research in three areas: Inlets, Ducts and Nozzles; Turbomachinery; and Chemically Reacting Flows. The specific validation activities are concerned with shock-boundary layer interactions, vortex generator effects, large low speed centrifugal compressor measurements, transonic fan shock structure, rotor/stator kinetic energy distributions, stator wake shedding characteristics, boundary layer transition, multiphase flow and reacting shear layers. These experiments are intended to provide CFD validation data for the internal flow fields within aerospace propulsion system components.

Povinelli, Louis A.↗

CFD validation experiments for internal flows

Computational Fluid Dynamics (CFD) validation experiments at NASA Lewis are described. The material presented summarized the research in 3 areas: Inlets, ducts and nozzles; Turbomachinery; and Chemically reacting flows. The specific validation activities are concerned with shock boundary layer interactions, vortex generator effects, large low speed centrifugal compressor measurements, transonic fan shock structure, rotor/stator kinetic energy distributions, stator wake shedding characteristics, boundary layer transition, multiphase flow and reacting shear layers. These experiments are intended to provide CFD validation data for the internal flow fields within aerospace propulsion system components.

Povinelli, Louis A.↗

Prognostic analysis of high-flow nasal cannula therapy and non-invasive ventilation in mild to moderate hypoxemia patients and construction of a machine learning model for 48-h intubation prediction—a retrospective analysis of the MIMIC database

Background This study aims to investigate the clinical outcome between high-flow nasal cannula (HFNC) and non-invasive ventilation (NIV) therapy in mild to moderate hypoxemic patients on the first ICU day and to develop a predictive model of 48-h intubation. Methods The study included adult patients from the MIMIC III and IV databases who first initiated HFNC or NIV therapy due to mild to moderate hypoxemia (100 < PaO2/FiO2 ≤ 300). The 48-h and 30-day intubation rates were compared using cross-sectional and survival analysis. Nine machine learning and six ensemble algorithms were deployed to construct the 48-h intubation predictive models, of which the optimal model was determined by its prediction accuracy. The top 10 risk and protective factors were identified using the Shapley interpretation algorithm. Result A total of 123,042 patients were screened, of which, 673 were from the MIMIC IV database for ventilation therapy comparison (HFNC n = 363, NIV n = 310) and 48-h intubation predictive model construction (training dataset n = 471, internal validation set n = 202) and 408 were from the MIMIC III database for external validation. The NIV group had a lower intubation rate (23.1% vs. 16.1%, p = 0.001), ICU 28-day mortality (18.5% vs. 11.6%, p = 0.014), and in-hospital mortality (19.6% vs. 11.9%, p = 0.007) compared to the HFNC group. Survival analysis showed that the total and 48-h intubation rates were not significantly different. The ensemble AdaBoost decision tree model (internal and external validation set AUROC 0.878, 0.726) had the best predictive accuracy performance. The model Shapley algorithm showed Sequential Organ Failure Assessment (SOFA), acute physiology scores (APSIII), the minimum and maximum lactate value as risk factors for early failure and age, the maximum PaCO 2 and PH value, Glasgow Coma Scale (GCS), the minimum PaO 2 /FiO 2 ratio, and PaO 2 value as protective factors. Conclusion NIV was associated with lower intubation rate and ICU 28-day and in-hospital mortality. Further survival analysis reinforced that the effect of NIV on the intubation rate might partly be attributed to the other impact factors. The ensemble AdaBoost decision tree model may assist clinicians in making clinical decisions, and early organ function support to improve patients’ SOFA, APSIII, GCS, PaCO 2 , PaO 2 , PH, PaO 2 /FiO 2 ratio, and lactate values can reduce the early failure rate and improve patient prognosis.

Fu, Wei↗

In Silico Prediction of the Toxicity of Nitroaromatic Compounds: Application of Ensemble Learning QSAR Approach

In this work, a dataset of more than 200 nitroaromatic compounds is used to develop Quantitative Structure–Activity Relationship (QSAR) models for the estimation of in vivo toxicity based on 50% lethal dose to rats (LD 50 ). An initial set of 4885 molecular descriptors was generated and applied to build Support Vector Regression (SVR) models. The best two SVR models, SVR_A and SVR_B, were selected to build an Ensemble Model by means of Multiple Linear Regression (MLR). The obtained Ensemble Model showed improved performance over the base SVR models in the training set (R 2 = 0.88), validation set (R 2 = 0.95), and true external test set (R 2 = 0.92). The models were also internally validated by 5-fold cross-validation and Y-scrambling experiments, showing that the models have high levels of goodness-of-fit, robustness and predictivity. The contribution of descriptors to the toxicity in the models was assessed using the Accumulated Local Effect (ALE) technique. The proposed approach provides an important tool to assess toxicity of nitroaromatic compounds, based on the ensemble QSAR model and the structural relationship to toxicity by analyzed contribution of the involved descriptors.

54 ENVIRONMENTAL SCIENCES↗

EAQUATE: An International Experiment for Hyper-Spectral Atmospheric Sounding Validation

The international experiment called EAQUATE (European AQUA Thermodynamic Experiment) was held in September 2004 in Italy and the United Kingdom to demonstrate certain ground-based and airborne systems useful for validating hyperspectral satellite sounding observations. A range of flights over land and marine surfaces were conducted to coincide with overpasses of the AIRS instrument on the EOS Aqua platform. Direct radiance evaluation of AIRS using NAST-I and SHIS has shown excellent agreement. Comparisons of level 2 retrievals of temperature and water vapor from AIRS and NAST-I validated against high quality lidar and drop sonde data show that the 1K/1km and 10%/1km requirements for temperature and water vapor (respectively) are generally being met. The EAQUATE campaign has proven the need for synergistic measurements from a range of observing systems for satellite cal/val and has paved the way for future cal/val activities in support of IASI on the European Metop platform and CrIS on the US NPP/NPOESS platform.

Taylor, J. P.↗

Atomic Oxygen Durability Testing of an International Space Station Solar Array Validation Coupon

An International Space Station solar array validation coupon was exposed in a directed atomic oxygen beam for space environment durability testing at the NASA Lewis Research Center. Exposure to atomic oxygen and intermittent tensioning of the solar array were conducted to verify the solar array#s durability to low Earth orbital atomic oxygen and to the docking threat of plume loading both of which are anticipated over its expected mission life of fifteen years. The validation coupon was mounted on a specially designed rotisserie. The rotisserie mounting enabled the solar and anti-solar facing side of the array to be exposed to directed atomic oxygen in a sweeping arrival process replicating space exposure. The rotisserie mounting also enabled tensioning, in order to examine the durability of the array and its hinge to simulated plume loads. Flash testing to verify electrical performance of the solar array was performed with a solar simulator before and after the exposure to atomic oxygen and tensile loading. Results of the flash testing indicated little or no degradation in the solar array#s performance. Photographs were also taken of the array before and after the durability testing and are included along with comparisons and discussions in this report. The amount of atomic oxygen damage appeared minor with the exception of a very few isolated defects. There were also no indications that the simulated plume loadings had weakened or damaged the array, even though there was some erosion of Kapton due to atomic oxygen attack. Based on the results of this testing, it is apparent that the International Space Station#s solar arrays should survive the low Earth orbital atomic oxygen environment and docking threats which are anticipated over its expected mission life.

Forkapa, Mark J.↗

The Experimental and Numerical Investigation of Internal Heat Transfer for Supercritical Carbon Dioxide Cooling in a Staggered Pin Fin Array and Single-Jet Impingement

Over the past decade, the drive to reduce greenhouse gas emissions and to increase thermal efficiency for turbomachinery has invigorated the application of supercritical carbon dioxide (sCO2) power cycles for energy generation. Compared to the industry standard air cycles, sCO2 applications hold the potential for several advantages, including higher efficiencies, smaller footprints, and zero greenhouse gas emissions. However, like any turbomachinery application, the turbine inlet temperature must increase to increase thermal efficiency. This introduces the need for internal cooling features to avoid material failure as operating conditions rise. Two standard features include pin fin turbulators in the trailing edge and jet impingement in the airfoil's leading edge. Over the past several decades, these features have been the subject of extensive research. However, the move to the sCO2 operating environment creates the need to re-visit these features to quantify the heat transfer capabilities within this supercritical cooling environment. The first objective of this paper is to discuss the development of the experimental demonstration for internal heat transfer testing at 200 bar and 400 Celsius, which sits well within the CO2 supercritical region. Next, the heat transfer for pin fin turbulators and single-jet impingement in the sCO2 environment is compared to existing air data-derived correlations to quantify any deviations from literature correlations. Finally, the experimental process aims to validate internal cooling conjugate heat transfer numerical simulations for sCO2 turbines.

20 FOSSIL-FUELED POWER PLANTS↗

Blind Study Validating Parametric Costing Tools PRICE TruePlanning and SEER-H for NASA Science Missions

Two of the primary parametric costing tools used to estimate the development and production cost of future spacecraft hardware are PRICE TruePlanning - Space Missions Catalog by PRICE Systems, and SEER-H by Galorath. These are standard tools used by NASA and industry to estimate the cost of new aerospace hardware. However, no independent verification of the accuracy of these tools is publicly available. Both PRICE Systems and Galorath have completed internal validation studies of their parametric cost estimating tools; however, they only provided the results of the studies and did not detail the exact methods used to perform the validation. In the present study, cost estimators used PRICE TruePlanning and SEER-H to estimate the cost of twelve different past NASA science missions. The estimators were prevented from knowing the actual cost of the missions in an effort to minimize cognitive biases. In the present study, SEER had an average error of 23%, median error of -0.3%, with a standard deviation of 43%. PRICE had an average error of 52%, median error of 50%, and standard deviation of 45%. There were several factors independent of PRICE and SEER which may have affected the accuracy of the results in the present study including: uncertainty in the technical data used for the estimates, the methods used to estimate uncertainty in spacecraft component mass and numbers of prototypes, and the experience of the estimators.

Paul D Friz↗

Blind Study Validating Parametric Costing Tools PRICE True Planning and SEER-H for NASA Science Missions

Two of the primary parametric costing tools used to estimate the development and production cost of future spacecraft hardware are PRICE True Planning - Space Missions Catalog by PRICE Systems, and SEER-H by Galorath. These are standard tools used by NASA and industry to estimate the cost of new aerospace hardware. However, no independent verification of the accuracy of these tools is publicly available. Both PRICE Systems and Galorath have completed internal validation studies of their parametric cost estimating tools; however, they only provided the results of the studies and did not detail the exact methods used to perform the validation. In the present study, cost estimators used PRICE True Planning and SEER-H to estimate the cost of twelve different past NASA science missions. The estimators were prevented from knowing the actual cost of the missions in an effort to minimize cognitive biases. In the present study, SEER had an average error of 23%, median error of -0.3%, with a standard deviation of 43%. PRICE had an average error of 52%, median error of 50%, and standard deviation of 45%. There were several factors independent of PRICE and SEER which may have affected the accuracy of the results in the present study including: uncertainty in the technical data used for the estimates, the methods used to estimate uncertainty in spacecraft component mass and numbers of prototypes, and the experience of the estimators.

Friz, Paul D.↗

Blind Validation Study of PRICE TruePlanning and SEER-H

Two of the primary parametric costing tools used to estimate the development and production cost of future spacecraft hardware are PRICE TruePlanning by PRICE Systems, and System Estimation and Evaluation of Resources-Hardware (SEER-H) by Galorath. These are standard tools used by NASA and industry to estimate the cost of new aerospace hardware. This study is an independent verification of the accuracy of these tools and was originally published in 2018. The purpose of this presentation is to circulate the findings of that study among the NASA cost community. Both PRICE Systems and Galorath have completed internal validation studies of their parametric cost estimating tools; however, they only provided the results of the studies and did not detail the exact methods used to perform the validation. In the present study, cost estimators used PRICE TruePlanning and SEER-H to estimate the cost of twelve different past NASA science missions. The estimators were prevented from knowing the actual cost of the missions in an effort to minimize cognitive biases. In the present study, SEER-H had an average error of 23%, median error of -0.3%, with a standard deviation of 43%. PRICE had an average error of 52%, median error of 50%, and standard deviation of 45%. Nine of the twelve mission's actual costs fell within the 80% confidence interval of SEER's probabilistic estimates; however, none of the missions fell within PRICE’s confidence intervals. There were several factors independent of PRICE and SEER-H which may have affected the accuracy of the results in the present study including: uncertainty in the technical data used for the estimates, the methods used to estimate uncertainty in spacecraft component mass and numbers of prototypes, and the experience of the estimators.

SEER↗

Self-diagnosis of model suitability for continuous measurements of stream-dissolved organic carbon derived from in situ UV–visible spectroscopy

Application of high-frequency monitoring of dissolved organic carbon (DOC) is difficult in instances where training datasets are challenging to develop (e.g., remote locations) and the relationship between optical features and DOC concentration changes due to environmental or landscape shifts (e.g., climate or land-use change). We developed and compared three partial least squares (PLS) models using in situ water level measurements, conductivity, and UV–Vis spectral attenuation to predict DOC. Two site-specific models were developed using data from a hillslope-dominated forest or a low-relief wetland-pond-dominated stream catchment. The third model, using data from both sites, exhibited the best performance (DOC range = 4–15.5 mg C L −1 , mean = 8.38 mg C L −1 , training RMSE = 0.34 mg C L −1 , internal validation RMSE = 0.50 mg C L −1 , external validation RMSE = 2.43 mg C L −1 ). We further demonstrate using PLS model statistics to monitor performance and elucidate when and how models should be updated. These statistics, Hotelling's T 2 and squared prediction errors, are useful consistency checks for the predictions made and detect underlying inconsistencies that, if undetected, can reduce the robustness of DOC prediction. For example, via the T 2 statistic, we identified the summer–autumn transition as a period when DOC composition differed from what was represented in the training dataset. We also determined that elevated SUVA 254 values contributed to the overall bias observed in predictions made during the subsequent year as part of the external validation. This enabled the application of a bias correction that reduced the RMSE from 2.43 to 0.89 mg C L −1 . The method presented here could be applied to future monitoring programs enabling model updates to monitor DOC fluxes accurately from optical datasets (e.g., attenuance or fluorescence) in the face of developing datasets in remote locations or environmental change. In conclusion, implementation of this approach may also identify possible regime shifts or landscape and hydrologic change associated with climate and other environmental changes relevant to terrestrial to aquatic fluxes.

UV-VIS spectroscopy↗

Modeling spatial patterns of longleaf pine needle dispersal using long-term data

Abstract Background Predicting patterns of fire behavior and effects in frequent fire forests relies on an understanding of fine-scale spatial patterns of available fuels. Leaf litter is a significant canopy-derived fine fuel in fire-maintained forests. Litter dispersal is dependent on foliage production, stand structure, and wind direction, but the relative importance of these factors is unknown. Results Using a 10-year litterfall dataset collected within eighteen 4-ha longleaf pine ( Pinus palustris Mill.) plots varying in canopy spatial pattern, we compared four spatially explicit models of annual needle litter dispersal: a model based only on basal area, an overstory abundance index (OAI) model, both isotropic and anisotropic litter kernel models, and a null model that assumed no spatial relationship. The best model was the anisotropic model (R 2 = 0.656) that incorporated tree size, location, and prevailing wind direction, followed by the isotropic model (R 2 = 0.612), basal area model (R 2 = 0.488), OAI model (R 2 = 0.416), and the null model (R 2 = 0.08). Conclusions As with previous studies, the predictive capability of the litter models was robust when internally validated with a subset of the original dataset (R 2 = 0.196–0.549); however, the models were less robust when challenged with an independent dataset (R 2 = 0.122–0.319) from novel forest stands. Our model validation underscores the need for rigorous tests with independent, external datasets to confirm the validity of litter dispersal models. These models can be used in the application of prescribed fire to estimate fuel distribution and loading, as well as aid in the fine tuning of fire behavior models to better understand fire outcomes across a range of forest canopy structures.

54 ENVIRONMENTAL SCIENCES↗

Diurnal Variability of Vertical Structure from a TRMM Passive Microwave "Virtual Radar" Retrieval

Robust description of the diurnal cycle from TRMM observations is complicated by the limitations of Low Earth Orbit (LEO) sampling; from a 'climatological' perspective, sufficient sampling must exist to control for both spatial and seasonal variability, before tackling an additional diurnal component (e.g., with 8 additional 3-hourly or 24 1-hourly bins). For documentation of vertical structure, the narrow sample swath of the TRMM Precipitation Radar limits the resolution of any of these components. A neural-network based 'virtual radar" retrieval has been trained and internally validated, using multifrequency / multipolarization passive microwave(TM1) brightness temperatures and textures parameters and lightning (LIS) observations, as inputs, and PR volumetric reflectivity as targets (outputs). By training the algorithms (essentially highly multivariate, nonlinear regressions) on a very large sample of high-quality co-located data from the center of the TRMM swath, 3D radar reflectivity and derived parameters (VIL, IWC, Echo Tops, etc.) can be retrieved across the entire TMI swath, good to 8-9% over the dynamic range of parameters. As a step in the retrieval (and as an output of the process), each TMI multifrequency pixel (at 85 GHz resolution) is classified into one of the 25 archetypal radar profile vertical structure "types", previously identified using cluster analysis. The dynamic range of retrieved vertical structure appears to have higher fidelity than the current (Version 6) experimental GPROF hydrometeor vertical structure retrievals. This is attributable to correct representation of the prior probabilities of vertical structure variability in the neural network training data, unlike the GPROF cloud-resolving model training dataset used in the V6 algorithms. The LIS lightning inputs are supplementary inputs, and a separate offline neural network has been trained to impute (predict) LIS lightning from passive-microwave-only data. The virtual radar retrieval is thus, in principle, extensible to Aqua/AMSR-E and NPOESS/CMIS passive microwave instruments. The virtual radar approach yields a threefold increase in effective sampling from the mission, albeit of lower-quality "retrieved" data, reducing the variance of local estimates by one third (or the standard deviation by-0.57). In this talk, the variance reduction is leveraged to more finely resolve global diurnal variability in both space and time (local hour).

Boccippio, Dennis J.↗

Developing Atmospheric Retrieval Methods for Direct Imaging Spectroscopy of Gas Giants in Reflected Light I: Methane Abundances and Basic Cloud Properties

Reflected light spectroscopy and photometry of cool, directly imaged extrasolar giant planets are expected to be performed in the next decade by space-based telescopes equipped with optical wavelength coronagraphs and integral field spectrographs, such as the Wide-Field Infrared Survey Telescope (WFIRST). We are developing a new atmospheric retrieval methodology to help assess the science return and inform the instrument design for such future missions, and ultimately interpret the resulting observations. Our retrieval technique employs an albedo model coupled with both a Markov chain Monte Carlo Ensemble Sampler (emcee) and a multimodal nested sampling algorithm (MultiNest) to map the posterior distribution. This combination makes the global evidence calculation more robust for any given model, and highlights possible discrepancies in the likelihood maps. Here we apply this methodology to simulated spectra of cool giant planets. As a proof-of-concept, our current atmospheric model contains 1 or 2 cloud layers, methane as a major absorber, and a H2-He background gas. This 6-to-9 parameter model is appropriate for Jupiter-like planets and can be easily expanded in the future. In addition to deriving the marginal likelihood distribution and confidence intervals for the model parameters, we perform model selection to determine the significance of methane and cloud detection as a function of expected signal-to-noise, in the presence of spectral noise correlations. After internal validation, the method is applied to realistic reflected-light spectra of Jupiter, Saturn, and HD 99492 c, a likely observing target. We find that the presence or absence of clouds and methane can be determined with high accuracy, while parameters uncertainties are model-dependent.

spectroscopy↗

Assessment of Segmentation-Induced Deviations of Porosity Metrics in Powder Bed Fusion Additively Manufactured Components

Post processing X-ray computational tomography (CT) inspection data for additively manufactured (AM) components can induce deviations in defect quantification, affecting subsequent fatigue and failure predictions. To assess the influence and potential impact of segmentation-induced measurement deviations, this paper applies several segmentation techniques to X-ray CT data for powder bed fusion Ti-6Al-4V specimens exhibiting porosity conditions. X-ray CT reconstructions were segmented with varying techniques including Otsu’s thresholding, random forest, k-nearest neighbors, and the multilayer perceptron. Metrics such as pore size and global porosity were compared for internal validity. Then, top-down X-ray CT measurements of surface-breaking porosity were compared to optical profilometry for cross-validation.

Data fusion↗