Search NASA⌕ Search

SEARCH · Search NASA

Results for “testing statistics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 757 records · Page 42

Simultaneous Development and Robust Optimization of a Microstructure Dependent Material

Recent microstructure characterization techniques combined with Symbolic Regression(SR)analysis has been proven to generate white box plasticity models well suited for incorporation into FEA software.The current work builds upon those efforts and demonstrates the applicability of Sequential Monte-Carlo (SMC) methods within SR analysis to condense model development and robust optimization into a single, co-dependent process. In this project, SMC methods provide a mechanism through which the observed microstructure features and associated variability can be incorporated into the discovery phase of model development and simultaneously recover approximate parameter distributions through SR analysis. The demonstration utilized a data set consisting of tensile test results from a limited number of sample specimens with corresponding EBSD data from which microstructure features were characterized.The maximum threshold stress model in the Visco-Plastic Self-Consistent (VPSC) code developed by Los Alamos National Laboratories was calibrated using mechanical test data.Synthetic volume elements with statistically equivalent microstructure were generated with DREAM3Dbased on the observed EBSD data. VPSC was used to simulate the corresponding tensile test response for each of the synthetic volume elements. The simulated microstructure and tensile test data was used astraining datafor SMC-SR algorithm and the resulting model was validated with data from the original empirical data set.

Karl Garbrecht↗

Expanded Performance Characterization of the NASA HERMeS Hall Thruster

This work presents a summary of a detailed performance assessment of the Hall Effect Rocket with Magnetic Shielding (HERMeS) Technology Demonstration Unit 3 (TDU-3) thruster at the throttle conditions most relevant for the Gateway Power and Propulsion Element (PPE). First, an assessment was performed of TDU-3 performance and stability at the previously-tested throttle points of 300 V/6.25 kW, 400 V/8.33 kW, 500 V/10.42 kW, and 600 V/12.5 kW that confirmed TDU-3 was operating nominally and consistent with historical baselines. TDU-3 performance and stability were then assessed throughout the PPE throttle range of 2.6-12 kW and shown to mirror the characteristics observed at the previously tested conditions. Finally, a statistical estimate for the thrust specification at the expanded set of throttle conditions was computed and found to yield similar tolerance ranges to those calculated for the heritage conditions. Taken together, these results provide strong evidence that the demonstrated performance characteristics of the HERMeS design are preserved at the expanded set of throttle conditions relevant to application on PPE.

HERMeS↗

Extended Performance Characterization of the NASA HERMeS Hall Thruster

This work presents a summary of a detailed performance assessment of the Hall Effect Rocket with Magnetic Shielding (HERMeS) Technology Demonstration Unit 3 (TDU-3) thruster at the throttle conditions most relevant for the Gateway Power and Propulsion Element (PPE). First, an assessment was performed of TDU-3 performance and stability at the previously-tested throttle points of 300 V/6.25 kW, 400 V/8.33 kW, 500 V/10.42 kW, and 600 V/12.5 kW that confirmed TDU-3 was operating nominally and consistent with historical baselines. TDU-3 performance and stability were then assessed throughout the PPE throttle range of 2.6-12 kW and shown to mirror the characteristics observed at the previously tested conditions. Finally, a statistical estimate for the thrust specification at the expanded set of throttle conditions was computed and found to yield similar tolerance ranges to those calculated for the heritage conditions. Taken together, these results provide strong evidence that the demonstrated performance characteristics of the HERMeS design are preserved at the expanded set of throttle conditions relevant to application on PPE.

HERMeS↗

A Ricin Forensic Profiling Approach Based on a Complex Set of Biomarkers

A forensic method for the retrospective determination of preparation methods used for illicit ricin toxin production was developed. The method was based on a complex set of biomarkers, including carbohydrates, fatty acids, seed storage proteins, in combination with data on ricin and Ricinus communis agglutinin. The analyses were performed on samples prepared from four castor bean plant ( Ricinus communis ) cultivars by four different sample preparation methods (PM1 – PM4) ranging from simple disintegration of the castor beans to multi-step preparation methods including different protein precipitation methods. Comprehensive analytical data was collected by use of a range of analytical methods and robust orthogonal partial least squares-discriminant analysis- models (OPLS-DA) were constructed based on the calibration set. By the use of a decision tree and two OPLS-DA models, the sample preparation methods of a test set of ricin preparations were determined. The model statistics of the two models were good and a 100% rate of correct predictions of the test set was achieved.

Fredriksson, Sten-Åke (ORCID:0000000237131990)↗

Calculation of guaranteed mean power from wind turbine generators

A method for calculating the 'guaranteed mean' power output of a wind turbine generator is proposed. The term 'mean power' refers to the average power generated at specified wind speeds during short-term tests. Correlation of anemometers, the method of bins for analyzing non-steady data, the PROP Code for predicting turbine power, and statistical analysis of deviations in test data from theory are discussed. Guaranteed mean power density for the Clayton Mod-OA system was found to be 8 watts per square meter less than theoretical power density at all power levels, with a confidence level of 0.999. This amounts to 4 percent of rated power.

Spera, D. A.↗

Anomalies in the topology of the temperature fluctuations in the cosmic microwave background: An analysis of the NPIPE and FFP10 data releases

We present a topological analysis of the temperature fluctuation maps from the Planck 2020 Data Release 4 NPIPE dataset and the Planck 2018 Data Release 3 FFP10 dataset. We performed a multiscale analysis in terms of the homology characteristics of the maps, invoking relative homology to account for the analysis in the presence of masks. We performed our analysis for a range of smoothing scales spanning sub- and super-horizon scales corresponding to a full width at half maximum (FWHM) of 5',10',20',40',80',160',320', and 640', and employed simulations based on the standard model for comparison, which assumes the initial fluctuation field to be an isotropic and homogeneous Gaussian random field. Examining the behavior of topological components, represented by the 0D homology group, we find the observations to be approximately 2σ or less deviant from the simulations for all resolutions and scales for the NPIPE dataset. For the FFP10 dataset, we detect a 2.96σ deviation between the observations and simulations at N = 128, FWHM = 80'. For the topological loops, represented by the first homology group, the simulations and observations are consistent within 2σ for most resolutions and scales for both the datasets. However, for the NPIPE dataset, we observe a high deviation between the observation and simulations in the number of loops at FWHM = 320', but at a low dimensionless threshold ν = –2.5. Under a Gaussian assumption, this would amount to a deviation of ~4σ. However, the distribution in this bin is manifestly non-Gaussian and does not obey Poisson statistics either. In the absence of a true theoretical understanding, we simply note that the significance is higher than what may be resolved by 600 simulations, yielding an empirical p-value of at most 0.0016. Specifically in this case, our tests indicate that the numbers arise from a statistically stable regime, despite being based on small numbers. For the FFP10 dataset, the differences are not as strong as for the NPIPE dataset, indicating a 2.77σ deviation at this resolution and threshold. The Euler characteristic, which is the alternating sum of the ranks of relative homology groups, reflects the deviations in the components and loops. To assess the significance of combined levels for a given scale, we employed the empirical and theoretical versions of the χ 2 test as well as the nonparametric Tukey depth test. Although all statistics exhibit a stable distribution, we favor the empirical version of the χ 2 test in the final interpretation, as it indicates the most conservative differences. For the NPIPE dataset, we find that the components and loops differ at more than 95%, but agree within the 99% confidence level with respect to the base model at N = 32, FWHM = 320'. The Euler characteristic at this resolution displays a per mil deviation. In contrast, the FFP10 dataset shows that the observations are consistent with the base model within the 95% confidence level, at this and smaller scales. This is consistent with the observations of the Planck analysis pipeline via Minkowski functionals. For the largest smoothing scale, N = 16, FWHM = 640', both datasets exhibit an anomalous behavior of the loops, where FFP10 data exhibit a deviation that is larger by an order of magnitude than that of the NPIPE dataset. In contrast, the values for the topological components and the Euler characteristic agree between observations and model to within a confidence level of 99%. However, for the largest scales, the statistics are based on low numbers and may have to be regarded with caution. Even though both datasets exhibit mild to significant discrepancies, they also exhibit contrasting behaviors at various instances. Therefore, we do not find it feasible to convincingly accept or reject the null hypothesis. Disregarding the large-scale anomalies that persist at similar scales in WMAP and Planck, observations of the cosmic microwave background are largely consistent with the standard cosmological model within 2σ.

79 ASTRONOMY AND ASTROPHYSICS↗

The DESI-Lensing Mock Challenge: large-scale cosmological analysis of 3x2-pt statistics

The current generation of large galaxy surveys will test the cosmological model by combining multiple types of observational probes. Realising the statistical promise of these new datasets requires rigorous attention to all aspects of analysis including cosmological measurements, modelling, covariance and parameter likelihood. In this paper we present the results of an end-to-end simulation study designed to test the analysis pipeline for the combination of the Dark Energy Spectroscopic Instrument (DESI) Year 1 galaxy redshift dataset and separate weak gravitational lensing information from the Kilo-Degree Survey, Dark Energy Survey and Hyper-Suprime-Cam Survey. Our analysis employs the 3x2-pt correlation functions including cosmic shear and galaxy-galaxy lensing, together with the projected correlation function of the spectroscopic DESI lenses. We build realistic simulations of these datasets including galaxy halo occupation distributions, photometric redshift errors, weights, multiplicative shear calibration biases and magnification. We calculate the analytical covariance of these correlation functions including the Gaussian, noise and super-sample contributions, and show that our covariance determination agrees with estimates based on the ensemble of simulations. We use a Bayesian inference platform to demonstrate that we can recover the fiducial cosmological parameters of the simulation within the statistical error margin of the experiment, investigating the sensitivity to scale cuts. This study is the first in a sequence of papers in which we present and validate the large-scale 3x2-pt cosmological analysis of DESI-Y1.

79 ASTRONOMY AND ASTROPHYSICS↗

Regression Analysis of Long-Term Profile Ozone Data Set from BUV Instruments

We have produced a profile merged ozone data set (MOD) based on the SBUV/SBUV2 series of nadir-viewing satellite backscatter instruments, covering the period from November 1978 - December 2003. In 2004, data from the Nimbus 7 SBUV and NOAA 9, ll, and 16 SBUV/2 instruments were reprocessed using the Version 8 (V8) algorithm and most recent calibrations. More recently, data from the Nimbus 4 BUT instrument, which was operational from 1970 - 1977, were also reprocessed using the V8 algorithm. As part of the V8 profile calibration, the Nimbus 7 and NOAA 9 (1993-1997 only) instrument calibrations have been adjusted to match the NOAA 11 calibration, which was established based on comparisons with SSBUV shuttle flight data. Differences between NOAA 11, Nimbus 7 and NOAA 9 profile zonal means are within plus or minus 5% at all levels when averaged over the respective periods of data overlap. NOAA 16 SBUV/2 data have insufficient overlap with NOAA 11, so its calibration is based on pre-flight information. Mean differences over 4 months of overlap are within plus or minus 7%. Given the level of agreement between the data sets, we simply average the ozone values during periods of instrument overlap to produce the MOD profile data set. Initial comparisons of coincident matches of N4 BUV and Arosa Umkehr data show mean differences of 0.5 (0.5)% at 30km; 7.5 (0.5)% at 35 km; and 11 (0.7)% at 40 km, where the number in parentheses is the standard error of the mean. In this study, we use the MOD profile data set (1978-2003) to estimate the change in profile ozone due to changing stratospheric chlorine levels. We use a standard linear regression model with proxies for the seasonal cycle, solar cycle, QBO, and ozone trend. To account for the non-linearity of stratospheric chlorine levels since the late 1990s, we use a time series of Effective Chlorine, defined as the global average of Chlorine + 50 * Bromine at 1 hPa, as the trend proxy. The Effective Chlorine data are taken from the 3-D Goddard CTM. We will show the latest trend results using this statistical model. In addition, the Nimbus 4 BUV data offer an opportunity to test the physical properties of our statistical model. From ground-based comparisons we will establish an uncertainty range for the Nimbus 4 data. We then extrapolate our statistical model fit backwards in time and compare to the Nimbus 4 data. We compare the characteristics of the residual, defined as the difference between the data and statistical regression fit, during the Nimbus 4 time period and the 1978-2003 period over which the statistical model coefficients were estimated, and present these results.

Stolarski, Richard S.↗

Robustness of Asymptotic Optimal Accelerated Test Plans to Small-Sample Settings

Accelerated testing is a form of testing commonly employed in industry, especially at Sandia National Laboratories, in order to extract information about product lifetime behavior that cannot be acquired under normal operating conditions due to the extensive durations of the intended life- times. As of the writing of this report, there is no known guideline for the design of accelerated tests at the laboratories and yet there is an entire field in the statistical literature on the optimal design of accelerated tests covering a wide range of limitations and constraints available for reference. A potential roadblock to incorporating this knowledge is the common difficulty at Sandia of having severely limited data available for such testing. Nearly all of the methodology presented in the statistical literature is based on asymptotic or large-sample theory and so may not be appropriate for the testing conditions present at Sandia. This report investigates optimal accelerated test plans derived based on small-sample or exact methodology for both a basic test setting as well as a more realistic setting and compares the resulting test plans to those derived based on the large- sample methodology from the literature. It is shown that the large-sample test plans are actually quite robust to the presence of limited data and are generally more flexible than the small-sample test plans.

36 MATERIALS SCIENCE↗

Low responsiveness of machine learning models to critical or deteriorating health conditions

Machine learning (ML) based mortality prediction models can be immensely useful in intensive care units. Such a model should generate warnings to alert physicians when a patient’s condition rapidly deteriorates, or their vitals are in highly abnormal ranges. Before clinical deployment, it is important to comprehensively assess a model’s ability to recognize critical patient conditions. We develop multiple medical ML testing approaches, including a gradient ascent method and neural activation map. We systematically assess these machine learning models’ ability to respond to serious medical conditions using additional test cases, some of which are time series. Guided by medical doctors, our evaluation involves multiple machine learning models, resampling techniques, and four datasets for two clinical prediction tasks. We identify serious deficiencies in the models’ responsiveness, with the models being unable to recognize severely impaired medical conditions or rapidly deteriorating health. For in-hospital mortality prediction, the models tested using our synthesized cases fail to recognize 66% of the injuries. In some instances, the models fail to generate adequate mortality risk scores for all test cases. Our study identifies similar kinds of deficiencies in the responsiveness of 5-year breast and lung cancer prediction models. Using generated test cases, we find that statistical machine-learning models trained solely from patient data are grossly insufficient and have many dangerous blind spots. Most of the ML models tested fail to respond adequately to critically ill patients. How to incorporate medical knowledge into clinical machine learning models is an important future research direction.

60 APPLIED LIFE SCIENCES↗

Helicopter Acoustic Flight Test with Altitude Variation and Maneuvers

A cooperative flight test campaign between NASA and the U.S. Army was performed from September 2014 to February 2015. The purposes of the testing were to: investigate the effects of altitude variation on noise generation, investigate the effects of gross weight variation on noise generation, establish the statistical variability in acoustic flight testing of helicopters, and characterize the effects of transient maneuvers on radiated noise for a medium-lift utility helicopter. This test was performed at three test sites (0, 4000, and 7000 feet above mean sea level) with two aircraft (AS350 SD1 and EH-60L) tested at each site. This report provides an overview of the test, documents the data acquired and describes the formats of the stored data.

Watts, Michael E.↗

Statistical Determination of Fatigue Strength

A cyclic test method for the fatigue strength of metals is proposed. In this method, fatigue curves characterizing the stress dependence of the number of cycles to failure are plotted and statistically described. The number of cycles to failure is described by a Weibull distribution, using the lowest guaranteed values.

42 ENGINEERING↗

Liquid Rocket Engine Testing Overview

Contents include the following: Objectives and motivation for testing. Technology, Research and Development Test and Evaluation (RDT&E), evolutionary. Representative Liquid Rocket Engine (LRE) test compaigns. Apollo, shuttle, Expandable Launch Vehicles (ELV) propulsion. Overview of test facilities for liquid rocket engines. Boost, upper stage (sea-level and altitude). Statistics (historical) of Liquid Rocket Engine Testing. LOX/LH, LOX/RP, other development. Test project enablers: engineering tools, operations, processes, infrastructure.

Rahman, Shamim↗

Test Population Selection from Weibull-Based, Monte Carlo Simulations of Fatigue Life

Fatigue life is probabilistic and not deterministic. Experimentally establishing the fatigue life of materials, components, and systems is both time consuming and costly. As a result, conclusions regarding fatigue life are often inferred from a statistically insufficient number of physical tests. A proposed methodology for comparing life results as a function of variability due to Weibull parameters, variability between successive trials, and variability due to size of the experimental population is presented. Using Monte Carlo simulation of randomly selected lives from a large Weibull distribution, the variation in the L10 fatigue life of aluminum alloy AL6061 rotating rod fatigue tests was determined as a function of population size. These results were compared to the L10 fatigue lives of small (10 each) populations from AL2024, AL7075 and AL6061. For aluminum alloy AL6061, a simple algebraic relationship was established for the upper and lower L10 fatigue life limits as a function of the number of specimens failed. For most engineering applications where less than 30 percent variability can be tolerated in the maximum and minimum values, at least 30 to 35 test samples are necessary. The variability of test results based on small sample sizes can be greater than actual differences, if any, that exists between materials and can result in erroneous conclusions. The fatigue life of AL2024 is statistically longer than AL6061 and AL7075. However, there is no statistical difference between the fatigue lives of AL6061 and AL7075 even though AL7075 had a fatigue life 30 percent greater than AL6061.

Vlcek, Brian L.↗

Test Population Selection from Weibull-Based, Monte Carlo Simulations of Fatigue Life

Fatigue life is probabilistic and not deterministic. Experimentally establishing the fatigue life of materials, components, and systems is both time consuming and costly. As a result, conclusions regarding fatigue life are often inferred from a statistically insufficient number of physical tests. A proposed methodology for comparing life results as a function of variability due to Weibull parameters, variability between successive trials, and variability due to size of the experimental population is presented. Using Monte Carlo simulation of randomly selected lives from a large Weibull distribution, the variation in the L10 fatigue life of aluminum alloy AL6061 rotating rod fatigue tests was determined as a function of population size. These results were compared to the L10 fatigue lives of small (10 each) populations from AL2024, AL7075 and AL6061. For aluminum alloy AL6061, a simple algebraic relationship was established for the upper and lower L10 fatigue life limits as a function of the number of specimens failed. For most engineering applications where less than 30 percent variability can be tolerated in the maximum and minimum values, at least 30 to 35 test samples are necessary. The variability of test results based on small sample sizes can be greater than actual differences, if any, that exists between materials and can result in erroneous conclusions. The fatigue life of AL2024 is statistically longer than AL6061 and AL7075. However, there is no statistical difference between the fatigue lives of AL6061 and AL7075 even though AL7075 had a fatigue life 30 percent greater than AL6061.

Vlcek, Brian L.↗

Active Subspace Methods for Data-Intensive Inverse Problems (Final Report)

Fostered by advances in computing, simulation and data science have emerged as the third and fourth paradigms enabling scientific discovery. Many mathematical challenges arise when integrating these paradigms to produce new knowledge. One of the most important challenges is the inverse or calibration problem: find the inputs of a simulation such that its outputs agree with a given set of measurements. As both the cost of the complex simulations and the volume of measurement data sets increase, intuition-driven trial-and-error methods for calibration quickly become untenable; modern scientists need more computationally efficient and mathematically rigorous methods. When measurements and simulated predictions contain uncertainty, the Bayesian perspective on statistical inference provides a popular mathematical framework for developing computational methods to solve inverse problems. Markov chain Monte Carlo (MCMC) methods are a mainstay of this framework; they characterize a probability distribution on the space of model inputs that is conditioned on the given measurements. However, despite recent advances, MCMC methods remain impractical for expensive simulations with more than a handful of inputs. At its heart, MCMC searches the space of inputs for corresponding outputs that match measurements, and so it is afflicted with the curse of dimensionality. Additionally, MCMC methods become less efficient as the volume of the measurement data increases, particularly if the measurements are redundant. To make MCMC—and Bayesian methods, in general—amenable to today’s computer models with millions of inputs and terabytes of measurement data, one must devise methods to reduce the dimension of both the simulation inputs and the measurements. For many models used in practice, the outputs vary primarily on a low-dimensional manifold of the nominally high-dimensional input space. Recently developed active subspace methods discover this low-dimensional manifold and exploit it to enable analyses typically suited for low-dimensional settings (e.g., statistical inversion of complex systems). We hypothesize that active subspace methods can provide the necessary dimension reduction, enabling existing Bayesian methods to find solutions for a large class of otherwise intractable statistical inverse problems. We will test the hypothesis with a series of mathematical analyses and computational experiments that will proceed along three research veins: (i) integrating active subspace methods into methods for statistical inversion, (ii) addressing current challenges in active subspace methods, and (iii) incorporating methods for nonlinear dimensionality reduction into active subspace methods. We will apply the methods we develop to real inverse problems in chemical kinetics and turbulent flame modeling. Our team combines expertise in methods for inverse problems, dimension reduction, and reduced order modeling. With a history of active collaboration and thoughtful engagement, we bring enthusiasm and fresh perspectives to a well studied subject. We are excited by the potential opportunity to work together to extend and apply active subspace methods to data-intensive inverse problems.

97 MATHEMATICS AND COMPUTING↗

Active Subspace Methods for Data-Intensive Inverse Problems

Fostered by advances in computing, simulation and data science have emerged as the third and fourth paradigms enabling scientific discovery. Many mathematical challenges arise when integrating these paradigms to produce new knowledge. One of the most important challenges is the inverse or calibration problem: find the inputs of a simulation such that its outputs agree with a given set of measurements. As both the cost of the complex simulations and the volume of measurement data sets increase, intuition-driven trial-and-error methods for calibration quickly become untenable; modern scientists need more computationally efficient and mathematically rigorous methods. When measurements and simulated predictions contain uncertainty, the Bayesian perspective on statistical inference provides a popular mathematical framework for developing computational methods to solve inverse problems. Markov chain Monte Carlo (MCMC) methods are a mainstay of this framework; they characterize a probability distribution on the space of model inputs that is conditioned on the given measurements. However, despite recent advances, MCMC methods remain impractical for expensive simulations with more than a handful of inputs. At its heart, MCMC searches the space of inputs for corresponding outputs that match measurements, and so it is afflicted with the curse of dimensionality. Additionally, MCMC methods become less efficient as the volume of the measurement data increases, particularly if the measurements are redundant. To make MCMC—and Bayesian methods, in general—amenable to today’s computer models with millions of inputs and terabytes of measurement data, one must devise methods to reduce the dimension of both the simulation inputs and the measurements. For many models used in practice, the outputs vary primarily on a low-dimensional manifold of the nominally high-dimensional input space. Recently developed active subspace methods discover this low-dimensional manifold and exploit it to enable analyses typically suited for low-dimensional settings (e.g., statistical inversion of complex systems). We hypothesize that active subspace methods can provide the necessary dimension reduction, enabling existing Bayesian methods to find solutions for a large class of otherwise intractable statistical inverse problems. We will test the hypothesis with a series of mathematical analyses and computational experiments that will proceed along three research veins: (i) integrating active subspace methods into methods for statistical inversion, (ii) addressing current challenges in active subspace methods, and (iii) incorporating methods for nonlinear dimensionality reduction into active subspace methods. We will apply the methods we develop to real inverse problems in chemical kinetics and turbulent flame modeling. Our team combines expertise in methods for inverse problems, dimension reduction, and reduced order modeling. With a history of active collaboration and thoughtful engagement, we bring enthusiasm and fresh perspectives to a well studied subject. We are excited by the potential opportunity to work together to extend and apply active subspace methods to data-intensive inverse problems.

97 MATHEMATICS AND COMPUTING↗

Failure risk assessment by analysis and testing

The sources of information on which to base an evaluation of reliability or failure risk of an aerospace flight system are (1) experience from tests and flights and (2) engineering analysis. It is rarely feasible to establish high reliability at high confidence by testing aerospace systems or components. Moreover, failure prediction by conventional, deterministic methods of engineering analysis can become arbitrary and subject to serious misinterpretation when uncertain or approximate information is used to establish analysis parameter values and to calibrate the accuracy of engineering models. The limitations of testing to evaluate failure risk are discussed, and a statistical approach which incorporates both engineering analysis and testing is presented.

Moore, N.↗