Search NASA⌕ Search

SEARCH · Search NASA

Results for “Statistical Testing Approach”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 325 records · Page 18

Improving seasonal precipitation forecasts in the Western United States through statistical downscaling

Abstract Seasonal precipitation forecasts in the western United States are critical resources for water resource management, especially during winter. While current seasonal forecasting systems provide monthly precipitation forecasts operationally, their coarse resolution limits their effectiveness in capturing the localized precipitation patterns and snowpack conditions essential for water resource managers in the mountainous regions. Here, analog statistical downscaling is demonstrated as an effective approach to enhance the spatial resolution of operational seasonal forecasts provided by the North American Multi-Model Ensemble. Downscaling was performed by building an analog ‘library’, in which corresponding model forecasts and observed values during the training period were stored. In the testing period, unseen model forecasts referenced the closest historical forecast from the analog library and applied the corresponding observational value for each point. This analysis indicates that downscaled products can capture localized features more accurately than the original coarse resolution forecasts, reducing forecast error across the western United States. Moreover, downscaling individual ensemble members—rather than downscaling the ensemble mean—further reduces forecasting error for their multi-model ensemble mean products. The greatest error reductions in the downscaled product, measured by root mean squared error (RMSE), were observed at low to mid-elevations (500–2000 meters), with 50%–70% improvement relative to the original forecasts. In the higher elevations (2000 meters and above), changes in RMSE relative to the original forecast were limited to 10%–30% improvements. The improvement is more substantial for forecast systems with 10 ensemble members compared to that with 4 members, but this relationship does not hold for the system with 24 ensemble members. These findings show that analog statistical downscaling can effectively address the spatial limitations of seasonal precipitation forecasts with minimal computational cost, providing a valuable framework for enhancing coarse resolution forecasting products while providing insights into the timing of ensemble mean calculations during the downscaling process.

Vernon, B. (ORCID:0009000891670689)↗

Machine learning assisted bayesian inference of mix and hot-spot conditions in NIF implosions

Experiments on the National Ignition Facility (NIF) have provided clear evidence of ablator material mixing into the Hot-Spot, leading to degraded performance. However, inferring the amount of mix and Hot-Spot conditions from typical experimental observations (e.g. x-ray spectra and images) is highly challenging. Here, we have developed an analysis method that utilizes machine learning assisted Bayesian inference to find the probability distributions of the Hot-Spot and mix conditions. This approach uses a neural network, trained on an idealized 2-dimensional representation of the Hot-Spot and mix distribution, and Bayesian inference to find the statistical distributions of Hot-Spot conditions that provide a match with observations. We have tested this method with synthetic data from simulations.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Simulator study of conventional general aviation instrument displays in path-following tasks with emphasis on pilot-induced oscillations

A study of the use of conventional general aviation instruments by general aviation pilots in a six degree of freedom, fixed base simulator was conducted. The tasks performed were tracking a VOR radial and making an ILS approach to landing. A special feature of the tests was that the sensitivity of the displacement indicating instruments (the RMI, CDI, and HSI) was kept constant at values corresponding to 5 n. mi. and 1.25 n. mi. from the station. Both statistical and pilot model analyses of the data were made. The results show that performance in path following improved with increases in display sensitivity up to the highest sensitivity tested. At this maximum test sensitivity, which corresponds to the sensitivity existing at 1.25 n. mi. for the ILS glide slope transmitter, tracking accuracy was no better than it was at 5 n. mi. from the station and the pilot aircraft system exhibited a marked reduction in damping. In some cases, a pilot induced, long period unstable oscillation occurred.

Adams, J. J.↗

Exploration of multifidelity UQ sampling strategies for computer network applications.

Network modeling is a powerful tool to enable rapid analysis of complex systems that can be challenging to study directly using physical testing. Here, two approaches are considered: emulation and simulation. The former runs real software on virtualized hardware, while the latter mimics the behavior of network components and their interactions in software. Although emulation provides an accurate representation of physical networks, this approach alone cannot guarantee the characterization of the system under realistic operative conditions. Operative conditions for physical networks are often characterized by intrinsic variability (payload size, packet latency, etc.) or a lack of precise knowledge regarding the network configuration (bandwidth, delays, etc.); therefore uncertainty quantification (UQ) strategies should be also employed. UQ strategies require multiple evaluations of the system with a number of evaluation instances that roughly increases with the problem dimensionality, i.e., the number of uncertain parameters. It follows that a typical UQ workflow for network modeling based on emulation can easily become unattainable due to its prohibitive computational cost. In this paper, a multifidelity sampling approach is discussed and applied to network modeling problems. The main idea is to optimally fuse information coming from simulations, which are a low-fidelity version of the emulation problem of interest, in order to decrease the estimator variance. By reducing the estimator variance in a sampling approach it is usually possible to obtain more reliable statistics and therefore a more reliable system characterization. Several network problems of increasing difficulty are presented. For each of them, the performance of the multifidelity estimator is compared with respect to the single fidelity counterpart, namely, Monte Carlo sampling. For all the test problems studied in this work, the multifidelity estimator demonstrated an increased efficiency with respect to MC.

97 MATHEMATICS AND COMPUTING↗

A Voxel-Based Approach to Explore Local Dose Differences Associated With Radiation-Induced Lung Damage

Purpose: To apply a voxel-based (VB) approach aimed at exploring local dose differences associated with late radiation-induced lung damage (RILD). Methods and Materials: An interinstitutional database of 98 patients who were Hodgkin lymphoma (HL) survivors treated with postchemotherapy supradiaphragmatic radiation therapy was analyzed in the study. Eighteen patients experienced late RILD, classified according to the Radiation Therapy Oncology Group scoring system. Each patient's computed tomographic (CT) scan was normalized to a single reference case anatomy (common coordinate system, CCS) through a log-diffeomorphic approach. The obtained deformation fields were used to map the dose of each patient into the CCS. The coregistration robustness and the dose mapping accuracy were evaluated by geometric and dose scores. Two different statistical mapping schemes for nonparametric multiple permutation inference on dose maps were applied, and the corresponding P<.05 significance lung subregions were generated. A receiver operating characteristic (ROC)-based test was performed on the mean dose extracted from each subregion. Results: The coregistration process resulted in a geometrically robust and accurate dose warping. A significantly higher dose was consistently delivered to RILD patients in voxel clusters near the peripheral medial-basal portion of the lungs. The area under the ROC curves (AUC) from the mean dose of the voxel clusters was higher than the corresponding AUC derived from the total lung mean dose. Conclusions: We implemented a framework including a robust registration process and a VB approach accounting for the multiple comparison problem in dose-response modeling, and applied it to a cohort of HL survivors to explore a local dose–RILD relationship in the lungs. Patients with RILD received a significantly greater dose in parenchymal regions where low doses (∼6 Gy) were delivered. Interestingly, the relation between differences in the high-dose range and RILD seems to lack a clear spatial signature.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

New research assessing the effect of engine operating conditions on regulated emissions of a 4-stroke motorcycle by test bench measurements

In the latest years the effect of powered two-wheelers on air polluting emissions is generally noteworthy all over the world, notwithstanding advances in internal combustion engines allowed to reduce considerably both fuel consumption and exhaust emissions of SI engines. Nowadays, in fact, these vehicles represent common means of quotidian moving, serving to meet daily urban transport necessities with a significant environmental impact on air quality. Besides, the emissive behavior of the two-wheelers measured under fixed legislative driving standards (and not on the local driving conditions) might not be sufficiently representative of real world motorcycle riding. The purpose of this investigation is a deeper research on emissive levels of in-use motorcycles equipped with last generation SI engines under real world driving behavior. In order to analyze the effect of vehicle instantaneous speed and acceleration on emissive behavior, instantaneous emissions of CO, HC and NO{sub X} were measured in the exhaust of a four-stroke motorcycle, equipped with a three-way catalyst and belonging to the Euro-3 legislative category. Experimental tests were executed on a chassis dynamometer bench in the laboratories of the National Research Council (Italy), during the Type Approval test cycle, at constant speed and under real-world driving cycles. This analytical-experimental investigation was executed with a methodology that improves vehicles emission assessment in comparison with the modeling approaches that are based on fixed legislative driving standards. The statistical processing results so obtained are very useful also in order to improve the database of emission models commonly used for estimating emissions from road transport sector, then they can be used to evaluate the environmental impact of last generation medium-size motorcycles under real driving behaviors.

54 ENVIRONMENTAL SCIENCES↗

A practical method to test the validity of the standard Gumbel distribution in logit-based multinomial choice models of travel behavior

Most multinomial choice models (e.g., the multinomial logit model) adopted in practice assume an extreme-value Gumbel distribution for the random components (error terms) of utility functions. This distributional assumption offers a closed-form likelihood expression when the utility maximization principle is applied to model choice behaviors. As a result, model coefficients can be easily estimated using the standard maximum likelihood estimation method. However, maximum likelihood estimators are consistent and efficient only if distributional assumptions on the random error terms are valid. It is therefore critical to test the validity of underlying distributional assumptions on the error terms that form the basis of parameter estimation and policy evaluation. In this paper, a practical yet statistically rigorous method is proposed to test the validity of the distributional assumption on the random components of utility functions in both the multinomial logit (MNL) model and multiple discrete-continuous extreme value (MDCEV) model. Based on a semi-nonparametric approach, a closed-form likelihood function that nests the MNL or MDCEV model being tested is derived. The proposed method allows traditional likelihood ratio tests to be used to test violations of the standard Gumbel distribution assumption. Simulation experiments are conducted to demonstrate that the proposed test yields acceptable Type-I and Type-II error probabilities at commonly available sample sizes. The test is then applied to three real-world discrete and discrete-continuous choice models. For all three models, the proposed test rejects the validity of the standard Gumbel distribution in most utility functions, calling for the development of robust choice models that overcome adverse effects of violations of distributional assumptions on the error terms in random utility functions.

30 DIRECT ENERGY CONVERSION↗

Fatigue criterion to system design, life and reliability

A generalized methodology to structural life prediction, design, and reliability based upon a fatigue criterion is advanced. The life prediction methodology is based in part on work of W. Weibull and G. Lundberg and A. Palmgren. The approach incorporates the computed life of elemental stress volumes of a complex machine element to predict system life. The results of coupon fatigue testing can be incorporated into the analysis allowing for life prediction and component or structural renewal rates with reasonable statistical certainty.

Zaretsky, E. V.↗

Fatigue criterion to system design, life and reliability

A generalized methodology to structural life prediction, design, and reliability based upon a fatigue criterion is advanced. The life prediction methodology is based in part on work of Weibull and Lundberg and Palmgren. The approach incorporates the computed life of elemental stress volumes of a complex machine element to predict system life. The results of coupon fatigue testing can be incorporated into the analysis allowing for life prediction and component or structural renewal rates with reasonable statistical certainty.

Zaretsky, E. V.↗

Practical Aspects of Multiple-Input Design for Aircraft System Identification Flight Tests

Practical aspects of multiple-input design methods for aircraft system identification flight tests are discussed. The orthogonal optimized multisine multiple-input design technique is explained and demonstrated, along with useful variations. Practical implementation and application issues associated with applying orthogonal optimized multisine inputs in flight are explained, and solutions are demonstrated using flight data from full-scale and subscale aircraft, as well as simulated flight data from an F-16 nonlinear simulation. Topics include truncated and extended excitation inputs, variations for control effectors with limited movement capabilities, aircraft dynamic response to excitation inputs, pilot implementation, statistical uncertainty quantification, multiple-input efficiency and effectiveness, flight testing with active feedback control, frequency-response estimation, control actuator modeling, and control interaction effects. Examples demonstrate that the orthogonal optimized multisine input design method is a practical, versatile, and effective approach for designing multiple excitation inputs for aircraft system identification flight tests.

Multiple-Input Design↗

Enabling High-Energy, High-Voltage Lithium-Ion Cells: Standardization of Coin-Cell Assembly, Electrochemical Testing, and Evaluation of Full Cells

Coin-cells are often the test format of choice for laboratories engaged in battery research and development as they provide a convenient platform for rapid testing of new materials on a small scale. However, reliable, reproducible data via the coin-cell format is inherently difficult, particularly in the full-cell configuration. In addition, statistical evaluation to prove the consistency and reliability of such data is often neglected. Herein we report on several studies aimed at formalizing physical process parameters and coin-cell construction related to full cells. Statistical analysis and performance benchmarking approaches are advocated as a means to more confidently track changes in cell performance. Finally, we show that trends in the electrochemical data obtained from coin-cells can be reliable and informative when standardized approaches are implemented in a consistent manner.

25 ENERGY STORAGE↗

A hybrid point-particle force model that combines physical and data-driven approaches

Here this study improves upon the physics-based pairwise interaction extended point-particle (PIEP) model. The PIEP model leverages our physical understanding to predict fluid mediated interactions between solid particles. By considering the relative location of neighboring particles, the PIEP model is able to provide better predictions than existing drag models. While the current physical PIEP model is a powerful tool, its assumptions lead to increased error in flows with higher particle volume fractions. To reduce this error, a regression algorithm makes direct use of the results of direct numerical simulations (DNS) of an array of monodisperse solid particles subjected to uniform ambient flow at varying Reynolds numbers. The resulting statistical model and the physical PIEP model are superimposed to construct a hybrid, physics-based data-driven PIEP model. It must be noted that the performance of a pure data-driven approach without the model-form provided by the physical PIEP model is substantially inferior. The hybrid model's predictive capabilities are analyzed using additional DNS data that was not part of training the data-driven model. In every case tested, the hybrid models resulting from the regression were capable of (1) improving upon the physical PIEP model's prediction and (2) recovering underlying relevant physics from the DNS data. As the particle volume fraction increases, the physical PIEP model loses the ability to approximate the forces experienced by the particles, but the statistical model continues to produce accurate approximations.

97 MATHEMATICS AND COMPUTING↗

Bayesian Framework for Bioburden Density Estimation in Planetary Protection

To comply with the international planetary protection policy set forth by the Committee on Space Research and NASA Agency level requirements, spacecraft destined to biologically sensitive planetary bodies have to minimize terrestrial biological contamination. Analysis, testing and inspection are the standard forward verification activities that are used to demonstrate compliance with the biological contamination requirements. For testing of spacecraft surface areas, a swab or wipe sample is collected from surfaces prior to last access and subsequently processed in the lab using NASA Approved Planetary Protection Methods for Culture Based Assays. Raw data resulting from this assay is then statistically treated employing a mathematical paradigm stemming from the 1970’s Viking Lander Project to generate the bioburden density and total microbial bioburden present. This standard approach arbitrarily accounts for error and provides an upper conservative bound as it reports the maximum number of spores estimated to be present on flight hardware surfaces. A bioburden density estimate factors in the following variables: the observed bioburden count, representative volume processed, sampling efficiencies. Notably, to account for error in the approach, a 0 observed count is arbitrarily changed to a count of 1 for each hardware grouping. The data generated by spacecraft bioburden verification campaigns in the past have resulted in <80% of wipes and <90% of swabs containing a bioburden count of 0. As such, having a robust and well documented statistical approach for dealing with the probability of low incident rates is necessary to be able to estimate spacecraft bioburden. Being able to statistically describe the bioburden distribution and associated confidence level is a gamechanger for the development of bioburden allocations during mission design and will allow for tighter management of risk throughout spacecraft build. Thus, Empirical Bayes statistical approach was evaluated to estimate the microbial bioburden on spacecraft to mitigate the aforementioned mathematical concerns and provide a probabilistic bioburden distribution of the flight hardware surface. For application of this approach to performing bioburden calculations, a range of non-informative prior assumptions on hardware surfaces are explored for Bayesian analyses while informative priors using posterior distributions from prior assays are utilized for Empirical Bayes analyses. Several non-informative priors are currently under investigation to assess fitness including use of these priors to serve as a foundation to build off of NASA specification values or a basis of risk to account for unknowns during the integration and testing process. Informative priors under consideration are generated using sampled bioburden values from hardware originating within like processing environments (e.g. vendor cleaning process or similar assembly process), temporal spacecraft status events as a prediction for hardware cleanliness of future samples, and heritage system bioburden actuals to predict allocation for subsequent missions. Informative priors and probabilistic bioburden distributions are then validated using data sets from the Mars Exploration Rover, Mars Science Laboratory, and InSight missions. Using Empirical Bayes approach to generate a probabilistic bioburden distribution as demonstrated through mission use cases provides a valid approach for use in the end-to-end requirements verification process.

97 - MATHEMATICS AND COMPUTING↗

More Data Needed for Failure Rate Estimation, Validation, and Uncertainty Reduction

The currently planned schedule for advanced Environmental Control and Life Support System (ECLSS) development and test activities to support human exploration missions is unlikely to generate sufficient data to enable statistically-supportable, precise Orbital Replacement Unit (ORU) failure rate estimates to meet existing crew safety expectations. Accurate and precise failure rate estimates are critical for missions beyond Low Earth Orbit (LEO) because current risk mitigation approaches –namely regular resupply and rapid abort capabilities –will not be available. Safe operations will depend on mission planners’ ability to accurately forecast spares demand and efficiently provide the necessary resources. However, even after more than a decade of International Space Station (ISS) ECLSS operations, a significant amount of uncertainty remains in failure rate estimates. Uncertain or inaccurate failure rates result in increased risk and spares mass. A Bayesian estimation approach, such as the one currently implemented by the ISS Program, can reduce uncertainty by incorporating engineering judgement into failure rate estimates. However, experience on the ISS and with other complex systems shows that these prior failure rate estimates are often inaccurate. In addition, prior estimates are typically point values; some level of uncertainty must be added to convert these into probability distributions for Bayesian updating, and there are several potential methods for doing so. Due to the low rate of data collection, any inaccuracy in theseprior estimates currently hasa strong influence on the end result. This paper examines the challenges associated with failure rate estimation, validation, and uncertainty reduction in the context of ECLSS development for beyond-LEO missions. A variety of techniques for generating and updating Bayesian priors are discussed and evaluated using both real-world and simulated data. Potential solutions for improving failure rate estimation, including testing additional units, are analyzed and discussed, and a set of recommendations are provided for next-generation system development activities.

Reliability↗

More Data Needed for Failure Rate Estimation, Validation, and Uncertainty Reduction

The currently planned schedule for advanced Environmental Control and Life Support System (ECLSS) development and test activities to support human exploration missions is unlikely to generate sufficient data to enable statistically-supportable, precise Orbital Replacement Unit (ORU) failure rate estimates to meet existing crew safety expectations. Accurate and precise failure rate estimates are critical for missions beyond Low Earth Orbit (LEO) because current risk mitigation approaches –namely regular resupply and rapid abort capabilities –will not be available. Safe operations will depend on mission planners’ ability to accurately forecast spares demand and efficiently provide the necessary resources. However, even after more than a decade of International Space Station (ISS) ECLSS operations, a significant amount of uncertainty remains in failure rate estimates. Uncertain or inaccurate failure rates result in increased risk and spares mass. A Bayesian estimation approach, such as the one currently implemented by the ISS Program, can reduce uncertainty by incorporating engineering judgement into failure rate estimates. However, experience on the ISS and with other complex systems shows that these prior failure rate estimates are often inaccurate. In addition, prior estimates are typically point values; some level of uncertainty must be added to convert these into probability distributions for Bayesian updating, and there are several potential methods for doing so. Due to the low rate of data collection, any inaccuracy in theseprior estimates currently hasa strong influence on the end result. This paper examines the challenges associated with failure rate estimation, validation, and uncertainty reduction in the context of ECLSS development for beyond-LEO missions. A variety of techniques for generating and updating Bayesian priors are discussed and evaluated using both real-world and simulated data. Potential solutions for improving failure rate estimation, including testing additional units, are analyzed and discussed, and a set of recommendations are provided for next-generation system development activities.

Reliability↗

An improved approach for flight readiness assessment

An improved methodology for quantitatively evaluating failure risk for a spaceflight system in order to assess flight readiness is presented. This methodology is of particular value when information relevant to failure prediction, including test experience and knowledge of parameters used in engineering analyses of failure phenomena, is limited. In this approach, engineering analysis models that characterize specific failure modes based on the physics and mechanics of the failure phenomena are used in a prescribed probabilistic structure to generate a failure probability distribution that is modified by test and flight experience in a Bayesian statistical procedure. The probabilistic structure and statistical methodology are generally applicable to any failure mode for which quantitative engineering analysis can be employed to characterize the failure phenomenon and are particularly well suited for use under the constraints on information availability that are typical of such spaceflight systems as the Space Shuttle and planetary spacecraft.

Moore, N. R.↗

Separating Gravitational Wave Signals from Instrument Artifacts

Central to the gravitational wave detection problem is the challenge of separating features in the data produced by astrophysical sources from features produced by the detector. Matched filtering provides an optimal solution for Gaussian noise, but in practice, transient noise excursions or "glitches" complicate the analysis. Detector diagnostics and coincidence tests can be used to veto many glitches which may otherwise be misinterpreted as gravitational wave signals. The glitches that remain can lead to long tails in the matched filter search statistics and drive up the detection threshold. Here we describe a Bayesian approach that incorporates a more realistic model for the instrument noise allowing for fluctuating noise levels that vary independently across frequency bands, and deterministic "glitch fitting" using wavelets as "glitch templates", the number of which is determined by a trans-dimensional Markov chain Monte Carlo algorithm. We demonstrate the method's effectiveness on simulated data containing low amplitude gravitational wave signals from inspiraling binary black hole systems, and simulated non-stationary and non-Gaussian noise comprised of a Gaussian component with the standard LIGO/Virgo spectrum, and injected glitches of various amplitude, prevalence, and variety. Glitch fitting allows us to detect significantly weaker signals than standard techniques.

Littenberg, Tyson B.↗

Processing Aleatory and Epistemic Uncertainties in Experimental Data From Sparse Replicate Tests of Stochastic Systems for Real-Space Model Validation

This paper presents a practical methodology for propagating and processing uncertainties associated with random measurement and estimation errors (that vary from test-to-test) and systematic measurement and estimation errors (uncertain but similar from test-to-test) in inputs and outputs of replicate tests to characterize response variability of stochastically varying test units. Also treated are test condition control variability from test-to-test and sampling uncertainty due to limited numbers of replicate tests. These aleatory variabilities and epistemic uncertainties result in uncertainty on computed statistics of output response quantities. The methodology was developed in the context of processing experimental data for “real-space” (RS) model validation comparisons against model-predicted statistics and uncertainty thereof. The methodology is flexible and sufficient for many types of experimental and data uncertainty, offering the most extensive data uncertainty quantification (UQ) treatment of any model validation method the authors are aware of. It handles both interval and probabilistic uncertainty descriptions and can be performed with relatively little computational cost through use of simple and effective dimension- and order-adaptive polynomial response surfaces in a Monte Carlo (MC) uncertainty propagation approach. A key feature of the progressively upgraded response surfaces is that they enable estimation of propagation error contributed by the surrogate model. Sensitivity analysis of the relative contributions of the various uncertainty sources to the total uncertainty of statistical estimates is also presented. Finally, the methodologies are demonstrated on real experimental validation data involving all the mentioned sources and types of error and uncertainty in five replicate tests of pressure vessels heated and pressurized to failure. Simple spreadsheet procedures are used for all processing operations.

97 MATHEMATICS AND COMPUTING↗