SEARCH · Search NASA
Results for “Bayesian Testing”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
The NANOGrav Nine-Year Data Set: Limits on the Isotropic Stochastic Gravitational Wave Background
We compute upper limits on the nanohertz-frequency isotropic stochastic gravitational wave background (GWB) using the 9 year data set from the North American Nanohertz Observatory for Gravitational Waves (NANOGrav) collaboration. Well-tested Bayesian techniques are used to set upper limits on the dimensionless strain amplitude (at a frequency of 1 yr(exp -1) for a GWB from supermassive black hole binaries of A(sub gw) less than 1.5 x 10(exp -15). We also parameterize the GWB spectrum with a broken power-law model by placing priors on the strain amplitude derived from simulations of Sesana and McWilliams et al. Using Bayesian model selection we find that the data favor a broken power law to a pure power law with odds ratios of 2.2 and 22 to one for the Sesana and McWilliams prior models, respectively. Using the broken power-law analysis we construct posterior distributions on environmental factors that drive the binary to the GW-driven regime including the stellar mass density for stellar-scattering, mass accretion rate for circumbinary disk interaction, and orbital eccentricity for eccentric binaries, marking the first time that the shape of the GWB spectrum has been used to make astrophysical inferences. Returning to a power-law model, we place stringent limits on the energy density of relic GWs, OMEGA(sub gw) (f) h squared less than 4.2 x 10(exp -10). Our limit on the cosmic string GWB, OMEGA(sub gw) (f) h squared less than 2.2 x 10(exp -10), translates to a conservative limit on the cosmic string tension with G mu less than 3.3 x 10(exp -8), a factor of four better than the joint Planck and high-l‚ cosmic microwave background data from other experiments.
Automated Monitoring with a BCP Fault-Decision Test
The Bayesian conditional probability (BCP) technique is a statistical fault-decision technique that is suitable as the mathematical basis of the fault-manager module in the automated-monitoring system and method described in the immediately preceding article. Within the automated-monitoring system, the fault-manager module operates in conjunction with the fault-detector module, which can be based on any one of several fault-detection techniques; examples include a threshold-limit-comparison technique or the BSP or SPRT technique mentioned in the preceding article. The present BCP technique is used to evaluate a series of one or more fault-detection events for the purpose of filtering out occasional false alarms produced by many types of statistical fault-detection procedures. The BCP technique increases the probability that an automated monitoring system produces a correct decision regarding the presence or absence of a fault. Because occasional false alarms are an inevitable consequence of the SPRT, BSP, or any other statistically based fault-detection test, there is a need for a logical procedure to distinguish between true and false alarms. Heretofore, it has been common practice to make a fault decision on an ad hoc basis for example by following a multiple-observation voting strategy in which a signal is declared to be indicative of a fault if m of the last n observations produced a fault-detection alarm. The BCP technique was developed to obtain results more reliable than those afforded by a voting strategy. The BCP technique involves a test in which one applies Bayesian inference techniques to a series of one or more single-observation alarms produced by a fault-detection test. One considers the last n decisions generated by a fault-detection test in order to evaluate the conditional probability that a failure is indicated (see figure). Each new decision reached by a fault-detection test is treated as a new piece of evidence about the state of the monitored asset, and the conditional probability of failure for the system is updated on the basis of this new evidence. The conditional probability of failure is compared with a predefined limit. For a probability below the limit, the asset is declared to be healthy. For a probability above the limit, the asset is declared to be faulty.
Model verification of large structural systems
A computer program for the application of parameter identification on the structural dynamic models of space shuttle and other large models with hundreds of degrees of freedom is described. Finite element, dynamic, analytic, and modal models are used to represent the structural system. The interface with math models is such that output from any structural analysis program applied to any structural configuration can be used directly. Processed data from either sine-sweep tests or resonant dwell tests are directly usable. The program uses measured modal data to condition the prior analystic model so as to improve the frequency match between model and test. A Bayesian estimator generates an improved analytical model and a linear estimator is used in an iterative fashion on highly nonlinear equations. Mass and stiffness scaling parameters are generated for an improved finite element model, and the optimum set of parameters is obtained in one step.
Evaluating Biosignatures for Life Detection
The goal here is to develop a conceptual framework for evaluating the ability of different biosignatures to provide evidence for the presence of life in planned missions or observational studies. The focus is on intrinsic characteristics of biosignatures in a given space environment rather than on their detection, which depends on the current state of technology. Suitable evaluation procedures are based on an extensive body of previous work on related problems in which different options are evaluated to make decisions in business, engineering, medical fields and social or political arena. In these cases, three approaches have proven to be particularly useful. Two of them, signal detection theory (SDT) and Bayesian hypothesis testing are based on probabilities. The third approach, multi-attribute utility (MAU) is based on utility theory.
A Bayesian approach to parameter and reliability estimation in the Poisson distribution.
For life testing procedures, a Bayesian analysis is developed with respect to a random intensity parameter in the Poisson distribution. Bayes estimators are derived for the Poisson parameter and the reliability function based on uniform and gamma prior distributions of that parameter. A Monte Carlo procedure is implemented to make possible an empirical mean-squared error comparison between Bayes and existing minimum variance unbiased, as well as maximum likelihood, estimators. As expected, the Bayes estimators have mean-squared errors that are appreciably smaller than those of the other two.
Sequential decision analysis for nonstationary stochastic processes
A formulation of the problem of making decisions concerning the state of nonstationary stochastic processes is given. An optimal decision rule, for the case in which the stochastic process is independent of the decisions made, is derived. It is shown that this rule is a generalization of the Bayesian likelihood ratio test; and an analog to Wald's sequential likelihood ratio test is given, in which the optimal thresholds may vary with time.
Reliability evaluation methodology for NASA applications
Liquid rocket engine technology has been characterized by the development of complex systems containing large number of subsystems, components, and parts. The trend to even larger and more complex system is continuing. The liquid rocket engineers have been focusing mainly on performance driven designs to increase payload delivery of a launch vehicle for a given mission. In otherwords, although the failure of a single inexpensive part or component may cause the failure of the system, reliability in general has not been considered as one of the system parameters like cost or performance. Up till now, quantification of reliability has not been a consideration during system design and development in the liquid rocket industry. Engineers and managers have long been aware of the fact that the reliability of the system increases during development, but no serious attempts have been made to quantify reliability. As a result, a method to quantify reliability during design and development is needed. This includes application of probabilistic models which utilize both engineering analysis and test data. Classical methods require the use of operating data for reliability demonstration. In contrast, the method described in this paper is based on similarity, analysis, and testing combined with Bayesian statistical analysis.
A computer program for model verification of dynamic systems
Dynamic model verification is the process whereby an analytical model of a dynamic system is compared with experimental data, and then qualified for future use in predicting system response in a different dynamic environment. There are various ways to conduct model verification. The approach adopted in MOVER II employs Bayesian statistical parameter estimation. Unlike curve fitting whose objective is to minimize the difference between some analytical function and a given quantity of test data (or curve), Bayesian estimation attempts also to minimize the difference between the parameter values of that function (the model) and their initial estimates, in a least squares sense. The objectives of dynamic model verification, therefore, are to produce a model which: (1) is in agreement with test data, (2) will assist in the interpretation of test data, (3) can be used to help verify a design, (4) will reliably predict performance, and (5) in the case of space structures, facilitate dynamic control.
Monte Carlo Bayesian Inference on a Statistical Model of Sub-gridcolumn Moisture Variability Using High-resolution Cloud Observations : Sensitivity Tests and Results - Part II
Part I presented a Monte Carlo Bayesian method for constraining a complex statistical model of GCM sub-gridcolumn moisture variability using high-resolution MODIS cloud data, thereby permitting large-scale model parameter estimation and cloud data assimilation. This part performs some basic testing of this new approach, verifying that it does indeed significantly reduce mean and standard deviation biases with respect to the assimilated MODIS cloud optical depth, brightness temperature and cloud top pressure, and that it also improves the simulated rotational-Ramman scattering cloud optical centroid pressure (OCP) against independent (non-assimilated) retrievals from the OMI instrument. Of particular interest, the Monte Carlo method does show skill in the especially difficult case where the background state is clear but cloudy observations exist. In traditional linearized data assimilation methods, a subsaturated background cannot produce clouds via any infinitesimal equilibrium perturbation, but the Monte Carlo approach allows finite jumps into regions of non-zero cloud probability. In the example provided, the method is able to restore marine stratocumulus near the Californian coast where the background state has a clear swath. This paper also examines a number of algorithmic and physical sensitivities of the new method and provides guidance for its cost-effective implementation. One obvious difficulty for the method, and other cloud data assimilation methods as well, is the lack of information content in the cloud observables on cloud vertical structure, beyond cloud top pressure and optical thickness, thus necessitating strong dependence on the background vertical moisture structure. It is found that a simple flow-dependent correlation modification due to Riishojgaard (1998) provides some help in this respect, by better honoring inversion structures in the background state.
Statistical Symbolic Execution with Informed Sampling
Symbolic execution techniques have been proposed recently for the probabilistic analysis of programs. These techniques seek to quantify the likelihood of reaching program events of interest, e.g., assert violations. They have many promising applications but have scalability issues due to high computational demand. To address this challenge, we propose a statistical symbolic execution technique that performs Monte Carlo sampling of the symbolic program paths and uses the obtained information for Bayesian estimation and hypothesis testing with respect to the probability of reaching the target events. To speed up the convergence of the statistical analysis, we propose Informed Sampling, an iterative symbolic execution that first explores the paths that have high statistical significance, prunes them from the state space and guides the execution towards less likely paths. The technique combines Bayesian estimation with a partial exact analysis for the pruned paths leading to provably improved convergence of the statistical analysis. We have implemented statistical symbolic execution with in- formed sampling in the Symbolic PathFinder tool. We show experimentally that the informed sampling obtains more precise results and converges faster than a purely statistical analysis and may also be more efficient than an exact symbolic analysis. When the latter does not terminate symbolic execution with informed sampling can give meaningful results under the same time and memory limits.
Quantifying parameters for Bayesian prior assumptions when estimating the probability of failure of software
Software reliability has become increasingly important, especially in life-critical situations. The ability to measure the results of testing and to quantify software reliability is needed. If this is accomplished, a certain minimum amount of reliability for a piece of software can be specified, and testing and/or other analysis may be done until that minimum number has been attained. There are many models for estimating software reliability. The accuracy of these models has been challenged and many revisions for the models and recalibration techniques have been devised. Of particular interest is the method of estimating the probability of failure of software when no failures have yet occurred in its current version as described by Miller. This model uses black box testing with formulae based on Bayesian estimation. The focus is on three interrelated issues: estimating the probability of failure when testing has revealed no errors; modifying this estimation when the input use distribution does not match the test distribution; and combining the results from random testing with other relevant information to obtain a possibly more accurate estimate of the probability of failure. Obtaining relevant information about the software and combining the results for a better estimate for the Miller model are discussed.
The effect of modeling dose uncertainty on low-boom community noise dose-response curves
In logistic dose-response modeling, failing to account for uncertainty in estimated doses can cause an artificial flattening or attenuation of the slope of the summary curve. In Lee et al. [J. Acoust. Soc. Am. 147(4), pp. 2222-2234 (2020)], data from two NASA low-amplitude sonic boom community noise survey tests were modeled using a Bayesian multilevel logistic regression (MLR) statistical model that assumed there was no uncertainty in the noise dose estimates. However, in these community tests, the noise dose uncertainty was estimated by Page et al. [NASA/CR-2014-218180 and NASA/CR-2020-220589/Volume I] using a leave-one-out method. In the current work, a term was added to extend the Bayesian MLR model to account for the estimated noise dose uncertainty quantified in the Page et al. analyses. This uncertainty term was included in two ways, either as classical or as Berkson uncertainty, and yield similar results. When the uncertainty is accounted for in the Bayesian MLR model, the dose-response curves become 5-10% steeper, but the difference in the noise dose that elicits a 5% highly annoyed response is small (less than 1 dB). This result is encouraging for future X-59 community tests whose survey area will be sparsely populated with noise monitors.
Composite Overwrapped Pressure Vessels (COPV): Flight Rationale for the Space Shuttle Program
Each Orbiter Vehicle (Space Shuttle Program) contains up to 24 Kevlar49/Epoxy Composite Overwrapped Pressure Vessels (COPV) for storage of pressurized gases. In the wake of the Columbia accident and the ensuing Return To Flight (RTF) activities, Orbiter engineers reexamined COPV flight certification. The original COPV design calculations were updated to include recently declassified Kevlar COPV test data from Lawrence Livermore National Laboratory (LLNL) and to incorporate changes in how the Space Shuttle was operated as opposed to orinigially envisioned. 2005 estimates for the probability of a catastrophic failure over the life of the program (from STS-1 through STS-107) were one-in-five. To address this unacceptable risk, the Orbiter Project Office (OPO) initiated a comprehensive investigation to understand and mitigate this risk. First, the team considered and eventually deemed unfeasible procuring and replacing all existing flight COPVs. OPO replaced the two vessels with the highest risk with existing flight spare units. Second, OPO instituted operational improvements in ground procedures to signficiantly reduce risk, without adversely affecting Shuttle capability. Third, OPO developed a comprehensive model to quantify the likelihood of occurrance. A fully-instrumented burst test (recording a lower burst pressure than expected) on a flight-certified vessel provided critical understanding of the behavior of Orbiter COPVs. A more accurate model was based on a newly-compiled comprehensive database of Kevlar data from LLNL and elsewhere. Considering hardware changes, operational improvements and reliability model refinements, the mean reliability was determined to be 0.998 for the remainder of the Shuttle Program (from 2007, for STS- 118 thru STS-135). Since limited hardware resources precluded full model validation through multiple tests, additional model confidence was sought through the first-ever Accelerated Stress Rupture Test (ASRT) of a flown flight article. A Bayesian statistical approach was developed to interpret possible test results. Since the lifetime observed in the ASRT exceeded initial estimates by one to two orders of magnitude, the Space Shuttle Program deemed there was significant conservatism in the model and accepted continued operation with existing flight hardware. Given the variability in tank-to-tank original prooftest response, a non-destructive evaluation (NDE) technique utilizing Raman Spectroscopy was developed to directly measure COPV residual stress state. Preliminary results showed that patterns of low fiber elastic strains over the outside vessel surface, together with measured permanent volume growth during proof, could be directly correlated to increased fiber stress ratios on the inside fibers adjacent to the liner, and thus reduced reliability.
Bayesian estimation of life parameters in the Weibull distribution.
Development of a Bayesian analysis of the scale and shape parameters in the Weibull distribution and the corresponding reliability function with respect to the usual life-testing procedures. For the scale parameter theta, Bayesian estimates of theta and reliability are obtained for the uniform, exponential, and inverted gamma prior probability densities. Bhattacharya's results (1967) for the one-parameter exponential life-testing distribution are reduced to a special case of these results. A fully Bayesian analysis of both the scale and shape parameters is developed by assuming independent prior distributions; since in the latter case, analytical tractability is not possible, Bayesian estimates are obtained through a conjunction of Monte Carlo simulation and numerical-integration techniques. In both cases, a computer simulation is carried out, and a comparison is made between the Bayesian and the corresponding minimum-variance unbiased, or maximum likelihood, estimates. As expected, the Bayesian estimates are superior.
Composite Overwrapped Pressure Vessels (COPV): Developing Flight Rationale for the Space Shuttle Program
Introducing composite vessels into the Space Shuttle Program represented a significant technical achievement. Each Orbiter vehicle contains 24 (nominally) Kevlar tanks for storage of pressurized helium (for propulsion) and nitrogen (for life support). The use of composite cylinders saved 752 pounds per Orbiter vehicle compared with all-metal tanks. The weight savings is significant considering each Shuttle flight can deliver 54,000 pounds of payload to the International Space Station. In the wake of the Columbia accident and the ensuing Return to Flight activities, the Space Shuttle Program, in 2005, re-examined COPV hardware certification. Incorporating COPV data that had been generated over the last 30 years and recognizing differences between initial Shuttle Program requirements and current operation, a new failure mode was identified, as composite stress rupture was deemed credible. The Orbiter Project undertook a comprehensive investigation to quantify and mitigate this risk. First, the engineering team considered and later deemed as unfeasible the option to replace existing all flight tanks. Second, operational improvements to flight procedures were instituted to reduce the flight risk and the danger to personnel. Third, an Orbiter reliability model was developed to quantify flight risk. Laser profilometry inspection of several flight COPVs identified deep (up to 20 mil) depressions on the tank interior. A comprehensive analysis was performed and it confirmed that these observed depressions were far less than the criterion which was established as necessary to lead to liner buckling. Existing fleet vessels were exonerated from this failure mechanism. Because full validation of the Orbiter Reliability Model was not possible given limited hardware resources, an Accelerated Stress Rupture Test of a flown flight vessel was performed to provide increased confidence. A Bayesian statistical approach was developed to evaluate possible test results with respect to the model credibility and thus flight rationale for continued operation of the Space Shuttle with existing flight hardware. A non-destructive evaluation (NDE) technique utilizing Raman Spectroscopy was developed to directly measure the overwrap residual stress state. Preliminary results provide optimistic results that patterns of fluctuation in fiber elastic strains over the outside vessel surface could be directly correlated with increased fiber stress ratios and thus reduced reliability.
Non-Targeted Effects and the Dose Response for Heavy Ion Tumorigenesis
BACKGROUND: There is no human epidemiology data available to estimate the heavy ion cancer risks experienced by astronauts in space. Studies of tumor induction in mice are a necessary step to estimate risks to astronauts. Previous experimental data can be better utilized to model dose response for heavy ion tumorigenesis and plan future low dose studies. DOSE RESPONSE MODELS: The Harderian Gland data of Alpen et al.[1-3] was re-analyzed [4] using non-linear least square regression. The data set measured the induction of Harderian gland tumors in mice by high-energy protons, helium, neon, iron, niobium and lanthanum with LET s ranging from 0.4 to 950 keV/micron. We were able to strengthen the individual ion models by combining data for all ions into a model that relates both radiation dose and LET for the ion to tumor prevalence. We compared models based on Targeted Effects (TE) to one motivated by Non-targeted Effects (NTE) that included a bystander term that increased tumor induction at low doses non-linearly. When comparing fitted models to the experimental data, we considered the adjusted R2, the Akaike Information Criteria (AIC), and the Bayesian Information Criteria (BIC) to test for Goodness of fit.In the adjusted R2test, the model with the highest R2values provides a better fit to the available data. In the AIC and BIC tests, the model with the smaller values of the summary value provides the better fit. The non-linear NTE models fit the combined data better than the TE models that are linear at low doses. We evaluated the differences in the relative biological effectiveness (RBE) and found the NTE model provides a higher RBE at low dose compared to the TE model. POWER ANALYSIS: The final NTE model estimates were used to simulate example data to consider the design of new experiments to detect NTE at low dose for validation. Power and sample sizes were calculated for a variety of radiation qualities including some not considered in the Harderian Gland data set and with different background tumor incidences. We considered different experimental designs with varying number of doses and varying low doses dependant on the LET of the radiation. The optimal design to detect a NTE for an individual ion had 4 doses equally spaced below a maximal dose where bending due to cell sterilization was < 2%. For example at 100 keV/micron we would irradiate at 0.03 Gy, 0.065 Gy, 0.13 Gy, and 0.26 Gy and require 850 mice including a control dose for a sensitivity to detect NTE with 80% power. Sample sizes could be improved by combining ions similar to the methods used with the Harderian Gland data.
An improved approach for flight readiness assessment
An improved methodology for quantitatively evaluating failure risk for a spaceflight system in order to assess flight readiness is presented. This methodology is of particular value when information relevant to failure prediction, including test experience and knowledge of parameters used in engineering analyses of failure phenomena, is limited. In this approach, engineering analysis models that characterize specific failure modes based on the physics and mechanics of the failure phenomena are used in a prescribed probabilistic structure to generate a failure probability distribution that is modified by test and flight experience in a Bayesian statistical procedure. The probabilistic structure and statistical methodology are generally applicable to any failure mode for which quantitative engineering analysis can be employed to characterize the failure phenomenon and are particularly well suited for use under the constraints on information availability that are typical of such spaceflight systems as the Space Shuttle and planetary spacecraft.