Search NASA⌕ Search

SEARCH · Search NASA

Results for “Statistical Testing Approach”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 505 records · Page 28

Evaluating the Benefits of Bayesian Hierarchical Methods for Analyzing Heterogeneous Environmental Datasets: A Case Study of Marine Organic Carbon Fluxes

Large compilations of heterogeneous environmental observations are increasingly available as public databases, allowing researchers to test hypotheses across datasets. Statistical complexities arise when analyzing compiled data due to unbalanced spatial sampling, variable environmental context, mixed measurement techniques, and other reasons. Hierarchical Bayesian modeling is increasingly used in environmental science to describe these complexities, however few studies explicitly compare the utility of hierarchical Bayesian models to simpler and more commonly applied methods. Here we demonstrate the utility of the hierarchical Bayesian approach with application to a large compiled environmental dataset consisting of 5,741 marine vertical organic carbon flux observations from 407 sampling locations spanning eight biomes across the global ocean. We fit a global scale Bayesian hierarchical model that describes the vertical profile of organic carbon flux with depth. Profile parameters within a particular biome are assumed to share a common deviation from the global mean profile. Individual station-level parameters are then modeled as deviations from the common biome-level profile. The hierarchical approach is shown to have several benefits over simpler and more common data aggregation methods. First, the hierarchical approach avoids statistical complexities introduced due to unbalanced sampling and allows for flexible incorporation of spatial heterogeneitites in model parameters. Second, the hierarchical approach uses the whole dataset simultaneously to fit the model parameters which shares information across datasets and reduces the uncertainty up to 95% in individual profiles. Third, the Bayesian approach incorporates prior scientific information about model parameters; for example, the non-negativity of chemical concentrations or mass-balance, which we apply here. We explicitly quantify each of these properties in turn. We emphasize the generality of the hierarchical Bayesian approach for diverse environmental applications and its increasing feasibility for large datasets due to recent developments in Markov Chain Monte Carlo algorithms and easy-to-use high-level software implementations.

54 ENVIRONMENTAL SCIENCES↗

The effect of limiting aerodynamic and structural coupling in models of mistuned bladed disk vibration

A model has been developed for studying the effect of mistuning on bladed disk vibration which has the unique feature that the extent of aerodynamic and structural interaction which it simulates can be readily varied from full coupling of all blades on the disk to coupling of each blade with only its nearest neighbors. Simulations utilizing the resulting algorithm show that limited coupling models may be used to predict the statistical distribution of blade amplitudes that characterizes the mistuning effect, which in turn determines stage durability. This approach is used to study the effect of changing various system parameters on amplitude scatter. Gas density, the number of blades on the disk, disk stiffness, and the engine order of the excitation are considered. The results are used to draw some conclusions about how to improve laboratory tests and component design.

Basu, P.↗

Identification and interpretation of patterns in rocket engine data

A prototype software system was constructed to detect anomalous Space Shuttle Main Engine (SSME) behavior in the early stages of fault development significantly earlier than the indication provided by either redline detection mechanism or human expert analysis. The major task of the research project is to analyze ground test data, to identify patterns associated with the anomalous engine behavior, and to develop a pattern identification and detection system on the basis of this analysis. A prototype expert system which was developed on both PC and Symbolics 3670 lisp machine for detecting anomalies in turbopump vibration data was checked with data from ground tests 902-473, 902-501, 902-519, and 904-097 of the Space Shuttle Main Engine. The neural networks method was also applied to supplement the statistical method utilized in the prototype system to investigate the feasibility in detecting anomalies in turbopump vibration of SSME. In most cases the anomalies detected by the expert system agree with those reported by NASA. On the neural networks approach, the results are given the successful detection rate higher than 95 percent to identify either normal or abnormal running condition based on the experimental data as well as numerical simulation.

Lo, C. F.↗

Improving Metallic Thermal Protection System Hypervelocity Impact Resistance Through Design of Experiments Approach

A design of experiments approach has been implemented using computational hypervelocity impact simulations to determine the most effective place to add mass to an existing metallic Thermal Protection System (TPS) to improve hypervelocity impact protection. Simulations were performed using axisymmetric models in CTH, a shock-physics code developed by Sandia National Laboratories, and validated by comparison with existing test data. The axisymmetric models were then used in a statistical sensitivity analysis to determine the influence of five design parameters on degree of hypervelocity particle dispersion. Several damage metrics were identified and evaluated. Damage metrics related to the extent of substructure damage were seen to produce misleading results, however damage metrics related to the degree of dispersion of the hypervelocity particle produced results that corresponded to physical intuition. Based on analysis of variance results it was concluded that the most effective way to increase hypervelocity impact resistance is to increase the thickness of the outer foil layer. Increasing the spacing between the outer surface and the substructure is also very effective at increasing dispersion.

Poteet, Carl C.↗

Free-Energy Calculations. A Mathematical Perspective

Ion channels are pore-forming assemblies of transmembrane proteins that mediate and regulate ion transport through cell walls. They are ubiquitous to all life forms. In humans and other higher organisms they play the central role in conducting nerve impulses. They are also essential to cardiac processes, muscle contraction and epithelial transport. Ion channels from lower organisms can act as toxins or antimicrobial agents, and in a number of cases are involved in infectious diseases. Because of their important and diverse biological functions they are frequent targets of drug action. Also, simple natural or synthetic channels find numerous applications in biotechnology. For these reasons, studies of ion channels are at the forefront of biophysics, structural biology and cellular biology. In the last decade, the increased availability of X-ray structures has greatly advanced our understanding of ion channels. However, their mechanism of action remains elusive. This is because, in order to assist controlled ion transport, ion channels are dynamic by nature, but X-ray crystallography captures the channel in a single, sometimes non-native state. To explain how ion channels work, X-ray structures have to be supplemented with dynamic information. In principle, molecular dynamics (MD) simulations can aid in providing this information, as this is precisely what MD has been designed to do. However, MD simulations suffer from their own problems, such as inability to access sufficiently long time scales or limited accuracy of force fields. To assess the reliability of MD simulations it is only natural to turn to the main function of channels - conducting ions - and compare calculated ionic conductance with electrophysiological data, mainly single channel recordings, obtained under similar conditions. If this comparison is satisfactory it would greatly increase our confidence that both the structures and our computational methodologies are sufficiently accurate. Channel conductance, defined as the ratio of ionic current through the channel to applied voltage, can be calculated in MD simulations by way of applying an external electric field to the system and counting the number of ions that traverse the channel per unit time. If the current is small, a voltage significantly higher than the experimental one needs to be applied to collect sufficient statistics of ion crossing events. Then, the calculated conductance has to be extrapolated to the experimental voltage using procedures of unknown accuracy. Instead, we propose an alternative approach that applies if ion transport through channels can be described with sufficient accuracy by the one-dimensional diffusion equation in the potential given by the free energy profile and applied voltage. Then, it is possible to test the assumptions of the equation, recover the full voltage/current dependence, determine the reliability of the calculated conductance and reconstruct the underlying (equilibrium) free energy profile, all from MD simulations at a single voltage. We will present the underlying theory, model calculations that test this theory and simulations on ion conductance through a channel that has been extensively studied experimentally. To our knowledge this is the first case in which the complete, experimentally measured dependence of the current on applied voltage has been reconstructed from MD simulations.

free energy↗

MARGInS Model-Based Analysis of Realizable Goals in Systems

The high complexity of modern aircraft and spacecraft requires elaborate Verification and Validation (V&V) approaches to make sure that such complex systems work properly and reliably. MARGInS is a framework for the analysis, understanding, and prediction of the behavior of a complex, hybrid system. MARGInS contains a set of machine learning and statistical algorithms for multivariate clustering, treatment learning, critical factor determination, time-series analysis, event prediction, and safety-boundary detection and characterization. The framework supports system testing and can be configured to find novel features in test suites, determine classes of behavior, propose new experiments that can efficiently explore and characterize the boundaries between classes of system behavior, and to create visualizations and reports.

He, Yuning↗

Exploring an Ensemble-Based Approach to Atmospheric Climate Modeling and Testing at Scale

A strict throughput requirement has placed a cap on the degree to which we can depend on the execution of single, long, fine spatial grid simulations to explore global atmospheric climate behavior. Alternatively, running an ensemble of short simulations is computationally more efficient. We test the null hypothesis that the climate statistics of a full-complexity atmospheric model derived from an ensemble of independent short simulation is equivalent to that from an equilibrated long simulation. The climate of short simulation ensembles is statistically distinguishable from that of a long simulation in terms of the distribution of global annual means, largely due to the presence of low-frequency atmospheric intrinsic variability in the long simulation. We also find that model climate statistics of the simulation ensemble are sensitive to the choice of compiler optimizations. While some answer-changing optimization choices do not effect the climate state in terms of mean, variability and extremes, aggressive optimizations can result in significantly different climate states.

54 ENVIRONMENTAL SCIENCES↗

Statistical Sampling Considerations in Multifamily Housing

In the past 30 years, some governmental agencies and private-sector owners of multifamily housing properties have limited radon screening to partial screening (e.g., the testing of a subset of the multifamily housing dwelling units). While sampling methods have differed, most have required that if any one unit was found to be “positive,” or have radon levels ≥4 pCi/L, all remaining untested housing units would be tested (e.g. trip-wire approach).

63 RADIATION, THERMAL, AND OTHER ENVIRON. POLLUTAN↗

An application of survival analysis to disruption prediction via Random Forests</strong

We report one of the most pressing challenges facing the fusion community is adequately mitigating or, even better, avoiding disruptions of tokamak plasmas. However, before this can be done, disruptions must first be predicted with sufficient warning time to actuate a response. The established field of survival analysis provides a convenient statistical framework for time-to-event (i.e. time-to-disruption) studies. This paper demonstrates the integration of an existing disruption prediction machine learning algorithm with the Kaplan–Meier estimator of survival probability. Specifically discussed are the implied warning times from binary classification of disruption databases and the interpretation of output signals from Random Forest algorithms trained and tested on these databases. This survival analysis approach is applied to both smooth and noisy test data to highlight important features of the survival and hazard functions. In addition, this method is applied to three Alcator C-Mod plasma discharges and compared to a threshold-based scheme for triggering alarms. In one case, both techniques successfully predict the disruption; although, in another, neither warns of the impending disruption with enough time to mitigate. For the final discharge, the survival analysis approach could avoid the false alarm triggered by the threshold method. Limitations of this analysis and opportunities for future work are also presented.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Heavily doped transparent-emitter regions in junction solar cells, diodes, and transistors

The paper presents an analytical treatment of transparent-emitter devices, particularly solar cells, that is more complete than previously available treatments. The proposed approach includes the effects of bandgap narrowing, Fermi-Dirac statistics, built-in field due to impurity profile, and a finite surface recombination velocity at the emitter surface. It is demonstrated that the transparent-emitter model can predict experimental values of Voc observed on n(plus)-p thin diffused junction silicon solar cells made on low-resistivity (0.1 ohm-cm) substrates. A test is included for the self-consistent validity of the transparent-emitter model. This test compares the calculated transit time of minority carriers across the emitter with the Auger-impact minority-carrier lifetime within the emitter region.

Shibib, M. A.↗

A Comparative Study of Probability Collectives Based Multi-agent Systems and Genetic Algorithms

We compare Genetic Algorithms (GA's) with Probability Collectives (PC), a new framework for distributed optimization and control. In contrast to GA's, PC-based methods do not update populations of solutions. Instead they update an explicitly parameterized probability distribution p over the space of solutions. That updating of p arises as the optimization of a functional of p. The functional is chosen so that any p that optimizes it should be p peaked about good solutions. The PC approach works in both continuous and discrete problems. It does not suffer from the resolution limitation of the finite bit length encoding of parameters into GA alleles. It also has deep connections with both game theory and statistical physics. We review the PC approach using its motivation as the information theoretic formulation of bounded rationality for multi-agent systems. It is then compared with GA's on a diverse set of problems. To handle high dimensional surfaces, in the PC method investigated here p is restricted to a product distribution. Each distribution in that product is controlled by a separate agent. The test functions were selected for their difficulty using either traditional gradient descent or genetic algorithms. On those functions the PC-based approach significantly outperforms traditional GA's in both rate of descent, trapping in false minima, and long term optimization.

Huang, Chien-Feng↗

HOPE: An On-Line Piloted Handling Qualities Experiment Data Book

A novel on-line database for capturing most of the information obtained during piloted handling qualities experiments (either flight or simulated) is described. The Hyperlinked Overview of Piloted Evaluations (HOPE) web application is based on an open-source object-oriented Web-based front end (Ruby-on-Rails) that can be used with a variety of back-end relational database engines. The hyperlinked, on-line data book approach allows an easily-traversed way of looking at a variety of collected data, including pilot ratings, pilot information, vehicle and configuration characteristics, test maneuvers, and individual flight test cards and repeat runs. It allows for on-line retrieval of pilot comments, both audio and transcribed, as well as time history data retrieval and video playback. Pilot questionnaires are recorded as are pilot biographies. Simple statistics are calculated for each selected group of pilot ratings, allowing multiple ways to aggregate the data set (by pilot, by task, or by vehicle configuration, for example). Any number of per-run or per-task metrics can be captured in the database. The entire run metrics dataset can be downloaded in comma-separated text for further analysis off-line. It is expected that this tool will be made available upon request

Jackson, E. B.↗

Combining RNA-SEQ Datasets from NASA GENELAB: An Evaluation of Correction Methods

Background: Conducting space biology experiments aboard the International Space Station, particularly those utilizing complex model organisms like mice, is expensive and difficult due to limited crew availability, hardware, and space. As a result, sample numbers from these studies are low, reducing the statistical power of any one experiment. Aggregating spaceflight datasets serves as a method to increase sample numbers, allowing for novel insights through bioinformatic analysis of ‘omics data from merged datasets. However, aggregating datasets can introduce unwanted variation including 1) differences in sample handling, processing, and sequencing platforms between datasets (technical variation) as well as 2) differences in experimental design between datasets. Methods: In the present study, NASA GeneLab-hosted RNAseq datasets from mouse liver tissues were used to evaluate several statistical methods to correct for this unwanted variation through two approaches, reference-based and standard. The following correction algorithms were applied with (reference-based) and/or without (standard) considering Universal Mouse RNA Reference samples: ComBat and ComBat_seq from the SVA package, the median polish, empirical Bayes, and ANOVA-based algorithms from the MBatch package, and negative binomial regression normalization in the DESeq2 package. For each approach, after the correction algorithm was applied, differential gene expression (DGE) analysis of flight and ground control samples was performed with the combined data. The robustness of each tool was evaluated using BatchQC to determine statistical differences between datasets before and after correction, Principal Component Analysis to evaluate global gene expression in samples before and after correction, and by comparing DGE analysis of individual datasets and combined datasets before and after correction. Results: The results showed that the reference-based approach introduced several additional (and likely artificial) differentially expressed genes when compared with the respective standard approach. Conclusions: Of the methods tested, standard ComBat_seq and DESeq2 were identified as the most robust correction methods for combining spaceflight mouse liver RNAseq datasets hosted on GeneLab.

Finsam Samson↗

Single-pixel interior filling function approach for detecting and correcting errors in particle tracking

We present a general method for detecting and correcting biases in the outputs of particle-tracking experiments. Our approach is based on the histogram of estimated positions within pixels, which we term the single-pixel interior filling function (SPIFF). We use the deviation of the SPIFF from a uniform distribution to test the veracity of tracking analyses from different algorithms. Unbiased SPIFFs correspond to uniform pixel filling, whereas biased ones exhibit pixel locking, in which the estimated particle positions concentrate toward the centers of pixels. Although pixel locking is a well-known phenomenon, we go beyond existing methods to show how the SPIFF can be used to correct errors. Here, the key is that the SPIFF aggregates statistical information from many single-particle images and localizations that are gathered over time or across an ensemble, and this information augments the single-particle data. We explicitly consider two cases that give rise to significant errors in estimated particle locations: undersampling the point spread function due to small emitter size and intensity overlap of proximal objects. In these situations, we show how errors in positions can be corrected essentially completely with little added computational cost. Additional situations and applications to experimental data are explored in SI Appendix. In the presence of experimental-like shot noise, the precision of the SPIFF-based correction achieves (and can even exceed) the unbiased Cramér–Rao lower bound. We expect the SPIFF approach to be useful in a wide range of localization applications, including single-molecule imaging and particle tracking, in fields ranging from biology to materials science to astronomy.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Approach to Quality Assurance for Complex Environmental Modeling - 20407

A strong Quality Assurance (QA) program for complex environmental modeling is essential for regulatory and public acceptance and trust, but it does not have to be onerous. Neptune and Company, Inc. (Neptune) has developed a strong QA program that improves transparency, traceability, reproducibility, and therefore, defensibility and trust. Neptune's QA program has evolved over the past 27 years, transitioning from an ad hoc QA program, to a program that is currently Nuclear Quality Assurance-1 (NQA-1) compliant, and will soon be NQA-1/DOE approved for DOE EM modeling work. In addition, Neptune is a qualified laboratory assessor and qualified auditor for reference materials, a proficiency testing provider, and fully complies with the Environmental Protection Agency (EPA) QA program. Neptune's President and CEO, Kelly Black, was recently named the Chairperson of the International Organization for Standardization (ISO) Technical Committee 69, Application of Statistical Methods. A strong QA program has been developed for Neptune's radiological performance assessment (PA) program. Although pieces of their QA program are currently in development, their current program includes document control using Subversion, issue tracking and work flow tracking using JIRA, and transparency and traceability via a system of 'calc sheets' for documentation of all data analysis and modeling combined with 'check print' documentation of QA checking, and rigorous model testing and configuration control. All work is reviewed by an independent subject matter expert who is not associated with the collection and assembly of information, for an internal peer review. The program is enforced with a handful of Standard Operation Procedures (SOPs), Work Instructions, QA Project Plans (QAPPs), and Quality Management Plans (QMPs) that are updated frequently, with required annual training and acknowledgment. In addition, effective communication of modeling approaches and results to clients and stakeholders are integral to their QA program. Many of their models are built using the GoldSim modeling platform, for which their models are well-known for their level of transparency, documentation, and QA traceability. This level of QA is also applied to their process-level models. Neptune has taken some lessons learned from the extremely rigorous QA program required for the Yucca Mountain Project (YMP), and imposed the YMP requirements of complete traceability, transparency, and reproducibility, but avoided the inflexibility of a QA program that likely contributed to the suspect e-mails in 2005 that resulted in loss of trust in the YMP, and in nuclear waste disposal (and nuclear power) in general. There are far too many examples of loss of public trust due to poor QA that could have been easily avoided with a simple and straight-forward QA program combined with stakeholder engagement. The purpose of this paper is to share features of Neptune's Radiological Performance Assessment program QA program, and some of their QA related lessons learned over the past 27 years of QA for complex environmental modeling. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

A Data-Driven Passive Islanding Detection Scheme

This paper proposes a novel statistical parameters based data driven approach for passive islanding detection. The proposed scheme relies in the voltage data, measured at the point of common coupling, and is comprised of three stages. The Stage-1 quickly detects what is termed as coarse islanding, Stage-2 computes the decaying DC detector (DDCD), and 3) finally, the digital relay logic (DRL) is devised in Stage-3. The proposed scheme is tested as per the IEEE-1547, UL 1741 and IEC-62116 standards. Various tests are investigated on the modified IEEE-13 bus system, modeled in PSCAD/EMTDC, under stringent test scenarios such as power mismatch, induction motor starting effect, capacitor bank switching and various level of loads changes, considered for both single and multiple DG systems. The test results reveal that the proposed scheme can detect islanding in less than 2 cycles, and results in nearly zero non detection zone (NDZ). Also, the scheme can very well be used for single-phase microgrids too as it relies only on the single phase voltage information.

anti-islanding protection↗

Optimization of the deep neural network parameters for generating homogenized fuel assembly data for nodal codes

Homogenized fuel assembly (FA) data is a typical input data for nodal codes. Generating that data, however, could be time-consuming. One of promising ways to mitigate the computational burden of generating macroscopic cross-sections is to use trained artificial neural network (ANN) models for predicting nuclear data. However, there is a challenge to make the model support variable FA geometry. In this work, two most common types of FA were combined in one ANN model. Since there could be multiple ways of converting 2-dimensional FA data into 1-dimensional input vector for ANN, three different approaches of data flattening were evaluated. The input parameters included each fuel pin enrichment, fuel temperature, moderator temperature and boron concentration. The output parameters were 2-group macroscopic cross-sections (XS) and pin power distribution (HFF). A fully connected deep neural network (DNN) model was trained and tested using pre-generated data obtained with lattice physics code STREAM. The results of this study showed no statistically significant difference in the accuracy of XS and HFF generation for all 3 tested input vector orders. This means that fully connected DNN for XS generation demonstrated input sequence invariance. Results of comparing predicted XS data with reference solutions were found sufficiently close considering the reduction of computation time offered by ANN. Mean relative difference (MRD) for all output XS parameters was found below 0.7%, while HFF MRD was found higher compared to XS values, in some cases slightly exceeding 1%, mostly near guide tube locations. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

An application of survival analysis to disruption prediction via Random Forests

One of the most pressing challenges facing the fusion community is adequately mitigating or, even better, avoiding disruptions of tokamak plasmas. However, before this can be done, disruptions must first be predicted with sufficient warning time to actuate a response. The established field of survival analysis provides a convenient statistical framework for time-to-event (i.e. time-to-disruption) studies. This paper demonstrates the integration of an existing disruption prediction machine learning algorithm with the Kaplan-Meier estimator of survival probability. Specifically discussed are the implied warning times from binary classification of disruption databases and the interpretation of output signals from Random Forest algorithms trained and tested on these databases. This survival analysis approach is applied to both smooth and noisy test data to highlight important features of the survival and hazard functions. In addition, this method is applied to three Alcator C-Mod plasma discharges and compared to a threshold-based scheme for triggering alarms. In one case, both techniques successfully predict the disruption; although, in another, neither warns of the impending disruption with enough time to mitigate. For the final discharge, the survival analysis approach could avoid the false alarm triggered by the threshold method. Limitations of this analysis and opportunities for future work are also presented.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗