SEARCH · Search NASA

Results for “statistical metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

All sources NASA NTRS DOE OSTI

NASA NTRS2020

Recent Developments in Tolerancing Methods for Imaging Spectrometers

We discuss detailed tolerancing methods developed for imaging spectrometers at NASA Jet Propulsion Laboratory, California Institute of Technology using the Earth Surface Mineral Dust Source Investigation (EMIT) imaging spectrometer as an illustrative example. We tolerance five metrics simultaneously: along-track response function, cross-track response function, spectral response function, spectral centroid uniformity, and spatial centroid uniformity. A method to calculate tolerancing sensitivities for each metric directly, a method to statistically combine Monte Carlo files from multiple tolerancing runs, and an example summary error budgets that communicate the key and driving tolerances for each metric are discussed. These methods facilitate rapid and semi-automated assessment of the predicted performance of imaging spectrometer systems from design through to assembly and launch life cycle, using metrics that are directly relevant to the extraction of accurate spectroscopic data from these instruments.

Mouroulis, Pantazis↗

NASA NTRS1987

Robust statistical methods for automated outlier detection

The computational challenge of automating outlier, or blunder point, detection in radio metric data requires the use of nonstandard statistical methods because the outliers have a deleterious effect on standard least squares methods. The particular nonstandard methods most applicable to the task are the robust statistical techniques that have undergone intense development since the 1960s. These new methods are by design more resistant to the effects of outliers than standard methods. Because the topic may be unfamiliar, a brief introduction to the philosophy and methods of robust statistics is presented. Then the application of these methods to the automated outlier detection problem is detailed for some specific examples encountered in practice.

Jee, J. R.↗

NASA NTRS1983

Metric analysis and data validation across FORTRAN projects

The desire to predict the effort in developing or explaining the quality of software has led to the proposal of several metrics. As a step toward validating these metrics, the Software Engineering Laboratory (SEL) has analyzed the software science metrics, cyclomatic complexity, and various standard program measures for their relation to effort (including design through acceptance testing), development errors (both discrete and weighted according to the amount of time to locate and fix), and one another. The data investigated are collected from a project FORTRAN environment and examined across several projects at once, within individual projects and by reporting accuracy checks demonstrating the need to validate a database. When the data comes from individual programmers or certain validated projects, the metrics' correlations with actual effort seem to be strongest. For modules developed entirely by individual programmers, the validity ratios induce a statistically significant ordering of several of the metrics' correlations. When comparing the strongest correlations, neither software science's E metric cyclomatic complexity not source lines of code appears to relate convincingly better with effort than the others.

Basili, Victor R.↗

NASA NTRS2010

Regression Model Term Selection for the Analysis of Strain-Gage Balance Calibration Data

The paper discusses the selection of regression model terms for the analysis of wind tunnel strain-gage balance calibration data. Different function class combinations are presented that may be used to analyze calibration data using either a non-iterative or an iterative method. The role of the intercept term in a regression model of calibration data is reviewed. In addition, useful algorithms and metrics originating from linear algebra and statistics are recommended that will help an analyst (i) to identify and avoid both linear and near-linear dependencies between regression model terms and (ii) to make sure that the selected regression model of the calibration data uses only statistically significant terms. Three different tests are suggested that may be used to objectively assess the predictive capability of the final regression model of the calibration data. These tests use both the original data points and regression model independent confirmation points. Finally, data from a simplified manual calibration of the Ames MK40 balance is used to illustrate the application of some of the metrics and tests to a realistic calibration data set.

Ulbrich, Norbert Manfred↗

NASA NTRS2017

Filter Tuning Using the Chi-Squared Statistic

This paper examines the use of the Chi-square statistic as a means of evaluating filter performance. The goal of the process is to characterize the filter performance in the metric of covariance realism. The Chi-squared statistic is the value calculated to determine the realism of a covariance based on the prediction accuracy and the covariance values at a given point in time. Once calculated, it is the distribution of this statistic that provides insight on the accuracy of the covariance. The process of tuning an Extended Kalman Filter (EKF) for Aqua and Aura support is described, including examination of the measurement errors of available observation types, and methods of dealing with potentially volatile atmospheric drag modeling. Predictive accuracy and the distribution of the Chi-squared statistic, calculated from EKF solutions, are assessed.

Filter Tuning↗

NASA NTRS2022

Using Trajectory Smoothness Metrics to Identify Drones in Radar Track Data

The identification of unmanned aircraft systems (UAS) using trajectory data is considered. Specifically, a number of smoothness metrics are proposed, which can be used to distinguish UAS from other aerial objects even when they are engaged in accelerative maneuvers (non-constant-velocity flight). The metrics are evaluated on a data set from a UAS sense-and-avoid field test, which contains track data of aerial objects recorded by a vehicle-board radar system during a flight test. The metrics are found to effectively differentiate UAS from other objects such as birds for this data set. In addition, an initial statistical performance analysis of one of the smoothness metrics is undertaken, using 15 data sets deriving from multiple flight tests. The smoothness metric is shown to identify the target UAS with 95% accuracy (95% true positive rate), while achieving a false positive rate of less than 9%.

Sandip Roy↗

NASA NTRSDate not supplied

Uncertainty Estimates for Sonic-Boom Pressure Signatures and Loudness Carpets

A non-intrusive uncertainty quantification method is applied to computational analysis of supersonic, low-boom aircraft. The mean and standard deviation statistics of the pressure waveforms and loudness metrics are evaluated through use of numerical quadrature. The probability density function (p.d.f.) of these outputs is evaluated via kernel density estimation. The simulations use an inviscid, embedded-boundary Cartesian-mesh flow solver in the nearfield combined with an augmented Burgers’ equation solver for propagation in the farfield. The results show that the p.d.f. of the waveform is bimodal at shocks, which makes the mean and standard deviation statistics inappropriate. Despite this limitation, we show that the moment statistics can provide effective assessment of discrepancies when comparing with experimental data. This is demonstrated by presenting uncertainty analysis of a wind-tunnel test and showing that we significantly improve the predictions when we include the test uncertainties in the simulation. Normal distributions are obtained for the ground signature and loudness metrics, which is primarily due to the careful shaping of the low-boom waveform. Separation of variables and error control are used to reduce computational cost. We demonstrate that this is an efficient approach in the sense of balancing numerical errors in the statistics quadrature with discretization errors in the solvers.

ARMD↗

NASA NTRSDate not supplied

SLS Integrated Modal Test Uncertainty Quantification using the Hybrid Parametric Variation Method

Uncertainty in structural loading during launch is a significant concern in the development of spacecraft and launch vehicles. Small variations in launch vehicle and payload mode shapes and their interaction can result in significant variation in system loads. In many cases involving large aerospace systems it is difficult, not economical, or impossible to perform a system modal test. However, it is still vital to obtain test results that can be compared with analytical predictions to validate models. Instead, the “Building Block Approach” is used in which system components are tested individually. Component models are correlated and updated to agree as best they can with test results. The Space Launch System consists of a number of components that are assembled into a launch vehicle. Finite element models of the components are developed, reduced to Hurty/Craig-Bampton models and assembled to represent different phases of flight. The only opportunity to obtain modal test data from an assembled Space Launch System will be during the Integrated Modal Test. There is always uncertainty in every model, which flows into uncertainty in predicted system results. Uncertainty Quantification is used to determine statistical bounds on prediction accuracy based on model uncertainty. For the Space Launch System, model uncertainty is at the Hurty/Craig-Bampton component level. Uncertainty in the Hurty/Craig-Bampton components is quantified using the hybrid parametric variation approach that combines parametric and nonparametric uncertainty. Uncertainty in model form is one of the biggest contributors to uncertainty in complex built-up structures. This type of uncertainty cannot be represented by variations infinite element model input parameters and thus cannot be included in a parametric approach. However, model-form uncertainty can be modeled using a nonparametric approach based on random matrix theory. The hybrid parametric variation method requires the selection of dispersion values for the Hurty/Craig-Bampton fixed-interface eigenvalues, and the Hurty/Craig-Bampton stiffness matrices. Component test/analysis frequency error is used to identify the fixed-interface eigenvalue dispersions, while test/analysis cross-orthogonality is used to identify stiffness dispersion values. The hybrid parametric variation uncertainty quantification approach is applied to the Space Launch System Integrated Modal Test configuration. Monte Carlo analysis is performed, and statistics are determined for modal correlation metrics, frequency response from Integrated Modal Test shakers to selected accelerometers, as well as other metrics for determining how well target modes are excited and identified. If the predicted uncertainty envelopes future Integrated Modal Test results, then there will be increased confidence in the utility of the component-based hybrid parametric variation uncertainty quantification approach.

Uncertainty Quantification↗

NASA NTRS2004

The coverage of elliptical orbits using Ergodic theory

One of the key performance metrics for satellite constellations is the statistics of the visibility periods between the satellites and poins on the ground. Associated with this are other desirable communications statistis such as data through-put, link qualities, etc. Typically, the computation of coverage statistics requires the propagation of the trajectories. In this paper a new algorithm using differential geometry enables us to extend the Ergodic theory to eliptical orbits.

Lo, Martin W.↗

NASA NTRS2004

Parametric Studies of Flow Separation using Air Injection

Boundary Layer separation causes the airfoil to stall and therefore imposes dramatic performance degradation on the airfoil. In recent years, flow separation control has been one of the active research areas in the field of aerodynamics due to its promising performance improvements on the lifting device. These active flow separation control techniques include steady and unsteady air injection as well as suction on the airfoil surface etc. This paper will be focusing on the steady and unsteady air injection on the airfoil. Although wind tunnel experiments revealed that the performance improvements on the airfoil using injection techniques, the details of how the key variables such as air injection slot geometry and air injection angle etc impact the effectiveness of flow separation control via air injection has not been studied. A parametric study of both steady and unsteady air injection active flow control will be the main objective for this summer. For steady injection, the key variables include the slot geometry, orientation, spacing, air injection velocity as well as the injection angle. For unsteady injection, the injection frequency will also be investigated. Key metrics such as lift coefficient, drag coefficient, total pressure loss and total injection mass will be used to measure the effectiveness of the control technique. A design of experiments using the Box-Behnken Design is set up in order to determine how each of the variables affects each of the key metrics. Design of experiment is used so that the number of experimental runs will be at minimum and still be able to predict which variables are the key contributors to the responses. The experiments will then be conducted in the 1ft by 1ft wind tunnel according to the design of experiment settings. The data obtained from the experiments will be imported into JMP, statistical software, to generate sets of response surface equations which represent the statistical empirical model for each of the metrics as a function of the key variables. Next, the variables such as the slot geometry can be optimized using the build-in optimizer within JMP. Finally, a wind tunnel testing will be conducted using the optimized slot geometry and other key variables to verify the empirical statistical model. The long term goal for this effort is to assess the impacts of active flow control using air injection at system level as one of the task plan included in the NASAs URETI program with Georgia Institute of Technology.

Zhang, Wei↗

NASA NTRS2011

Validating LES for Jet Aeroacoustics

Engineers charged with making jet aircraft quieter have long dreamed of being able to see exactly how turbulent eddies produce sound and this dream is now coming true with the advent of large eddy simulation (LES). Two obvious challenges remain: validating the LES codes at the resolution required to see the fluid-acoustic coupling, and the interpretation of the massive datasets that result in having dreams come true. This paper primarily addresses the former, the use of advanced experimental techniques such as particle image velocimetry (PIV) and Raman and Rayleigh scattering, to validate the computer codes and procedures used to create LES solutions. It also addresses the latter problem in discussing what are relevant measures critical for aeroacoustics that should be used in validating LES codes. These new diagnostic techniques deliver measurements and flow statistics of increasing sophistication and capability, but what of their accuracy? And what are the measures to be used in validation? This paper argues that the issue of accuracy be addressed by cross-facility and cross-disciplinary examination of modern datasets along with increased reporting of internal quality checks in PIV analysis. Further, it is argued that the appropriate validation metrics for aeroacoustic applications are increasingly complicated statistics that have been shown in aeroacoustic theory to be critical to flow-generated sound.

Bridges, James↗

NASA NTRS2017

Mind the Gap: Exploring the Underground of the NASA Space Cancer Risk Model

The REID quantifies the lifetime risk of death from radiation-induced cancer in an exposed astronaut. The NASA Space Cancer Risk (NSCR) 2012 mode incorporates elements from physics, biology, epidemiology, and statistics to generate the REID distribution. The current model quantifies the space radiation environment, radiation quality, and dose-rate effects to estimate a NASA-weighted dose. This weighted dose is mapped to the excess risk of radiation-induced cancer mortality from acute exposures to gamma rays and then transferred to an astronaut population. Finally, the REID is determined by integrating this risk over the individual's lifetime. The calculated upper 95% confidence limit of the REID is used to restrict an astronaut's permissible mission duration (PMD) for a proposed mission. As a statistical quantity characterized by broad, subjective uncertainties, REID estimates for space missions result in wide distributions. Currently, the upper 95% confidence level is over 350% larger than the mean REID value, which can severely limit an astronaut's PMD. The model incorporates inputs from multiple scientific disciplines in the risk estimation process. Physics and particle transport models calculate how radiation moves through space, penetrates spacecraft, and makes its way to the human beings onboard. Epidemiological studies of exposures from atomic bombings, medical treatments, and power plants are used to quantify health risks from acute and chronic low linear energy transfer (LET) ionizing radiation. Biological studies in cellular and animal models using radiation at various LETs and energies inform quality metrics for ions present in space radiation. Statistical methodologies unite these elements, controlling for mathematical and scientific uncertainty and variability. Despite current progress, these research platforms contain knowledge gaps contributing to the large uncertainties still present in the model. The NASA Space Radiation Program Element (SRPE) defines the knowledge gaps that impact our understanding of the cancer risks. These gaps are outlined in NASA's Human Research Roadmap [4], which identifies the research questions and actions recommended for reducing the uncertainty in the current NSCR model and for formulation of future models. The greatest contributors to uncertainty in the current model include radiation quality, dose rate effects, and the transfer of exposure-based risk from other populations to an astronaut population. Future formulations of the risk model may benefit from including other potential sources of uncertainty such as space dosimetry, errors in human epidemiology data, and the impact of microgravity and other spaceflight stressors. Here, we discuss the current capabilities of the NSCR-2012 model and several immediate research needs, highlighting areas expected to have an operational impact on the current model schema. The following subway-style route map outlines the NSCR-2012 model (Green Line), emphasizing the research gaps in the Human Research Roadmap for risk of radiation-induced carcinogenesis (Stops on Dashed Lines). The map diagrams how these research gaps feed specific portions of the model.

Chappell, L. J.↗

NASA NTRS2011

Validating LES for Jet Aeroacoustics

Engineers charged with making jet aircraft quieter have long dreamed of being able to see exactly how turbulent eddies produce sound and this dream is now coming true with the advent of large eddy simulation (LES). Two obvious challenges remain: validating the LES codes at the resolution required to see the fluid-acoustic coupling, and the interpretation of the massive datasets that are produced. This paper addresses the former, the use of advanced experimental techniques such as particle image velocimetry (PIV) and Raman and Rayleigh scattering, to validate the computer codes and procedures used to create LES solutions. This paper argues that the issue of accuracy of the experimental measurements be addressed by cross-facility and cross-disciplinary examination of modern datasets along with increased reporting of internal quality checks in PIV analysis. Further, it argues that the appropriate validation metrics for aeroacoustic applications are increasingly complicated statistics that have been shown in aeroacoustic theory to be critical to flow-generated sound, such as two-point space-time velocity correlations. A brief review of data sources available is presented along with examples illustrating cross-facility and internal quality checks required of the data before it should be accepted for validation of LES.

Bridges, James↗

NASA NTRS1984

The independence of software metrics taken at different life-cycle stages

Over the past few years a large number of software metrics have been proposed and, in varying degrees, a number of these metrics have been subjected to empirical validation which demonstrated the utility of the metrics in the software development process. Attempts to classify these metrics and to determine if the metrics in these different classes appear to be measuring distinct attributes of the software product are studied. Statistical analysis is used to determine the degree of relationship among the metrics.

Kafura, D.↗

NASA NTRS2013

Metric Learning to Enhance Hyperspectral Image Segmentation

Unsupervised hyperspectral image segmentation can reveal spatial trends that show the physical structure of the scene to an analyst. They highlight borders and reveal areas of homogeneity and change. Segmentations are independently helpful for object recognition, and assist with automated production of symbolic maps. Additionally, a good segmentation can dramatically reduce the number of effective spectra in an image, enabling analyses that would otherwise be computationally prohibitive. Specifically, using an over-segmentation of the image instead of individual pixels can reduce noise and potentially improve the results of statistical post-analysis. In this innovation, a metric learning approach is presented to improve the performance of unsupervised hyperspectral image segmentation. The prototype demonstrations attempt a superpixel segmentation in which the image is conservatively over-segmented; that is, the single surface features may be split into multiple segments, but each individual segment, or superpixel, is ensured to have homogenous mineralogy.

Thompson, David R.↗

NASA NTRSDate not supplied

Turbulence Effects on Shaped Booms: Propagation Simulations Using KZKFourier

Upcoming X-59 aircraft flight tests as part of NASA’s Quesst Mission are expected to occur in a range of atmospheric conditions. Sensitivity of ground waveform acoustic metrics to turbulent perturbations during propagation complicates the determination of noise levels. A series of simulations through turbulence was executed using the KZKFourier model of Stout et al. to develop a database of results for expanding the functionality of NASA tools for estimating turbulence effects on shaped-boom ground waveforms. For 45 cases covering a seven-factor design space, multiple realizations of turbulence were generated to characterize statistical turbulence effects on levels of six acoustic metrics. An individual simulation in the study produced over one thousand waveforms across a virtual microphone array. Different measures were used to evaluate refinement of results with increasing number of simulations. Data suggest that mean effects and variability were most strongly influenced by propagation distance and velocity fluctuation intensity. Although local increases in acoustic metrics were common, the overall average result in all cases was a reduction in metric levels. Sensitivity of acoustic metrics varied, with mean reductions of up to 2 dB in Perceived Level occurring across the 45 cases.

turbulence↗

NASA NTRS1994

Understanding software faults and their role in software reliability modeling

This study is a direct result of an on-going project to model the reliability of a large real-time control avionics system. In previous modeling efforts with this system, hardware reliability models were applied in modeling the reliability behavior of this system. In an attempt to enhance the performance of the adapted reliability models, certain software attributes were introduced in these models to control for differences between programs and also sequential executions of the same program. As the basic nature of the software attributes that affect software reliability become better understood in the modeling process, this information begins to have important implications on the software development process. A significant problem arises when raw attribute measures are to be used in statistical models as predictors, for example, of measures of software quality. This is because many of the metrics are highly correlated. Consider the two attributes: lines of code, LOC, and number of program statements, Stmts. In this case, it is quite obvious that a program with a high value of LOC probably will also have a relatively high value of Stmts. In the case of low level languages, such as assembly language programs, there might be a one-to-one relationship between the statement count and the lines of code. When there is a complete absence of linear relationship among the metrics, they are said to be orthogonal or uncorrelated. Usually the lack of orthogonality is not serious enough to affect a statistical analysis. However, for the purposes of some statistical analysis such as multiple regression, the software metrics are so strongly interrelated that the regression results may be ambiguous and possibly even misleading. Typically, it is difficult to estimate the unique effects of individual software metrics in the regression equation. The estimated values of the coefficients are very sensitive to slight changes in the data and to the addition or deletion of variables in the regression equation. Since most of the existing metrics have common elements and are linear combinations of these common elements, it seems reasonable to investigate the structure of the underlying common factors or components that make up the raw metrics. The technique we have chosen to use to explore this structure is a procedure called principal components analysis. Principal components analysis is a decomposition technique that may be used to detect and analyze collinearity in software metrics. When confronted with a large number of metrics measuring a single construct, it may be desirable to represent the set by some smaller number of variables that convey all, or most, of the information in the original set. Principal components are linear transformations of a set of random variables that summarize the information contained in the variables. The transformations are chosen so that the first component accounts for the maximal amount of variation of the measures of any possible linear transform; the second component accounts for the maximal amount of residual variation; and so on. The principal components are constructed so that they represent transformed scores on dimensions that are orthogonal. Through the use of principal components analysis, it is possible to have a set of highly related software attributes mapped into a small number of uncorrelated attribute domains. This definitively solves the problem of multi-collinearity in subsequent regression analysis. There are many software metrics in the literature, but principal component analysis reveals that there are few distinct sources of variation, i.e. dimensions, in this set of metrics. It would appear perfectly reasonable to characterize the measurable attributes of a program with a simple function of a small number of orthogonal metrics each of which represents a distinct software attribute domain.

Munson, John C.↗

NASA NTRS2016

Uncertainty Assessment of the NASA Earth Exchange Global Daily Downscaled Climate Projections (NEX-GDDP) Dataset

The NASA Earth Exchange Global Daily Downscaled Projections (NEX-GDDP) dataset is comprised of downscaled climate projections that are derived from 21 General Circulation Model (GCM) runs conducted under the Coupled Model Intercomparison Project Phase 5 (CMIP5) and across two of the four greenhouse gas emissions scenarios (RCP4.5 and RCP8.5). Each of the climate projections includes daily maximum temperature, minimum temperature, and precipitation for the periods from 1950 through 2100 and the spatial resolution is 0.25 degrees (approximately 25 km x 25 km). The GDDP dataset has received warm welcome from the science community in conducting studies of climate change impacts at local to regional scales, but a comprehensive evaluation of its uncertainties is still missing. In this study, we apply the Perfect Model Experiment framework (Dixon et al. 2016) to quantify the key sources of uncertainties from the observational baseline dataset, the downscaling algorithm, and some intrinsic assumptions (e.g., the stationary assumption) inherent to the statistical downscaling techniques. We developed a set of metrics to evaluate downscaling errors resulted from bias-correction ("quantile-mapping"), spatial disaggregation, as well as the temporal-spatial non-stationarity of climate variability. Our results highlight the spatial disaggregation (or interpolation) errors, which dominate the overall uncertainties of the GDDP dataset, especially over heterogeneous and complex terrains (e.g., mountains and coastal area). In comparison, the temporal errors in the GDDP dataset tend to be more constrained. Our results also indicate that the downscaled daily precipitation also has relatively larger uncertainties than the temperature fields, reflecting the rather stochastic nature of precipitation in space. Therefore, our results provide insights in improving statistical downscaling algorithms and products in the future.

climate projection↗