Search NASA⌕ Search

SEARCH · Search NASA

Results for “metric standardization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
NASA NTRS2010

A Singular Perturbation Approach for Time-Domain Assessment of Phase Margin

This paper considers the problem of time-domain assessment of the Phase Margin (PM) of a Single Input Single Output (SISO) Linear Time-Invariant (LTI) system using a singular perturbation approach, where a SISO LTI fast loop system, whose phase lag increases monotonically with frequency, is introduced into the loop as a singular perturbation with a singular perturbation (time-scale separation) parameter Epsilon. First, a bijective relationship between the Singular Perturbation Margin (SPM) max and the PM of the nominal (slow) system is established with an approximation error on the order of Epsilon(exp 2). In proving this result, relationships between the singular perturbation parameter Epsilon, PM of the perturbed system, PM and SPM of the nominal system, and the (monotonically increasing) phase of the fast system are also revealed. These results make it possible to assess the PM of the nominal system in the time-domain for SISO LTI systems using the SPM with a standardized testing system called "PM-gauge," as demonstrated by examples. PM is a widely used stability margin for LTI control system design and certification. Unfortunately, it is not applicable to Linear Time-Varying (LTV) and Nonlinear Time-Varying (NLTV) systems. The approach developed here can be used to establish a theoretical as well as practical metric of stability margin for LTV and NLTV systems using a standardized SPM that is backward compatible with PM.

Zhu, J. Jim↗
NASA NTRS1982

Spanwise loading distribution and wake velocity surveys of a semi-span wing

The spanwise distribution of bound circulation on a semi-span wing and the flow velocities in its wake were measured in a wind tunnel. Particular attention was given to documenting the flow velocities in and around the development tip vortex. A two-component laser velocimeter was used to make the velocity measurements. The spanwise distribution of bound circulation, three components of the time-averaged velocities throughout the near wake their standard deviations, and the integrated forces and moments on a metric tip as measured by an internal strain gage balance are presented without discussion.

Felker, F. F., III↗
NASA NTRS1991

Performance analysis of LAN bridges and routers

Bridges and routers are used to interconnect Local Area Networks (LANs). The performance of these devices is important since they can become bottlenecks in large multi-segment networks. Performance metrics and test methodology for bridges and routers were not standardized. Performance data reported by vendors is not applicable to the actual scenarios encountered in an operational network. However, vendor-provided data can be used to calibrate models of bridges and routers that, along with other models, yield performance data for a network. Several tools are available for modeling bridges and routers - Network II.5 was used. The results of the analysis of some bridges and routers are presented.

Hajare, Ankur R.↗
NASA NTRS2015

Improving Climate Projections Using "Intelligent" Ensembles

Recent changes in the climate system have led to growing concern, especially in communities which are highly vulnerable to resource shortages and weather extremes. There is an urgent need for better climate information to develop solutions and strategies for adapting to a changing climate. Climate models provide excellent tools for studying the current state of climate and making future projections. However, these models are subject to biases created by structural uncertainties. Performance metrics-or the systematic determination of model biases-succinctly quantify aspects of climate model behavior. Efforts to standardize climate model experiments and collect simulation data-such as the Coupled Model Intercomparison Project (CMIP)-provide the means to directly compare and assess model performance. Performance metrics have been used to show that some models reproduce present-day climate better than others. Simulation data from multiple models are often used to add value to projections by creating a consensus projection from the model ensemble, in which each model is given an equal weight. It has been shown that the ensemble mean generally outperforms any single model. It is possible to use unequal weights to produce ensemble means, in which models are weighted based on performance (called "intelligent" ensembles). Can performance metrics be used to improve climate projections? Previous work introduced a framework for comparing the utility of model performance metrics, showing that the best metrics are related to the variance of top-of-atmosphere outgoing longwave radiation. These metrics improve present-day climate simulations of Earth's energy budget using the "intelligent" ensemble method. The current project identifies several approaches for testing whether performance metrics can be applied to future simulations to create "intelligent" ensemble-mean climate projections. It is shown that certain performance metrics test key climate processes in the models, and that these metrics can be used to evaluate model quality in both current and future climate states. This information will be used to produce new consensus projections and provide communities with improved climate projections for urgent decision-making.

Baker, Noel C.↗
NASA NTRS2015

Improving Climate Projections Using "Intelligent" Ensembles

Recent changes in the climate system have led to growing concern, especially in communities which are highly vulnerable to resource shortages and weather extremes. There is an urgent need for better climate information to develop solutions and strategies for adapting to a changing climate. Climate models provide excellent tools for studying the current state of climate and making future projections. However, these models are subject to biases created by structural uncertainties. Performance metrics-or the systematic determination of model biases-succinctly quantify aspects of climate model behavior. Efforts to standardize climate model experiments and collect simulation data-such as the Coupled Model Intercomparison Project (CMIP)-provide the means to directly compare and assess model performance. Performance metrics have been used to show that some models reproduce present-day climate better than others. Simulation data from multiple models are often used to add value to projections by creating a consensus projection from the model ensemble, in which each model is given an equal weight. It has been shown that the ensemble mean generally outperforms any single model. It is possible to use unequal weights to produce ensemble means, in which models are weighted based on performance (called "intelligent" ensembles). Can performance metrics be used to improve climate projections? Previous work introduced a framework for comparing the utility of model performance metrics, showing that the best metrics are related to the variance of top-of-atmosphere outgoing longwave radiation. These metrics improve present-day climate simulations of Earth's energy budget using the "intelligent" ensemble method. The current project identifies several approaches for testing whether performance metrics can be applied to future simulations to create "intelligent" ensemble-mean climate projections. It is shown that certain performance metrics test key climate processes in the models, and that these metrics can be used to evaluate model quality in both current and future climate states. This information will be used to produce new consensus projections and provide communities with improved climate projections for urgent decision-making.

Baker, Noel C.↗
NASA NTRS2011

NASA Occupant Protection Standards Development

Current National Aeronautics and Space Administration (NASA) occupant protection standards and requirements are based on extrapolations of biodynamic models, which were based on human tests performed under pre-Space Shuttle human flight programs where the occupants were in different suit and seat configurations than is expected for the Multi Purpose Crew Vehicle (MPCV) and Commercial Crew programs. As a result, there is limited statistical validity to the occupant protection standards. Furthermore, the current standards and requirements have not been validated in relevant spaceflight suit, seat configurations or loading conditions. The objectives of this study were to develop new standards and requirements for occupant protection and rigorously validate these new standards with sub-injurious human testing. To accomplish these objectives we began by determining which critical injuries NASA would like to protect for. We then defined the anthropomorphic test device (ATD) and the associated injury metrics of interest. Finally, we conducted a literature review of available data for the Test Device for Human Occupant Restraint New Technology (THOR-NT) ATD to determine injury assessment reference values (IARV) to serve as a baseline for further development. To better understand NASA s environment, we propose conducting sub-injurious human testing in spaceflight seat and suit configurations with spaceflight dynamic loads, with a sufficiently high number of subjects to validate no injury during nominal landing loads. In addition to validate nominal loads, the THOR-NT ATD will be tested in the same conditions as the human volunteers, allowing correlation between human and ATD responses covering the Orion nominal landing environment and commercial vehicle expected nominal environments. All testing will be conducted without the suit and with the suit to ascertain the contribution of the suit to human and ATD responses. In addition to the testing campaign proposed, additional data analysis is proposed to mine existing human injury and response data from other sources, including military volunteer testing, automotive Crash Injury Research Engineering Network (CIREN), and IndyCar impact and injury data. These data sources can allow a better extrapolation of the ATD responses to off-nominal conditions above the nominal range that can safely be tested. These elements will be used to develop injury risk functions for each of the injury metrics measured from the ATD. These risk functions would serve as the basis for the NASA standards. Finally, we propose defining standard test methodology for evaluating future spacecraft designs against the IARVs, including developing a star-rating system to allow crew safety comparisons between vehicles.

Somers, Jeffrey T.↗
NASA NTRS1992

Metric transition

This report describes NASA's metric transition in terms of seven major program elements. Six are technical areas involving research, technology development, and operations; they are managed by specific Program Offices at NASA Headquarters. The final program element, Institutional Management, covers both NASA-wide functional management under control of NASA Headquarters and metric capability development at the individual NASA Field Installations. This area addresses issues common to all NASA program elements, including: Federal, state, and local coordination; standards; private industry initiatives; public-awareness initiatives; and employee training. The concluding section identifies current barriers and impediments to metric transition; NASA has no specific recommendations for consideration by the Congress.

Source record↗
NASA NTRS2009

On Applying the Prognostic Performance Metrics

Prognostics performance evaluation has gained significant attention in the past few years. As prognostics technology matures and more sophisticated methods for prognostic uncertainty management are developed, a standardized methodology for performance evaluation becomes extremely important to guide improvement efforts in a constructive manner. This paper is in continuation of previous efforts where several new evaluation metrics tailored for prognostics were introduced and were shown to effectively evaluate various algorithms as compared to other conventional metrics. Specifically, this paper presents a detailed discussion on how these metrics should be interpreted and used. Several shortcomings identified, while applying these metrics to a variety of real applications, are also summarized along with discussions that attempt to alleviate these problems. Further, these metrics have been enhanced to include the capability of incorporating probability distribution information from prognostic algorithms as opposed to evaluation based on point estimates only. Several methods have been suggested and guidelines have been provided to help choose one method over another based on probability distribution characteristics. These approaches also offer a convenient and intuitive visualization of algorithm performance with respect to some of these new metrics like prognostic horizon and alpha-lambda performance, and also quantify the corresponding performance while incorporating the uncertainty information.

Saxena, Abhinav↗
NASA NTRS2009

Regression Model Optimization for the Analysis of Experimental Data

A candidate math model search algorithm was developed at Ames Research Center that determines a recommended math model for the multivariate regression analysis of experimental data. The search algorithm is applicable to classical regression analysis problems as well as wind tunnel strain gage balance calibration analysis applications. The algorithm compares the predictive capability of different regression models using the standard deviation of the PRESS residuals of the responses as a search metric. This search metric is minimized during the search. Singular value decomposition is used during the search to reject math models that lead to a singular solution of the regression analysis problem. Two threshold dependent constraints are also applied. The first constraint rejects math models with insignificant terms. The second constraint rejects math models with near-linear dependencies between terms. The math term hierarchy rule may also be applied as an optional constraint during or after the candidate math model search. The final term selection of the recommended math model depends on the regressor and response values of the data set, the user s function class combination choice, the user s constraint selections, and the result of the search metric minimization. A frequently used regression analysis example from the literature is used to illustrate the application of the search algorithm to experimental data.

Ulbrich, N.↗
NASA NTRS2015

Pilot Evaluation of a UAS Detect-and-Avoid System's Effectiveness in Remaining Well Clear

Unmanned aircraft will equip with a detect-and-avoid (DAA) system that allows them to comply with the requirement to see and avoid other aircraft, an important layer in the overall set of procedural, strategic and tactical separation methods designed to prevent mid-air collisions. Although the effectiveness of the DAA system will be set to a minimum threshold by regulators, different combinations of algorithms, displays and procedures could be used to meet that minimum. The research presented in this paper indicates the effectiveness of the combined pilot-DAA system as a function of the DAA design requirements and provides data that may be used to model the behavior of pilots when employing such systems. Over the course of two simulations 21 professional UAS pilots evaluated eight different DAA system designs and metrics were collected on their ability to maintain the well clear separation standard. The independent variables were the time horizon at which pilots were alerted to potential losses of well clear, the location of the traffic display, and the tools and informational elements available on the display to aid the pilot in detecting and resolving those potential losses. In the second experiment the UAS encountered two categories of aircraft: those equipped with simulated transponders that could be seen dozens of miles away and those without that were only detectable by a simulated radar within a range of six nautical miles. The data indicate that integrating the traffic display with the primary mission map directly in front of the pilot reduced the frequency of losses of well clear. Improved detection and resolution tools, including explicit maneuver guidance and a trial planning capability, had less of an effect in reducing the frequency of losses but significantly reduced the time in loss when they occurred. The amount of warning time provided to the pilot had a strong effect on their ability to remain well clear: when alerts were first presented with less than about 15 seconds to a predicted loss of well clear pilots were able to maneuver successfully in only 26 percent of encounters, whereas they were about 83 percent successful when they had more than 15 seconds. Pilots' ability to separate from the two categories of aircraft was nearly the same after accounting for the amount of alert time provided in each encounter, although the limited surveillance volume for the non-transponder equipped aircraft meant alerts tended to occur later and therefore were more difficult to resolve.

loss of well clear↗
NASA NTRS2016

Survey of Quantitative Research Metrics to Assess Pilot Performance in Upset Recovery

Accidents attributable to in-flight loss of control are the primary cause for fatal commercial jet accidents worldwide. The National Aeronautics and Space Administration (NASA) conducted a literature review to determine and identify the quantitative standards for assessing upset recovery performance. This review contains current recovery procedures for both military and commercial aviation and includes the metrics researchers use to assess aircraft recovery performance. Metrics include time to first input, recognition time and recovery time and whether that input was correct or incorrect. Other metrics included are: the state of the autopilot and autothrottle, control wheel/sidestick movement resulting in pitch and roll, and inputs to the throttle and rudder. In addition, airplane state measures, such as roll reversals, altitude loss/gain, maximum vertical speed, maximum/minimum air speed, maximum bank angle and maximum g loading are reviewed as well.

Le Vie, Lisa R.↗
NASA NTRS1973

Image quality against metric capability of an aerial camera

Glass plate targets containing multiple contrast three bar patterns and a military standard resolution target were photographed through a light-source collimator-camera system. The glass plate targets and images were then scanned. Through a computer plotting program selected portions of the scanned items were presented. The plots were then measured and analyzed to obtain information of image quality parameters and effects. The measurement errors are discussed together with aspects of resolution determination.

Ghosh, S. K.↗
NASA NTRSDate not supplied

Applying NASA-STD-7009 Standard for Models and Simulations to Surrogate and Other Statistical Models

NASA-STD-7009A, Standard for Models and Simulations, contains a worthy and insightfully-crafted credibility assessment. That set of metrics is intended to accumulate and communicate information that can be critical for users and their customers in evaluating findings due to a model or simulation. Was the model designed for the present use? Is it validated in the domain it’s being applied to? And so on. The standard was originally focused on physics-based models, but it can be applied more generally. It can – and should – be applied to high-impact statistical models, including surrogate models based on larger models and sims and regression models based directly on data. A key issue with many of such models is their relative simplicity compared to most M&S despite their use in high-risk engineering decisions. This report will make a case for using this Standard for a wider class of models, and how it might actually add value in excess of its cost for not only stakeholders, but modelers themselves. A rough example showing compliance for a statistical model is included.

K L Johnson↗
NASA NTRS1982

Design Aids for Real-Time Systems (DARTS)

Design-Aids for Real-Time Systems (DARTS) is a tool that assists in defining embedded computer systems through tree structured graphics, military standard documentation support, and various analyses including automated Software Science parameter counting and metrics calculation. These analyses provide both static and dynamic design quality feedback which can potentially aid in producing efficient, high quality software systems.

Szulewski, P. A.↗
NASA NTRS1996

DCTune Perceptual Optimization of Compressed Dental X-Rays

In current dental practice, x-rays of completed dental work are often sent to the insurer for verification. It is faster and cheaper to transmit instead digital scans of the x-rays. Further economies result if the images are sent in compressed form. DCTune is a technology for optimizing DCT (digital communication technology) quantization matrices to yield maximum perceptual quality for a given bit-rate, or minimum bit-rate for a given perceptual quality. Perceptual optimization of DCT color quantization matrices. In addition, the technology provides a means of setting the perceptual quality of compressed imagery in a systematic way. The purpose of this research was, with respect to dental x-rays, 1) to verify the advantage of DCTune over standard JPEG (Joint Photographic Experts Group), 2) to verify the quality control feature of DCTune, and 3) to discover regularities in the optimized matrices of a set of images. We optimized matrices for a total of 20 images at two resolutions (150 and 300 dpi) and four bit-rates (0.25, 0.5, 0.75, 1.0 bits/pixel), and examined structural regularities in the resulting matrices. We also conducted psychophysical studies (1) to discover the DCTune quality level at which the images became 'visually lossless,' and (2) to rate the relative quality of DCTune and standard JPEG images at various bitrates. Results include: (1) At both resolutions, DCTune quality is a linear function of bit-rate. (2) DCTune quantization matrices for all images at all bitrates and resolutions are modeled well by an inverse Gaussian, with parameters of amplitude and width. (3) As bit-rate is varied, optimal values of both amplitude and width covary in an approximately linear fashion. (4) Both amplitude and width vary in systematic and orderly fashion with either bit-rate or DCTune quality; simple mathematical functions serve to describe these relationships. (5) In going from 150 to 300 dpi, amplitude parameters are substantially lower and widths larger at corresponding bit-rates or qualities. (6) Visually lossless compression occurs at a DCTune quality value of about 1. (7) At 0.25 bits/pixel, comparative ratings give DCTune a substantial advantage over standard JPEG. As visually lossless bit-rates are approached, this advantage of necessity diminishes. We have concluded that DCTune optimized quantization matrices provide better visual quality than standard JPEG. Meaningful quality levels may be specified by means of the DCTune metric. Optimized matrices are very similar across the class of dental x-rays, suggesting the possibility of a 'class-optimal' matrix. DCTune technology appears to provide some value in the context of compressed dental x-rays.

Watson, Andrew B.↗
NASA NTRSDate not supplied

Applying Sensor Uncertainty Mitigation Schemes to Detect-and-Avoid Systems

Impact of sensor noise on the performance of Detect-And-Avoid (DAA) systems can be reduced by implementing various mitigation schemes. This paper evaluates the Sensor Uncertainty Mitigation (SUM) method, implemented in the Detect and Avoid Alerting Logic for Unmanned Systems (DAIDALUS) algorithm, a reference implementation in the DAA minimum operational performance standards. DAIDALUS SUM performance is evaluated using a few safety and operational suitability metrics and compared with more traditional approaches using static safety buffers. A large number of encounters representative of low-speed unmanned aircraft against non-cooperative manned aircraft are simulated and evaluated. An air-to-air radar model produces representative sensor noise for the DAA system. Results show that increasing the tunable parameters for horizontal and vertical uncertainty in DAIDALUS SUM improves the safety metric at the cost of increasing the number of system alerts leading to increased workload. A range of SUM parameters is recommended as suitable values for the type of operations considered for this work. General trends and optimal SUM configurations were found to be nearly the same for two large and very different encounter data sets.

detect and avoid↗
NASA NTRS2009

Evaluating Algorithm Performance Metrics Tailored for Prognostics

Prognostics has taken a center stage in Condition Based Maintenance (CBM) where it is desired to estimate Remaining Useful Life (RUL) of the system so that remedial measures may be taken in advance to avoid catastrophic events or unwanted downtimes. Validation of such predictions is an important but difficult proposition and a lack of appropriate evaluation methods renders prognostics meaningless. Evaluation methods currently used in the research community are not standardized and in many cases do not sufficiently assess key performance aspects expected out of a prognostics algorithm. In this paper we introduce several new evaluation metrics tailored for prognostics and show that they can effectively evaluate various algorithms as compared to other conventional metrics. Specifically four algorithms namely; Relevance Vector Machine (RVM), Gaussian Process Regression (GPR), Artificial Neural Network (ANN), and Polynomial Regression (PR) are compared. These algorithms vary in complexity and their ability to manage uncertainty around predicted estimates. Results show that the new metrics rank these algorithms in different manner and depending on the requirements and constraints suitable metrics may be chosen. Beyond these results, these metrics offer ideas about how metrics suitable to prognostics may be designed so that the evaluation procedure can be standardized. 1

Saxena, Abhinav↗
NASA NTRSDate not supplied

A Psychoacoustic Test for Urban Air Mobility Vehicle Sound Quality

This paper describes a psychoacoustic test in the Exterior Effects Room (EER) at the NASA Langley Research Center. The test investigated the degree to which sound quality metrics (sharpness, tonality, etc.) are predictive of annoyance to notional sounds of Urban Air Mobility (UAM) vehicles (i.e., air taxis). A suite of 136 unique (4.6 second duration) UAM rotor noise stimuli was generated. These stimuli were based on aeroacoustic predictions of a NASA reference UAM quadrotor aircraft under two flight conditions. The synthesizer changed rotor noise parameters such as the blade passage frequency, the relative level of broadband self-noise, and the relative level of tonal motor noise. With loudness constant, the synthesis parameters impacted sound quality in a way that created a spread of predictors both in synthesizer parameters and in sound quality metrics. Forty subjects listened to the suite of UAM noise stimuli in the EER and judged each sound individually on a standard scale of annoyance. Additionally, a subset of the UAM noise stimuli were compared to a reference sound that varied in loudness. From these responses, the relative effect of changes in loudness or changes in other sound quality metrics on annoyance was evaluated. This paper covers background and motivation for the test, details of how the sound stimuli were generated, and details of the test design and execution. Test results investigate how sound quality may affect perceived annoyance to UAM vehicle noise, indicating the importance of sharpness, tonality, impulsiveness, and roughness on annoyance to UAM noise.

UAM↗