SEARCH · Search NASA

Results for “test metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10

All sources NASA NTRS DOE OSTI

NASA NTRS2018

Development of a Ground Test and Analysis Protocol to Support NASA's NextSTEP Phase 2 Habitation Concepts

The NASA Next Space Technologies for Exploration Partnerships (NextSTEP) program is a public-private partnership model that seeks commercial development of deep space exploration capabilities to support extensive human spaceflight missions around and beyond cislunar space. NASA first issued the Phase 1 NextSTEP Broad Agency Announcement to U.S. industries in 2014, which called for innovative cislunar habitation concepts that leveraged commercialization plans for low Earth orbit. These habitats will be part of the Deep Space Gateway (DSG), the cislunar space station planned by NASA for construction in the 2020s. In 2016, Phase 2 of the NextSTEP program selected five commercial partners to develop ground prototypes. A team of NASA research engineers and subject matter experts have been tasked with developing the ground test protocol that will serve as the primary means by which these Phase 2 prototype habitats will be evaluated. Since 2008, this core test team has successfully conducted multiple spaceflight analog mission evaluations utilizing a consistent set of operational products, tools, methods, and metrics to enable the iterative development, testing, analysis, and validation of evolving exploration architectures, operations concepts, and vehicle designs. The purpose of implementing a similar evaluation process for the NextSTEP Phase 2 Habitation Concepts is to consistently evaluate the different commercial partner ground prototypes to provide data-driven, actionable recommendations for Phase 3.

Beaton, Kara H.↗

NASA NTRS2017

Drawbar Pull (DP) Procedures for Off-Road Vehicle Testing

As NASA strives to explore the surface of the Moon and Mars, there is a continued need for improved tire and vehicle development. When tires or vehicles are being designed for off-road conditions where significant thrust generation is required, such as climbing out of craters on the Moon, it is important to use a standard test method for evaluating their tractive performance. The drawbar pull (DP) test is a way of measuring the net thrust generated by tires or a vehicle with respect to performance metrics such as travel reduction, sinkage, or power efficiency. DP testing may be done using a single tire on a traction rig, or with a set of tires on a vehicle; this report focuses on vehicle DP tests. Though vehicle DP tests have been used for decades, there are no standard procedures that apply to exploration vehicles. This report summarizes previous methods employed, shows the sensitivity of certain test parameters, and provides a body of knowledge for developing standard testing procedures. The focus of this work is on lunar applications, but these test methods can be applied to terrestrial and planetary conditions as well. Section 1.0 of this report discusses the utility of DP testing for off-road vehicle evaluation and the metrics used. Section 2.0 focuses on test-terrain preparation, using the example case of lunar terrain. There is a review of lunar terrain analogs implemented in the past and a discussion on the lunar terrain conditions created at the NASA Glenn Research Center, including methods of evaluating the terrain strength variation and consistency from test to test. Section 3.0 provides details of the vehicle test procedures. These consist of a review of past methods, a comprehensive study on the sensitivity of test parameters, and a summary of the procedures used for DP testing at Glenn.

wheel↗

NASA NTRS2017

Using Dispersed Modes During Model Correlation

The model correlation process for the modal characteristics of a launch vehicle is well established. After a test, parameters within the nominal model are adjusted to reflect structural dynamics revealed during testing. However, a full model correlation process for a complex structure can take months of man-hours and many computational resources. If the analyst only has weeks, or even days, of time in which to correlate the nominal model to the experimental results, then the traditional correlation process is not suitable. This paper describes using model dispersions to assist the model correlation process and decrease the overall cost of the process. The process creates thousands of model dispersions from the nominal model prior to the test and then compares each of them to the test data. Using mode shape and frequency error metrics, one dispersion is selected as the best match to the test data. This dispersion is further improved by using a commercial model correlation software. In the three examples shown in this paper, this dispersion based model correlation process performs well when compared to models correlated using traditional techniques and saves time in the post-test analysis.

Stewart, Eric C.↗

NASA NTRS2008

Advanced Diagnostic and Prognostic Testbed (ADAPT) Testability Analysis Report

As system designs become more complex, determining the best locations to add sensors and test points for the purpose of testing and monitoring these designs becomes more difficult. Not only must the designer take into consideration all real and potential faults of the system, he or she must also find efficient ways of detecting and isolating those faults. Because sensors and cabling take up valuable space and weight on a system, and given constraints on bandwidth and power, it is even more difficult to add sensors into these complex designs after the design has been completed. As a result, a number of software tools have been developed to assist the system designer in proper placement of these sensors during the system design phase of a project. One of the key functions provided by many of these software programs is a testability analysis of the system essentially an evaluation of how observable the system behavior is using available tests. During the design phase, testability metrics can help guide the designer in improving the inherent testability of the design. This may include adding, removing, or modifying tests; breaking up feedback loops, or changing the system to reduce fault propagation. Given a set of test requirements, the analysis can also help to verify that the system will meet those requirements. Of course, a testability analysis requires that a software model of the physical system is available. For the analysis to be most effective in guiding system design, this model should ideally be constructed in parallel with these efforts. The purpose of this paper is to present the final testability results of the Advanced Diagnostic and Prognostic Testbed (ADAPT) after the system model was completed. The tool chosen to build the model and to perform the testability analysis with is the Testability Engineering and Maintenance System Designer (TEAMS-Designer). The TEAMS toolset is intended to be a solution to span all phases of the system, from design and development through health management and maintenance. TEAMS-Designer is the model-building and testability analysis software in that suite.

Ossenfort, John↗

NASA NTRS2018

Landsat 9 TIRS-2 Performance Results Based on Subsystem-Level Testing

Landsat 9 is the next in the series of Landsat satellites and has a complement of two pushbroom imagers: Operational Land Imager-2 (OLI-2) that samples the solar reflective spectrum with nine channels and Thermal Infrared Sensor-2 (TIRS-2) samples the thermal infrared spectrum with two channels. The first builds of these sensors, OLI and TIRS, were launched on Landsat 8 in 2013 and Landsat 9 is expected to launch in December 2020. TIRS-2 is designed and built to continue the Landsat data record and satisfy the needs of the remote sensing community. There are two sets of requirements considered for planning the component, subsystem and instrument level tests for TIRS-2: performance requirements and Special Calibration Test Requirements (SCTR). The performance requirements specify key spectral, spatial, radiometric, and operational parameters of TIRS-2 while the SCTRs specify parameters of how the instrument is tested. Several requirements can only be verified at the instrument level, but many performance metrics can be assessed earlier in prelaunch testing at the subsystem level. A test program called TIRS Imaging Performance and Cryoshell Evaluation (TIPCE) was developed to characterize TIRS-2 spectral, spatial, and scattered-light rejection performance at the telescope and detector subsystem level. There were three thermal vacuum campaigns in TIPCE that occurred from November 2017 to March 2018. This work shows results of TIPCE data analysis which provide confidence that key requirements will be met at instrument level with a few minor waivers. A full complement of performance testing will be done at the TIRS-2 instrument level for final verification in late 2018 through Spring 2019.

scatter↗

NASA NTRSDate not supplied

The Inspectability Metric: A Formalized System Of Measurement Enabling The Design For Inspection Framework

Nondestructive evaluation (NDE) engineers are often confronted with structural design choices that present challenges to meeting inspection requirements. These challenges, at best, increase the resources needed to design an inspection solution and, at worst, require resource intensive redesign of the structure. If the inspectability of the structure can be determined early in the design cycle, these challenging inspection scenarios can be avoided. The emergence of additive manufacturing has further compounded this problem by enabling the creation of highly optimized structures with no regard to inspection constraints. Design for inspection (DFI) offers a framework to integrate nondestructive evaluation (NDE) into the design process to alleviate the mechanisms that produce uninspectable designs. DFI is the concept of including inspectability in a multi-objective optimization framework so that it can be considered in parallel to other metrics such as mass and manufacturability. This allows rapid evaluation of the trade-off between design metrics to find solutions that meet the inspection needs of a particular material system, structural concept, or vehicle program. To enable DFI, there must be a system by which the inspectability of a structure can be measured. This system must be agile to produce results quickly, it must be versatile to work with the type of incomplete information one would encounter early in the design process (such as lack of inspection requirements), and it must be delivered in a form that is easily understood by designers. To meet this need, this presentation introduces the novel inspectability metric as a system to measure inspectability. The inspectability metric is a standardized, automation friendly procedure that uses simulations to determine inspectability. Along with guidelines to properly process designs and integrate with existing workflows, the inspectability metric provides a suite of simulation tests to interrogate the ability to find defects and the sensitivity to variability. The testing rubric is designed to maximize the coverage of the parameter space while minimizing the number of simulations needed. The inspectability metric has been in development in collaboration with industry partners to ensure compatibility with modern simulation tools and aerospace design workflows. In this study, we will demonstrate how the inspectability metric is able to determine the inspectability of multiple types of structures, including aerospace composites and additively manufactured parts. We will then show how the inspectability score can be plugged into existing design optimization tasks, such as structural sizing algorithms or design for manufacturing (DFM) frameworks.

Design for inspection↗

NASA NTRS2007

Feasibility of Turing-Style Tests for Autonomous Aerial Vehicle "Intelligence"

A new approach is suggested to define and evaluate key metrics as to autonomous aerial vehicle performance. This approach entails the conceptual definition of a "Turing Test" for UAVs. Such a "UAV Turing test" would be conducted by means of mission simulations and/or tailored flight demonstrations of vehicles under the guidance of their autonomous system software. These autonomous vehicle mission simulations and flight demonstrations would also have to be benchmarked against missions "flown" with pilots/human-operators in the loop. In turn, scoring criteria for such testing could be based upon both quantitative mission success metrics (unique to each mission) and by turning to analog "handling quality" metrics similar to the well-known Cooper-Harper pilot ratings used for manned aircraft. Autonomous aerial vehicles would be considered to have successfully passed this "UAV Turing Test" if the aggregate mission success metrics and handling qualities for the autonomous aerial vehicle matched or exceeded the equivalent metrics for missions conducted with pilots/human-operators in the loop. Alternatively, an independent, knowledgeable observer could provide the "UAV Turing Test" ratings of whether a vehicle is autonomous or "piloted." This observer ideally would, in the more sophisticated mission simulations, also have the enhanced capability of being able to override the scripted mission scenario and instigate failure modes and change of flight profile/plans. If a majority of mission tasks are rated as "piloted" by the observer, when in reality the vehicle/simulation is fully- or semi- autonomously controlled, then the vehicle/simulation "passes" the "UAV Turing Test." In this regards, this second "UAV Turing Test" approach is more consistent with Turing s original "imitation game" proposal. The overall feasibility, and important considerations and limitations, of such an approach for judging/evaluating autonomous aerial vehicle "intelligence" will be discussed from a theoretical perspective.

Young, Larry A.↗

NASA NTRS1972

Photogrammetry using Apollo 16 orbital photography, part B

Discussion is made of the Apollo 15 and 16 metric and panoramic cameras which provided photographs for accurate topographic portrayal of the lunar surface using photogrammetric methods. Nine stereoscopic models of Apollo 16 metric photographs and three models of panoramic photographs were evaluated photogrammetrically in support of the Apollo 16 geologic investigations. Four of the models were used to collect profile data for crater morphology studies; three models were used to collect evaluation data for the frequency distributions of lunar slopes; one model was used to prepare a map of the Apollo 16 traverse area; and one model was used to determine elevations of the Cayley Formation. The remaining three models were used to test photogrammetric techniques using oblique metric and panoramic camera photographs. Two preliminary contour maps were compiled and a high-oblique metric photograph was rectified.

Wu, S. S. C.↗

NASA NTRS1982

Metric half-span model support system

A model support system used to support a model in a wind tunnel test section is described. The model comprises a metric, or measured, half-span supported by a nonmetric, or nonmeasured half-span which is connected to a sting support. Moments and forces acting on the metric half-span are measured without interference from the support system during a wind tunnel test.

Jackson, C. M., Jr.↗

NASA NTRS1982

A study of image quality for radar image processing

Methods developed for image quality metrics are reviewed with focus on basic interpretation or recognition elements including: tone or color; shape; pattern; size; shadow; texture; site; association or context; and resolution. Seven metrics are believed to show promise as a way of characterizing the quality of an image: (1) the dynamic range of intensities in the displayed image; (2) the system signal-to-noise ratio; (3) the system spatial bandwidth or bandpass; (4) the system resolution or acutance; (5) the normalized-mean-square-error as a measure of geometric fidelity; (6) the perceptual mean square error; and (7) the radar threshold quality factor. Selective levels of degradation are being applied to simulated synthetic radar images to test the validity of these metrics.

King, R. W.↗

NASA NTRS2017

Initial Investigation into the Psychoacoustic Properties of Small Unmanned Aerial System Noise

For the past several years, researchers at NASA Langley have been engaged in a series of projects to study the degree to which existing facilities and capabilities, originally created for work on full-scale aircraft, are extensible to smaller scales --those of the small unmanned aerial systems (sUAS, also UAVs and, colloquially, `drones') that have been showing up in the nation's airspace of late. This paper follows an e ort that has led to an initial human{subject psychoacoustic test regarding the annoyance generated by sUAS noise. This e ort spans three phases: 1. The collection of the sounds through field recordings. 2. The formulation and execution of a psychoacoustic test using those recordings. 3. The initial analysis of the data from that test. The data suggests a lack of parity between the noise of the recorded sUAS and that of a set of road vehicles that were also recorded and included in the test, as measured by a set of contemporary noise metrics. Future work, including the possibility of further human subject testing, is discussed in light of this suggestion.

Christian, Andrew↗

NASA NTRS1991

Dual strain gage balance system for measuring light loads

A dual strain gage balance system for measuring normal and axial forces and pitching moment of a metric airfoil model imparted by aerodynamic loads applied to the airfoil model during wind tunnel testing includes a pair of non-metric panels being rigidly connected to and extending towards each other from opposite sides of the wind tunnel, and a pair of strain gage balances, each connected to one of the non-metric panels and to one of the opposite ends of the metric airfoil model for mounting the metric airfoil model between the pair of non-metric panels. Each strain gage balance has a first measuring section for mounting a first strain gage bridge for measuring normal force and pitching moment and a second measuring section for mounting a second strain gage bridge for measuring axial force.

Roberts, Paul W.↗

NASA NTRS2015

JPSS-1 VIIRS Pre-Launch Radiometric Performance

The first Joint Polar Satellite System (JPSS-1 or J1) mission is scheduled to launch in January 2017, and will be very similar to the Suomi-National Polar-orbiting Partnership (SNPP) mission. The Visible Infrared Imaging Radiometer Suite (VIIRS) on board the J1 spacecraft completed its sensor level performance testing in December 2014. VIIRS instrument is expected to provide valuable information about the Earth environment and properties on a daily basis, using a wide-swath (3,040 km) cross-track scanning radiometer. The design covers the wavelength spectrum from reflective to long-wave infrared through 22 spectral bands, from 0.412 m to 12.01 m, and has spatial resolutions of 370 m and 740 m at nadir for imaging and moderate bands, respectively. This paper will provide an overview of pre-launch J1 VIIRS performance testing and methodologies, describing the at-launch baseline radiometric performance as well as the metrics needed to calibrate the instrument once on orbit. Key sensor performance metrics include the sensor signal to noise ratios (SNRs), dynamic range, reflective and emissive bands calibration performance, polarization sensitivity, bands spectral performance, response-vs-scan (RVS), near field response, and stray light rejection. A set of performance metrics generated during the pre-launch testing program will be compared to the sensor requirements and to SNPP VIIRS pre-launch performance.

Oudrari, Hassan↗

DOE OSTI2025

Optimizing cost-effective and benchmarked industry standards to quantify nutrient bioextraction by seaweed

Interest in the utility of seaweed farms to mitigate coastal eutrophication, or nutrient loading, has grown commensurate with the recent rise of the farmed seaweed industry in the U.S. But economic valuation of this ecosystem service remains elusive in part because of challenges in quantifying this spatiotemporally variable biological process with reproducible and comparable metrics. Regulatory bodies that permit wastewater discharge or lease area for aquaculture farms require water quality testing and reporting of dissolved total nitrogen (N) in nearshore marine environments. These metrics must meet EPA standards for testing and reporting (e.g., Total Kjeldahl Nitrogen - TKN). However, these metrics are inherently highly variable over space and time in dynamic nearshore systems, and expensive to evaluate with sufficient breadth to constrain this variance, creating a critical bottleneck to direct quantification of farmed seaweed net uptake rates in situ.

59 BASIC BIOLOGICAL SCIENCES↗

NASA NTRS2023

Atmospheric Correction Inter-comparison eXercise, ACIX-II Land: An Assessment of Amospheric Correction Processors for Landsat 8 and Sentinel-2 Over Land

The correction of the atmospheric effects on optical satellite images is essential for quantitative and multi-temporal remote sensing applications. In order to study the performance of the state-of-the-art methods in an integrated way, a voluntary and open-access benchmark Atmospheric Correction Inter-comparison eXercise (ACIX) was initiated in 2016 in the frame of Committee on Earth Observation Satellites (CEOS) Working Group on Calibration & Validation (WGCV). The first exercise was extended in a second edition wherein twelve atmospheric correction (AC) processors, a substantially larger testing dataset and additional validation metrics were involved. The sites for the inter-comparison analysis were defined by investigating the full catalogue of the Aerosol Robotic Network (AERONET) sites for coincident measurements with satellites' overpass. Although there were more than one hundred sites for Copernicus Sentinel-2 and Landsat 8 acquisitions, the analysis presented in this paper concerns only the common matchups amongst all processors, reducing the number to 79 and 62 sites respectively. Aerosol Optical Depth (AOD) and Water Vapour (WV) retrievals were consequently validated based on the available AERONET observations. The processors mostly succeeded in retrieving AOD for relatively light to medium aerosol loading (AOD < 0.2) with uncertainties <0.08, while the overall uncertainty values were typically 0.23 ± 0.15. Better performances were observed for WV retrievals with >90% of the results falling within the suggested empirical specifications and with the Root Mean Square Error (RMSE) being mostly <0.25 g/cm2. Regarding Surface Reflectance (SR) validation two main approaches were followed. For the first one, a simulated SR reference dataset was computed over all of the test sites by using the 6SV (Second Simulation of the Satellite Signal in the Solar Spectrum vector code) full radiative transfer modelling (RTM) and AERONET measurements for the required aerosol variables and water vapour content. The performance assessment demonstrated that the retrievals were not biased for most of the bands. The uncertainties ranged from approximately 0.003 to 0.01 (excluding B01) for the best performing processors in both sensors' analyses. For the second one, measurements from the radiometric calibration network RadCalNet over La Crau (France) and Gobabeb (Namibia) were involved in the validation. The performance of the processors was in general consistent across all bands for both sensors and with low standard deviations (<0.04) between on-site and estimated surface reflectance. Overall, our study provides a good insight of AC algorithms' performance to developers and users, pointing out similarities and differences for AOD, WV and SR retrievals. Such validation though still lacks of ground-based measurements of known uncertainty to better assess and characterize the uncertainties in SR retrievals.

Atmospheric correction↗

NASA NTRS1990

Multiversion software reliability through fault-avoidance and fault-tolerance

In this project we have proposed to investigate a number of experimental and theoretical issues associated with the practical use of multi-version software in providing dependable software through fault-avoidance and fault-elimination, as well as run-time tolerance of software faults. In the period reported here we have working on the following: We have continued collection of data on the relationships between software faults and reliability, and the coverage provided by the testing process as measured by different metrics (including data flow metrics). We continued work on software reliability estimation methods based on non-random sampling, and the relationship between software reliability and code coverage provided through testing. We have continued studying back-to-back testing as an efficient mechanism for removal of uncorrelated faults, and common-cause faults of variable span. We have also been studying back-to-back testing as a tool for improvement of the software change process, including regression testing. We continued investigating existing, and worked on formulation of new fault-tolerance models. In particular, we have partly finished evaluation of Consensus Voting in the presence of correlated failures, and are in the process of finishing evaluation of Consensus Recovery Block (CRB) under failure correlation. We find both approaches far superior to commonly employed fixed agreement number voting (usually majority voting). We have also finished a cost analysis of the CRB approach.

Vouk, Mladen A.↗

NASA NTRS2009

[A Handling Qualities Metric for Damaged Aircraft]

In recent flight tests of F-15 Intelligent Flight Control System (IFCS), software simulated aircraft control surface failures were inserted to evaluate the IFCS adaptive systems. The failure commanded the left stabilator to a fixed position. The adaptive system uses a neural network that is designed to change control law gains, in the event of damage (real or simulated), that allows the aircraft to fly as it had before the damage. The performance of the adaptive system was assessed in terms of its ability to re-establish good onboard model tracking and its ability to decouple roll and pitch response.

Cogan, Bruce↗

NASA NTRSDate not supplied

A Psychoacoustic Test for Urban Air Mobility Vehicle Sound Quality

This paper describes a psychoacoustic test in the Exterior Effects Room (EER) at the NASA Langley Research Center. The test investigated the degree to which sound quality metrics (sharpness, tonality, etc.) are predictive of annoyance to notional sounds of Urban Air Mobility (UAM) vehicles (i.e., air taxis). A suite of 136 unique (4.6 second duration) UAM rotor noise stimuli was generated. These stimuli were based on aeroacoustic predictions of a NASA reference UAM quadrotor aircraft under two flight conditions. The synthesizer changed rotor noise parameters such as the blade passage frequency, the relative level of broadband self-noise, and the relative level of tonal motor noise. With loudness constant, the synthesis parameters impacted sound quality in a way that created a spread of predictors both in synthesizer parameters and in sound quality metrics. Forty subjects listened to the suite of UAM noise stimuli in the EER and judged each sound individually on a standard scale of annoyance. Additionally, a subset of the UAM noise stimuli were compared to a reference sound that varied in loudness. From these responses, the relative effect of changes in loudness or changes in other sound quality metrics on annoyance was evaluated. This paper covers background and motivation for the test, details of how the sound stimuli were generated, and details of the test design and execution. Test results investigate how sound quality may affect perceived annoyance to UAM vehicle noise, indicating the importance of sharpness, tonality, impulsiveness, and roughness on annoyance to UAM noise.

UAM↗