Search NASA⌕ Search

SEARCH · Search NASA

Results for “Testing metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 343 records · Page 19
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

The Inspectability Metric: A Formalized System Of Measurement Enabling The Design For Inspection Framework

Nondestructive evaluation (NDE) engineers are often confronted with structural design choices that present challenges to meeting inspection requirements. These challenges, at best, increase the resources needed to design an inspection solution and, at worst, require resource intensive redesign of the structure. If the inspectability of the structure can be determined early in the design cycle, these challenging inspection scenarios can be avoided. The emergence of additive manufacturing has further compounded this problem by enabling the creation of highly optimized structures with no regard to inspection constraints. Design for inspection (DFI) offers a framework to integrate nondestructive evaluation (NDE) into the design process to alleviate the mechanisms that produce uninspectable designs. DFI is the concept of including inspectability in a multi-objective optimization framework so that it can be considered in parallel to other metrics such as mass and manufacturability. This allows rapid evaluation of the trade-off between design metrics to find solutions that meet the inspection needs of a particular material system, structural concept, or vehicle program. To enable DFI, there must be a system by which the inspectability of a structure can be measured. This system must be agile to produce results quickly, it must be versatile to work with the type of incomplete information one would encounter early in the design process (such as lack of inspection requirements), and it must be delivered in a form that is easily understood by designers. To meet this need, this presentation introduces the novel inspectability metric as a system to measure inspectability. The inspectability metric is a standardized, automation friendly procedure that uses simulations to determine inspectability. Along with guidelines to properly process designs and integrate with existing workflows, the inspectability metric provides a suite of simulation tests to interrogate the ability to find defects and the sensitivity to variability. The testing rubric is designed to maximize the coverage of the parameter space while minimizing the number of simulations needed. The inspectability metric has been in development in collaboration with industry partners to ensure compatibility with modern simulation tools and aerospace design workflows. In this study, we will demonstrate how the inspectability metric is able to determine the inspectability of multiple types of structures, including aerospace composites and additively manufactured parts. We will then show how the inspectability score can be plugged into existing design optimization tasks, such as structural sizing algorithms or design for manufacturing (DFM) frameworks.

Design for inspection↗
NASA NTRS2007

Feasibility of Turing-Style Tests for Autonomous Aerial Vehicle "Intelligence"

A new approach is suggested to define and evaluate key metrics as to autonomous aerial vehicle performance. This approach entails the conceptual definition of a "Turing Test" for UAVs. Such a "UAV Turing test" would be conducted by means of mission simulations and/or tailored flight demonstrations of vehicles under the guidance of their autonomous system software. These autonomous vehicle mission simulations and flight demonstrations would also have to be benchmarked against missions "flown" with pilots/human-operators in the loop. In turn, scoring criteria for such testing could be based upon both quantitative mission success metrics (unique to each mission) and by turning to analog "handling quality" metrics similar to the well-known Cooper-Harper pilot ratings used for manned aircraft. Autonomous aerial vehicles would be considered to have successfully passed this "UAV Turing Test" if the aggregate mission success metrics and handling qualities for the autonomous aerial vehicle matched or exceeded the equivalent metrics for missions conducted with pilots/human-operators in the loop. Alternatively, an independent, knowledgeable observer could provide the "UAV Turing Test" ratings of whether a vehicle is autonomous or "piloted." This observer ideally would, in the more sophisticated mission simulations, also have the enhanced capability of being able to override the scripted mission scenario and instigate failure modes and change of flight profile/plans. If a majority of mission tasks are rated as "piloted" by the observer, when in reality the vehicle/simulation is fully- or semi- autonomously controlled, then the vehicle/simulation "passes" the "UAV Turing Test." In this regards, this second "UAV Turing Test" approach is more consistent with Turing s original "imitation game" proposal. The overall feasibility, and important considerations and limitations, of such an approach for judging/evaluating autonomous aerial vehicle "intelligence" will be discussed from a theoretical perspective.

Young, Larry A.↗
DOE OSTI2017

A status review of photovoltaic power conversion equipment reliability, safety, and quality assurance protocols

Data indicate that the inverter is the element of the photovoltaic plant that has the highest number of service calls and the greatest operation and maintenance cost burden. This paper describes the projects and relevant background needed in developing design qualification standards that would serve to establish a minimum level of reliability, along with a review of photovoltaic inverter quality and safety standards, most of which are in their infancy. We compare stresses and levels for accelerated testing of inverters proposed in the standard drafts, and those proposed by manufacturers and purchasers of inverters. We also review bases for the methods, stress types, and stress levels for durability testing of key inverter components. Many of the test protocols appear to need more comprehensive inclusion of stress factors existing in the natural environment such as wind driven rain, dust, and grid disturbances. Further understanding of how temperature, humidity ingress, and voltage bias affect the inverters and their components is also required. We provide data indicating inconsistent quality of the inverters and the durability of components leading to greater cost for the photovoltaic plant operator. Accordingly, the recommendation for data collection within quality standards for obtaining cost of ownership metrics is made. Design validation testing using realistic operation, environmental, and connection conditions, including under end-use field conditions with feedback for continuous improvement is recommended for inclusion within a quality standard.

14 SOLAR ENERGY↗
NASA NTRS1972

Photogrammetry using Apollo 16 orbital photography, part B

Discussion is made of the Apollo 15 and 16 metric and panoramic cameras which provided photographs for accurate topographic portrayal of the lunar surface using photogrammetric methods. Nine stereoscopic models of Apollo 16 metric photographs and three models of panoramic photographs were evaluated photogrammetrically in support of the Apollo 16 geologic investigations. Four of the models were used to collect profile data for crater morphology studies; three models were used to collect evaluation data for the frequency distributions of lunar slopes; one model was used to prepare a map of the Apollo 16 traverse area; and one model was used to determine elevations of the Cayley Formation. The remaining three models were used to test photogrammetric techniques using oblique metric and panoramic camera photographs. Two preliminary contour maps were compiled and a high-oblique metric photograph was rectified.

Wu, S. S. C.↗
NASA NTRS1982

Metric half-span model support system

A model support system used to support a model in a wind tunnel test section is described. The model comprises a metric, or measured, half-span supported by a nonmetric, or nonmeasured half-span which is connected to a sting support. Moments and forces acting on the metric half-span are measured without interference from the support system during a wind tunnel test.

Jackson, C. M., Jr.↗
DOE OSTI2022

Performance Monitoring Program: Developing Comparative Metrics for Fitness-for-Duty Programs

To comply with U.S. Nuclear Regulatory Commission (NRC) regulations, licensees and entities authorized under Title 10 of the Code of Federal Regulations (CFR) Part 26 Section 26.3 (§ 26.3) are required to have fitness-for-duty (FFD) programs. Under § 26.3, the expectation of these FFD programs is to provide reasonable assurance that individuals who are granted unescorted access to nuclear power reactor protected areas and Category I fuel cycle facility material control areas are trustworthy, will perform their tasks in a reliable manner, are not under the influence of any substance, legal or illegal, that may impair their ability to perform their duties, and are not mentally or physically impaired from any cause that can adversely affect their ability to safely and competently perform their duties. Pacific Northwest National Laboratory (PNNL) was tasked with developing a performance monitoring program to risk inform NRC inspection and policy regarding quantitative FFD performance data. To meet this need, PNNL developed methodologies that could be implemented within a performance monitoring program. Throughout this report, these methodologies are referred to as comparative metrics. The NRC provided PNNL with 2016–2019 data from annual reporting forms and single positive test forms provided by licensees and other entities. For most of the comparative metrics, the analyses required customized processing, such as creating filtering fields, adding data fields and summaries, and joining datasets. The comparative metrics developed include: random testing rate, random policy violation rate, pre-access policy violation rate, subversion attempt rate, and number of policy violations by labor category. The comparative metrics can be used for parsing and visualizing FFD program data and monitoring FFD performance at the labor category, facility, licensee, and industry levels to risk inform NRC inspection and policy with regard to the FFD data currently collected from licensees and other entities that implement Part 26 requirements. Furthermore, these developed comparative metrics allow a more in-depth look at the FFD programs for the industry to discern trends and patterns that may warrant changes at an industry level and to inform policy decisions. These analyses should be refreshed as new FFD data become available.

99 GENERAL AND MISCELLANEOUS↗
NASA NTRS1982

A study of image quality for radar image processing

Methods developed for image quality metrics are reviewed with focus on basic interpretation or recognition elements including: tone or color; shape; pattern; size; shadow; texture; site; association or context; and resolution. Seven metrics are believed to show promise as a way of characterizing the quality of an image: (1) the dynamic range of intensities in the displayed image; (2) the system signal-to-noise ratio; (3) the system spatial bandwidth or bandpass; (4) the system resolution or acutance; (5) the normalized-mean-square-error as a measure of geometric fidelity; (6) the perceptual mean square error; and (7) the radar threshold quality factor. Selective levels of degradation are being applied to simulated synthetic radar images to test the validity of these metrics.

King, R. W.↗
NASA NTRS2017

Initial Investigation into the Psychoacoustic Properties of Small Unmanned Aerial System Noise

For the past several years, researchers at NASA Langley have been engaged in a series of projects to study the degree to which existing facilities and capabilities, originally created for work on full-scale aircraft, are extensible to smaller scales --those of the small unmanned aerial systems (sUAS, also UAVs and, colloquially, `drones') that have been showing up in the nation's airspace of late. This paper follows an e ort that has led to an initial human{subject psychoacoustic test regarding the annoyance generated by sUAS noise. This e ort spans three phases: 1. The collection of the sounds through field recordings. 2. The formulation and execution of a psychoacoustic test using those recordings. 3. The initial analysis of the data from that test. The data suggests a lack of parity between the noise of the recorded sUAS and that of a set of road vehicles that were also recorded and included in the test, as measured by a set of contemporary noise metrics. Future work, including the possibility of further human subject testing, is discussed in light of this suggestion.

Christian, Andrew↗
NASA NTRS1991

Dual strain gage balance system for measuring light loads

A dual strain gage balance system for measuring normal and axial forces and pitching moment of a metric airfoil model imparted by aerodynamic loads applied to the airfoil model during wind tunnel testing includes a pair of non-metric panels being rigidly connected to and extending towards each other from opposite sides of the wind tunnel, and a pair of strain gage balances, each connected to one of the non-metric panels and to one of the opposite ends of the metric airfoil model for mounting the metric airfoil model between the pair of non-metric panels. Each strain gage balance has a first measuring section for mounting a first strain gage bridge for measuring normal force and pitching moment and a second measuring section for mounting a second strain gage bridge for measuring axial force.

Roberts, Paul W.↗
NASA NTRS2015

JPSS-1 VIIRS Pre-Launch Radiometric Performance

The first Joint Polar Satellite System (JPSS-1 or J1) mission is scheduled to launch in January 2017, and will be very similar to the Suomi-National Polar-orbiting Partnership (SNPP) mission. The Visible Infrared Imaging Radiometer Suite (VIIRS) on board the J1 spacecraft completed its sensor level performance testing in December 2014. VIIRS instrument is expected to provide valuable information about the Earth environment and properties on a daily basis, using a wide-swath (3,040 km) cross-track scanning radiometer. The design covers the wavelength spectrum from reflective to long-wave infrared through 22 spectral bands, from 0.412 m to 12.01 m, and has spatial resolutions of 370 m and 740 m at nadir for imaging and moderate bands, respectively. This paper will provide an overview of pre-launch J1 VIIRS performance testing and methodologies, describing the at-launch baseline radiometric performance as well as the metrics needed to calibrate the instrument once on orbit. Key sensor performance metrics include the sensor signal to noise ratios (SNRs), dynamic range, reflective and emissive bands calibration performance, polarization sensitivity, bands spectral performance, response-vs-scan (RVS), near field response, and stray light rejection. A set of performance metrics generated during the pre-launch testing program will be compared to the sensor requirements and to SNPP VIIRS pre-launch performance.

Oudrari, Hassan↗
DOE OSTI2025

Optimizing cost-effective and benchmarked industry standards to quantify nutrient bioextraction by seaweed

Interest in the utility of seaweed farms to mitigate coastal eutrophication, or nutrient loading, has grown commensurate with the recent rise of the farmed seaweed industry in the U.S. But economic valuation of this ecosystem service remains elusive in part because of challenges in quantifying this spatiotemporally variable biological process with reproducible and comparable metrics. Regulatory bodies that permit wastewater discharge or lease area for aquaculture farms require water quality testing and reporting of dissolved total nitrogen (N) in nearshore marine environments. These metrics must meet EPA standards for testing and reporting (e.g., Total Kjeldahl Nitrogen - TKN). However, these metrics are inherently highly variable over space and time in dynamic nearshore systems, and expensive to evaluate with sufficient breadth to constrain this variance, creating a critical bottleneck to direct quantification of farmed seaweed net uptake rates in situ.

59 BASIC BIOLOGICAL SCIENCES↗
NASA NTRS2023

Atmospheric Correction Inter-comparison eXercise, ACIX-II Land: An Assessment of Amospheric Correction Processors for Landsat 8 and Sentinel-2 Over Land

The correction of the atmospheric effects on optical satellite images is essential for quantitative and multi-temporal remote sensing applications. In order to study the performance of the state-of-the-art methods in an integrated way, a voluntary and open-access benchmark Atmospheric Correction Inter-comparison eXercise (ACIX) was initiated in 2016 in the frame of Committee on Earth Observation Satellites (CEOS) Working Group on Calibration & Validation (WGCV). The first exercise was extended in a second edition wherein twelve atmospheric correction (AC) processors, a substantially larger testing dataset and additional validation metrics were involved. The sites for the inter-comparison analysis were defined by investigating the full catalogue of the Aerosol Robotic Network (AERONET) sites for coincident measurements with satellites' overpass. Although there were more than one hundred sites for Copernicus Sentinel-2 and Landsat 8 acquisitions, the analysis presented in this paper concerns only the common matchups amongst all processors, reducing the number to 79 and 62 sites respectively. Aerosol Optical Depth (AOD) and Water Vapour (WV) retrievals were consequently validated based on the available AERONET observations. The processors mostly succeeded in retrieving AOD for relatively light to medium aerosol loading (AOD < 0.2) with uncertainties <0.08, while the overall uncertainty values were typically 0.23 ± 0.15. Better performances were observed for WV retrievals with >90% of the results falling within the suggested empirical specifications and with the Root Mean Square Error (RMSE) being mostly <0.25 g/cm2. Regarding Surface Reflectance (SR) validation two main approaches were followed. For the first one, a simulated SR reference dataset was computed over all of the test sites by using the 6SV (Second Simulation of the Satellite Signal in the Solar Spectrum vector code) full radiative transfer modelling (RTM) and AERONET measurements for the required aerosol variables and water vapour content. The performance assessment demonstrated that the retrievals were not biased for most of the bands. The uncertainties ranged from approximately 0.003 to 0.01 (excluding B01) for the best performing processors in both sensors' analyses. For the second one, measurements from the radiometric calibration network RadCalNet over La Crau (France) and Gobabeb (Namibia) were involved in the validation. The performance of the processors was in general consistent across all bands for both sensors and with low standard deviations (<0.04) between on-site and estimated surface reflectance. Overall, our study provides a good insight of AC algorithms' performance to developers and users, pointing out similarities and differences for AOD, WV and SR retrievals. Such validation though still lacks of ground-based measurements of known uncertainty to better assess and characterize the uncertainties in SR retrievals.

Atmospheric correction↗
NASA NTRS1990

Multiversion software reliability through fault-avoidance and fault-tolerance

In this project we have proposed to investigate a number of experimental and theoretical issues associated with the practical use of multi-version software in providing dependable software through fault-avoidance and fault-elimination, as well as run-time tolerance of software faults. In the period reported here we have working on the following: We have continued collection of data on the relationships between software faults and reliability, and the coverage provided by the testing process as measured by different metrics (including data flow metrics). We continued work on software reliability estimation methods based on non-random sampling, and the relationship between software reliability and code coverage provided through testing. We have continued studying back-to-back testing as an efficient mechanism for removal of uncorrelated faults, and common-cause faults of variable span. We have also been studying back-to-back testing as a tool for improvement of the software change process, including regression testing. We continued investigating existing, and worked on formulation of new fault-tolerance models. In particular, we have partly finished evaluation of Consensus Voting in the presence of correlated failures, and are in the process of finishing evaluation of Consensus Recovery Block (CRB) under failure correlation. We find both approaches far superior to commonly employed fixed agreement number voting (usually majority voting). We have also finished a cost analysis of the CRB approach.

Vouk, Mladen A.↗
DOE OSTI2021

Joint Work Plan for DPC Filler Testing

The US generates approximately 2,000 Metric Tons of Heavy Metal (MTHM) of commercial spent fuel (SNF) each year and currently stores ~ 85,000 MTHM of commercial SNF at 70+ reactor sites, the disposal of which is the responsibility of the US Department of Energy (DOE). SNF is initially stored in spent nuclear fueld pools (SFPs). SFPs were initially constructed by US utilities for temporary fuel storage, but with no final disposal pathway available, SFPs are reaching capacity. To allow continued operation of the nation's commercial nuclear reactor fleet, utilities started transferring SNF from SFPs (wet storage) to dry cask storage systems, typically using dual-purpose (storage and transportation) canisters (DPCs). And while DPCs were designed, licensed and loaded to meet Nuclear Regulatory Commission (NRC) requirements that preclude the possibility of a critical event during SNF storage and transport, they were not designed or loaded to preclude the possibility of a criticality event during the regulated post-closure period following disposal, which could be up to 1,000,000 years (Price, 2019). DPC filler option criteria are detailed and materials that exhibit these attributes are explored. This document is an update of the SNL Joint Workplan on Filler Investigations for DPCs.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
NASA NTRS2009

[A Handling Qualities Metric for Damaged Aircraft]

In recent flight tests of F-15 Intelligent Flight Control System (IFCS), software simulated aircraft control surface failures were inserted to evaluate the IFCS adaptive systems. The failure commanded the left stabilator to a fixed position. The adaptive system uses a neural network that is designed to change control law gains, in the event of damage (real or simulated), that allows the aircraft to fly as it had before the damage. The performance of the adaptive system was assessed in terms of its ability to re-establish good onboard model tracking and its ability to decouple roll and pitch response.

Cogan, Bruce↗
DOE OSTI2014

Testing the no-hair theorem with event horizon telescope observations of Sagittarius A*

The advent of the Event Horizon Telescope (EHT), a millimeter-wave very long baseline interferometric array, has enabled spatially resolved studies of the subhorizon-scale structure for a handful of supermassive black holes. Among these, the supermassive black hole at the center of the Milky Way, Sagittarius A* (Sgr A*), presents the largest angular cross section. Thus far, these studies have focused on measurements of the black hole spin and the validation of low-luminosity accretion models. However, a critical input in the analysis of EHT data is the structure of the black hole spacetime, and thus these observations provide the novel opportunity to test the applicability of the Kerr metric to astrophysical black holes. Here we present the first simulated images of a radiatively inefficient accretion flow (RIAF) around Sgr A* employing a quasi-Kerr metric that contains an independent quadrupole moment in addition to the mass and spin that fully characterize a black hole in general relativity. We show that these images can be significantly different from the images of an RIAF around a Kerr black hole with the same spin and demonstrate the feasibility of testing the no-hair theorem by constraining the quadrupolar deviation from the Kerr metric with existing EHT data. Equally important, we find that the disk inclination and spin orientation angles are robust to the inclusion of additional parameters, providing confidence in previous estimations assuming the Kerr metric based on EHT observations. However, at present, the limits on potential modifications of the Kerr metric remain weak.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Tidal Turbine Test, Downeast Turbines, July 12, 2021

Downeast Turbines tested a tidal turbine prototype with novel rotor/channel system and lateral effluent discharge apparatus (LEDA), during five days of testing in the flume at Alden Lab. Three days of testing (July 12-14, 2021) focused on turbine power metrics of torque and rpm, which were low, and then two days of follow-up testing (July 27-28, 2021) focused on LEDA performance metrics of pressure differential and rates of volumetric flow, with encouraging results. Next step is to to characterize, and even optimize, configurations of the LEDA, using 3D-CFD as a helpful tool, to refine its shape and explore its limits of performance as a means of effluent discharge that augments performance of an instream turbine. An improved configuration of the LEDA will be re-combined with the rotor/channel system of the turbine prototype, and ultimately, the rotor will be re-sized (enlarged), to better match the LEDA's performance capabilities in drawing through a rate of volumetric flow. This submission is Downeast Turbines' Post Access Report for the test event. It includes the files described here (next below), and several reference links. "Downeast TEAMER-Post-Access-Report...docx" is a document file containing the report. "Appendices A, B, and C" are included in this file, and so are "Figures #1-7." "Appendix D - Test Data Workbooks.zip" is an archive file containing all post access data (raw data tables, calculating tables, and graphs), presented in fourteen Excel workbooks as described in the report. "Appendix E - Post Access Figures.zip" is an archive file containing "Figures #8-52," of the report.

16 TIDAL AND WAVE POWER↗
DOE OSTI2023

MODELING A SPENT NUCLEAR FUEL CASK SEISMIC TEST

The US Department of Energy Spent Fuel and Waste Science and Technology (SFWST) program is planning to conduct a series of full-scale shake table tests to simulate hypothetical earthquake conditions and record the response of surrogate spent nuclear fuel (SNF) assemblies in a canister dry storage system mockup. The shake table motions will represent a range of hypothetical earthquake conditions at hypothetical locations in the continental US to generally define the range of mechanical loads that SNF can be expected to experience during extended dry storage periods. The earthquake conditions will represent seismic hazards in the 2,000-to-20,000-year return period range. The test will use instrumented pressurized water reactor fuel assemblies with surrogate mass inside the fuel rods instead of radioactive fuel pellets. Dummy assemblies with similar mass and dimensions to actual fuel assemblies will occupy the rest of the fuel assembly locations in a SNF canister that holds 32 fuel assemblies. The canister will be located inside a mockup vertical concrete cask. Instrumentation will record the motion of the major components of the complex dynamic system, and strain gauges will be used to record the cladding strain at select locations. Preparations for the test require modeling predictions to identify the range of response of the system and to help select specific earthquake cases to be simulated on the shake table from a large set of potential cases. This paper describes the pretest nonlinear finite element modeling efforts that have been completed to date, including cask system level modeling and fuel assembly modeling in LS-DYNA. The cask system level models are critical for anticipating sliding or tipping of an unanchored cask during the test. The fuel assembly model is needed to estimate the range of cladding strain response and fuel assembly structural response to be expected during the test and cladding strain measurements will be key metrics for model validation and the development of modeling best practices after the test is concluded.

Klymyshyn, Nicholas A.↗