Search NASA⌕ Search

SEARCH · Search NASA

Results for “Testing metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 361 records · Page 20

A study of image quality for radar image processing

Methods developed for image quality metrics are reviewed with focus on basic interpretation or recognition elements including: tone or color; shape; pattern; size; shadow; texture; site; association or context; and resolution. Seven metrics are believed to show promise as a way of characterizing the quality of an image: (1) the dynamic range of intensities in the displayed image; (2) the system signal-to-noise ratio; (3) the system spatial bandwidth or bandpass; (4) the system resolution or acutance; (5) the normalized-mean-square-error as a measure of geometric fidelity; (6) the perceptual mean square error; and (7) the radar threshold quality factor. Selective levels of degradation are being applied to simulated synthetic radar images to test the validity of these metrics.

King, R. W.↗

Initial Investigation into the Psychoacoustic Properties of Small Unmanned Aerial System Noise

For the past several years, researchers at NASA Langley have been engaged in a series of projects to study the degree to which existing facilities and capabilities, originally created for work on full-scale aircraft, are extensible to smaller scales --those of the small unmanned aerial systems (sUAS, also UAVs and, colloquially, `drones') that have been showing up in the nation's airspace of late. This paper follows an e ort that has led to an initial human{subject psychoacoustic test regarding the annoyance generated by sUAS noise. This e ort spans three phases: 1. The collection of the sounds through field recordings. 2. The formulation and execution of a psychoacoustic test using those recordings. 3. The initial analysis of the data from that test. The data suggests a lack of parity between the noise of the recorded sUAS and that of a set of road vehicles that were also recorded and included in the test, as measured by a set of contemporary noise metrics. Future work, including the possibility of further human subject testing, is discussed in light of this suggestion.

Christian, Andrew↗

Dual strain gage balance system for measuring light loads

A dual strain gage balance system for measuring normal and axial forces and pitching moment of a metric airfoil model imparted by aerodynamic loads applied to the airfoil model during wind tunnel testing includes a pair of non-metric panels being rigidly connected to and extending towards each other from opposite sides of the wind tunnel, and a pair of strain gage balances, each connected to one of the non-metric panels and to one of the opposite ends of the metric airfoil model for mounting the metric airfoil model between the pair of non-metric panels. Each strain gage balance has a first measuring section for mounting a first strain gage bridge for measuring normal force and pitching moment and a second measuring section for mounting a second strain gage bridge for measuring axial force.

Roberts, Paul W.↗

JPSS-1 VIIRS Pre-Launch Radiometric Performance

The first Joint Polar Satellite System (JPSS-1 or J1) mission is scheduled to launch in January 2017, and will be very similar to the Suomi-National Polar-orbiting Partnership (SNPP) mission. The Visible Infrared Imaging Radiometer Suite (VIIRS) on board the J1 spacecraft completed its sensor level performance testing in December 2014. VIIRS instrument is expected to provide valuable information about the Earth environment and properties on a daily basis, using a wide-swath (3,040 km) cross-track scanning radiometer. The design covers the wavelength spectrum from reflective to long-wave infrared through 22 spectral bands, from 0.412 m to 12.01 m, and has spatial resolutions of 370 m and 740 m at nadir for imaging and moderate bands, respectively. This paper will provide an overview of pre-launch J1 VIIRS performance testing and methodologies, describing the at-launch baseline radiometric performance as well as the metrics needed to calibrate the instrument once on orbit. Key sensor performance metrics include the sensor signal to noise ratios (SNRs), dynamic range, reflective and emissive bands calibration performance, polarization sensitivity, bands spectral performance, response-vs-scan (RVS), near field response, and stray light rejection. A set of performance metrics generated during the pre-launch testing program will be compared to the sensor requirements and to SNPP VIIRS pre-launch performance.

Oudrari, Hassan↗

Optimizing cost-effective and benchmarked industry standards to quantify nutrient bioextraction by seaweed

Interest in the utility of seaweed farms to mitigate coastal eutrophication, or nutrient loading, has grown commensurate with the recent rise of the farmed seaweed industry in the U.S. But economic valuation of this ecosystem service remains elusive in part because of challenges in quantifying this spatiotemporally variable biological process with reproducible and comparable metrics. Regulatory bodies that permit wastewater discharge or lease area for aquaculture farms require water quality testing and reporting of dissolved total nitrogen (N) in nearshore marine environments. These metrics must meet EPA standards for testing and reporting (e.g., Total Kjeldahl Nitrogen - TKN). However, these metrics are inherently highly variable over space and time in dynamic nearshore systems, and expensive to evaluate with sufficient breadth to constrain this variance, creating a critical bottleneck to direct quantification of farmed seaweed net uptake rates in situ.

59 BASIC BIOLOGICAL SCIENCES↗

Atmospheric Correction Inter-comparison eXercise, ACIX-II Land: An Assessment of Amospheric Correction Processors for Landsat 8 and Sentinel-2 Over Land

The correction of the atmospheric effects on optical satellite images is essential for quantitative and multi-temporal remote sensing applications. In order to study the performance of the state-of-the-art methods in an integrated way, a voluntary and open-access benchmark Atmospheric Correction Inter-comparison eXercise (ACIX) was initiated in 2016 in the frame of Committee on Earth Observation Satellites (CEOS) Working Group on Calibration & Validation (WGCV). The first exercise was extended in a second edition wherein twelve atmospheric correction (AC) processors, a substantially larger testing dataset and additional validation metrics were involved. The sites for the inter-comparison analysis were defined by investigating the full catalogue of the Aerosol Robotic Network (AERONET) sites for coincident measurements with satellites' overpass. Although there were more than one hundred sites for Copernicus Sentinel-2 and Landsat 8 acquisitions, the analysis presented in this paper concerns only the common matchups amongst all processors, reducing the number to 79 and 62 sites respectively. Aerosol Optical Depth (AOD) and Water Vapour (WV) retrievals were consequently validated based on the available AERONET observations. The processors mostly succeeded in retrieving AOD for relatively light to medium aerosol loading (AOD < 0.2) with uncertainties <0.08, while the overall uncertainty values were typically 0.23 ± 0.15. Better performances were observed for WV retrievals with >90% of the results falling within the suggested empirical specifications and with the Root Mean Square Error (RMSE) being mostly <0.25 g/cm2. Regarding Surface Reflectance (SR) validation two main approaches were followed. For the first one, a simulated SR reference dataset was computed over all of the test sites by using the 6SV (Second Simulation of the Satellite Signal in the Solar Spectrum vector code) full radiative transfer modelling (RTM) and AERONET measurements for the required aerosol variables and water vapour content. The performance assessment demonstrated that the retrievals were not biased for most of the bands. The uncertainties ranged from approximately 0.003 to 0.01 (excluding B01) for the best performing processors in both sensors' analyses. For the second one, measurements from the radiometric calibration network RadCalNet over La Crau (France) and Gobabeb (Namibia) were involved in the validation. The performance of the processors was in general consistent across all bands for both sensors and with low standard deviations (<0.04) between on-site and estimated surface reflectance. Overall, our study provides a good insight of AC algorithms' performance to developers and users, pointing out similarities and differences for AOD, WV and SR retrievals. Such validation though still lacks of ground-based measurements of known uncertainty to better assess and characterize the uncertainties in SR retrievals.

Atmospheric correction↗

Multiversion software reliability through fault-avoidance and fault-tolerance

In this project we have proposed to investigate a number of experimental and theoretical issues associated with the practical use of multi-version software in providing dependable software through fault-avoidance and fault-elimination, as well as run-time tolerance of software faults. In the period reported here we have working on the following: We have continued collection of data on the relationships between software faults and reliability, and the coverage provided by the testing process as measured by different metrics (including data flow metrics). We continued work on software reliability estimation methods based on non-random sampling, and the relationship between software reliability and code coverage provided through testing. We have continued studying back-to-back testing as an efficient mechanism for removal of uncorrelated faults, and common-cause faults of variable span. We have also been studying back-to-back testing as a tool for improvement of the software change process, including regression testing. We continued investigating existing, and worked on formulation of new fault-tolerance models. In particular, we have partly finished evaluation of Consensus Voting in the presence of correlated failures, and are in the process of finishing evaluation of Consensus Recovery Block (CRB) under failure correlation. We find both approaches far superior to commonly employed fixed agreement number voting (usually majority voting). We have also finished a cost analysis of the CRB approach.

Vouk, Mladen A.↗

Joint Work Plan for DPC Filler Testing

The US generates approximately 2,000 Metric Tons of Heavy Metal (MTHM) of commercial spent fuel (SNF) each year and currently stores ~ 85,000 MTHM of commercial SNF at 70+ reactor sites, the disposal of which is the responsibility of the US Department of Energy (DOE). SNF is initially stored in spent nuclear fueld pools (SFPs). SFPs were initially constructed by US utilities for temporary fuel storage, but with no final disposal pathway available, SFPs are reaching capacity. To allow continued operation of the nation's commercial nuclear reactor fleet, utilities started transferring SNF from SFPs (wet storage) to dry cask storage systems, typically using dual-purpose (storage and transportation) canisters (DPCs). And while DPCs were designed, licensed and loaded to meet Nuclear Regulatory Commission (NRC) requirements that preclude the possibility of a critical event during SNF storage and transport, they were not designed or loaded to preclude the possibility of a criticality event during the regulated post-closure period following disposal, which could be up to 1,000,000 years (Price, 2019). DPC filler option criteria are detailed and materials that exhibit these attributes are explored. This document is an update of the SNL Joint Workplan on Filler Investigations for DPCs.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

[A Handling Qualities Metric for Damaged Aircraft]

In recent flight tests of F-15 Intelligent Flight Control System (IFCS), software simulated aircraft control surface failures were inserted to evaluate the IFCS adaptive systems. The failure commanded the left stabilator to a fixed position. The adaptive system uses a neural network that is designed to change control law gains, in the event of damage (real or simulated), that allows the aircraft to fly as it had before the damage. The performance of the adaptive system was assessed in terms of its ability to re-establish good onboard model tracking and its ability to decouple roll and pitch response.

Cogan, Bruce↗

Testing the no-hair theorem with event horizon telescope observations of Sagittarius A*

The advent of the Event Horizon Telescope (EHT), a millimeter-wave very long baseline interferometric array, has enabled spatially resolved studies of the subhorizon-scale structure for a handful of supermassive black holes. Among these, the supermassive black hole at the center of the Milky Way, Sagittarius A* (Sgr A*), presents the largest angular cross section. Thus far, these studies have focused on measurements of the black hole spin and the validation of low-luminosity accretion models. However, a critical input in the analysis of EHT data is the structure of the black hole spacetime, and thus these observations provide the novel opportunity to test the applicability of the Kerr metric to astrophysical black holes. Here we present the first simulated images of a radiatively inefficient accretion flow (RIAF) around Sgr A* employing a quasi-Kerr metric that contains an independent quadrupole moment in addition to the mass and spin that fully characterize a black hole in general relativity. We show that these images can be significantly different from the images of an RIAF around a Kerr black hole with the same spin and demonstrate the feasibility of testing the no-hair theorem by constraining the quadrupolar deviation from the Kerr metric with existing EHT data. Equally important, we find that the disk inclination and spin orientation angles are robust to the inclusion of additional parameters, providing confidence in previous estimations assuming the Kerr metric based on EHT observations. However, at present, the limits on potential modifications of the Kerr metric remain weak.

79 ASTRONOMY AND ASTROPHYSICS↗

Tidal Turbine Test, Downeast Turbines, July 12, 2021

Downeast Turbines tested a tidal turbine prototype with novel rotor/channel system and lateral effluent discharge apparatus (LEDA), during five days of testing in the flume at Alden Lab. Three days of testing (July 12-14, 2021) focused on turbine power metrics of torque and rpm, which were low, and then two days of follow-up testing (July 27-28, 2021) focused on LEDA performance metrics of pressure differential and rates of volumetric flow, with encouraging results. Next step is to to characterize, and even optimize, configurations of the LEDA, using 3D-CFD as a helpful tool, to refine its shape and explore its limits of performance as a means of effluent discharge that augments performance of an instream turbine. An improved configuration of the LEDA will be re-combined with the rotor/channel system of the turbine prototype, and ultimately, the rotor will be re-sized (enlarged), to better match the LEDA's performance capabilities in drawing through a rate of volumetric flow. This submission is Downeast Turbines' Post Access Report for the test event. It includes the files described here (next below), and several reference links. "Downeast TEAMER-Post-Access-Report...docx" is a document file containing the report. "Appendices A, B, and C" are included in this file, and so are "Figures #1-7." "Appendix D - Test Data Workbooks.zip" is an archive file containing all post access data (raw data tables, calculating tables, and graphs), presented in fourteen Excel workbooks as described in the report. "Appendix E - Post Access Figures.zip" is an archive file containing "Figures #8-52," of the report.

16 TIDAL AND WAVE POWER↗

MODELING A SPENT NUCLEAR FUEL CASK SEISMIC TEST

The US Department of Energy Spent Fuel and Waste Science and Technology (SFWST) program is planning to conduct a series of full-scale shake table tests to simulate hypothetical earthquake conditions and record the response of surrogate spent nuclear fuel (SNF) assemblies in a canister dry storage system mockup. The shake table motions will represent a range of hypothetical earthquake conditions at hypothetical locations in the continental US to generally define the range of mechanical loads that SNF can be expected to experience during extended dry storage periods. The earthquake conditions will represent seismic hazards in the 2,000-to-20,000-year return period range. The test will use instrumented pressurized water reactor fuel assemblies with surrogate mass inside the fuel rods instead of radioactive fuel pellets. Dummy assemblies with similar mass and dimensions to actual fuel assemblies will occupy the rest of the fuel assembly locations in a SNF canister that holds 32 fuel assemblies. The canister will be located inside a mockup vertical concrete cask. Instrumentation will record the motion of the major components of the complex dynamic system, and strain gauges will be used to record the cladding strain at select locations. Preparations for the test require modeling predictions to identify the range of response of the system and to help select specific earthquake cases to be simulated on the shake table from a large set of potential cases. This paper describes the pretest nonlinear finite element modeling efforts that have been completed to date, including cask system level modeling and fuel assembly modeling in LS-DYNA. The cask system level models are critical for anticipating sliding or tipping of an unanchored cask during the test. The fuel assembly model is needed to estimate the range of cladding strain response and fuel assembly structural response to be expected during the test and cladding strain measurements will be key metrics for model validation and the development of modeling best practices after the test is concluded.

Klymyshyn, Nicholas A.↗

A Psychoacoustic Test for Urban Air Mobility Vehicle Sound Quality

This paper describes a psychoacoustic test in the Exterior Effects Room (EER) at the NASA Langley Research Center. The test investigated the degree to which sound quality metrics (sharpness, tonality, etc.) are predictive of annoyance to notional sounds of Urban Air Mobility (UAM) vehicles (i.e., air taxis). A suite of 136 unique (4.6 second duration) UAM rotor noise stimuli was generated. These stimuli were based on aeroacoustic predictions of a NASA reference UAM quadrotor aircraft under two flight conditions. The synthesizer changed rotor noise parameters such as the blade passage frequency, the relative level of broadband self-noise, and the relative level of tonal motor noise. With loudness constant, the synthesis parameters impacted sound quality in a way that created a spread of predictors both in synthesizer parameters and in sound quality metrics. Forty subjects listened to the suite of UAM noise stimuli in the EER and judged each sound individually on a standard scale of annoyance. Additionally, a subset of the UAM noise stimuli were compared to a reference sound that varied in loudness. From these responses, the relative effect of changes in loudness or changes in other sound quality metrics on annoyance was evaluated. This paper covers background and motivation for the test, details of how the sound stimuli were generated, and details of the test design and execution. Test results investigate how sound quality may affect perceived annoyance to UAM vehicle noise, indicating the importance of sharpness, tonality, impulsiveness, and roughness on annoyance to UAM noise.

UAM↗

Suited and Unsuited Hybrid III Impact Testing and Finite Element Model Characterization

NASA spacecraft design requirements for occupant protection are a combination of the Brinkley Dynamic Response Criteria and injury assessment reference values (IARV) extracted from anthropomorphic test devices (ATD). For the ATD IARVs, the requirements specify the use of the 5th percentile female Hybrid III and the 95th percentile male Hybrid III. Each of these ATDs is required to be fitted with an articulating pelvis (also known as the aerospace pelvis) and a straight spine. The articulating pelvis is necessary for the ATD to fit into spacecraft seats, while the straight spine is required as injury metrics for vertical accelerations are better defined for this configuration. Sled testing of the Hybrid III 5th Percentile Female Anthropomorphic Test Device (ATD) was performed at Wright-Patterson Air Force Base (WAPFB). Two 5th Percentile ATDs were tested, the Air Force Research Lab (AFRL) and NASA owned Hybrid III ATDs with aerospace pelvises. Testing was also conducted with a NASA-owned 95th Percentile Male Hybrid III with aerospace pelvis at WPAFB. Testing was performed using an Orion seat prototype provided by Johnson Space Center (JSC). A 5-point harness comprised of 2 inch webbing was also provided by JSC. For suited runs, a small and extra-large Advanced Crew Escape System (ACES) suit and helmet were also provided by JSC. Impact vectors were combined frontal/spinal and rear/lateral. Some pure spinal and rear axis testing was also performed for model validation. Peak accelerations ranged between 15 and 20-g. This range was targeted because the ATD responses fell close to the IARV defined in the Human-Systems Integration Requirements (HSIR) document. Rise times varied between 70 and 110 ms to assess differences in ATD responses and model correlation for different impact energies. The purpose of the test series was to evaluate the Hybrid III ATD models in Orion-specific landing orientations both with and without a spacesuit. The results of these tests were used by the NASA Engineering and Safety Center (NESC) to validate the finite element model (FEM) of the Hybrid III 5th Percentile Female ATD. Physical test data was compared to analytical predictions from simulations, and modelling uncertainty factors have been determined for each injury metric. Additionally, the test data has been used to further improve the FEM, particularly in the areas of the ATD preload, harness, and suit and helmet effects.

Lawrence, C.↗

TU-AB-BRA-05: Repeatability of [F-18]-NaF PET Imaging Biomarkers for Bone Lesions: A Multicenter Study

Purpose: Quantifying the repeatability of imaging biomarkers is critical for assessing therapeutic response. While therapeutic efficacy has been traditionally quantified by SUV metrics, imaging texture features have shown potential for use as quantitative biomarkers. In this study we evaluated the repeatability of quantitative {sup 18}F-NaF PET-derived SUV metrics and texture features in bone lesions from patients in a multicenter study. Methods: Twenty-nine metastatic castrate-resistant prostate cancer patients received whole-body test-retest NaF PET/CT scans from one of three harmonized imaging centers. Bone lesions of volume greater than 1.5 cm{sup 3} were identified and automatically segmented using a SUV>15 threshold. From each lesion, 55 NaF PET-derived texture features (including first-order, co-occurrence, grey-level run-length, neighbor gray-level, and neighbor gray-tone difference matrix) were extracted. The test-retest repeatability of each SUV metric and texture feature was assessed with Bland-Altman analysis. Results: A total of 315 bone lesions were evaluated. Of the traditional SUV metrics, the repeatability coefficient (RC) was 12.6 SUV for SUVmax, 2.5 SUV for SUVmean, and 4.3 cm{sup 3} for volume. Their respective intralesion coefficients of variation (COVs) were 12%, 17%, and 6%. Of the texture features, COV was lowest for entropy (0.03%) and highest for kurtosis (105%). Lesion intraclass correlation coefficient (ICC) was lowest for maximum correlation coefficient (ICC=0.848), and highest for entropy (ICC=0.985). Across imaging centers, repeatability of texture features and SUV varied. For example, across imaging centers, COV for SUVmax ranged between 11–23%. Conclusion: Many NaF PET-derived SUV metrics and texture features for bone lesions demonstrated high repeatability, such as SUVmax, entropy, and volume. Several imaging texture features demonstrated poor repeatability, such as SUVtotal and SUVstd. These results can be used to establish response criteria for NaF PET-based treatment response assessment. Prostate Cancer Foundation (PCF)

60 APPLIED LIFE SCIENCES↗

An evaluation of the variable-resolution CESM for modeling California's climate

In this paper, the recently developed variable-resolution option within the Community Earth System Model (VR-CESM) is assessed for long-term regional climate modeling of California at 0.25° (~ 28 km) and 0.125° (~ 14 km) horizontal resolutions. The mean climatology of near-surface temperature and precipitation is analyzed and contrasted with reanalysis, gridded observational data sets, and a traditional regional climate model (RCM)—the Weather Research and Forecasting (WRF) model. Statistical metrics for model evaluation and tests for differential significance have been extensively applied. VR-CESM tended to produce a warmer summer (by about 1–3°C) and overestimated overall winter precipitation (about 25%–35%) compared to reference data sets when sea surface temperatures were prescribed. Increasing resolution from 0.25° to 0.125° did not produce a statistically significant improvement in the model results. By comparison, the analogous WRF climatology (constrained laterally and at the sea surface by ERA-Interim reanalysis) was ~1–3°C colder than the reference data sets, underestimated precipitation by ~20%–30% at 27 km resolution, and overestimated precipitation by ~ 65–85% at 9 km. Overall, VR-CESM produced comparable statistical biases to WRF in key climatological quantities. Moreover, this assessment highlights the value of variable-resolution global climate models (VRGCMs) in capturing fine-scale atmospheric processes, projecting future regional climate, and addressing the computational expense of uniform-resolution global climate models.

54 ENVIRONMENTAL SCIENCES↗

A multi‐variable framework for selecting WRF physics configurations at convection‐permitting scales: An Amazon wet‐season case study

Tropical convection over rainforests modulates atmospheric circulation and the energy and hydrological cycles across multiple scales. However, the scarcity of observations still limits our understanding of these processes. Although numerical models are utilized to investigate atmospheric physical processes, their performance depends on the choice of parameterizations and grid resolution. Here, in this work, we introduce and apply a multi‐variable, multi‐physics framework to evaluate and rank the Advanced Research Weather Research and Forecasting (WRF–ARW) configurations at 1‐km resolution over the central Amazon during the wet season. A 48‐member ensemble combines three land‐surface models (LSMs), four planetary boundary layer (PBL), and four microphysics (MP) schemes. Model performance for seven convection‐related near‐surface and boundary‐layer variables is assessed using Taylor diagrams and the Taylor skill score (TSS), analysis of variance (ANOVA)‐based sensitivity metrics, and non‐parametric rank tests. We then construct a combined, weighted TSS to identify configurations that are comparatively robust across variables. Results showed that most configurations reproduce near‐surface temperature, sensible and latent heat fluxes, and boundary‐layer height reasonably well, whereas humidity and rainfall remain challenging. LSM choice has the strongest impact on the surface fluxes and a secondary influence on near‐surface temperature and humidity, PBL schemes dominate boundary‐layer height, and MP schemes exert the largest control on rainfall. No single configuration is optimal for all variables, but the combination of Noah (LSM), Yonsei University (PBL), and Morrison (MP) emerges as the most robust configuration for this case, with the WRF single‐moment six‐class scheme (WSM6) providing a competitive, computationally cheaper MP alternative. The framework is general and can be applied to other regions, seasons, and convective regimes.

Amazon rainforest↗

Smoothing and flattening the universe through slow contraction versus inflation

In a systematic study, we use an equivalent pair of improved numerical relativity codes based on a tetrad-formulation of the classical Einstein-scalar field equations to examine whether slow contraction or inflation (or both) can resolve the homogeneity, isotropy and flatness problems. Our finding, based on a set of gauge/frame invariant diagnostics and the models considered, is that slow contraction robustly and rapidly smooths and flattens spacetime beginning from initial conditions that are outside the perturbative regime of the flat Friedmann-Robertson-Walker metric, whereas inflation fails these tests. We present new numerical evidence supporting the conjecture that the combination of ultralocal evolution and an effective equation-of-state with pressure much greater than energy density is the key to having robust and rapid smoothing. The opposite of ultralocality occurs in expanding spacetimes, which is the leading obstruction to smoothing following a big bang.

79 ASTRONOMY AND ASTROPHYSICS↗