Search NASA⌕ Search

SEARCH · Search NASA

Results for “Testing metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 253 records · Page 14

Software unit testing in Ada environment

A validation procedure for the Ada binding of the Graphical Kernel System (GKS) is being developed. PRIOR Data Sciences is also producing a version of the GKS written in Ada. These major software engineering projects will provide an opportunity to demonstrate a sound approach for software testing in an Ada environment. The GKS/Ada validation capability will be a collection of test programs and data, and test management guidelines. These products will be used to assess the correctness, completeness, and efficiency of any GKS/Ada implementation. The GKS/Ada developers will be able to obtain the validation software for their own use. It is anticipated that this validation software will eventually be taken over by an independent standards body to provide objective assessments of GKS/Ada implementations, using an approach similar to the validation testing currently applied to Ada compilers. In the meantime, if requested, this validation software will be used to assess GKS/Ada products. The second project, implementation of GKS using the Ada language, is a conventional software engineering tasks. It represents a large body of Ada code and has some interesting testing problems associated with automatic testing of graphics routines. Here the normal test practices which include automated regression testing, independent quality assistance, test configuration management, and the application of software quality metrics will be employed. The software testing methods emphasize quality enhancement and automated procedures. Ada makes some aspects of testing easier, and introduces some concerns. These issues are addressed.

Warnock, Glenn↗

Experimental constraints on metric and non-metric theories of gravity

Experimental constraints on metric and non-metric theories of gravitation are reviewed. Tests of the Einstein Equivalence Principle indicate that only metric theories of gravity are likely to be viable. Solar system experiments constrain the parameters of the weak field, post-Newtonian limit to be close to the values predicted by general relativity. Future space experiments will provide further constraints on post-Newtonian gravity.

Will, Clifford M.↗

TH-E-BRF-02: 4D-CT Ventilation Image-Based IMRT Plans Are Dosimetrically Comparable to SPECT Ventilation Image-Based Plans

Purpose: An emerging lung ventilation imaging method based on 4D-CT can be used in radiotherapy to selectively avoid irradiating highly-functional lung regions, which may reduce pulmonary toxicity. Efforts to validate 4DCT ventilation imaging have been focused on comparison with other imaging modalities including SPECT and xenon CT. The purpose of this study was to compare 4D-CT ventilation image-based functional IMRT plans with SPECT ventilation image-based plans as reference. Methods: 4D-CT and SPECT ventilation scans were acquired for five thoracic cancer patients in an IRB-approved prospective clinical trial. The ventilation images were created by quantitative analysis of regional volume changes (a surrogate for ventilation) using deformable image registration of the 4D-CT images. A pair of 4D-CT ventilation and SPECT ventilation image-based IMRT plans was created for each patient. Regional ventilation information was incorporated into lung dose-volume objectives for IMRT optimization by assigning different weights on a voxel-by-voxel basis. The objectives and constraints of the other structures in the plan were kept identical. The differences in the dose-volume metrics have been evaluated and tested by a paired t-test. SPECT ventilation was used to calculate the lung functional dose-volume metrics (i.e., mean dose, V20 and effective dose) for both 4D-CT ventilation image-based and SPECT ventilation image-based plans. Results: Overall there were no statistically significant differences in any dose-volume metrics between the 4D-CT and SPECT ventilation imagebased plans. For example, the average functional mean lung dose of the 4D-CT plans was 26.1±9.15 (Gy), which was comparable to 25.2±8.60 (Gy) of the SPECT plans (p = 0.89). For other critical organs and PTV, nonsignificant differences were found as well. Conclusion: This study has demonstrated that 4D-CT ventilation image-based functional IMRT plans are dosimetrically comparable to SPECT ventilation image-based plans, providing evidence to use 4D-CT ventilation imaging for clinical applications. Supported in part by Free to Breathe Young Investigator Research Grant and NIH/NCI R01 CA 093626. The authors thank Philips Radiation Oncology Systems for the Pinnacle3 treatment planning systems.

60 APPLIED LIFE SCIENCES↗

Power Prediction of Airborne Wind Energy Systems Using Multivariate Machine Learning

Kites can be used to harvest wind energy at higher altitudes while using only a fraction of the material required for conventional wind turbines. In this work, we present the kite system of Kyushu University and demonstrate how experimental data can be used to train machine learning regression models. The system is designed for 7 kW traction power and comprises an inflatable wing with suspended kite control unit that is either tethered to a fixed ground anchor or to a towing vehicle to produce a controlled relative flow environment. A measurement unit was attached to the kite for data acquisition. To predict the generated tether force, we collected input–output samples from a set of well-designed experimental runs to act as our labeled training data in a supervised machine learning setting. We then identified a set of key input parameters which were found to be consistent with our sensitivity analysis using Pearson input–output correlation metrics. Finally, we designed and tested the accuracy of a neural network, among other multivariate regression models. The quality metrics of our models show great promise in accurately predicting the tether force for new input/feature combinations and potentially guide new designs for optimal power generation.

17 WIND ENERGY↗

SU-E-I-65: The Joint Commission's Requirements for Annual Diagnostic Physics Testing of Nuclear Medicine Equipment, and a Clinically Relevant Methodology for Testing Low-Contrast Resolution

Purpose: To introduce the Joint Commission's requirements for annual diagnostic physics testing of all nuclear medicine equipment, effective 7/1/2014, and to highlight an acceptable methodology for testing lowcontrast resolution of the nuclear medicine imaging system. Methods: The Joint Commission's required diagnostic physics evaluations are to be conducted for all of the image types produced clinically by each scanner. Other accrediting bodies, such as the ACR and the IAC, have similar imaging metrics, but do not emphasize testing low-contrast resolution as it relates clinically. The proposed method for testing low contrast resolution introduces quantitative metrics that are clinically relevant. The acquisition protocol and calculation of contrast levels will utilize a modified version of the protocol defined in AAPM Report #52. Results: Using the Rose criterion for lesion detection with a SNRpixel = 4.335 and a CNRlesion = 4, the minimum contrast levels for 25.4 mm and 31.8 mm cold spheres were calculated to be 0.317 and 0.283, respectively. These contrast levels are the minimum threshold that must be attained to guard against false positive lesion detection. Conclusion: Low contrast resolution, or detectability, can be properly tested in a manner that is clinically relevant by measuring the contrast level of cold spheres within a Jaszczak phantom using pixel values within ROI's placed in the background and cold sphere regions. The measured contrast levels are then compared to a minimum threshold calculated using the Rose criterion and a CNRlesion = 4. The measured contrast levels must either meet or exceed this minimum threshold to prove acceptable lesion detectability. This research and development activity was performed by the authors while employed at West Physics Consulting, LLC. It is presented with the consent of West Physics, which has authorized the dissemination of the information and/or techniques described in the work.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Comprehensive Evaluation for Protective Coatings: Optical, Electrical, Photoelectrochemical, and Spectroscopic Characterizations

Numerous efficient semiconductors suffer from instability in aqueous electrolytes. Strategies utilizing protective coatings have thus been developed to protect these photoabsorbers against corrosion while synergistically improving charge separation and reaction kinetics. Recently, various photoelectrochemical (PEC) protective coatings have been reported with suitable electronic properties to ensure low charge transport loss and reveal the fundamental photoabsorber efficiency. However, protocols for studying the critical figures of merit for protective coatings have yet to be established. For this reason, we propose four criteria for evaluating the performance of a protective coating for PEC water-splitting: stability, conductivity, optical transparency, and energetic matching. We then propose a flow chart that summarizes the recommended testing protocols for quantifying these four performance metrics. In particular, we lay out the stepwise testing protocols to evaluate the energetics matching at a semiconductor/coating/(catalyst)/liquid interface. Finally, we provide an outlook for the future benchmarking needs for coatings.

36 MATERIALS SCIENCE↗

Software development predictors, error analysis, reliability models and software metric analysis

The use of dynamic characteristics as predictors for software development was studied. It was found that there are some significant factors that could be useful as predictors. From a study on software errors and complexity, it was shown that meaningful results can be obtained which allow insight into software traits and the environment in which it is developed. Reliability models were studied. The research included the field of program testing because the validity of some reliability models depends on the answers to some unanswered questions about testing. In studying software metrics, data collected from seven software engineering laboratory (FORTRAN) projects were examined and three effort reporting accuracy checks were applied to demonstrate the need to validate a data base. Results are discussed.

Basili, Victor↗

Prioritizing Off-gas Metrics

Off-gas research within the Material Recovery and Waste Form Development (MRWFD) team is dispersed among four national laboratories and multiple universities and small businesses. Ensuring research is conducted consistently and in a manner conducive to direct comparison is a nontrivial task. The team assembled to realign goals and expectations by identifying distinct R&D phases, prioritizing crucial metrics, and outlining standardized tests specifically associated with Kr, Xe, and I capture.

11 - NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Evaluating Synthetic Smart Meter Locations For Communication System Modeling

An important part of good communication models for smart grid applications, particularly when wireless protocols are used, is the location information of the nodes. We’ve developed metrics to assist in evaluating whether a given model with such information is representative of feeders found in the world. These metrics are applied to a set of feeder models with known good location information for the smart meter locations and compared to a set of feeder models with similar location information that is not expected to be representative of real-world feeders. The comparison reveals that the suspect models do not pass statistical tests utilizing the developed metrics, providing initial validation of the analysis technique.

smart meter, feeder model, GridLAB-D↗

A building performance-based approach to determining energy resilience for grocery stores in the event of a power outage

Evaluating progress toward a built environment that is best equipped to serve communities during a regional power outage will require metrics that capture the energy resilience of the unique buildings and businesses most crucial to the well-being of those nearby. We focused on grocery stores as key buildings for which access, and thus energy resilience, is critical during a disaster when power is unavailable. We evaluated the energy resilience of these buildings by offering and testing building-scale metrics that assess business continuity potential during a power outage. Metrics proposed in this study are calculated based on the unique power loads characteristic to grocery stores, primarily refrigeration and maintaining safe indoor environmental conditions. Building simulations based on varying levels of backup power were carried out against occupant safety and comfort parameters to apply these metrics, with additional criteria imposed on grocery stores to capture the inventory and sales loss from food spoilage resulting from a lack of refrigeration power. Findings from this study demonstrate the feasibility of our proposed metrics and methodology to serve as a low-data burden means for stakeholders to evaluate the energy resilience of grocery stores, with greater implications in helping to understand the impact on community-scale energy resilience.

Siegel, Lino Sanchez↗

Advances in Integrally Formed Metallic Structures

NASA Langley Research Center (LaRC) is applying flow forming technology to aerospace aluminum alloys for the manufacture of integrally stiffened cylinders that meet aircraft fuselage or launch vehicle specifications. High-strength 2xxx series alloys, in various temper/product form conditions, are being compared with medium-strength, very formable AA 6061 as the benchmark. Screening of candidate materials traditionally relies on empirical forming trials, but more recently on metrics derived from constant nominal strain rate tensile testing. In this work, the established hierarchy of formability metrics is adopted as the baseline; tensile area reduction, work hardening exponent, and modulus of resilience. The suite of metrics can be expanded based on phenomenological and mechanistic considerations of plastic flow instabilities to incorporate anisotropy ratio, rate of work hardening, and dynamic strain aging effects. Separate research thrusts at LaRC are aimed at formulating a protocol for scalable production of fully formed, defect-free components: 1) quantifying the extended set of formability metrics from constant and jump strain rate tensile tests with digital image correlation (DIC); 2) using statistically designed experiments on a vertical spin/flow forming facility (WF VUD-600®) to isolate the effect of convenient process variables, such as axial feed rate and thickness reduction; and 3) predicting the effect of feed ratio (roller translation relative to mandrel rotation speeds) on material response via process modeling using the DEFORM® simulation software. Current results will be discussed in terms of correlating intrinsic material properties with the bulk workability characteristics to successfully fabricate cylinders with complex cross-sections.

Aluminum↗

Development of an All-Aqueous Thermally Regenerative Redox Flow Battery to Support Fossil Fuel Assets

Low-temperature thermal energy, a largely untapped resource, holds significant promise for large-scale electrical power generation globally. Various stationary sources, including industrial entities and thermal power plants, emit considerable low-temperature heat that currently remains unutilized. This energy is often overlooked because its low temperature renders it unsuitable for efficient power generation using conventional methods. However, current research is exploring diverse technologies capable of converting low-temperature heat into grid-scale power, aiming to enhance grid efficiency, further decarbonization initiatives, and facilitate a shift toward more decentralized power systems. One such innovative technology is the thermally regenerative battery (TRB), noted for its high power and energy densities compared to similar technologies, positioning it as a potential game-changer in power generation. TRBs integrate two scalable and well-established unit operations: a redox flow battery and a distillation column. This integration suggests that once an effective TRB chemistry is established, the pathway to commercialization could be expedited. The copper-based thermally regenerative ammonia battery (Cu aq -TRAB) stands out as the first TRB that circumvents electrodeposition/dissolution reactions, stabilizing Cu(I) and Cu(II) within the electrolyte and maintaining stability of all electroactive species in an aqueous phase. This stabilization has led to improvements in coulombic efficiency, open circuit potential, and copper solubility, thereby enhancing power density, energy density, and overall energy efficiency. Preliminary tests were conducted to determine the effects of various electrolyte species on the performance metrics of the battery, both theoretically and experimentally. These tests revealed that the solubility of copper in the Cu aq -TRAB electrolyte was constrained by the Cu(I)-NH 3 complex. Adjusting the background electrolyte to 5 M NH4Br and the ligand concentration to 4 M NH 3 enabled the copper concentration to reach a maximum of 0.6 M. This modification led to an estimated theoretical maximum energy density of 9.5 Wh L -1 for the Cu aq -TRAB. Additionally, full cell testing indicated a tradeoff between peak power and energy density with varying copper and ammonia concentrations. Increasing the applied current density during discharge linearly raised the average power output, with a minimal reduction in energy density due to a balance between higher ohmic overpotential and reduced time for undesirable ammonia crossover. Furthermore, a comprehensive numerical sensitivity analysis of the complete Cu aq -TRAB system was performed. This analysis aimed to assess how the battery and the distillation column responded to changes in system input parameters, providing insights into optimal research directions for enhancing system performance. The analysis revealed that at room temperature, battery power was significantly more sensitive to ohmic losses than to mass transfer, with reaction rates having minimal impact. This trend continued even at higher temperatures. Also, the thermal energy required for ammonia separation was studied, showing that increased temperatures generally reduced energy requirements, except in low-pressure scenarios above 65 °C. An investigation into membrane performance in the Cu aq -TRAB was undertaken, given the significant impact of ammonia transport control and ohmic losses on system performance. Various membranes were evaluated to identify key performance metrics. Among the tested membranes, Selemion CMVN exhibited the highest performance, with a peak power density of 84 mW cm -2 and average values of 26 ± 6.8 mW cm -2 for power density and 2.9 Wh L -1 for energy density at an applied current density of 50 mA cm -2 . An economic assessment indicated a levelized cost of storage at $410 per MWh under optimal conditions, highlighting the commercial potential of the Cu aq -TRAB when utilizing cost-effective, readily available materials.

25 ENERGY STORAGE↗

Optimal Transport for e/$\pi^0$ Particle Classification in LArTPC Neutrino Experiments

Separation of electron signals from $\pi^0$ backgrounds is crucial for neutrino oscillation measurements and searches for Beyond Standard Model (BSM) physics in current and future Liquid Argon Time Projection Chamber (LArTPC) experiments. e/$\pi^0$ separation has been a reconstruction challenge since both e and $\pi^0$ present as electromagnetic showers, and often only one out of the two showers produced by $\pi^0$ is reconstructed correctly. This research aims to improve the performance of e/$\pi^0$ separation using optimal transport (OT), by leveraging on the topological differences in the showers produced by the two particles. OT is a method which compares two distributions by finding the most efficient way to transform, or “move” from one to the other. This work uses the MicroBooNE open samples public dataset to test the e/$\pi^0$ separation performance of the method on events which incorporate realistic modeling of LArTPC detector response. Reconstructed 3D energy deposits are projected onto a plane perpendicular to the primary shower, allowing OT to better detect the topological differences between the two types of particles without the need to separately reconstruct all the showers in the events. Different distance metrics for OT are tested and preliminary results on e/$\pi^0$ separation are presented.

43 PARTICLE ACCELERATORS↗

Automated Testcase Generation for Numerical Support Functions in Embedded Systems

We present a tool for the automatic generation of test stimuli for small numerical support functions, e.g., code for trigonometric functions, quaternions, filters, or table lookup. Our tool is based on KLEE to produce a set of test stimuli for full path coverage. We use a method of iterative deepening over abstractions to deal with floating-point values. During actual testing the stimuli exercise the code against a reference implementation. We illustrate our approach with results of experiments with low-level trigonometric functions, interpolation routines, and mathematical support functions from an open source UAS autopilot.

Metrics↗

Activities of the NASA Urban Air Mobility Noise Working Group (UNWG)

The NASA-led Urban Air Mobility Noise Working Group was established in 2018 following an exploratory meeting of stakeholders from traditional and emerging aviation sectors across industry, government agencies, academia, and community groups. Its mission is to create and support a diverse community of acoustics experts to identify, discuss, and address noise issues associated with UAM vehicles and their operations. The UNWG is comprised of four subgroups including Tools and Technologies, Ground and Flight Testing, Human Response and Metrics, and Regulation and Policy. In 2020, NASA published the white paper entitled “Urban Air Mobility Noise: Current Practice, Gaps, and Recommendations,” that presents a set of high-level goals and makes recommendations to address gaps in the current practice. This presentation is about recent activities of each of the subgroups to address those recommendations, including work towards establishment of an aeroacoustic test database for model validation, development of a set of measurement guidelines to quantify vehicle acoustic emission, a remotely administered human response study for quantifying annoyance to UAM operations, and status on noise standard, regulation, policy, and community engagement developments.

Urban air mobility↗

Validation of an expert system intended for research in distributed artificial intelligence

The expert system discussed in this paper is designed to function as a testbed for research on cooperating expert systems. Cooperating expert systems are members of an organization which dictates the manner in which the expert systems will interact when solving a problem. The Blackbox Expert described in this paper has been constructed using the C Language Integrated Production System (CLIPS), C++, and X windowing environment. CLIPS is embedded in a C++ program which provides objects that are used to maintain the state of the Blackbox puzzle. These objects are accessed by CLIPS rules through user-defined functions calls. The performance of the Blackbox Expert is validated by experimentation. A group of people are asked to solve a set of test cases for the Blackbox puzzle. A metric has been devised which evaluates the 'correctness' of a solution proposed for a test case of Blackbox. Using this metric and the solutions proposed by the humans, each person receives a rating for their ability to solve the Blackbox puzzle. The Blackbox Expert solves the same set of test cases and is assigned a rating for its ability. Then the rating obtained by the Blackbox Expert is compared with the ratings of the people, thus establishing the skill level of our expert system.

Grossner, C.↗

Development of a standardized battery of performance tests for the assessment of noise stress effects

A set of 36 relatively independent categories of human performance were identified. These categories encompass human performance in the cognitive, perceptual, and psychomotor areas, and include diagnostic measures and sensitive performance metrics. Then a prototype standardized test battery was constructed, and research was conducted to obtain information on the sensitivity of the tests to stress, the sensitivity of selected categories of performance degradation, the time course of stress effects on each of the selected tests, and the learning curves associated with each test. A research project utilizing a three factor partially repeated analysis of covariance design was conducted in which 60 male subjects were exposed to variations in noise level and quality during performance testing. Effects of randomly intermittent noise on performance of the reaction time tests were observed, but most of the other performance tests showed consistent stability. The results of 14 analyses of covariance of the data taken from the performance of the 60 subjects on the prototype standardized test battery provided information which will enable the final development and test of a standardized test battery and the associated development of differential sensitivity metrics and diagnostic classificatory system.

Theologus, G. C.↗

Automated, high-accuracy classification of textured microstructures using a convolutional neural network

Crystallographic texture is an important descriptor of material properties but requires time-intensive electron backscatter diffraction (EBSD) for identifying grain orientations. While some metrics such as grain size or grain aspect ratio can distinguish textured microstructures from untextured microstructures after significant grain growth, such morphological differences are not always visually observable. This paper explores the use of deep learning to classify experimentally measured textured microstructures without knowledge of crystallographic orientation. A deep convolutional neural network is used to extract high-order morphological features from binary images to distinguish textured microstructures from untextured microstructures. The convolutional neural network results are compared with a statistical Kolmogorov–Smirnov tests with traditional morphological metrics for describing microstructures. Results show that the convolutional neural network achieves a significantly improved classification accuracy, particularly at early stages of grain growth, highlighting the capability of deep learning to identify the subtle morphological patterns resulting from texture. The results demonstrate the potential of a convolutional neural network as a tool for reliable and automated microstructure classification with minimal preprocessing.

36 MATERIALS SCIENCE↗