Search NASA⌕ Search

SEARCH · Search NASA

Results for “Testing metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 523 records · Page 29

Multifamily Envelope Leakage Model

The objective of the 2013 research project was to develop the model for predicting fully guarded test results (FGT), using unguarded test data and specific building features of apartment units. The model developed has a coefficient of determination R2 value of 0.53 with a root mean square error (RMSE) of 0.13. Both statistical metrics indicate that the model is relatively strong. When tested against data that was not included in the development of the model, prediction accuracy was within 19%, which is reasonable given that seasonal differences in blower door measurements can vary by as much as 25%.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Building America Case Study: Predicting Envelope Leakage in Attached Dwellings (Fact Sheet)

'The cost for blower testing is high, because it is labor intensive, and it may disrupt occupants in multiple units. This high cost and disruption deters program participants, and dissuades them from pursuing energy improvements that would trigger air leakage testing, such as improvements to the building envelope.' This statement found in a 2012 report by Heschong Mahone Group emphasizes the importance of reducing the cost and complexity of blower testing in multifamily buildings. Energy efficiency opportunities are being bypassed. The cost of single blower testing is on the order of $300. The cost for guarded blower door testing, the more appropriate test for assessing energy savings opportunities, could easily be six times that and that's only if you have the equipment and simultaneous access to multiple apartments. Thus, the proper test is simply not performed. The objective of the 2013 research project was to develop the model for predicting fully guarded test results (FGT), using unguarded test data and specific building features of apartment units. The model developed has a coefficient of determination R2 value of 0.53 with a root mean square error (RMSE) of 0.13. Both statistical metrics indicate that the model is relatively strong. When tested against data that was not included in the development of the model, prediction accuracy was within 19%, which is reasonable given that seasonal differences in blower door measurements can vary by as much as 25%.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

{sup 18}F-Fluorodeoxyglucose Positron Emission Tomography Can Quantify and Predict Esophageal Injury During Radiation Therapy

Purpose: We sought to investigate the ability of mid-treatment {sup 18}F-fluorodeoxyglucose positron emission tomography (PET) studies to objectively and spatially quantify esophageal injury in vivo from radiation therapy for non-small cell lung cancer. Methods and Materials: This retrospective study was approved by the local institutional review board, with written informed consent obtained before enrollment. We normalized {sup 18}F-fluorodeoxyglucose PET uptake to each patient's low-irradiated region (<5 Gy) of the esophagus, as a radiation response measure. Spatially localized metrics of normalized uptake (normalized standard uptake value [nSUV]) were derived for 79 patients undergoing concurrent chemoradiation therapy for non-small cell lung cancer. We used nSUV metrics to classify esophagitis grade at the time of the PET study, as well as maximum severity by treatment completion, according to National Cancer Institute Common Terminology Criteria for Adverse Events, using multivariate least absolute shrinkage and selection operator (LASSO) logistic regression and repeated 3-fold cross validation (training, validation, and test folds). This 3-fold cross-validation LASSO model procedure was used to predict toxicity progression from 43 asymptomatic patients during the PET study. Dose-volume metrics were also tested in both the multivariate classification and the symptom progression prediction analyses. Classification performance was quantified with the area under the curve (AUC) from receiver operating characteristic analysis on the test set from the 3-fold analyses. Results: Statistical analysis showed increasing nSUV is related to esophagitis severity. Axial-averaged maximum nSUV for 1 esophageal slice and esophageal length with at least 40% of axial-averaged nSUV both had AUCs of 0.85 for classifying grade 2 or higher esophagitis at the time of the PET study and AUCs of 0.91 and 0.92, respectively, for maximum grade 2 or higher by treatment completion. Symptom progression was predicted with an AUC of 0.75. Dose metrics performed poorly at classifying esophagitis (AUC of 0.52, grade 2 or higher mid treatment) or predicting symptom progression (AUC of 0.67). Conclusions: Normalized uptake can objectively, locally, and noninvasively quantify esophagitis during radiation therapy and predict eventual symptoms from asymptomatic patients. Normalized uptake may provide patient-specific dose-response information not discernible from dose.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Usability: Human Research Program - Space Human Factors and Habitability

The Usability project addresses the need for research in the area of metrics and methodologies used in hardware and software usability testing in order to define quantifiable and verifiable usability requirements. A usability test is a human-in-the-loop evaluation where a participant works through a realistic set of representative tasks using the hardware/software under investigation. The purpose of this research is to define metrics and methodologies for measuring and verifying usability in the aerospace domain in accordance with FY09 focus on errors, consistency, and mobility/maneuverability. Usability metrics must be predictive of success with the interfaces, must be easy to obtain and/or calculate, and must meet the intent of current Human Systems Integration Requirements (HSIR). Methodologies must work within the constraints of the aerospace domain, be cost and time efficient, and be able to be applied without extensive specialized training.

Sandor, Aniko↗

Reconstructing the metric of the local Universe from number counts observations

Number counts observations available with new surveys such as the Euclid mission will be an important source of information about the metric of the Universe. We compute the low red-shift expansion for the energy density and the density contrast using an exact spherically symmetric solution in presence of a cosmological constant. At low red-shift the expansion is more precise than linear perturbation theory prediction. We then use the local expansion to reconstruct the metric from the monopole of the density contrast. We test the inversion method using numerical calculations and find a good agreement within the regime of validity of the red-shift expansion. The method could be applied to observational data to reconstruct the metric of the local Universe with a level of precision higher than the one achievable using perturbation theory.

79 ASTRONOMY AND ASTROPHYSICS↗

It's All Relative: A Validation of Radiation Quality Comparison Metrics

The difference between high-LET and low-LET radiation is quantified by a measure called relative biological effectiveness (RBE). RBE is defined as the ratio of the dose of a reference radiation to that of a test radiation to achieve the same effect level, and thus, is described either as an iso-effector dose-to-dose ratio. A single dose point is not sufficient to calculate an RBE value; therefore, studies with only one dose point usually calculate an effect-to-effect ratio. While not formally used in radiation protection, these iso-dose values may still be informative. Shuryak, et al 2017 investigated the use of an iso-dose metric termed "radiation effects ratio" (RER) and used both RBE and RER to estimate high-LET risks. To apply RBE or RER to risk prediction, the selected metric must be uniquely defined. That is, the calculated value must be consistent within a model given a constant set of constraints and assumptions, regardless of how effects are defined using statistical transformations from raw endpoint data. We first test the RBE and the RER to determine whether they are uniquely defined using transformations applied to raw data. Then, we test whether both metrics can predict heavy ion response data after simulated effect size scaling between human populations or when converting animal to human endpoints.

Chappell, Lori J.↗

Statistical quality indicators for electron-density maps

A likelihood-based metric for scoring the local agreement of a structure model with the observed electron density is described. The commonly used validation metrics for the local agreement of a structure model with the observed electron density, namely the real-space R (RSR) and the real-space correlation coefficient (RSCC), are reviewed. It is argued that the primary goal of all validation techniques is to verify the accuracy of the model, since precision is an inherent property of the crystal and the data. It is demonstrated that the principal weakness of both of the above metrics is their inability to distinguish the accuracy of the model from its precision. Furthermore, neither of these metrics in their usual implementation indicate the statistical significance of the result. The statistical properties of electron-density maps are reviewed and an improved alternative likelihood-based metric is suggested. This leads naturally to a χ{sup 2} significance test of the difference density using the real-space difference density Z score (RSZD). This is a metric purely of the local model accuracy, as required for effective model validation and structure optimization by practising crystallographers prior to submission of a structure model to the PDB. A new real-space observed density Z score (RSZO) is also proposed; this is a metric purely of the model precision, as a substitute for other precision metrics such as the B factor.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Weather-Corrected Performance Ratio

Photovoltaic (PV) system performance depends on both the quality of the system and the weather. One simple way to communicate the system performance is to use the performance ratio (PR): the ratio of the electricity generated to the electricity that would have been generated if the plant consistently converted sunlight to electricity at the level expected from the DC nameplate rating. The annual system yield for flat-plate PV systems is estimated by the product of the annual insolation in the plane of the array, the nameplate rating of the system, and the PR, which provides an attractive way to estimate expected annual system yield. Unfortunately, the PR is, again, a function of both the PV system efficiency and the weather. If the PR is measured during the winter or during the summer, substantially different values may be obtained, making this metric insufficient to use as the basis for a performance guarantee when precise confidence intervals are required. This technical report defines a way to modify the PR calculation to neutralize biases that may be introduced by variations in the weather, while still reporting a PR that reflects the annual PR at that site given the project design and the project weather file. This resulting weather-corrected PR gives more consistent results throughout the year, enabling its use as a metric for performance guarantees while still retaining the familiarity this metric brings to the industry and the value of its use in predicting actual annual system yield. A testing protocol is also presented to illustrate the use of this new metric with the intent of providing a reference starting point for contractual content.

14 SOLAR ENERGY↗

An Overview of Controls and Flying Qualities Technology on the F/A-18 High Alpha Research Vehicle

The NASA F/A-18 High Alpha Research Vehicle (HARV) has been the flight test bed of a focused technology effort to significantly increase maneuvering capability at high angles of attack. Development and flight test of control law design methodologies, handling qualities metrics, performance guidelines, and flight evaluation maneuvers are described. The HARV has been modified to include two research control effectors, thrust vectoring, and actuated forebody strakes in order to provide increased control power at high angles of attack. A research flight control system has been used to provide a flexible, easily modified capability for high-angle-of-attack research controls. Different control law design techniques have been implemented and flight-tested, including eigenstructure assignment, variable gain output feedback, pseudo controls, and model-following. Extensive piloted simulation has been used to develop nonlinear performance guide-lines and handling qualities criteria for high angles of attack. This paper reviews the development and evaluation of technologies useful for high-angle-of-attack control. Design, development, and flight test of the research flight control system, control laws, flying qualities specifications, and flight test maneuvers are described. Flight test results are used to illustrate some of the lessons learned during flight test and handling qualities evaluations.

Pahle, Joseph W.↗

Performance Analysis and Simulaion of the Hydraulic Scram System in TREAT Reactor

The Transient Reactor Test Facility (TREAT) at Idaho National Laboratory (INL) serves a vital role in nuclear fuel safety research, enabling transient experiments that simulate reactivity excursions and accident scenarios. Central to these operations is the transient control rod drive system (TCRDS), which drives rapid motion of the transient control rods such that TREAT can simulate rapid power changes typical of reactor accidents. The reliability and performance of this system are critical for protecting both fuel specimens and reactor infrastructure. This study presents the initial phase of a two-year investigation into the dynamics and reliability of the TREAT hydraulic TCRDS. Conducted in collaboration with INL, the research employs a combined computational and experimental approach to analyze the system's response time, pressure transients, and potential failure modes. Emphasis is placed on understanding how fluid characteristics influence the TCRDS’s ability to achieve both rapid power changes and mechanical stability. The TRDS and the skid that powers it will be analyzed throughout this investigation. Computational modeling using computational fluid dynamics (CFD) will simulate the hydraulic response under varying conditions. In parallel, experimental testing planned at INL will validate these models and capture key performance metrics. This paper outlines the system design, analytical framework, and modeling strategies that form the foundation for later testing. Ultimately, this work aims to support improvements to the TCRDS’s design and reliability, contributing to the broader goal of enhancing nuclear fuel safety and sustaining TREAT’s mission as a premier nuclear fuel test facility.

21 - SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLAN↗

Medium-temperature phase change material integration in domestic heat pump water heaters for improved thermal energy storage

In this review, we examine state-of-the-art developments in integrating phase change materials (PCMs) for thermal energy storage (TES) in domestic heat pump water heaters (HPWHs). The component design optimization and control optimization of HPWHs and TES are reviewed for insight into improving the thermal capacity and efficiency of a PCM-integrated HPWH. The state-of-the-art review is categorized by the stage of development of the PCM for deployment in HPWHs. To select appropriate PCMs for HPWHs, a six-factor down-selection process is used to determine the best material(s) for integration in HPWHs with appropriate heat exchanger design. Ultimately, food-grade PCMs appear to be the best candidate for integration of TES in domestic HPWHs because they are nontoxic, highly cyclable, and have heat transfer properties accommodable to water heating when integrated in a manner to overcome the thermal conductivity limitations of the material. A key parameter of water heating performance is thermal heating power of the PCM component, which is not often reported. Many studies report significant improvement in capacity and efficiency. Furthermore, many performance metrics are identified from the literature to quantify the system performance, but agreement across studies is not found. Unified energy factor and first hour rating performance tests are required for commercially available HPWHs, and these performance metrics could homogenize the literature. Ultimately, we find that select PCMs are ready for deployment with HPWHs for performance improvement, and component design and control optimizations are new avenues of research and development required for a commercially viable PCM-integrated HPWH system.

25 ENERGY STORAGE↗

Utilization of an agility assessment module in analysis and optimization of preliminary fighter configuration

A study has been conducted to develop and to analyze a FORTRAN computer code for performing agility analysis on fighter aircraft configurations. This program is one of the modules of the NASA Ames ACSYNT (AirCraft SYNThesis) design code. The background of the agility research in the aircraft industry and a survey of a few agility metrics are discussed. The methodology, techniques, and models developed for the code are presented. The validity of the existing code was evaluated by comparing with existing flight test data. A FORTRAN program was developed for a specific metric, PM (Pointing Margin), as part of the agility module. Example trade studies using the agility module along with ACSYNT were conducted using a McDonnell Douglas F/A-18 Hornet aircraft model. Tile sensitivity of thrust loading, wing loading, and thrust vectoring on agility criteria were investigated. The module can compare the agility potential between different configurations and has capability to optimize agility performance in the preliminary design process. This research provides a new and useful design tool for analyzing fighter performance during air combat engagements in the preliminary design.

Ngan, Angelen↗

Software Verification and Validation Plan Activities, 2011, Project Number: N6423, SAPHIRE Version 8

The SV&V Plan experienced changes over the past year to bring it into the operational software life cycle of SAPHIRE 8 and to maintain its sections on design features. Peer review of the SVVP with the former IV&V members identified the need for the operational use of metrics as a tool for quality maintenance and improvement. New tests were added to the SVVP to verify the operation of the new design features incorporated into SAPHIRE 8. Other additions to the SVVP were the addition of software metrics and the PDR and CDR processes. Audit support was provided for the NRC Technical Manager and Project Manager for the NRC OIG Audit performed throughout 2011. The SVVP is considered to be an up to date reference and useful roadmap of verification and validation activities going forward.

42 ENGINEERING↗

Design and Initial Verification Testing of an In Situ Load Retention Aging Fixture

Load retention is a key metric used to evaluate long-term mechanical functionality of polymeric materials. Its measurement currently requires periodically removing the aging fixtures from the ovens, removing the specimens from the fixtures, and performing load testing on the specimens. The ability to perform load measurement in situ (i.e., without having to remove the specimens from the fixtures, and possibly even the ovens) would facilitate data acquisition/analysis, improve data quality, and enable uninterrupted compression of specimens which is more representative of the actual application.

36 MATERIALS SCIENCE↗

Applications of Similarity Analysis of Reactivity-initiated Accident Experiments in TREAT

Experimental testing capabilities have been created to perform reactivity-initiated accident experiments in the Transient Reactor Test Facility for fuel safety testing of Accident Tolerant Fuel concepts and to extend the burnup limits for our current light water reactor fuel designs. Completely prototypic test conditions are not possible and compromises naturally have to be made. To date, the representativity of the small scale safety testing to a similar event in a commercial reactor is made by comparing a few specific phenomena of interest. The goal of this work is to apply similarity analysis as another metric to judge the representativity of in-pile safety tests to full-scaled accident scenario.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

SU-G-JeP1-05: Clinical Impact of MLC Tracking for Lung SABR

Purpose: The objective of this study was to investigate the dosimetric benefits of multi-leaf collimator (MLC) tracking for lung SABR treatments in end-to-end clinically realistic planning and delivery scenarios. Methods: The clinical benefits of MLC tracking were assessed using previously delivered treatment plans and physical experiments. The 10 most recent single lesion lung SABR patients were re-planned following a 4D-GTV-based real-time adaptive protocol (PTV defined as the end-of-exhalation GTV plus 5.0 mm margins). The plans were delivered on a Trilogy Varian linac. Electromagnetic transponders (Calypso, Varian Medical Systems, USA) were embedded into a programmable moving phantom (HexaMotion platform) tracked with the Varian Calypso system. For each physical experiment, the MLC positions were collected and used as input for dose reconstruction. For both planned and physical experiments, the OAR dose metrics from the conventional and real-time adaptive SABR plans (Mean Lung Dose (MLD), V20 for lung, and near-maximum dose (D2%) for spine and heart) were statistically compared. The Wilcoxon test was used to compare plan and physical experiment dose metrics. Results: While maintaining target coverage, percentage reductions in dose metrics to the OARs were observed for both planned and physical experiments. Comparing the two plans showed MLD percentage reduction (MLDr) of 25.4% (absolute differences of 1.41 Gy) and 28.9% (1.29%) for the V20r. D2% percentage reduction for spine and heart were respectively 27.9% (0.3 Gy) and 20.2% (0.3 Gy). For the physical experiments, MLDr was 23.9% (1.3 Gy), and V20r 37.4% (1.6%). D2% reduction for spine and heart were respectively 27.3% (0.3 Gy) and 19.6% (0.3 Gy). For both plans and physical experiments, significant OAR dose differences (p<0.05) were found between the conventional SABR and real-time adaptive plans. Conclusion: Application of MLC tracking for lung SABR patients has the potential to reduce the dose to OARs during radiation therapy.

60 APPLIED LIFE SCIENCES↗

Geometry-Based Observability Metric

The Satellite Servicing Capabilities Office (SSCO) is currently developing and testing Goddard s Natural Feature Image Recognition (GNFIR) software for autonomous rendezvous and docking missions. GNFIR has flight heritage and is still being developed and tailored for future missions with non-cooperative targets: (1) DEXTRE Pointing Package System on the International Space Station, (2) Relative Navigation System (RNS) on the Space Shuttle for the fourth Hubble Servicing Mission.

Eaton, Colin↗

International Benchmarking: A Global Perspective of Workplace Drug and Alcohol Testing

Workplace drug and alcohol testing has been widely implemented in diverse industries and is designed to detect the use of and deter the influence of legal and illegal substances that would impair the ability to perform work that has significant security and safety implications. Across the globe, patterns of drug and alcohol use and abuse and the regulations controlling drug and alcohol use vary by location (e.g., regions, countries, etc.). The prevalence of workplace drug and alcohol testing and protection of individuals’ rights in the workplace similarly tend to exhibit contextual differences. With the advancements in drug-testing technologies and the recent issuance of guidance for additional testing matrices (e.g., oral fluid, hair, etc.) by the U.S. Department of Health and Human Services, gaining a global perspective on the current state of drug and alcohol testing practices and regulations is important for leveraging such advancements to improve the effectiveness of drug and alcohol testing. The U.S. Nuclear Regulatory Commission (NRC) tasked the Pacific Northwest National Laboratory (PNNL) fitness for duty (FFD) team to conduct a benchmarking analysis of international drug and alcohol testing regulations and programs to identify similarities and differences in workplace testing programs and the basis for any differences. The benchmarking study focuses on a policy analysis and benchmarking metrics comparison among different countries and regions. The policy analysis includes the following three types of relevant international and national policies relevant to drug and alcohol testing in the nuclear industry: 1. National drug and alcohol policy 2. Workplace drug and alcohol testing 3. FFD regulations for Nuclear Power Plants The international drug and alcohol benchmarking metrics focusses on the use of blood, breath, urine, hair, and oral fluid in testing; drug and drug metabolite cutoff values; point-of-collection testing devices; roadside testing methodologies; innovations in drug testing; and the extent of availability of such data. Gaining an understanding of workplace drug and alcohol testing in other countries as well as information about new testing methods, matrices, tools, and technologies that other countries have adopted provides NRC with international comparative information regarding the technical basis for such programs and methods as it relates to the protection of worker and public health and safety.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗