Search NASA⌕ Search

SEARCH · Search NASA

Results for “Testing metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 559 records · Page 31

SU-C-202-02: A Comprehensive Evaluation of Adaptive Daily Planning for Cervical Cancer HDR Brachytherapy

Purpose: To evaluate adaptive daily planning for cervical cancer patients who underwent high-dose-rate intra-cavitary brachytherapy (HDR-ICBT). Methods: This study included 22 cervical cancer patients who underwent 5 fractions of HDR ICBT. Regions of interest (ROIs) including high-risk clinical tumor volume (HR-CTV) and organs-at-risk (OARs) were manually contoured on daily CT images. All patients were treated with adaptive daily plans, which involved ROI delineation and dose optimization at each treatment fraction. Single treatment plans were retrospectively generated by applying the first treatment fraction’s dwell times adjusted for decay and dwell positions of the applicator to subsequent treatment fractions. Various existing similarity metrics were calculated for the ROIs to quantify interfractional organ variations. A novel similarity score (JRARM) was established, which combined both volumetric overlap metrics (DSC, JSC, and RVD) and distance metrics (ASD, MSD, and RMSD). Linear regression was performed to determine a relationship between inter-fractional organ variations of various similarity metrics and D2cc variations from both plans. Wilcoxon Signed Rank Tests were used to assess adaptive daily plans and single plans by comparing EQD2 D2cc (α/β=3) for OARs. Results: For inter-fractional organ variations, the sigmoid demonstrated the greatest variations based on the JRARM and DSC similarity metrics. Comparisons between paired ROIs showed differences in JRARM scores and DSCs at each treatment fraction. RVD, MSD, and RMSD were found to be significantly correlated to D2cc variations for bladder and sigmoid. The comparison between plans found that adaptive daily planning provided lower EQD2 D2cc of OARs than single planning, specifically for the sigmoid (p=0.015). Conclusion: Substantial inter-fractional organ motion can occur during HDR-BT, which may significantly affect D2cc of OARs. Adaptive daily planning provides improved dose sparing for OARs compared to single planning.

60 APPLIED LIFE SCIENCES↗

2011 SAPHIRE 8 Software Quality Assurance Status Report

The Software Quality Assurance engineer position was created in fiscal year 2011 to better maintain and improve the quality of the SAPHIRE 8 development program. This year's Software Quality Assurance tasks concentrated on developing the framework of the SQA program. This report reviews the accomplishments and recommendations for each of the subtasks set forth for JCN V6059: (1) Reviews, Tests, and Code Walkthroughs; (2) Data Dictionary; (3) Metrics; (4) Requirements Traceability Matrix; (5) Provide Oversight on SAPHIRE QA Activities; and (6) Support NRC Presentations and Meetings.

42 ENGINEERING↗

Physical Security Model Development of an Electrochemical Facility.

Nuclear facilities in the U.S. and around the world face increasing challenges in meeting evolving physical security requirements while keeping costs reasonable. The addition of security features after a facility has been designed and without attention to optimization (the approach of the past) can easily lead to cost overruns. Instead, security should be considered at the beginning of the design process in order to provide robust, yet efficient physical security designs. The purpose of this work is to demonstrate how modeling and simulation can be used to optimize the design of physical protection systems. A suite of tools, including Scribe3D and Blender, were used to model up a generic electrochemical reprocessing facility. Physical protection elements such as sensors, portal monitors, barriers, and guard forces were added to the model based on best practices for physical security. One outsider theft scenario was examined with 4-8 adversaries to determine security metrics. This work fits into a larger Virtual Test Bed 2020 Milestone in the Material Protection, Accounting, and Control Technologies (MPACT) program through the Department of Energy (DOE). The purpose of the milestone is to demonstrate how a series of experimental and modeling capabilities across the DOE complex provide the capabilities to demonstrate complete Safeguards and Security by Design (SSBD) for nuclear facilities. ACKNOWLEDGEMENTS This work was funded by the Materials Protection, Accounting, and Control Technologies (MPACT) working group as part of the Nuclear Technology Research and Development Program under the U.S. Department of Energy, Office of Nuclear Energy.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗

Manufacturing Supply Chain Development for Modular Solar-Thermochemical Conversion Platform - CRADA 387 (Final Report)

Modular chemical process intensification (MCPI) is an emerging field where chemical processing is performed using small-scale modular equipment instead of conventional large centralized chemical plants. Conventional chemical plants benefit from economies of scale that encourage scale-up to ever larger plants. A goal of MCPI is to develop technology that intensifies processing so that equipment can be dramatically smaller and integrated into modular systems. Scale-up occurs by adding more modules in parallel rather than making the equipment larger. A key concept is that equipment and modules can ultimately be cheaper by leveraging economies of mass production, analogous to the automotive industry, in manufacturing the equipment. This project made significant progress toward this outcome by meeting the RAPID institute metric to reduce equipment cost by 20% for each doubling in manufacturing volume. The MCPI application was thermochemical technology that is being commercialized by STARS Technology Corporation, one of the CRADA partners. The technology converts solar and renewable power to chemical energy to produce renewable hydrogen, fuels, and chemicals. The benefit to the public is reduction in greenhouse gases that are contributing to climate change. The project transitioned the steam methane reforming (SMR) reactor from conventional fabrication methods to additive manufacturing (AM) direct metal laser sintering (DMLS) process. This is projected to reduce the cost of making a reactor by 58% when producing 100 reactors per year. Innovations in the DMLS process produced a patented design that reduces reactor weight by 60%. Reductions in material costs and processing time extend the DMLS advantage to higher production volumes. The new design promises to be 38% cheaper than the conventional processes at 1000 units per year. The resulting 87% reduction in the steam methane reforming (SMR) module cost in scaling from current costs meets the RAPID metric. The project was successful in producing and testing the first ever additively manufactured SMR reactors. A reactor achieved over 82% efficiency in converting electric power to chemical energy, which is a world record for an inductively heated SMR. The project has contributed to the design and assembly of a first demonstration plant that is headed to a hydrogen bus filling station in Thousand Palms, CA.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Digital Tools for the Preventive Conservation of Built Heritage: The Church of Santa Ana in Seville

Historic Building Information Modelling (HBIM) plays a pivotal role in heritage conservation endeavours, offering a robust framework for digitally documenting existing structures and supporting conservation practices. However, HBIM’s efficacy hinges upon the implementation of case-specific approaches to address the requirements and resources of each individual asset and context. This paper defines a flexible and generalisable workflow that encompasses various aspects (i.e., documentation, surveying, vulnerability assessment) to support risk-informed decision making in heritage management tailored to the peculiar conservation needs of the structure. This methodology includes an initial investigation covering historical data collection, metric and condition surveys and non-destructive testing. The second stage includes Finite Element Method (FEM) modelling and structural analysis. All data generated and processed are managed in a multi-purpose HBIM model. The methodology is tested on a relevant case study, namely, the church of Santa Ana in Seville, chosen for its historical significance, intricacy and susceptibility to seismic action. The defined level of detail of the HBIM model is sufficient to inform the structural analysis, being balanced by a more accurate representation of the alterations, through linked orthophotos and a comprehensive list of alphanumerical parameters. This ensures an adequate level of information, optimising the trade-off between model complexity, investigation time requirements, computational burden and reliability in the decision-making process. Field testing and FEM analysis provide valuable insight into the main sources of vulnerability in the building, including the connection between the tower and nave and the slenderness of the columns.

Chaves, Estefanía↗

Comparison of RCCI Operation with and without EGR over the Full Operating Map ofa Heavy-Duty Diesel Engine

Dual-fuel combustion using port-injection of low reactivity fuel combined with direct injection of a higher reactivity fuel, otherwise known as Reactivity Controlled Compression Ignition (RCCI), has been shown as a method to achieve high efficiency combustion with moderate peak pressure rise rates, low engine-out soot and NOx emissions. A key requirement for extending to high-load operation is reduce the reactivity of the premixed charge prior to the diesel injection. One way to accomplish this is to use a very low reactivity fuel such as natural gas. In this work, experimental testing was conducted on a 13L multi-cylinder heavy-duty diesel engine modified to operate using RCCI combustion with port injection of natural gas and direct injection of diesel fuel. Natural gas/diesel RCCI engine operation is compared over the EPA Heavy-Duty 13 mode supplemental emissions test with and without EGR. Emissions and efficiency metrics were examined over the entire engine map for both operating modes. It was found that the use of EGR lowered combustion noise to less than 97 dBa and lowered the cycle averaged NOx emissions by 48%, with only a slight increase in soot and 0.5 point decrease in brake thermal efficiency. Thus, operation with EGR offered the lowest total fluid consumption when considering the use of a selective catalytic reduction system for NOx aftertreatment.

Hanson, Reed↗

Sensitivity Analysis of Drivers Water Shortage in the Los Angeles Region During Drought

The code and detailed step-by-step instructions for generating the model output data, processing results, and analysis and plotting are provided at https://github.com/IMMM-SFA/Ferencz_et_al_2026_ER_Water. The PyArtes model is a python adaptation of the Artes model. PyArtes uses many of the same input data and optimization model architecture as Artes. Documentation for the PyArtes model is provided in the Supplement to the paper. The primary data product are simulated monthly water shortages for indoor and outdoor demand under a large ensemble of drought scenarios (>13,000). The droughts are hypothetical and are not based on historical time series data of supply sources - though historical data did help inform ranges explored for supply parameters. Demands are informed by recent 2017-2021 water supply data. Demands used for the model can be accessed at https://github.com/IMMM-SFA/Ferencz_et_al_2026_ER_Water. Simulations resolve demand for over 90 water providers in the study region. The results report 36 months of water shortage data for each indoor and outdoor demand node. The study also developed a multilayer perceptron (MLP) neural network trained on a subset of the simulated shortage ensemble to emulate worst annual water shortage for a given set of parameter multipliers -- provided the parameter values fall within the ranges sampled in the ensemble. Emulated water shortages for synthetic ensembles are in the MLP-generated shortages folder. The MLP model was used to generate larger ensembles to support Sobol analysis that would have been extremely computationally expensive to simulate. Datasets provided in this repository*: Simulated shortages. These results are used for the analysis for Figures 5, 8, and 9 in the paper, and also to train the MLP emulator. .zip file containing outputs for the 13,312 scenario ensemble. Separate .csv files for indoor and outdoor shortage for each scenario. Rows = demand ids (~100), Columns = months (36) Units = acre-feet/month of shortage (shortage = monthly demand - supply). 1 acft = 1233.48 m^3 .csv files of aggregated shortages derived from the 13,312 ensemble Rows = scenarios (13,312), Columns = demand ids (~100) Units = acre-feet/year (either worst annual shortage or total shortage over the 3-year drought) .csv file of the parameter multipliers scenarios for the ensemble .csv file of the parameter ranges and baseline values the multipliers were applied to MLP-generated shortages. These results are used for Figures 4, 6, and 7 in the paper. mwd higher folder: scenario ensembles, emulated worst year total shortages (acft), and Sobol results Emulated shortages. Rows = scenarios, columns = demand ids, units acft Sobol results. Rows = demand ids, columns Sobol (S1, ST, or 95% confidence interval) value for each parameter mwd lower folder: scenario ensembles, emulated worst year total shortages (acft), and Sobol results same organization as mwd higher MLP performance: performance metrics (R^2, RMSE, BIAS, MAPE) for the testing subset (20% or 2,662 scenarios) and simulated vs emulated worst year shortage (acre-feet/year) for every demand node, MWD wholesale regions, and the entire study region (LAC). Supporting data for figures. Figure plotting scripts in the associated GitHub repo. These files support analysis and visualization. Geospatial Data used for plotting simulated water shortages and Sobol results. Dictionary of full names for demand nodes in the model and estimates of water supply by source type informed by Artes input files and California Urban Water Management Planning data: https://water.ca.gov/Programs/Water-Use-And-Efficiency/Urban-Water-Use-Efficiency/Urban-Water-Management-Plans *Readme files provided for each folder.

drought↗

Quality metrics for sensor images

Methods are needed for evaluating the quality of augmented visual displays (AVID). Computational quality metrics will help summarize, interpolate, and extrapolate the results of human performance tests with displays. The FLM Vision group at NASA Ames has been developing computational models of visual processing and using them to develop computational metrics for similar problems. For example, display modeling systems use metrics for comparing proposed displays, halftoning optimizing methods use metrics to evaluate the difference between the halftone and the original, and image compression methods minimize the predicted visibility of compression artifacts. The visual discrimination models take as input two arbitrary images A and B and compute an estimate of the probability that a human observer will report that A is different from B. If A is an image that one desires to display and B is the actual displayed image, such an estimate can be regarded as an image quality metric reflecting how well B approximates A. There are additional complexities associated with the problem of evaluating the quality of radar and IR enhanced displays for AVID tasks. One important problem is the question of whether intruding obstacles are detectable in such displays. Although the discrimination model can handle detection situations by making B the original image A plus the intrusion, this detection model makes the inappropriate assumption that the observer knows where the intrusion will be. Effects of signal uncertainty need to be added to our models. A pilot needs to make decisions rapidly. The models need to predict not just the probability of a correct decision, but the probability of a correct decision by the time the decision needs to be made. That is, the models need to predict latency as well as accuracy. Luce and Green have generated models for auditory detection latencies. Similar models are needed for visual detection. Most image quality models are designed for static imagery. Watson has been developing a general spatial-temporal vision model to optimize video compression techniques. These models need to be adapted and calibrated for AVID applications.

Ahumada, AL↗

Recent Advances to Estimation of Fixed-Interface Modal Models Using Dynamic Substructuring

In 2010, Allen & Mayes proposed to estimate the fixed-interface modes of a structure by measuring the modes of the structure bolted to a fixture and then applying constraints to the fixture using the transmission simulator method. While the method proved useful, and has indeed been used in studies since that point, a few peculiarities were noted. First, in some cases the estimated fixed-base natural frequencies were observed to converge very slowly to the true values (in simulated experiments) as the number of constraints was increased. To formulate these constraints, prior studies used only the free-interface modes of the fixture or the measured modes of the assembly. This work extends that to consider other sets of constraints, showing improved results. Furthermore, in some prior studies it has been observed that there were errors of more than 10% in the natural frequencies even when the fixture motion was hundreds of times smaller than the motion of the structure of interest (and so it had presumably been removed). This work explores this phenomenon, seeking to use the strain energy in the fixture, to the extent that it can be estimated using a test-analysis model for the fixture, as a metric to predict frequency error. The proposed methods are explored by applying them to simulated measurements from a beam and from the NASA Space Launch System coupled to the Mobile Launcher.

Allen, Mathew S.↗

Microgreens Food Safety Evaluation

Microgreens have been recently identified as a new type of pick and eat salad crop that can be utilized in space crop production systems. The majority of traditionally grown leafy green crops can be grown as microgreens, in addition to crops such as legumes (e.g. pea shoots), sunflower, buckwheat, most herbs, and corn, presenting hundreds of microgreen crop options. With a wide variety of flavors, exceptional nutritional density, short growth cycles (7-14 days), and unmatched volume optimization potential, microgreens present as an interesting option for sustainable production of nutritious and flavorful crops in space. The food safety aspects of microgreens have been investigated by USDA in recent years, however, an assessment for crew consumption purposes has not yet been conducted to capture the variety of microgreen cultivar types grown in spaceflight relevant environmental conditions and hardware that could be a part of the astronaut diet. Seed sanitation methods will be developed for each microgreen cultivar to sanitize the seed coat and ensure viable germination and plant growth. Food safety analysis will be performed on 24 different fast-growing cultivars of microgreens by performing aerobic plate counts (APC), fungal counts, and select pathogen screening of edible plant tissue before and after treatment with ProSan sanitizing produce wash. Additional avenues of investigation into microgreens food safety will be undertaken to understand impacts of plant height at time of harvest (able to be manipulated by light spectrum), seed density at time of planting, and role of blue light in food safety metrics of microgreens (n=6-12 microgreen cultivars tested, depending on variable). These findings will be included in the Space Crop Production Hazard Analysis Critical Control Point (HACCP) plan. This new work is currently postponed due to Covid-19.

M E Hummerick↗

Assessing the Effectiveness of Environmental Test Programs

NASA's Code QE Test Effectiveness Program is funding a series of applied research activities focused on utilizing the principles of physics and engineering of failure along with those of engineering economics to assess and improve the value added by various validation and verification activities, with the primary emphasis being on testing.

test↗

Performance Risk Model Validation with Operationally Relevant Tasks

Human Research Program aims to develop methods to support astronauts’ health and productivity during spaceflight. The Crew Health and Performance Probabilistic Risk Assessment (CHP-PRA) team uses powerful computational methods to predict mission risk in both domains: medical and performance. Here, we show how CHP-PRA uses the Performance Risk Model (PRisM) to quantify the performance risk and show an application of the model on operationally relevant tasks. There are various metrics adopted across performance researchers that PRisM can accommodate. For data analysis, interpretation, and integration, we use a method of unifying data from multiple sources by converting each to a single metric. We consult subject matter experts prior to integrating the converted data into PRisM. The method we use is inspired by the Cooper-Harper rating scale [1]. Using this unified metric, we can easily combine data from various tests and lab groups. We explain our conversion method in detail and show how it pertains to the process of testing and validation of PRisM on operational tasks. We conducted an initial validation in collaboration with the Behavioral Health and Performance (BHP) lab. We test PRisM using data on their operationally relevant task ROBoT-r, a track-and-capture task for grappling incoming resupply vehicles [2]. Several other labs at NASA Johnson Space Center worked together to design 7 Functional Task Tests (FTTs) in pursuit of simulating the tasks required after landing on a planetary surface and after return to Earth [3]. Here we use the results from both ROBoT-r and the 7 FTTs and compare their experiment data to PRisM’s computational output to demonstrate how PRisM can support operations by predicting crew performance on future missions.

performance modeling↗

Accelerated Aging of Fast Pyrolysis Bio-Oils

Carbonyl content is a better metric for tracking aging than viscosity. Existing aging test (24 hours @ 80 °C) is too severe, and correlates to more room temperature aging than would realistically occur. New aging test proposed: 2 hours @ 80 °C correlates to 1-3 months of room temperature storage. Aging is slowed in cold storage, but still occurs.

aging↗

Integration of Dynamical System Scaling to RAVEN and Facility Application

As part of the design optimization and model validation effort within the hybrid energy systems program, the research to implement dynamical system scaling (DSS) code to RAVEN is funded under the Integrated Energy System program in collaboration with the Digital Reactor Technology & Development department. The DSS data processing algorithm has been coded within the RAVEN framework along with other metrics and postprocessing models. The implemented code was tested using a gravity-driven draining tank draining case modeled in RELAP5-3D and generated data was successfully postprocessed based on the scaling analyses defined by DSS. Postprocessed data concluded less agreement for the tank exit velocity and indicated that the modeled input deck for RELAP5-3D requires modification to enforce minimal pressure differential effects to the draining process.

97 MATHEMATICS AND COMPUTING↗

Launch Vehicle Production and Operations Cost Metrics

Traditionally, launch vehicle cost has been evaluated based on $/Kg to orbit. This metric is calculated based on assumptions not typically met by a specific mission. These assumptions include the specified orbit whether Low Earth Orbit (LEO), Geostationary Earth Orbit (GEO), or both. The metric also assumes the payload utilizes the full lift mass of the launch vehicle, which is rarely true even with secondary payloads.1,2,3 Other approaches for cost metrics have been evaluated including unit cost of the launch vehicle and an approach to consider the full program production and operations costs.4 Unit cost considers the variable cost of the vehicle and the definition of variable costs are discussed. The full program production and operation costs include both the variable costs and the manufacturing base. This metric also distinguishes operations costs from production costs, including pre-flight operational testing. Operations costs also consider the costs of flight operations, including control center operation and maintenance. Each of these 3 cost metrics show different sensitivities to various aspects of launch vehicle cost drivers. The comparison of these metrics provides the strengths and weaknesses of each yielding an assessment useful for cost metric selection for launch vehicle programs.

Watson, Michael D.↗

CPES-QSM: A Quantitative Method Towards the Secure Operation of Cyber-Physical Energy Systems

Power systems are evolving into cyber-physical energy systems (CPES) mainly due to the integration of modern communication and Internet-of-Things (IoT) devices. CPES security evaluation is challenging since the physical and cyber layers are often not considered holistically. Existing literature focuses on only optimizing the operation of either the physical or cyber layer while ignoring the interactions between them. This paper proposes a metric, the Cyber-Physical Energy System Quantitative Security Metric (CPES-QSM), that quantifies the interaction between the cyber and physical layers across three domains: electrical, cyber-risk, and network topology. A method for incorporating the proposed cyber-metric into operational decisions is also proposed by formulating a cyber-constrained AC optimal power flow (C-ACOPF) that considers the status of all the CPES layers. The C-ACOPF considers the vulnerabilities of physical and cyber networks by incorporating factors such as voltage stability, contingencies, graph-theory, and IoT cyber risks, while using a multi-criteria decision-making technique. We note that simulation studies are conducted using standard IEEE test systems to evaluate the effectiveness of the proposed metric and the C-ACOPF formulation.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Metric Learning for Hyperspectral Image Segmentation

We present a metric learning approach to improve the performance of unsupervised hyperspectral image segmentation. Unsupervised spatial segmentation can assist both user visualization and automatic recognition of surface features. Analysts can use spatially-continuous segments to decrease noise levels and/or localize feature boundaries. However, existing segmentation methods use tasks-agnostic measures of similarity. Here we learn task-specific similarity measures from training data, improving segment fidelity to classes of interest. Multiclass Linear Discriminate Analysis produces a linear transform that optimally separates a labeled set of training classes. The defines a distance metric that generalized to a new scenes, enabling graph-based segmentation that emphasizes key spectral features. We describe tests based on data from the Compact Reconnaissance Imaging Spectrometer (CRISM) in which learned metrics improve segment homogeneity with respect to mineralogical classes.

Compact Reconnaissance Imaging Spectrometer (CRISM↗

The evaluation of several agility metrics for fighter aircraft using optimal trajectory analysis

Several functional agility metrics, including the combat cycle time metric, dynamic speed turn plots, and relative energy state metric, are used to compare turning performance for generic F-18, X-29, and X-31-type aircraft models. These three-degree-of-freedom models have characteristics similar to the real aircraft. The performance comparisons are made using data from optimal test trajectories to reduce sensitivities to different pilot input techniques and to reduce the effects of control system limiters. The turn performance for all three aircraft is calculated for simulated minimum time 180 deg heading captures from simulation data. Comparisons of the three aircraft give more insight into turn performance than would be available from traditional measures of performance. Using the optimal test technique yields significant performance improvements as measured by the metrics. These performance improvements were found without significant increases in turn radius.

Ryan, George W., III↗