Search NASA⌕ Search

SEARCH · Search NASA

Results for “Testing metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 271 records · Page 15

2011 Ground Testing Highlights Article

Two tests supporting development of the launch abort system for the Orion MultiPurpose Crew Vehicle were run in the NASA Ames Unitary Plan wind tunnel last year. The first test used a fully metric model to examine the stability and controllability of the Launch Abort Vehicle during potential abort scenarios for Mach numbers ranging from 0.3 to 2.5. The aerodynamic effects of the Abort Motor and Attitude Control Motor plumes were simulated using high-pressure air flowing through independent paths. The aerodynamic effects of the proximity to the launch vehicle during the early moments of an abort were simulated with a remotely actuated Service Module that allowed the position relative to the Crew Module to be varied appropriately. The second test simulated the acoustic environment around the Launch Abort Vehicle caused by the plumes from the 400,000-pound thrust, solid-fueled Abort Motor. To obtain the proper acoustic characteristics of the hot rocket plumes for the flight vehicle, heated Helium was used. A custom Helium supply system was developed for the test consisting of 2 jumbo high-pressure Helium trailers, a twelve-tube accumulator, and a 13MW gas-fired heater borrowed from the Propulsion Simulation Laboratory at NASA Glenn Research Center. The test provided fluctuating surface pressure measurements at over 200 points on the vehicle surface that have now been used to define the ground-testing requirements for the Orion Launch Abort Vehicle.

Ross, James C.↗

Efficient and Reliable Power Takeoff for Ocean Wave Energy Harvesting

The project goal is to significantly improve the current ocean wave energy harvesting through innovative Power Take-off (PTO) design, advanced power electronics, and novel wave capture structures. The objective of the project is to design and demonstrate system-agnostic components for application across multiple MHK systems, and complete component designs, build scaled prototypes, and perform testing and analysis for metric validation of 25% increase in component rating/per unit cost and 50% reduction in failure rate. The major innovation of the PTO is the Mechanical Motion Rectifier (MMR) mechanism that rectifies the bi-directional oscillatory motion of the input from waves into a steady unidirectional rotation output to directly drive the electrical generator. Through this mechanism, the efficiency and the fatigue life of the PTO can be significantly improved to benefit the energy absorption and lifespan of the wave energy converters (WEC). During the period of performance, the component and system design are completed, the scaled prototypes are developed and performed testing. It is validated that the 25% increase in a component rating/per unit cost. The 50% reduction in failure rate is not directly validated by experiments, however, it can be explained qualitatively with analysis. Besides, 8 journal articles, 13 conference proceedings, 1 patent, 3 Master thesis and two Ph.D. dissertations are published based on the work related to this project. The list of all the publications can be found at the end of the project as an appendix. Over 30 students and postdocs were trained through this project. Three prototypes of 100W and 500W WECs and 10KW PTO were designed, built, and tested in ocean wave tank and using the NREL dynamometer. This project demonstrated 50-80% PTO efficiency, 90-98% power electronics efficiency, up to 66% capture width ratio in irregular waves, and 34% overall efficiency in regular waves.

16 TIDAL AND WAVE POWER↗

Lunar Proving Grounds Definition

The Lunar Surface Innovation Consortium (LSIC) is hosting a hybrid Lunar Proving Grounds Definition Workshop, July 12-13,2023, at the Johns Hopkins University Applied Physics Laboratory in Laurel, Maryland, and on Zoom. The topic of facilities needed for testing hardware destined for the Moon and the need for Earth-based ‘Lunar Proving Grounds’ for testing systems has come up across all six Focus Areas of LSIC. While facilities exist for component- and instrument- level technology maturation (e.g., up to system/subsystem demo in relevant environments), and there are potential flight opportunities for component maturation to flight-qualified and even flight-proved systems, the Artemis Program vision for a sustained presence and transition to industry (e.g., the Moon to Mars Objectives and the LSIC “Path to an Enduring Lunar Presence” white paper) suggests an architecture of integrated systems and systems of systems more complex than Apollo or the International Space Station. Some questions we aim to address through this workshop include: (1) How will validation and verification of these systems and interactions, including human-robotic operations, be accomplished? (2) What metrics need to be tested, and thus what capabilities will such a facility or facilities need? (3) Which functionalities can be tested separately and which need to synergize? What can be the role of digital engineering?

Proving Grounds↗

Comparison of Cognitive Performance Tests for Promethazine Pharmacodynamics in Human Subjects

The objective of this study is to compare cognitive function tests, Automated Neurological Assessment Metrics (ANAM) based Readiness Evaluation System (ARES(Registered TradeMark)) on a Palm Pilot and Windows based Spaceflight Cognitive Assessment Tool (WinSCAT(Registered TradeMark)) on a personal computer (PC) to assess performance effects of promethazine (PMZ) after administration to human subjects. In a randomized placebo-controlled cross-over design, subjects received 12.5, 25, and 50 mg intramuscular (IM) PMZ or a placebo and completed 14 sessions with WinSCAT(Registered TradeMark) (v. 1.26) and ARES(Registered TradeMark) (v. 1.27) consecutively for 72 h post dose. Maximum plasma concentrations (4.25, 6.25 and 13.33 ng/ml) were linear with dose and were achieved by 0.75, 8, and 24 h after dosing for the three doses, respectively. No significant differences in cognitive function after PMZ dosing were detected using WinSCAT(Registered TradeMark), however, tests from ARES(Registered TradeMark) demonstrated concentration dependent decrements in reaction time associated with PMZ dose.

Vaksman, Z.↗

Test Points for Online Monitoring of Quantum Circuits

Noisy Intermediate-Scale Quantum (NISQ) computers consisting of tens of inherently noisy quantum bits (qubits) suffer from reliability problems. Qubits and their gates are susceptible to various types of errors. Due to limited numbers of qubits and high error rates, quantum error correction cannot be applied. Physical constraints of quantum hardware including the error rates are used to guide the design and the layout of quantum circuits. The error rates determine the selection of qubits and their operations. The resulting circuit is executed on the quantum computer. This study explores the risk of unexpected changes in the error rates of NISQ computers post-calibration. We show that unexpected changes in error rates can alter the output state of a quantum circuit. To detect these changes, we propose the insertion of test points into the quantum circuit to enable online monitoring of the physical qubit behavior. We utilize classical, superposition, and uncompute test points. Furthermore, we use a gate error coverage metric to assess the quality of the tests. We verify the effectiveness of the proposed scheme on different IBM quantum computers (IBM Q), in addition to a noisy simulation that shows the scalability of the proposed approach.

97 MATHEMATICS AND COMPUTING↗

Software metrics: The key to quality software on the NCC project

Network Control Center (NCC) Project metrics are captured during the implementation and testing phases of the NCCDS software development lifecycle. The metrics data collection and reporting function has interfaces with all elements of the NCC project. Close collaboration with all project elements has resulted in the development of a defined and repeatable set of metrics processes. The resulting data are used to plan and monitor release activities on a weekly basis. The use of graphical outputs facilitates the interpretation of progress and status. The successful application of metrics throughout the NCC project has been instrumental in the delivery of quality software. The use of metrics on the NCC Project supports the needs of the technical and managerial staff. This paper describes the project, the functions supported by metrics, the data that are collected and reported, how the data are used, and the improvements in the quality of deliverable software since the metrics processes and products have been in use.

Burns, Patricia J.↗

Exxon Mobil-NETL Testing of DAC Materials

This joint initiative is aimed at extending our understanding of real DAC testing conditions at the NETL DAC testbed facility. Of particular focus will be: • Small Scale testing of powdered and formulated materials (supplied by ExxonMobil) to evaluate various performance metrics under DAC process cycles • Pilot scale testing of formulated materials at larger scales ExxonMobil will work with NETL to shake down equipment, validate testing methods, and define best practices for data analysis. Three tasks are proposed: 1. Validation of multi-cycle test data on powdered and formulated materials to benchmark various performance metrics of these materials. 2. Steam regeneration of powdered samples at small scale to define baseline performance under commercially relevant conditions. 3. Pilot scale testing of larger formulated materials to test commercially relevant samples under actual process cycles.

36 MATERIALS SCIENCE↗

Development of a Ground Test and Analysis Protocol for NASA's NextSTEP Phase 2 Habitation Concepts

The NASA Next Space Technologies for Exploration Partnerships (NextSTEP) program is a public-private partnership model that seeks commercial development of deep space exploration capabilities to support human spaceflight missions around and beyond cislunar space. NASA first issued the Phase 1 NextSTEP Broad Agency Announcement to U.S. industries in 2014, which called for innovative cislunar habitation concepts that leveraged commercialization plans for low-Earth orbit. These habitats will be part of the Deep Space Gateway (DSG), the cislunar space station planned by NASA for construction in the 2020s. In 2016, Phase 2 of the NextSTEP program selected five commercial partners to develop ground prototypes. A team of NASA research engineers and subject matter experts (SMEs) have been tasked with developing the ground-test protocol that will serve as the primary means by which these Phase 2 prototypes will be evaluated. Since 2008, this core test team has successfully conducted multiple spaceflight analog mission evaluations utilizing a consistent set of operational tools, methods, and metrics to enable the iterative development, testing, analysis, and validation of evolving exploration architectures, operations concepts, and vehicle designs. The purpose of implementing a similar evaluation process for the Phase 2 Habitation Concepts is to consistently evaluate different commercial partner ground prototypes to provide data-driven, actionable recommendations for Phase 3. This paper describes the process by which the ground test protocol was developed and the objectives, methods, and metrics by which the NextSTEP Phase 2 Habitation Concepts will be rigorously and systematically evaluated. The protocol has been developed using both a top-down and bottom-up approach. Top-down development began with the Human Exploration and Operations Mission Directorate (HEOMD) exploration objectives and ISS Exploration Capability Study Team (IECST) candidate flight objectives. Strategic questions and associated rationales, derived from these candidate architectural objectives, provide the framework by which the ground-test protocol will address the DSG stack elements and configurations, systems and subsystems, and habitation, science, and EVA functions. From these strategic questions, high-level functional requirements for the DSG were drafted and associated ground-test objectives and analysis protocols were established. Bottom-up development incorporated objectives from NASA SMEs in autonomy, avionics and software, communication, environmental control and life support systems, exercise, extravehicular activity, exploration medical operations, guidance navigation and control, human factors and behavioral performance, human factors and habitability, logistics, Mission Control Center operations, power, radiation, robotics, safety and mission assurance, science, simulation, structures, thermal, trash management, and vehicle health. Top-down and bottom-up objectives were integrated to form overall functional requirements - ground-test objectives and analysis mapping. From this mapping, ground-test objectives were organized into those that will be evaluated through inspection, demonstration, analysis, subsystem standalone testing, and human-in-the-loop (HITL) testing. For the HITL tests, mission-like timelines, procedures, and flight rules have been developed to directly meet ground test objectives and evaluate specific functional requirements. Data collected from these assessments will be analyzed to determine the acceptability of habitation element configurations and the combinations of capabilities that will result in the best habitation platform to be recommended by the test team for Phase 3.

Gernhardt, Michael L.↗

Towards inverse microstructure-centered materials design using generative phase-field modeling and deep variational autoencoders

The field of Integrated Computational Materials Engineering (ICME) combines a broad range of methods to study materials’ responses over a spectrum of length scales. A relatively unexplored aspect of microstructure-sensitive materials design is uncertainty propagation and quantification (UP/UQ) of materials’ microstructure, as well as establishing process-structure–property (PSP) relationships for inverse material design. In this study, an efficient UP technique built on the idea of changing probability measures and a deep generative unsupervised representative machine learning method for microstructure-based design of thermal conductivity of materials is proposed. Probability measures are used to represent microstructure space, and Wasserstein metrics are used to test the efficiency of the UP method. By using deep Variational AutoEncoder (VAE), we identify the correlations between the material/process parameters and the thermal conductivity of heterogeneous dual-phase microstructures. Through high-throughput screening, UP, and the deep-generative VAE method, PSP relationships that are too complex can be revealed by exploiting the materials’ design space with an emphasis on microstructures. As a last point, we demonstrate generative machine learning serves as a useful tool for inverse microstructure-centered materials design, and we demonstrate this by examining the inverse design of thermal conductivity in nano-structured materials. Here, the results reveal the effects of morphology, volume fraction, characteristic length scale, and the individual thermal diffusivity of phases on the thermal conductivity of dual-phase alloys. Our findings emphasize the advantages of high-throughput phase-field modeling and generative deep learning for linking PSP and inverse microstructure-centered materials design.

36 MATERIALS SCIENCE↗

Proof-of-concept chemometric approach for environmental forensic sourcing of crude oil samples using SPME-GC-MS

Environmental exposure to crude oil through seepage and spillage poses risks to the immediate environment and the broader ecosystem as areas along the oil distribution path are affected by the influx of crude petroleum as well as the environmental, economic, and civil unrest that accompanies it. There is a large financial burden associated with the lost resources, including the cost of rehabilitation, and the affected sources of revenue for communities affected by oil spills. As such, it is crucial to determine the responsible parties. This work outlines an environmental forensics approach to determining the source of an un-weathered crude oil sample. The researchers employed solid phase microextraction coupled with gas chromatography mass spectrometry (SPME-GC-MS) to capture and analyze the gaseous components emitted by crude oil samples sourced from five locations. Samples were analyzed using Spearman's rank correlation and 3D covariance analysis. Both chemometric approaches yielded optimal performance results with no misclassifications, true positive rate (TPR) = 100 % and false positive rate (FPR) = 0 %. The similarity metrics calculated by each test noted clear delineations between the values of same-source and differently sourced samples. The Spearman's rank correlation test and 3D covariance calculations both demonstrated the ability to correctly identify sample source origin in this dataset. Finally, the authors outline an approach to the future application of these tests and suggest their joint use in future crude oil sourcing endeavors.

3D covariance mapping↗

Testing Hidden Assumptions of Representativeness in Reach-Scale Studies of Hyporheic Exchange

Field studies of hyporheic exchange in mountain systems are often conducted using short study reaches and a limited number of observations. It is common practice to assume these study reaches represent hyporheic exchange at larger scales or different sites and to infer general relationships among potential causal mechanisms from the limited number of observations. However, these assumptions of representativeness are rarely tested. In this study, we develop numerical models from four segments of mountain streams in different geomorphologic settings and extract shorter reaches to test how representative exchange metrics are in shorter reaches compared to their reference segments. We also map the locations of the representative reaches to determine if a pattern exists based on location. Finally, we compare variance of these shorter within-site reaches to 29 additional reaches across the same basin to understand the impacts of inferring causal mechanisms, for example, the expectation that wide and narrow valley bottoms will yield different hyporheic exchange patterns. Our results show that the location and length strategy of the study reach must be considered before assuming an exchange metric to be representative of anything other than the exact segment studied. Further, it is necessary to quantify within and between site variations before making causal inferences based on observable characteristics, such as valley width or stream morphology. Our findings have implications for future field practices and how those practices are translated into models.

54 ENVIRONMENTAL SCIENCES↗

Measuring the Kerr spin parameter of regular black holes from their shadow

In a previous paper, one of us has showed that, at least in some cases, the Kerr-nature of astrophysical black hole candidates is extremely difficult to test and current techniques, even in presence of excellent data not available today, cannot distinguish a Kerr black hole from a Bardeen one, despite the substantial difference of the two backgrounds. In this paper, we investigate if the detection of the ''shadow'' of nearby super-massive black hole candidates by near future mm/sub-mm very long baseline interferometry experiments can do the job. More specifically, we consider the measurement of the Kerr spin parameter of the Bardeen and Hayward regular black holes from their shadow, and we then compare the result with the estimate inferred from the Kα iron line and from the frequency of the innermost stable circular orbit. For non-rotating black holes, the shadow approach provides different values, and therefore the Kerr black hole hypothesis can potentially be tested. For near extremal objects, all the approaches give quite similar results, and therefore it is not possible to constrain deviations from the Kerr solution. The present work confirms that it is definitively challenging to test this kind of metrics, even with future facilities. However, the detection of a source that looks like a fast-rotating Kerr black hole can put meaningful constraints on the nature of the compact object.

79 ASTRONOMY AND ASTROPHYSICS↗

Variation in stem mortality rates determines patterns of above-ground biomass in Amazonian forests: implications for dynamic global vegetation models

Understanding the processes that determine above-ground biomass (AGB) in Amazonian forests is important for predicting the sensitivity of these ecosystems to environmental change and for designing and evaluating dynamic global vegetation models (DGVMs). AGB is determined by inputs from woody productivity [woody net primary productivity (NPP)] and the rate at which carbon is lost through tree mortality. Here, we test whether two direct metrics of tree mortality (the absolute rate of woody biomass loss and the rate of stem mortality) and/or woody NPP, control variation in AGB among 167 plots in intact forest across Amazonia. We then compare these relationships and the observed variation in AGB and woody NPP with the predictions of four DGVMs. The observations show that stem mortality rates, rather than absolute rates of woody biomass loss, are the most important predictor of AGB, which is consistent with the importance of stand size structure for determining spatial variation in AGB. The relationship between stem mortality rates and AGB varies among different regions of Amazonia, indicating that variation in wood density and height/diameter relationships also influences AGB. In contrast to previous findings, we find that woody NPP is not correlated with stem mortality rates and is weakly positively correlated with AGB. Across the four models, basin-wide average AGB is similar to the mean of the observations. However, the models consistently overestimate woody NPP and poorly represent the spatial patterns of both AGB and woody NPP estimated using plot data. In marked contrast to the observations, DGVMs typically show strong positive relationships between woody NPP and AGB. Resolving these differences will require incorporating forest size structure, mechanistic models of stem mortality and variation in functional composition in DGVMs.

54 ENVIRONMENTAL SCIENCES↗

Indirect Tool Condition Monitoring Using Ensemble Machine Learning Techniques

Abstract Tool condition monitoring (TCM) has become a research area of interest due to its potential to significantly reduce manufacturing costs while increasing process visibility and efficiency. Machine learning (ML) is one analysis technique which has demonstrated advantages for TCM applications. However, the commonly studied individual ML models lack generalizability to new machining and environmental conditions, as well as robustness to the unbalanced datasets which are common in TCM. Ensemble ML models have demonstrated superior performance in other fields, but have only begun to be evaluated for TCM. As a result, it is not well understood how their TCM performance compares to that of individual models, or how homogeneous and heterogeneous ensemble models’ performances compare to one another. To fill in these research gaps, milling experiments were conducted using various cutting conditions, and the model groups were compared across several performance metrics. Statistical t-tests were also used to evaluate the significance of model performance differences. Through the analysis of four individual ML models and five ensemble models, all based on the processes’ sound, spindle power, and axial load signals, it was found that on average, the ensemble models performed better than the individual models, and that the homogeneous ensembles outperformed the heterogeneous ensembles.

Engineering↗

SU-E-T-03: 3D GPU-Accelerated Secondary Checks of Radiation Therapy Treatment Plans

Purpose: Redundant treatment verifications in conformal and intensity-modulated radiation therapy techniques are traditionally performed with single point calculations. New solutions can replace these checks with 3D treatment plan verifications. This work describes a software tool (Mobius3D, Mobius Medical Systems) that uses a GPU-accelerated collapsed cone algorithm to perform 3D independent verifications of TPS calculations. Methods: Mobius3D comes with reference beam models for common linear accelerators. The system uses an independently developed collapsed cone algorithm updated with recent enhancements. 144 isotropically-spaced cones are used for each voxel for calculations. These complex calculations can be sped up by using GPUs. Mobius3D calculate dose using DICOM information coming from TPS (CT, RT Struct, RT Plan RT Dose). DVH-metrics and 3D gamma tests can be used to compare both TPS and secondary calculations. 170 patients treated with all common techniques as 3DCFRT (including wedged), static and dynamic IMRT and VMAT have been successfully verified with this solution. Results: Calculation times are between 3–5 minutes for 3DCFRT treatments and 15–20 for most complex dMLC and VMAT plans. For all PTVs mean dose and 90% coverage differences are (1.12±0.97)% and (0.68±1.19)%, respectively. Mean dose discrepancies for all OARs is (0.64±1.00)%. 3D gamma (global, 3%/3 mm) analysis shows a mean passing rate of (97.8 ± 3.0)% for PTVs and (99.0±3.0)% for OARs. 3D gamma pasing rate for all voxels in CT has a mean value of (98.5±1.6)%. Conclusion: Mobius3D is a powerful tool to verify all modalities of radiation therapy treatments. Dose discrepancies calculated by this system are in good agreement with TPS. The use of reference beam data results in time savings and can be used to avoid the propagation of errors in original beam data into our QA system. GPU calculations permit enhanced collapsed cone calculations with reasonable calculation times.

60 APPLIED LIFE SCIENCES↗

Protocol for Uniformly Measuring and Expressing the Performance of Energy Storage Systems

The Protocol for Uniformly Measuring and Expressing the Performance of Energy Storage Systems (PNNL-22010) was first issued in November 2012 as a first step toward providing a foundational basis for developing an initial standard for the uniform measurement and expression of energy storage system (ESS) performance. Based on experiences with the application and use of that document, and to include additional ESS applications and associated duty cycles, test procedures and performance metrics, a first revision of the November 2012 Protocol was issued in June 2014 (PNNL 22010 Rev. 1). As an update of the 2014 revision 1 to the Protocol, this document (the March 2016 revision 2 to the Protocol) is intended to supersede the June 2014 revision 1 to the Protocol and provide a more user-friendly yet more robust and comprehensive basis for measuring and expressing ESS performance.

25 ENERGY STORAGE↗

Protocol for Uniformly Measuring and Expressing the Performance of Energy Storage Systems

The Protocol for Uniformly Measuring and Expressing the Performance of Energy Storage Systems (PNNL-22010) was first issued in November 2012 as a first step toward providing a foundational basis for developing an initial standard for the uniform measurement and expression of energy storage system (ESS) performance. Based on experiences with the application and use of that document, and to include additional ESS applications and associated duty cycles, test procedures and performance metrics, a first revision of the November 2012 Protocol was issued in June 2014 (PNNL-22010 Rev. 1). As an update of the 2014 revision 1 to the Protocol, this document (the March 2016 revision 2 to the Protocol) is intended to supersede the June 2014 revision 1 to the Protocol and provide a more user-friendly yet more robust and comprehensive basis for measuring and expressing ESS performance. This foreword1 provides general and specific details about what additions, revisions, and enhancements have been made to the June 2014 Protocol and the rationale for them in arriving at this March 2016 Protocol (PNNL-22010 Rev. 2).

25 ENERGY STORAGE↗