Search NASA⌕ Search

SEARCH · Search NASA

Results for “Testing metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 541 records · Page 30

Experimental and Measurement Uncertainty Associated with Characterizing Slurry Mixing Performance of Pulsating Jets at Multiple Scales

Understanding how uncertainty manifests itself in complex experiments is important for developing the testing protocol and interpreting the experimental results. This paper describes experimental and measurement uncertainties, and how they can depend on the order of performing experimental tests. Experiments with pulse-jet mixers in tanks at three scales were conducted to characterize the performance of transient-developing periodic flows in Newtonian slurries. Other test parameters included the simulant, solids concentration, and nozzle exit velocity. Critical suspension velocity and cloud height were the metrics used to characterize Newtonian slurry flow associated with mobilization and mixing. During testing, near-replicate and near-repeat tests were conducted. The experimental results were used to quantify the combined experimental and measurement uncertainties using standard deviations and percent relative standard deviations (%RSD) The uncertainties in critical suspension velocity and cloud height tend to increase with the values of these responses. Hence, the %RSD values are the more appropriate summary measure of near-replicate testing and measurement uncertainty.

pulse jet mixing↗

Proactive Frequency Stability Scheme: A Distributed Framework Based on Particle Filters and Synchrophasors

The reactive nature of traditional under-frequency load shedding schemes can lead to delayed response and unnecessary loss of load. This work presents a proactive framework for power system frequency stability. Bayesian filters and synchrophasors are leveraged to produce predictions after disturbances are detected. By being able to estimate the future state of frequency corrective actions can be taken before the system reaches a critical condition. This proactive approach makes it possible to optimize the response to a disturbance, which results in a decrease in the amount of compensation utilized. The framework is tested via Matlab simulations based on Kundur’s Two-Area System, and the IEEE 14-Bus System. Performance metrics are provided and evaluated against other contemporary solutions found in literature. During testing this framework outperformed other solutions by drastically reducing the amount of load dropped during compensation.

42 ENGINEERING↗

EVSE Characterization: V2G EVSE Comparison

As part of the U.S. Department of Energy EVs@Scale consortium Next-Generation Profiles project, results and analysis from the characterization of high-power conductive and wireless charging infrastructure are presented. This characterization is conducted over a wide range of direct current (DC) current and DC voltage operation for nominal test conditions and off-nominal test conditions. Test plans and procedures were developed to define the test configurations and requirements, measurement parameters, and test procedures used throughout testing. Results from a 2024 study conducted on electric vehicle supply equipment (EVSE) characterization by the Idaho National Laboratory (INL) include two bi-directional vehicle-to-grid (V2G) capable EVSEs. These EVSE are referred to as V2G-EVSE9 and V2G-EVSE10. Laboratory testing is conducted at nominal test conditions to characterize the power transfer capabilities, efficiency, power factor, and other power quality metrics of the two DC EVSEs capable of V2G bi-directional power transfer. Results from testing show the performance is consistent for V2G-EVSE9 and V2G-EVSE10 when comparing charging to discharging performance, except for V2G-EVSE10 for power transfer when operating above 70% of the rated DC current. V2G-EVSE10 efficiency is >98% while charging and <91% while discharging at the same operating conditions, near maximum-rated current, at 300VDC. In contrast, V2G-EVSE9 results are consistent for charging and discharging. This EVSE is nearly 96% efficient while charging or discharging when operating over 50% of rated AC power. V2G EVSE performance is also characterized during off-nominal AC grid conditions involving AC voltage deviation (426 VAC to 518 VAC), AC frequency deviation of +2% (58.8 Hz to 61.2 Hz), and AC voltage harmonics injection. Many test conditions have little-to-no impact on performance characteristics of the two EVSEs; however, there are a few notable findings with significant power transfer capability impacts. AC voltage harmonics injection resulted in negative impacts on power quality attributes for both EVSEs, but with no impact on power transfer capability. Off-nominal AC voltage and frequency conditions resulted in unstable or lack of power transfer capability for both EVSEs. V2G-EVSE9 is unable to transfer power when AC voltage is >300V L-N. V2G-EVSE10 is unable to transfer power when AC frequency deviation exceeds +0.8%. V2G energy management system transient response and latency are quantified during laboratory testing. V2G-EVSE9 and V2G-EVSE10 utilize cloud-based V2G energy management systems that command the power transfer level between the EVSE and EV. The latency and response characteristics of the entire systems (web-based user interface, V2G energy management system, cellular communications, and EVSE response) are quantified through laboratory testing for V2G-EVSE9 and V2G-EVSE10. V2G-EVSE9 latency ranges from 0.8 to 1.8 seconds, whereas V2G-EVSE10 latency ranges from 3.4 to 8.8 seconds. The ramp rate to a change in power transfer request also differs between the two EVSEs. V2G-EVSE10 ramp rate ranges from 50% to -250% of rated AC power per second, whereas V2G-EVSE9 rate ranges from 95% to -95% of rated AC power per second. At the highest rate of change in power transfer, V2G-EVSE10 can change from full charge power to full discharge power in less than one second. The V2G EVSE characterization presented in this report provides valuable insights and results for use by numerous entities. This includes modeling and simulation organizations, decision makers, fleet planning, industry stakeholders, and many others involved with the development and deployment of electrified transportation technologies. Additional high-power DC chargers, bidirectional chargers, and inductive power transfer EVSE characterization results are anticipated from additional EVSE brands and models, which will be detailed in future publications in support of the U.S. Department of Energy EVs@Scale consortium Next-Gen Profiles project.

25 ENERGY STORAGE↗

Draft Plan to Develop Non-Intrusive Load Monitoring Test Protocols

This document presents a Draft Plan proposed to develop a common test protocol that can be used to evaluate the performance requirements of Non-Intrusive Load Monitoring. Development on the test protocol will be focused on providing a consistent method that can be used to quantify and compare the performance characteristics of NILM products. Elements of the protocols include specifications for appliances to be used, metrics, instrumentation, and a procedure to simulate appliance behavior during tests. In addition, three priority use cases for NILM will be identified and their performance requirements will specified.

47 OTHER INSTRUMENTATION↗

Evaluation of Mid-Size Male Hybrid III Models for use in Spaceflight Occupant Protection Analysis

In an effort to improve occupant safety during dynamic phases of spaceflight, the National Aeronautics and Space Administration (NASA) has worked to develop occupant protection standards for future crewed spacecraft. One key aspect of these standards is the identification of injury mechanisms through anthropometric test devices (ATDs). Within this analysis, both physical and computational ATD evaluations are required to reasonably encompass the vast range of loading conditions any spaceflight crew may encounter. In this study the accuracy of publically available mid-size male HIII ATD finite element (FE) models are evaluated within applicable loading conditions against extensive sled testing performed on their physical counterparts. Methods: A series of sled tests were performed at the Wright Patterson Air force Base (WPAFB) employing variations of magnitude, duration, and impact direction to encompass the dynamic loading range for expected spaceflight. FE simulations were developed to the specifications of the test setup and driven using measured acceleration profiles. Both fast and detailed FE models of the mid-size male HIII were ran to quantify differences in their accuracy and thus assess the applicability of each within this field. Results: Preliminary results identify the dependence of model accuracy on loading direction, magnitude, and rate. Additionally the accuracy of individual response metrics are shown to vary across each model within evaluated test conditions. Causes for model inaccuracy are identified based on the observed relationships. Discussion: Computational modeling provides an essential component to ATD injury metric evaluation used to ensure the safety of future spaceflight occupants. The assessment of current ATD models lays the groundwork for how these models can be used appropriately in the future. Identification of limitations and possible paths for improvement aid in the development of these effective analysis tools.

Putnam, Jacob B.↗

Evaluation of Mid-Size Male Hybrid III Models for use in Spaceflight Occupant Protection Analysis

Introduction: In an effort to improve occupant safety during dynamic phases of spaceflight, the National Aeronautics and Space Administration (NASA) has worked to develop occupant protection standards for future crewed spacecraft. One key aspect of these standards is the identification of injury mechanisms through anthropometric test devices (ATDs). Within this analysis, both physical and computational ATD evaluations are required to reasonably encompass the vast range of loading conditions any spaceflight crew may encounter. In this study the accuracy of publically available mid-size male HIII ATD finite element (FE) models are evaluated within applicable loading conditions against extensive sled testing performed on their physical counterparts. Methods: A series of sled tests were performed at the Wright Patterson Air force Base (WPAFB) employing variations of magnitude, duration, and impact direction to encompass the dynamic loading range for expected spaceflight. FE simulations were developed to the specifications of the test setup and driven using measured acceleration profiles. Both fast and detailed FE models of the mid-size male HIII were ran to quantify differences in their accuracy and thus assess the applicability of each within this field. Results: Preliminary results identify the dependence of model accuracy on loading direction, magnitude, and rate. Additionally the accuracy of individual response metrics are shown to vary across each model within evaluated test conditions. Causes for model inaccuracy are identified based on the observed relationships. Discussion: Computational modeling provides an essential component to ATD injury metric evaluation used to ensure the safety of future spaceflight occupants. The assessment of current ATD models lays the groundwork for how these models can be used appropriately in the future. Identification of limitations and possible paths for improvement aid in the development of these effective analysis tools.

Putnam, J.↗

Computed tomography-based texture analysis of bladder cancer: differentiating urothelial carcinoma from micropapillary carcinoma

Purpose: The purpose of the study is to determine the feasibility of using computed tomography-based texture analysis (CTTA) in differentiating between urothelial carcinomas (UC) of the bladder from micropapillary carcinomas (MPC) of the bladder. Methods: Regions of interests (ROIs) of computerized tomography (CT) images of 33 MPCs and 33 UCs were manually segmented and saved. Custom MATLAB code was used to extract voxel information corresponding to the ROI. The segmented tumors were input to a pre-existing radiomics platform with a CTTA panel. A total of 58 texture metrics were extracted using four different texture extraction techniques and statistically analyzed using a Wilcoxon rank-sum test to determine the differences between UCs and MPCs. Results: Of the 58 texture metrics extracted using the gray level co-occurrence matrix (GLCM) and gray level difference matrix (GLDM), 28 texture metrics were statistically significant (p < 0.05) for differences in tumor textures and 27 texture metrics were statistically significant (p < 0.05) for peritumoral fat textures. The remaining nine metrics extracted using histogram and fast Fourier transform analyses did not show significant differences between the textures of the tumors and their peritumoral fat. Conclusions: CTTA shows that MPC have a more heterogeneous texture compared to UC. As visual discrimination of MPC from UC from clinical CT scans are difficult, results from this study suggest that tumor heterogeneity extracted using GLCM and GLDM may be a good imaging aid in segregating MPC from UC. This tool can aid clinicians in further sub-classifying bladder cancers on routine imaging, a process which has potential to alter treatment and patient care.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Estimated Ambient Sonic Boom Metric Levels and X-59 Signal-to-Noise Ratios across the USA

NASA is building the X-59 Quiet Supersonic Technology aircraft to produce low noise sonic booms for a series of community noise surveys across the USA. Survey participants will rate their perception of the low-booms from supersonic X-59 flyovers. Several noise metrics are proposed to quantify the noise dose: A-, B-, D-, and E-weighted Sound Exposure Level, Stevens Perceived Level, and Indoor Sonic Boom Annoyance Predictor. Sparse measurements across the survey area will be used to estimate community noise exposure. The level of these low-booms may be comparable to the ambient noise level in some locations, leading to uncertainty in noise exposure estimations. This uncertainty may necessitate increased reliance on sonic boom propagation predictions for exposure estimation. Low-boom signal to ambient noise ratio is one way to quantify uncertainty in measured sonic boom levels. An empirical relationship between A-weighted ambient level and sonic boom metric levels is used in conjunction with the National Park Service’s L50 SPL map to estimate ambient noise levels expressed in terms of sonic boom noise metrics across the USA. These estimates of ambient levels will aid in X-59 community test planning and execution. The signal-to-noise ratio for the undertrack X-59 sonic boom is also estimated, and an example application of these data is presented for comparing potential noise monitor sites prior to a community noise test.

X-59↗

On Transferability of Performance Metrics for Redox-Active Molecules

Because of their lower material costs, redox-active organic molecules (ROMs) are increasingly used in flow cells. Quantification of their electrochemical stability is important for choosing between different battery chemistries. Ad hoc performance metrics such as cycle numbers have been used to guide materials discovery, but it is unclear whether such metrics are suitable for the intended use. Here, we suggest a kinetic model allowing a narrower definition of cyclability by interpreting it through intrinsic parameters. Using this model, we examine systematic data on cycling and calendar stabilities for dialkoxyarene ROMs in stationary cells. Even for these closely related ROMs, the cycling numbers strongly depend on minor changes in test protocols and structure, which argue against the universality and transferability of these commonly used metrics. Recommendations regarding the standardization of testing protocols are suggested in light of these findings.

25 ENERGY STORAGE↗

The James Webb Space Telescope: Observatory Status and Preparations for Launch

The James Webb Space Telescope (JWST) is a large (6.5 meter) segmented aperture telescope equipped with near- and mid-infrared instruments (0.6-28 microns), all of which are passively cooled to approximately 40 degrees Kelvin by a 5-layer sunshield while the mid-infrared instrument is actively cooled to 7 degrees Kelvin. The JWST telescope recently completed its cryogenic test program and the sunshield has been fully integrated and deployed. JWST is currently in the final stages of the test program at the Observatory level. The current estimated JWST performance metrics will be presented, such as the image quality, pointing stability, sensitivity, and stray light backgrounds. The JWST development status and future plans will be described for the final testing, launch, and commissioning.

McElwain, Michael↗

Geometrical aspects on the dark matter problem

In the present paper we apply Nash’s theory of perturbative geometry to the study of dark matter gravity in a higher-dimensional space–time. It is shown that the dark matter gravitational perturbations at local scale can be explained by the extrinsic curvature of the standard cosmology. In order to test our model, we use a spherically symmetric metric embedded in a five-dimensional bulk. As a result, considering a sample of 10 low surface brightness and 6 high surface brightness galaxies, we find a very good agreement with the observed rotation curves of smooth hybrid alpha-HI measurements. - Highlights: • The metric perturbation and the embedding lead naturally to a “brane-world”-like higher dimensional structure. • Nash’s theorem as a cornerstone of the formation of geometrical structures. • The dark matter gravitational perturbations at local scale can be explained by the extrinsic curvature. • A good agreement was found with the observed rotation curves of smooth hybrid alpha-HI measurements.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Variations in the Contouring of Organs at Risk: Test Case From a Patient With Oropharyngeal Cancer

Purpose: Anatomy contouring is critical in radiation therapy. Inaccuracy and variation in defining critical volumes will affect everything downstream: treatment planning, dose-volume histogram analysis, and contour-based visual guidance used in image-guided radiation therapy. This study quantified: (1) variation in the contouring of organs at risk (OAR) in a clinical test case and (2) corresponding effects on dosimetric metrics of highly conformal plans. Methods and Materials: A common CT data set with predefined targets from a patient with oropharyngeal cancer was provided to a population of clinics, which were asked to (1) contour OARs and (2) design an intensity-modulated radiation therapy plan. Thirty-two acceptable plans were submitted as DICOM RT data sets, each generated by a different clinical team. Using those data sets, we quantified: (1) the OAR contouring variation and (2) the impact this variation has on dosimetric metrics. New technologies were employed, including a software tool to quantify three-dimensional structure comparisons. Results: There was significant interclinician variation in OAR contouring. The degree of variation is organ-dependent. We found substantial dose differences resulting strictly from contouring variation (differences ranging from -289% to 56% for mean OAR dose; -22% to 35% for maximum dose). However, there appears to be a threshold in the OAR comparison metric beyond which the dose differences stabilize. Conclusions: The effects of interclinician variation in contouring organs-at-risk in the head and neck can be large and are organ-specific. Physicians need to be aware of the effect that variation in OAR contouring can play on the final treatment plan and not restrict their focus only to the target volumes.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

An Objective and Efficient Method for Assessing the Impact of Reduced-Precision Calculations On Solution Correctness

Recent studies have shown that reducing the precision of floating-point calculations in an atmospheric model can improve the models' computational performance without affecting model fidelity, but code changes are needed to accommodate lower precision, and a small number of calculations might need to retain higher precision to prevent undue round-off error. For complex and computationally expensive systems like the Energy Exascale Earth System Model (E3SM), a method is needed to objectively assess the quality of the lower-precision simulations and to quickly identify problematic code pieces This paper demonstrates that solution correctness can be evaluated through a simple and quantitative error metric based on time step convergence. The proposed test method can unambiguously detect the accumulation of precision error and objectively assess its impact on solution accuracy. Since fast physical processes are known to affect key features of the multi-year mean climate in an atmospheric model, we show that the convergence test applied to short simulations can provide useful information about the impact of reduced precision on a model’s long-term behavior. In contrast, the traditional way of climate model evaluation requires multi-year simulations and involves inspecting many physical quantities for statistical significance in the presence of natural variability. The simplicity, computational efficiency, and objectivity of the proposed method makes it very attractive for high-resolution model development. The method is also expected to be applicable to other models that numerically solve time evolution equations.

97 MATHEMATICS AND COMPUTING↗

Physical patterning of high-Q superconducting niobium resonators via ion beam etching

The development of superconducting quantum circuits increasingly involves the exploration of chemically distinct materials and complex multilayered structures. Accelerating this trend may benefit from low-damage, materials-agnostic patterning techniques that are compatible with a broad range of materials. Here, in this work, we investigate the utility of low-energy ion beam etching (IBE), a physical patterning technique, as an alternative to reactive ion etching for fabricating low-loss superconducting resonators. We use niobium (Nb) resonators as a test platform, leveraging their well-characterized performance metrics for benchmarking. To address IBE-induced surface redeposition, we introduce an in situ aluminum capping layer combined with targeted post-fabrication chemical treatment. This strategy yields resonators with internal quality factors as high as 6 × 10 5 in the single-photon regime at 50 mK. These results establish low-energy IBE as a promising patterning technique for superconducting devices, with the potential to accelerate development across chemically diverse and multilayered material platforms.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Demonstration of a 1820 channel multiplexer for transition-edge sensor bolometers

The scalability of most transition-edge sensor arrays is limited by the multiplexing technology, which combines their signals over a reduced number of wires and amplifiers. Here, in this Letter, we present and demonstrate a multiplexer design optimized for transition-edge sensor bolometers with 1820 sensors per readout unit, a factor of two more than the previous state-of-the-art. The design is optimized for cosmic microwave background imaging applications, and it builds on previous microwave superconducting quantum interference device multiplexers by doubling the available readout bandwidth to the full 4–8 GHz octave. Evaluating the key performance metrics of yield, sensitivity, and crosstalk through laboratory testing, we find an end-to-end operable detector yield of 78%, a typical nearest-neighbor crosstalk amplitude of ∼0.4%, and a median white noise level of 83 pA/$\sqrt{\textrm{Hz}}$ due to the multiplexer, corresponding to an estimated contribution of 4% to the total system noise for a ground-based cosmic microwave background telescope. Additionally, we identify a possible path toward reducing resonator loss for future designs with reduced noise. We expect these developments to alleviate the system complexity, cryogenic requirements, and cost of future large arrays of low temperature detectors.

cosmic microwave background↗

Planning for Resilient Power Distribution Systems using Risk-Based Quantification and Q-Learning

Grid hardening is one of the most effective approaches that reduce the component failures and restoration efforts thus increasing the resilience of the power systems against extreme events. However, hardening and upgrading the entire system is prohibitively expensive and hence the optimal design of a distribution network is challenging. This paper adopted a reinforcement learning algorithm to identify the optimal hardening strategy to enhance the resilience of power distribution systems. Adopting the Q-learning algorithm as the reinforcement learning technique, we found the sequential optimal action for hardening measures to enhance the grid's resilience for the given budget. To identify the optimal strategy through Q-learning, Conditional Value at Risk (CVaR) is used as a rewarding metric. A study on the IEEE 123-bus test feeder validate the effectiveness of the proposed model and show how to effectively allocate budget limited resources to plan a resilient power distribution network.

Paul, Shuva↗

Poster — Thur Eve — 58: Dosimetric validation of electronic compensation for radiotherapy treatment planning

In this study we investigate the deliverability of dosimetric plans generated by the irregular surface compensator (ISCOMP) algorithm for 6 MV photon beams in Eclipse (Varian Medical System, CA). In contrast to physical tissue compensation, the electronic ISCOMP uses MLCs to dynamically modulate the fluence of a photon beam in order to deliver a uniform dose at a user defined plane in tissue. This method can be used to shield critical organs that are located within the treatment portal or improve dose uniformity by tissue compensation in inhomogeneous regions. Three site specific plans and a set of test fields were evaluated using the γ-metric of 3%/ 3 mm on Varian EPID, MapCHECK, and Gafchromic EBT3 film with a clinical tolerance of >95% passing rates. Point dose measurements with an NRCC calibrated ionization chamber were also performed to verify the absolute dose delivered. In all cases the MapCHECK measured plans met the gamma criteria. The mean passing rate for the six EBT3 film field measurements was 96.2%, with only two fields at 93.4 and 94.0% passing rates. The EPID plans passed for fields encompassing the central ∼10 × 10 cm{sup 2} region of the detector; however for larger fields and greater off-axis distances discrepancies were observed and attributed to the profile corrections and modeling of backscatter in the portal dose calculation. The magnitude of the average percentage difference for 21 ion chamber point dose measurements and 17 different fields was 1.4 ± 0.9%, and the maximum percentage difference was −3.3%. These measurements qualify the algorithm for routine clinical use subject to the same pre-treatment patient specific QA as IMRT.

60 APPLIED LIFE SCIENCES↗