Search NASA⌕ Search

SEARCH · Search NASA

Results for “Testing metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 505 records · Page 28

Object-Based Evaluation of Dynamical and Statistical Downscaled Precipitation Products over CONUS

High-resolution precipitation data, generated through dynamical downscaling (DD) or statistical downscaling (SD) of global climate model output, provide critical information for regional climate assessment and adaptation planning. Most downscaling development and validation have focused on accurate gridscale precipitation construction and ignored the spatial structure of precipitation across model grids and at the event scale. However, many applications, e.g., hydrologic modeling and the analysis using the downscaled precipitation, require a reasonable representation of the spatial structure of precipitation within watersheds. Therefore, a set of standard metrics to evaluate the representation of the spatial structure of individual storms across diverse downscaled precipitation products is desired. To address this need, we conducted an object-based evaluation of precipitation in decades-long DD and SD products over the contiguous United States (CONUS). Specifically, we evaluate their ability to reproduce various features of precipitation objects in the observations: total volume, precipitation area, peak intensity, and spatial structure. Multiple metrics (bias, Perkins score, and nonparametric statistical tests) are used to quantify model performance. Our evaluation reveals notable variations in performance among individual products across different climate zones and seasons, as well as between extreme and nonextreme events. In general, most DD products exhibit balanced performance across the four precipitation object features, while SD products vary more significantly in their performance across products. Based on this comprehensive evaluation, we provide guidance on choosing downscaled products for specific regions, seasons, and precipitation object features. These findings and recommendations can inform precipitation-relevant modeling and analysis over CONUS, guide future downscaling technique developments, and provide actionable information for climate impact assessment and adaptation.

Environmental sciences↗

January 2022 Operational Highlights

The Indexing Lab is now complete and includes classified and unclassified computers, safes and work stations. Staff capture metadata and catalog materials, including classified reports, directors’ correspondence and Rocky Flats materials to ensure the holdings are discoverable and retrievable. Access protocols were updated for products related to the LANL Annual Assessment in the Online Vault (OLV) classified repository. The Annual Assessment is a yearly analysis of the weapons systems in the nation’s nuclear stockpile. The Standardization Quality Assessment (SQA) for the Online Vault (OLV) classified repository was completed and contained no findings. Beta testing, or customer validation, began for the metrics database that generates metrics reports and visualizations from NSRC customer projects.

97 MATHEMATICS AND COMPUTING↗

Hot Pressing of Reinforced Li-NMC All-Solid State Batteries with Sulfide Glass Electrolyte

The performance of solid-state batteries (SSBs) with sulfide solid-state electrolytes (SSEs) is limited because they are 10 - 30% porous. Porosity limits energy density of the composite cathode and provides a conduit for Li-metal deposits through the separator if operating specifications (e.g. current density, operating temperature, and pressure) are not strictly controlled. This project focused on demonstrating that hot press cell processing and appropriately formulated sulfide glass SSEs can help eliminate porosity to enable SSBs with energy density of ≥ 350 Wh/kg. The objective of the project was to research, develop, and test SSBs capable of achieving program performance metrics by implementing appropriately formulated sulfide glass SSEs and hot press cell processing in a dry room environment. In the composite cathode, hot pressing helps eliminate porosity to increase energy density by enabling thick composite cathodes with high active material loading. In the separator, hot pressing helps eliminate porosity that may otherwise provide a conduit for Li metal deposits to short the cell.

25 ENERGY STORAGE↗

Final Technical Report of a Feasibility Investigation of a Hydropower Flexibility Upgrade Kit

The project studied the feasibility of a concept for a flexibility upgrade kit called h-Dynamyk to improve the flexibility of U.S. hydropower plants and assessed its potential impact on the U.S. hydropower industry. This concept addresses the growing need for grid flexibility amid rising renewable energy adoption, such as wind and solar, which often introduce intermittency. The upgrade kit is intended to address challenges posed by aging turbine designs in meeting modern grid demands for flexibility services like frequency response, voltage control, spinning reserve, and black-start capabilities. Key features of the h-Dynamyk kit include siphon turbine assemblies, medium-voltage variable frequency drives (VFDs), and smart controls integrated with plant systems to optimize operations and extend equipment life. This nine-month discovery project successfully achieved its objectives by defining a baseline design and performance metrics for the h-Dynamyk flexibility upgrade kit, analyzing this baseline, and incorporating lessons learned from this analysis into a modified set of metrics and recommendations for future design and testing, which were further guided by field site studies.

13 HYDRO ENERGY↗

Operation at Reduced Atmospheric Pressure and Concept of Reliability Redundancy for Optimized Design of Insulation Systems

Electrified transportation is calling for insulation design criteria that is adequate to provide elevated levels of power density, power dynamics and reliability. Increasing voltage levels are expected to cause accelerated intrinsic and extrinsic aging effects which will not be easily predictable at the design stage due to a lack of suitable modeling. Designing reliable insulation systems would require finding solutions able to control accelerated aging due to an unpredictable increase of intrinsic stresses and the onset of extrinsic stresses as partial discharges. This paper proposes the concept of reliability redundancy for the insulation design of aerospace electrical asset components, which is also validated at lower-than-standard atmospheric pressure. The principle is that extrinsic-aging-free design might be achieved upon determining the aging stress or abnormal service stresses distribution and being sure that aging will not generate conditions that can incept extrinsic aging (partial discharges) during operation life. However, such information is never, in practice, fully available to insulation system designers. Hence, especially in critical applications such as electrified aircraft, aerospace, and combat ships a further level of reliability should be added to a partial-discharge-free design, which can consist of the use of corona-resistant materials and/or of life models able to consider the accelerated aging effect of partial discharges (or any other type of extrinsic-accelerated aging factor). Innovative life modeling considering both extrinsic and intrinsic aging stresses, insulating material testing to estimate model parameters, and a metric for quantifying the extent of corona (or partial discharge) resistance can lead to establishing feasibility and limit conditions for optimized or fully reliability-redundant design. It is shown in the paper that if an extrinsic-aging-free design is not feasible, and it is therefore replaced by a redundant design, a further level of reliability redundancy can be provided by effective condition monitoring plans.

Montanari, Gian Carlo↗

A Field Study of Nonintrusive Load Monitoring Devices and Implications for Load Disaggregation

Evaluations of nonintrusive load monitoring (NILM) algorithms and technologies have mostly occurred in constrained, artificial environments. However, few field evaluations of NILM products have taken place in actual buildings under normal operating conditions. This paper describes a field evaluation of a state-of-the-art NILM product, tested in eight homes. The match rate metric—a technique recommended by a technical advisory group—was used to measure the NILM’s success in identifying specific loads and the accuracy of the energy consumption estimates. A performance assessment protocol was also developed to address common issues with NILM mislabeling and ground-truth comparisons that have not been sufficiently addressed in past evaluations. The NILM product’s estimates were compared to the submetered consumption of eight major appliances. Overall, the product had good performance in disaggregating the energy consumption of the electric water heaters, which included both electric resistance and heat-pump water heaters, but only a fair accuracy with refrigerators, dryers, and air conditioners. The performance was poor for cooking equipment, furnace fans, clothes washers, and dishwashers. Moreover, the product was often unable to detect major loads in homes. Typically, two or more appliances were not detected in a home. At least two dryers, furnace fans, and air conditioners went undetected across the eight homes. On the other hand, the dishwasher was detected in all homes where available or monitored. The key findings were qualitatively compared to those of past field evaluations. Potential areas for improvement in NILM product performance were determined along with areas where complementary technologies may be able to aid in load-disaggregation applications.

47 OTHER INSTRUMENTATION↗

Reinventing The Design Process: Teams and Models

The future of space mission designing will be dramatically different from the past. Formerly, performance-driven paradigms emphasized data return with cost and schedule being secondary issues. Now and in the future, costs are capped and schedules fixed-these two variables must be treated as independent in the design process. Accordingly, JPL has redesigned its design process. At the conceptual level, design times have been reduced by properly defining the required design depth, improving the linkages between tools, and managing team dynamics. In implementation-phase design, system requirements will be held in crosscutting models, linked to subsystem design tools through a central database that captures the design and supplies needed configuration management and control. Mission goals will then be captured in timelining software that drives the models, testing their capability to execute the goals. Metrics are used to measure and control both processes and to ensure that design parameters converge through the design process within schedule constraints. This methodology manages margins controlled by acceptable risk levels. Thus, teams can evolve risk tolerance (and cost) as they would any engineering parameter. This new approach allows more design freedom for a longer time, which tends to encourage revolutionary and unexpected improvements in design.

Wall, Stephen D.↗

An On-Line Technology Information System (OTIS) for Advanced Life Support

An On-line Technology Information System (OTIS) is currently being developed for the Advanced Life Support (ALS) Program. This paper describes the preliminary development of OTIS, which is a system designed to provide centralized collection and organization of technology information. The lack of thorough, reliable and easily understood technology information is a major obstacle in effective assessment of technology development progress, trade studies, metric calculations, and technology selection for integrated testing. OTIS will provide a formalized, well-organized protocol to communicate thorough, accurate, current and relevant technology information between the hands-on technology developer and the ALS Community. The need for this type of information transfer system within the Solid Waste Management (SWM) element was recently identified and addressed. A SWM Technology Information Form (TIF) was developed specifically for collecting detailed technology information in the area of SWM. In the TIF, information is requested from SWM technology developers, based upon the Technology Readiness Level (TRL). Basic information is requested for low-TRL technologies, and more detailed information is requested as the TRL of the technology increases. A comparable form is also being developed for the wastewater processing element. In the future, similar forms will also be developed for the ALS elements of air revitalization, food processing, biomass production and thermal control. These ALS element-specific forms will be implemented in OTIS via a web-accessible interface,with the data stored in an object-oriented relational database (created in MySQLTM) located on a secure server at NASA Ames Research Center. With OTIS, ALS element leads and managers will be able to carry out informed research and development investment, thereby promoting technology through the TRL scale. OTIS will also allow analysts to make accurate evaluations of technology options. Additionally, the range and specificity of information solicited will help educate technology developers of programmatic needs.

Levri, Julie A.↗

Human Factors Assessment of Vibration Effects on Visual Performance During Launch

The Human Factors Assessment of Vibration Effects on Visual Performance During Launch (Visual Performance) investigation will determine visual performance limits during operational vibration and g-loads on the Space Shuttle, specifically through the determination of minimum readable font size during ascent using planned Orion display formats. Research Summary: The aim of the Human Factors Assessment of Vibration Effects on Visual Performance during Launch (Visual Performance) investigation is to provide supplementary data to that collected by the Thrust Oscillation Seat Detailed Technical Objective (DTO) 695 (Crew Seat DTO) which will measure seat acceleration and vibration from one flight deck and two middeck seats during ascent. While the Crew Seat DTO data alone are important in terms of providing a measure of vibration and g-loading, human performance data are required to fully interpret the operational consequences of the vibration values collected during Space Shuttle ascent. During launch, crewmembers will be requested to view placards with varying font sizes and indicate the minimum readable size. In combination with the Crew Seat DTO, the Visual Performance investigation will: Provide flight-validated evidence that will be used to establish vibration limits for visual performance during combined vibration and linear g-loading. o Provide flight data as inputs to ongoing ground-based simulations, which will further validate crew visual performance under vibration loading in a controlled environment. o Provide vibration and performance metrics to help validate procedures for ground tests and analyses of seats, suits, displays and controls, and human-in-the-loop performance.

Kritina Holden↗

Trajectory-Oriented Approach to Managing Traffic Complexity: Trajectory Flexibility Metrics and Algorithms and Preliminary Complexity Impact Assessment

This document describes exploratory research on a distributed, trajectory oriented approach for traffic complexity management. The approach is to manage traffic complexity based on preserving trajectory flexibility and minimizing constraints. In particular, the document presents metrics for trajectory flexibility; a method for estimating these metrics based on discrete time and degree of freedom assumptions; a planning algorithm using these metrics to preserve flexibility; and preliminary experiments testing the impact of preserving trajectory flexibility on traffic complexity. The document also describes an early demonstration capability of the trajectory flexibility preservation function in the NASA Autonomous Operations Planner (AOP) platform.

Idris, Husni↗

Recent Stirling Conversion Technology Developments and Operational Measurements at NASA Glenn Research Center

In support of the Advanced Stirling Radioisotope Generator (ASRG) project and other potential applications, NASA Glenn Research Center (GRC) has initiated convertor technology development efforts in the areas of acoustic emission, electromagnetic field mitigation, thermoacoustic Stirling conversion, and multiple-cylinder alpha arrangements of Stirling machines. The acoustic emission measurement effort was developed as a health monitoring metric for several Stirling convertors undergoing life testing. While accelerometers have been used in the past to monitor dynamic signature, the acoustic sensors were chosen to monitor cycle events, such gas bearing operation. Several electromagnetic interference (EMI) experiments were performed on a pair of Advanced Stirling Convertors (ASC). These tests demonstrated that a simple bucking coil was capable of reducing the alternating current (ac) magnetic field below the ASRG system specification. The thermoacoustic Stirling concept eliminates the displacer typically found in Stirling machines by making use of the pressure oscillations of a traveling acoustic wave. A 100 W-class thermoacoustic Stirling prototype manufactured by Northrop Grumman Space and Technology was received and tested. Another thermoacoustic prototype designed and fabricated by Sunpower, Inc., will be tested in the near future. A four cylinder free piston alpha prototype convertor was received from Sunpower, Inc. and has been tested at GRC. This hardware was used as a proof of concept to validate thermodynamic models and demonstrate stable operation of multiple-cylinder free-piston Stirling conversion. This paper will discuss each of these activities and the results they produced.

Oriti, Salvatore M.↗

Preliminary Assessment of Suomi-NPP VIIRS On-orbit Radiometric Performance

The Visible-Infrared Imaging Radiometer Suite (VIIRS) is a key instrument on-board the Suomi National Polar-orbiting Partnership (NPP) spacecraft that was launched on October 28th 2011. VIIRS was designed to provide moderate and imaging resolution of most of the globe twice daily. It is a wide-swath (3,040 km) cross-track scanning radiometer with spatial resolutions of 370.and 740 m at nadir for imaging and moderate bands, respectively. It has 22 spectral bands covering the spectrum between 0.412 11m and 12.01 11m, including 14 reflective solar bands (RSB), 7 thermal emissive bands (TEB), and 1 day-night band (ON B). VIIRS observations are used to generate 22 environmental data products (EORs). This paper will briefly describe NPP VIIRS calibration strategies performed by the independent government team, for the initial on-orbit Intensive Calibration and Validation (ICV) activities. In addition, this paper will provide an early assessment of the sensor on-orbit radiometric performance, such as the sensor signal to noise ratios (SNRs), dual gain transition verification, dynamic range and linearity, reflective bands calibration based on the solar diffuser (SO) and solar diffuser stability monitor (SOSM), and emissive bands calibration based on the on-board blackbody calibration (OBC). A comprehensive set of performance metrics generated during the pre-launch testing program will be compared to VIIRS on-orbit early performance, and a plan for future cal/val activities and performance enhancements will be presented.

Oudrari, Hassan↗

NASA Tech Briefs, Februrary 2013

Topics covered include: Measurements of Ultra-Stable Oscillator (USO) Allan Deviations in Space; Gaseous Nitrogen Orifice Mass Flow Calculator; Validation of Proposed Metrics for Two-Body Abrasion Scratch Test Analysis Standards; Rover Low Gain Antenna Qualification for Deep Space Thermal Environments; Automated, Ultra-Sterile Solid Sample Handling and Analysis on a Chip; Measuring and Estimating Normalized Contrast in Infrared Flash Thermography; Spectrally and Radiometrically Stable, Wideband, Onboard Calibration Source; High-Reliability Waveguide Vacuum/Pressure Window; Methods of Fabricating Scintillators With Radioisotopes for Beta Battery Applications; Magnetic Shield for Adiabatic Demagnetization Refrigerators (ADR); CMOS-Compatible SOI MESFETS for Radiation-Hardened DC-to-DC Converters; Silicon Heat Pipe Array; Adaptive Phase Delay Generator; High-Temperature, Lightweight, Self-Healing Ceramic Composites for Aircraft Engine Applications; Treatment to Control Adhesion of Silicone-Based Elastomers; High-Temperature Adhesives for Thermally Stable Aero-Assist Technologies; Rockballer Sample Acquisition Tool; Rock Gripper for Sampling, Mobility, Anchoring, and Manipulation; Advanced Magnetic Materials Methods and Numerical Models for Fluidization in Microgravity and Hypogravity; Data Transfer for Multiple Sensor Networks Over a Broad Temperature Range; Using Combustion Synthesis to Reinforce Berms and Other Regolith Structures; Visible-Infrared Hyperspectral Image Projector; Three-Axis Attitude Estimation With a High-Bandwidth Angular Rate Sensor Change_Detection.m; AGATE: Adversarial Game Analysis for Tactical Evaluation; Ionospheric Simulation System for Satellite Observations and Global Assimilative; Modeling Experiments (ISOGAME); An Extensible, User- Modifiable Framework for Planning Activities; Mission Operations Center (MOC) - Precipitation Processing System (PPS) Interface Software System (MPISS); Automated 3D Damaged Cavity Model Builder for Lower Surface Acreage Tile on Orbiter; Mixed Linear/Square-Root Encoded Single-Slope Ramp Provides Low-Noise ADC with High Linearity for Focal Plane Arrays; RUSHMAPS: Real-Time Uploadable Spherical Harmonic Moment Analysis for Particle Spectrometers; Powered Descent Guidance with General Thrust-Pointing Constraints; X-Ray Detection and Processing Models for Spacecraft Navigation and Timing; and Extreme Ionizing-Radiation-Resistant Bacterium

Source record↗

A Psychoacoustic Evaluation of Noise Signatures from Advanced Civil Transport Aircraft

The NASA Environmentally Responsible Aviation project has been successful in developing and demonstrating technologies for integrated aircraft systems that can simultaneously meet aggressive goals for fuel burn, noise and emissions. Some of the resulting systems substantially differ from the familiar tube and wing designs constituting the current civil transport fleet. This study attempts to explore whether or not the effective perceived noise level metric used in the NASA noise goal accurately reflects human subject response across the range of vehicles considered. Further, it seeks to determine, in a quantitative manner, if the sounds associated with the advanced aircraft are more or less preferable to the reference vehicles beyond any differences revealed by the metric. These explorations are made through psychoacoustic tests in a controlled laboratory environment using simulated stimuli developed from auralizations of selected vehicles based on systems noise assessments.

Rizzi, Stephen A.↗

Estimating the Noise Floor of Sonic Boom Metrics Across the USA

NASA is building the X-59 Quiet Supersonic Technology aircraft to produce low noise sonic booms for a series of community noise surveys across the USA. Survey participants will rate their perception of the low booms from supersonic X-59 flyovers. Several noise metrics are proposed to quantify the noise dose: A-, B-, D-, E-weighted Sound Exposure Level, Stevens Perceived Level, and Indoor Sonic Boom Annoyance Predictor. Sparse measurements across the survey area will be used to estimate community noise exposure. The level of these low booms may be comparable to the ambient noise level in some locations, leading to uncertainty in noise exposure estimations. This uncertainty may necessitate increased reliance on sonic boom propagation predictions for exposure estimation. Low-boom signal to ambient noise ratio is one way to quantify uncertainty in measured sonic boom levels. An empirical relationship between A-weighted ambient level and sonic boom metric levels is used in conjunction with the National Park Service’s L50 SPL map to estimate sonic boom metric ambient levels across the USA. The estimate of ambient sonic boom metric levels will aid in X-59 test planning and execution.

sonic boom↗

The Cusp Plasma Imaging Detector (Cupid) Cubesat Observatory: Instrumentation

The Cusp Plasma Imaging Detector (CuPID) CubeSat observatory is a 6U CubeSat designed to observe solar wind charge exchange in magnetospheric cusps to test competing theories of magnetic reconnection at the Earth’s magnetopause. The CuPID is equipped with three instruments, namely, a wide field-of-view (4.6° × 4.6°) soft x-ray telescope, a micro-dosimeter suite, and an engineering magnetometer optimized for the science operation. The instrument suite has been tested and calibrated in relevant environments, demonstrating successful design. The testing and calibration of these instruments produced metrics and coefficients that will be used to create the CuPID mission’s data product.

Magnetospheric dynamics↗

Noise Abatement Flight Test Data Report

An acoustic flight test of six helicopters conducted by NASA, FAA, and the US Army is described. Microphone arrays were used to collect acoustic data for source noise characterization, to investigate the acoustic impacts of maneuvering flight, and to develop and evaluate noise abatement operations for helicopters on approach. Atmospheric data and vehicle state data were also collected. Noise metrics are calculated and plotted for all conditions tested. All data described are publicly available upon request.

Rotorcraft↗

Lithium-Ion Battery Diagnostics Using Electrochemical Impedance via Machine-Learning

Diagnosing battery states such as health, state-of-charge, or temperature is crucial for ensuring the safety and reliability of electrochemical energy storage systems. While some states, such as temperature, may be measured using cheap sensors, accurate diagnosis of battery health metrics usually requires time-consuming performance measurements, making them infeasible for use in real-world operation. These health metrics can be measured during lab-testing and then estimated on-line using predictive life models or via state observer algorithms such as Kalman filters, but these predictive methods should be supplemented by actual measurement of battery health whenever possible to ensure reliability. Rapid measurement of battery health may be done by various types of fast diagnostic techniques such as electrochemical impedance spectroscopy (EIS), which can be performed in only a few minutes and require only a fraction of the energy and power needed for a full charge and discharge measurement. But there is a substantial challenge for estimating battery health using EIS data, as EIS is sensitive to cell temperature, state-of-charge, current, and resting time in addition to health. Thus, utilizing EIS data to predict battery capacity requires correcting for all these additional variables, a task that is extremely difficult to handle analytically. This talk utilizes machine-learning methods to estimate the effectiveness of battery capacity prediction from EIS data, leveraging a data set of hundreds of EIS measurements recorded at varying temperature and state-of-charge throughout a 500-day aging study of 32 commercial, large-format NMC-Graphite lithium-ion batteries. Using EIS as input to machine-learning models is complicated by the nonlinear response of impedance to battery health, temperature, and state-of-charge, as well as the collinearity between the impedance response at neighboring frequencies, which can easily lead to overfit models. To train robust models, features from EIS data need to be extracted from the data or some subset of critical frequencies selected. Many approaches for extracting and selecting features from EIS data from electrochemical analysis and machine-learning fields were identified for analysis: using the entire raw spectra; selection of one, two, or many frequencies from the entire spectra; selecting interesting points from the EIS measurement using domain knowledge; fitting EIS with an equivalent-circuit model; calculating statistics on the raw impedance values; and reducing the dimensionality of the data using unsupervised linear (principal component analysis) and non-linear (uniform manifold approximation and projection) methods. These approaches were rigorously compared using a machine-learning pipeline approach, training linear, Gaussian process, and random forest regression models and quantifying performance using cross-validation as well as a held-out test set. An artificial neural network model trained on the raw spectra was also tested. Promising pipelines were fine-tuned via Bayesian hyperparameter optimization using cross-validation loss and training with class-specific weights to counter data set imbalance. The most reliable method for utilizing impedance in this work was the selection of two optimal frequencies through an exhaustive search, resulting in about 2% mean absolute error on test data for both Gaussian process and random forest model architectures. Interrogation of a variety of models reveals critical frequencies of 100 Hz and 103 Hz for this data set, though the optimal set of frequencies is not necessarily intuitive, i.e., the best performing models are not simply those that use impedance at frequencies that have the highest correlation to the relative discharge capacity. The best performing model is an ensemble model, which is able to predict battery capacity with 1.9% mean absolute error for unseen cells using impedance recorded at a variety of temperatures and states-of-charge.

battery↗