Search NASA⌕ Search

SEARCH · Search NASA

Results for “Testing metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 631 records · Page 35

Machine Learning-Based Regression Models for Ironmaking Blast Furnace Automation

Computational fluid dynamics (CFD)-based simulation has been the traditional way to model complex industrial systems and processes. One very large and complex industrial system that has benefited from CFD-based simulations is the steel blast furnace system. The problem with the CFD-based simulation approach is that it tends to be very slow for generating data. The CFD-only approach may not be fast enough for use in real-time decisionmaking. To address this issue, in this work, the authors propose the use of machine learning techniques to train and test models based on data generated via CFD simulation. Regression models based on neural networks are compared with tree-boosting models. In particular, several areas (tuyere, raceway, and shaft) of the blast furnace are modeled using these approaches. The results of the model training and testing are presented and discussed. The obtained R 2 metrics are, in general, very high. The results appear promising and may help to improve the efficiency of operator and process engineer decisionmaking when running a blast furnace.

97 MATHEMATICS AND COMPUTING↗

Characterization of nonGaussian atmospheric turbulence for prediction of aircraft response statistics

Mathematical expressions were derived for the exceedance rates and probability density functions of aircraft response variables using a turbulence model that consists of a low frequency component plus a variance modulated Gaussian turbulence component. The functional form of experimentally observed concave exceedance curves was predicted theoretically, the strength of the concave contribution being governed by the coefficient of variation of the time fluctuating variance of the turbulence. Differences in the functional forms of response exceedance curves and probability densities also were shown to depend primarily on this same coefficient of variation. Criteria were established for the validity of the local stationary assumption that is required in the derivations of the exceedance curves and probability density functions. These criteria are shown to depend on the relative time scale of the fluctuations in the variance, the fluctuations in the turbulence itself, and on the nominal duration of the relevant aircraft impulse response function. Metrics that can be generated from turbulence recordings for testing the validity of the local stationary assumption were developed.

Mark, W. D.↗

Forecasting of Weather Effects for the Deep Space Network

This paper presents a proposed approach for Ka-band link management for deep space applications using daily weather forecasts and discusses the tools that will be employed for operations. Performance metrics are also presented. The proposed approach will be tested in a two-year experiment campaign.

Deep Space Network (DSN)↗

An Analysis of the Environments of Intense Convective Systems in West Africa in 2003

We investigated the local- and regional-scale thermodynamical and dynamical environments associated with intense convective systems in West Africa during 2003. We identified convective system cases from TRMM microwave imagery, classifying each case by the system minimum 85-GHz brightness temperature and by the estimated elapsed time of propagation from high terrain. The speed of the mid-level jet, the magnitude of the low-level shear, and the surface equivalent potential temperature (theta(sub e)) were greater for the intense cases compared to the non-intense cases, although the differences between the means tended to be small, less than 3K for surface theta(sub e). Hypothesis testing of a series of commonly used intensity prediction metrics resulted in significant results only for low-level metrics such as convective available potential energy and not for any of the mid- or upper-level metrics such as 700-hPa theta(sub e). None of the environmental variables or intensity metrics by themselves or in combination appeared to be reliable direct predictors of intensity. In the regional scale analysis, the majority of intense convective systems occurred in the surface baroclinic zone where surface theta(sub e) exceeded 344 K and the 700-hPa zonal wind speeds were less than -6/ms. Fewer intense cases compared to non-intense cases were associated with African easterly wave troughs. Fewer than 25% of our cases occurred in environments with detectable Saharan dust loads, and the results for intense and non-intense cases were similar. Our results for the regional analysis were consistent with the seasonal movement of the WAM and the intertropical front, regional differences in topography, and AEW energetics.

Nicholls, Stephen D.↗

INTEGRITY - Integrated Human Exploration Mission Simulation Facility

It is proposed to develop a high-fidelity ground facility to carry out long-duration human exploration mission simulations. These would not be merely computer simulations - they would in fact comprise a series of actual missions that just happen to stay on earth. These missions would include all elements of an actual mission, using actual technologies that would be used for the real mission. These missions would also include such elements as extravehicular activities, robotic systems, telepresence and teleoperation, surface drilling technology-all using a simulated planetary landscape. A sequence of missions would be defined that get progressively longer and more robust, perhaps a series of five or six missions over a span of 10 to 15 years ranging in duration from 180 days up to 1000 days. This high-fidelity ground facility would operate hand-in-hand with a host of other terrestrial analog sites such as the Antarctic, Haughton Crater, and the Arizona desert. Of course, all of these analog mission simulations will be conducted here on earth in 1-g, and NASA will still need the Shuttle and ISS to carry out all the microgravity and hypogravity science experiments and technology validations. The proposed missions would have sufficient definition such that definitive requirements could be derived from them to serve as direction for all the program elements of the mission. Additionally, specific milestones would be established for the "launch" date of each mission so that R&D programs would have both good requirements and solid milestones from which to .build their implementation plans. Mission aspects that could not be directly incorporated into the ground facility would be simulated via software. New management techniques would be developed for evaluation in this ground test facility program. These new techniques would have embedded metrics which would allow them to be continuously evaluated and adjusted so that by the time the sequence of missions is completed, the best management techniques will have been developed, implemented, and validated. A trained cadre of managers experienced with a large, complex program would then be available.

Henninger, Donald L.↗

Design and Testing of a Hall Effect Thruster with 3D Printed Channel and Propellant Distributor

The UAH-78AM is a low-power Hall effect thruster developed at the University of Alabama in Huntsville with channel walls and a propellant distributor manufactured using 3D printing. The goal of this project is to assess the feasibility of using unconventional materials to produce a low-cost functioning Hall effect thruster and consider how additive manufacturing can expand the design space and provide other benefits. A version of the thruster was tested at NASA Glenn Research Center to obtain performance metrics and to validate the ability of the thruster to produce thrust and sustain a discharge. An overview of the thruster design and transient performance measurements are presented here. Measured thrust ranged from 17.2 millinewtons to 30.4 millinewtons over a discharge power of 280 watts to 520 watts with an anode I (sub SP)(Specific Impulse) range of 870 seconds to 1450 seconds. Temperature limitations of materials used for the channel walls and propellant distributor limit the ability to run the thruster at thermal steady-state.

Hall thrusters↗

Infrared Imaging using Sparse Microresonator Arrays and Compressed Sensing

In this work, we present an uncooled infrared (IR) thermal imager featuring a sparse array of resonant pixels that can be monolithically integrated with the readout electronics for long-term operation in high-temperature environments. Each of the detector pixels is made of an IR-sensitive GaN microresonator developed to be operated at up to 500 0C. The pixels are employed in sparse array configuration to allow for routing the signal carrying traces and to ease the routing from pixel to circuit, as well as to reduce the power consumption at the expense of moderate post-processing time and increased complexity to recover the image. Three 128 by 128 sparse resonator arrays with the array densities of 10%, 25% and 37.5% are designed, and used in conjunction with the Compressed Sensing (CS) method to reconstruct complex test images from highly incomplete data. Capabilities and performance metrics of the recovery technique are explored.

Rais-Zadeh, Mina↗

Thermal Infrared Detector Sparse Array for NASA Planetary Applications

In this work, we present an uncooled infrared (IR) thermal imager featuring a sparse array of resonant pixels that can be monolithically integrated with the readout electronics for long-term operation in hightemperature environments. Each of the detector pixels is made of an IR-sensitive GaN microresonator developed to be operated at up to 500 0C. The pixels are employed in sparse array configuration to allow for routing the signal carrying traces and to ease the routing from pixel to circuit, as well as to reduce the power consumption. A 128´128 sparse resonator array with the array density of 37.5% is designed and used in conjunction with the Compressed Sensing method to reconstruct complex test images from highly incomplete data. Capabilities and performance metrics of the recovery technique are explored.

Rais-Zadeh, Mina↗

Sensor Fault Detection in Smart Extraterrestrial Habitats Using Unsupervised Learning

Various types of sensors are needed to monitor the health state of smart deep-space habitats. However, measured data can be affected by sensor faults, which influence the health management system and consequently the decision-making. In this paper, an unsupervised learning approach based on convolutional autoencoders (CAEs) is developed to detect anomalies in temperature and pressure sensors. The proposed method is systematically investigated using a habitat simulator (HabSim). Several illustrative examples are demonstrated in the nominal and hazardous states of the habitat, including micrometeorite impact and fire scenarios. The performance of the proposed method using CAEs is compared with that of existing methods using auto-associative neural networks (AANNs) and variational autoencoders. This comparison is based on typical evaluation metrics, including precision, recall, F1 score, training time, and testing time. The effect of temperature–pressure coupling on the detection performance of CAEs and AANNs is explored by training different data-driven models, including one with temperature sensors, one with pressure sensors, and one with both temperature and pressure sensors. The effect of the number of faulty sensors on the performance of CAEs is studied, as with an increase in the number of faulty sensors, redundant information among the sensors is reduced. The capability of CAEs to change the number of sensors without redesigning the network architecture and retraining the neural network is investigated and demonstrated. The capabilities and limitations of the proposed solution are discussed.

Zixin Wang↗

Beyond Widgets -- Systems Incentive Programs for Utilities

Utility incentive programs remain one of the most significant means of deploying commercialized, but underutilized building technologies to scale. However, these programs have been largely limited to component-based products (e.g., lamps, RTUs). While some utilities do provide ‘custom’ incentive programs with whole building and system level technical assistance, these programs require deeper levels of analysis, resulting in higher program costs. This results in custom programs being restricted to utilities with greater resources, and are typically applied mainly to large or energy-intensive facilities, leaving much of the market without cost effective access and incentives for these solutions. In addition, with increasingly stringent energy codes, cost effective component-based solutions that achieve significant savings are dwindling. Building systems (e.g., integrated façade, HVAC and/or lighting solutions) can deliver higher savings that translate into large sector-wide savings if deployed at the scale of these programs. However, systems application poses a number of challenges – baseline energy use must be defined and measured; the metrics for energy and performance must be defined and tested against; in addition, system savings must be validated under well understood conditions. This paper presents a sample of findings of a project to develop validated utility incentive program packages for three specific integrated building systems, in collaboration with Xcel Energy (CO, MN), ComEd, and a consortium of California Public Owned Utilities (CA POUs) (Northern California Power Agency(NCPA) and the Southern California Public Power Authority(SCPPA)). Furthermore, these program packages consist of system specifications, system performance, M&V protocols, streamlined assessment methods, market assessment and implementation guidance.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

SU-E-T-76: A Software System to Monitor VMAT Plan Complexity in a Large Radiotherapy Centre

Purpose: To develop a system that analyses and reports the complexity of Volumetric Modulated Arc Therapy (VMAT) plans to aid in the decision making for streamlining patient specific dosimetric quality assurance (QA) tests. Methods: A software system, Delcheck, was developed in-house to calculate VMAT plan and delivery complexity using the treatment delivery file. Delcheck has the functionality to calculate multiple plan complexity metrics including the Li-Xing Modulation Index (LI-MI), multiplicative combination of Leaf Travel and Modulation Complexity Score (LTMCSv), Monitor Units per prescribed dose (MU/D) and the delivery complexity index (MIt) that incorporates the modulation of dose rate, leaf speed and gantry speed. Delcheck includes database functionality to store and compare plan metrics for a specified treatment site. The overall plan and delivery complexity is assessed based on the 95% conformance limit of the complexity metrics as Similar, More or Less complex. The functionality of the software was tested using 42 prostate conventional, 10 prostate SBRT and 15 prostate bed VMAT plans generated for an Elekta linear accelerator. Results: The mean(σ) of LI-MI for conventional, SBRT and prostate bed plans were 1690(486), 3215.4(1294) and 3258(982) respectively. The LTMCSv of the studied categories were 0.334(0.05), 0.325(0.07) and 0.3112(0.09). The MU/D of the studied categories were 2.4(0.4), 2.7(0.7) and 2.5(0.5). The MIt of the studied categories were 21.6(3.4), 18.2(3.0) and 35.9(6.6). The values of the complexity metrics show that LI-MI appeared to resolve the plan complexity better than LTMCSv and MU/D. The MIt value increased as the delivery complexity increased. Conclusion: The developed software was shown to be working as expected. In studied treatment categories Prostate bed plans are more complex in both plan and delivery and SBRT is more complex in plan and less complex in delivery as demonstrated by LI-MI and MIt. This project was funded through a Cancer Council NSW Project Grant (RG14-11)

60 APPLIED LIFE SCIENCES↗

ISRU Soil Mechanics Vacuum Facility: Soil Bin Preparation and Simulant Strength Characterization

Testing in relevant environments is key to exploration mission hardware development. This is true on both the component level (in early development) and system level (in late development stages). During ISRU missions the hardware will interface with the soil (digging, roving, etc) in a vacuum environment. A relevant test environment will therefore involve a vacuum chamber with a controlled, conditioned simulant bed. However, in earth-based granular media, such as lunar soil simulant, gases trapped within the material pore structures and water adsorbed to all particle surfaces will release when exposed to vacuum. Early vacuum testing has shown that this gas release can occur violently, which loosens and weakens the simulant, altering the consolidation state. The Vacuum Facility #13, a mid-size chamber (3.66m tall, 1.5m inner diameter) at the NASA Glenn Research Center has been modified to create a soil mechanics test facility. A 0.64m deep by 0.914m square metric ton bed of lunar simulant was placed under vacuum using a variety of pumping techniques. Both GRC-3 and LHT-3M simulant types have been used. An electric cone penetrometer was used to measure simulant strength properties at vacuum including: cohesion, friction angle, bulk density and shear modulus. Simulant disruptions, caused by off gassing, affected the strength properties, but could be mitigated by reducing pump rate. No disruptions were observed at pressures below 2.5Torr, regardless of the pump rate. However, slow off gassing of the soil lead to long test times, a full week, to reach 10-5Torr. This work highlights the need for robotic machine-simulant hardware and operations in vacuum to expeditiously perform (sub-)systems tests.

Kleinhenz, Julie↗

SAPHIRE 8 Quality Assurance Software Metrics Report

The purpose of this review of software metrics is to examine the quality of the metrics gathered in the 2010 IV&V and to set an outline for results of updated metrics runs to be performed. We find from the review that the maintenance of accepted quality standards presented in the SAPHIRE 8 initial Independent Verification and Validation (IV&V) of April, 2010 is most easily achieved by continuing to utilize the tools used in that effort while adding a metric of bug tracking and resolution. Recommendations from the final IV&V were to continue periodic measurable metrics such as McCabe's complexity measure to ensure quality is maintained. The four software tools used to measure quality in the IV&V were CodeHealer, Coverage Validator, Memory Validator, Performance Validator, and Thread Validator. These are evaluated based on their capabilities. We attempted to run their latest revisions with the newer Delphi 2010 based SAPHIRE 8 code that has been developed and was successful with all of the Validator series of tools on small tests. Another recommendation from the IV&V was to incorporate a bug tracking and resolution metric. To improve our capability of producing this metric, we integrated our current web reporting system with the SpiraTest test management software purchased earlier this year to track requirements traceability.

97 MATHEMATICS AND COMPUTING↗

Delta-Gravity (Physics of Particles and Nuclei, v. 44(2))

We present a model of the gravitational field based on two symmetric tensors. The equations of motion of test particles are derived: Massive particles do not follow a geodesic but massless particles trajectories are null geodesics of an effective metric. Outside matter, the predictions of the model coincide exactly with General Relativity, so all classical tests are satisfied. In Cosmology, we get accelerated expansion without a cosmological constant. Additionally, we study the quantization of the model. The main result being that the Effective Action is finite and receives one loop corrections only.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

A Summary of Results from Vertical Drop Testing of Hybrid III and WIAMan ATDs

With the development and maturation of the Urban Air Mobility (UAM) market, many new types of electric vertical take-off and landing (eVTOL) vehicles will be flying in the national airspace carrying goods, people or conducting operations for a variety of missions. These types of vehicles are unlike current aircraft due to their novel design and operational profile. Several considerations must be examined in areas including noise, comfort and safety in order for these vehicles to be utilized and accepted into the current airspace system. Researchers at NASA Langley Research Center (LaRC) have conducted sub-scale and full-scale tests on representative eVTOL airframes and seats under a variety of dynamic impact conditions. These tests were conducted to generate data necessary to inform the development of standards in the areas specific to crashworthiness of eVTOL vehicle systems and safety. The data in this report relates to occupant responses obtained during a test campaign utilizing various makes, models, and sizes of Anthropomorphic Test Devices (ATD’s, a.k.a. crash test dummies) undergoing vertical impacts in a variety of seats. The data is intended to provide occupant behavior response and injury metrics for several anticipated impact scenarios that may occur in eVTOL operations. This report will present test data highlighting the effects of several variables on the test results. Discussions on the ATD sizes, along with comparisons between different ATD makes and types will be included. The performance of an in-house developed energy absorbing seat will be detailed, and discussions pertaining to the applicability in various loading conditions will be presented. Finally, a discussion as to the applicability of the tested results to eVTOL full-scale conditions will be included.

Dynamic Drop Testing↗

Development and Testing of an ISRU Soil Mechanics Vacuum Test Facility

For extraterrestrial missions, earth based testing in relevant environments is key to successful hardware development. This is true for both early component level development and system level integration. For In-Situ Resource Utilization (ISRU) on the moon, hardware must interface with the surface material, or regolith, in a vacuum environment. A relevant test environment will therefore involve a vacuum chamber with a controlled, properly conditioned bed of lunar regolith simulant. However, in earth-based granular media, such as lunar regolith simulant, gases trapped within the material pore structures and water adsorbed to all particle surfaces will release when exposed to vacuum. Early vacuum testing has shown that this gas release can occur violently, which loosens and weakens the simulant, altering the consolidation state. A mid-size chamber (3.66 m tall, 1.5 m inner diameter) at the NASA Glenn Research Center has been modified to create a soil mechanics test facility. A 0.64 m deep by 0.914 m square metric ton bed of lunar simulant was placed under vacuum using a variety of pumping techniques. Both GRC-3 and LHT-3M simulant types were used. Data obtained from an electric cone penetrometer can be used to determine strength properties at vacuum including: cohesion, friction angle, bulk density and shear modulus. Simulant disruptions, caused by off-gassing, affected the strength properties, but could be mitigated by reducing pump rate. No disruptions were observed at pressures below 2.5 Torr, regardless of the pump rate. The slow off-gassing of the soil at low pressure lead to long test times; a full week to reach 10(exp -5) Torr. Robotic soil manipulation would enable multiple ISRU hardware test within the same vacuum cycle. The feasibility of a robotically controlled auger and tamper was explored at vacuum conditions.

lunar regolith↗

Cumulative $\chi^2$ Metric for VALID for ENDF/B-VII.1 and ENDF/B-VIII.0 in SCALE 6.3b9

Nuclear data testing is generally performed in a manner similar to validation: a series of critical benchmark experiments is modeled, and the differences between calculated and expected results are examined to understand the performance of a code and cross section set. This process is repeated for multiple sets of experiments with different transport codes and sometimes with different datasets. Occasionally, data processing differences are also important in understanding the differences in calculated results. Relevant experts then meet periodically to compare results and set priorities for improvement in the next round of data updates. This process has been largely effective, despite the confounding factors of different benchmark sets, application areas, sponsors, nationalities, and the resulting different priorities.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

The MPC&A Questionnaire

The questionnaire is the instrument used for recording performance data on the nuclear material protection, control, and accountability (MPC&A) system at a nuclear facility. The performance information provides a basis for evaluating the effectiveness of the MPC&A system. The goal for the questionnaire is to provide an accurate representation of the performance of the MPC&A system as it currently exists in the facility. Performance grades for all basic MPC&A functions should realistically reflect the actual level of performance at the time the survey is conducted. The questionnaire was developed after testing and benchmarking the material control and accountability (MC&A) system effectiveness tool (MSET) in the United States. The benchmarking exercise at the Idaho National Laboratory (INL) proved extremely valuable for improving the content and quality of the early versions of the questionnaire. Members of the INL benchmark team identified many areas of the questionnaire where questions should be clarified and areas where additional questions should be incorporated. The questionnaire addresses all elements of the MC&A system. Specific parts pertain to the foundation for the facility's overall MPC&A system, and other parts pertain to the specific functions of the operational MPC&A system. The questionnaire includes performance metrics for each of the basic functions or tasks performed in the operational MPC&A system. All of those basic functions or tasks are represented as basic events in the MPC&A fault tree. Performance metrics are to be used during completion of the questionnaire to report what is actually being done in relation to what should be done in the performance of MPC&A functions.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗