Search NASA⌕ Search

SEARCH · Search NASA

Results for “Testing metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 703 records · Page 39

Titan Atmospheric Entry Radiative Heating

Detailed spectrally and spatially resolved radiance has been measured in the Electric Arc Shock Tube for conditions relevant to Titan entry, varying atmospheric composition, free-stream density (equivalent to altitude) and shock velocity. Permutations in atmospheric composition include 1.1, 2, 5 and 8.6 CH4 by mole with a balance of N2 and 1.5 CH4 0.5 Ar 98 N2 by mole, which is consistent with the current understanding of Titan's atmosphere. The effect of gas impurities identified in previous shock tube studies were also examined by testing in pure N2 and deliberate addition of air to the CH4N2 mixtures. The test campaign measured radiation at velocities from 4.7 kms to 8 kms and free-stream pressures from 0.1 to 0.47 Torr. These conditions cover a range of potential trajectories for flight missions, including a direct ballistic trajectory, a fly by or an extremely high speed entry. Radiances measured in this work are substantially larger compared to that reported both in past EAST test campaigns and other shock tube facilities. Depending on the metric used for comparison, the discrepancy can be as high as an order of magnitude. Potential causes for the discrepancy, such as the effect of oxygen due to Air leakage, gas composition and purity are discussed. The present work provides a new benchmark set of data to replace those published in previous studies.

Earth↗

Titan Atmospheric Entry Radiative Heating

Detailed spectrally and spatially resolved radiance has been measured in the Electric Arc Shock Tube for conditions relevant to Titan entry, varying atmospheric composition, free-stream density (equivalent to altitude) and shock velocity. Permutations in atmospheric composition include 1.1, 2, 5 and 8.6 CH4 by mole with a balance of N2 and 1.5 CH4 0.5 Ar 98 N2 by mole, which is consistent with the current understanding of Titan's atmosphere. The effect of gas impurities identified in previous shock tube studies were also examined by testing in pure N2 and deliberate addition of air to the CH4N2 mixtures. The test campaign measured radiation at velocities from 4.7 kms to 8 kms and free-stream pressures from 0.1 to 0.47 Torr. These conditions cover a range of potential trajectories for flight missions, including a direct ballistic trajectory, a fly by or an extremely high speed entry. Radiances measured in this work are substantially larger compared to that reported both in past EAST test campaigns and other shock tube facilities. Depending on the metric used for comparison, the discrepancy can be as high as an order of magnitude. Potential causes for the discrepancy, such as the effect of oxygen due to Air leakage, gas composition and purity are discussed. The present work provides a new benchmark set of data to replace those published in previous studies.

Brandis, A. M↗

Analyzing Natural Language Context in Human-Machine Teaming using Supervised Machine Learning

Building a foundation for trustworthiness and trust verification in multi-asset teaming is the research challenge of Autonomy Teaming and TRAjectories for Complex Trusted Operational Reliability (ATTRACTOR). The Design Reference Mission (DRM) for ATTRACTOR is a search and rescue mission objective governed by a multi-member team consisting of human and machine operators. A crucial component to the effort is the communication between humans and autonomous agents throughout both planning and execution stages of the mission. Intuitive communication methods and modalities are posited as critical enablers for certifying trust and trustworthiness. This paper reports on the data collection and analysis conducted in support of the Human Informed Natural-language GANs Evaluation (HINGE)project to attain explainable and trusted communication between human-machine assets. Two identically curated image description datasets were acquired for HINGE, both consisting of two unique input modalities (typed vs. verbal) and retrieved in two distinct contexts (general vs. specific). The gathered datasets were assessed and compared using Parts-of-Speech (POS)features, sentence similarity metrics, and linguistic analysis. Then, the datasets were modeled and tested separately and in combination with one another using machine learning algorithms. The comparison and testing results reveal a superior dataset, by which a preferred context and input is understood, for generating image representations of missing persons using a Generative Adversarial Network (GAN).

Bryan A Barrows↗

Surrogate Models for TREAT Transient Calculations

The Transient Reactor Test Facility (TREAT) at Idaho National Laboratory (INL) is currently preparing for resumption of operation after more than 20 years in a long-term standby state. In parallel with preparation for operations and a new transient testing program, the DOE-NE Advanced Modeling and Simulation program (NEAMS) has been supporting the development of three-dimensional transient multi-physics modeling capabilities within the reactor physics application MAMMOTH [1]. Recently, the MAMMOTH reactor physics team has begun development of uncertainty and sensitivity analysis for the TREAT core. This research will be used to support the analysis of reactor operation and experiment planning. Traditional approaches, i.e., Monte Carlo simulations, for such tasks require a large number of model evaluations under both steady state and transient conditions. However, exhaustive sets of sampling runs are simply not practical here since evaluations of each model can take a substantial amount of time. A solution to this problem is to build a surrogate model employing a smaller number of model evaluations. The surrogate model is designed to provide an accurate and fast approximation to the detailed TREAT model. In this manner, the surrogate model can be used in place of the TREAT transient model for uncertainty and sensitivity analysis at a fraction of the computational cost. The high dimensional model representation (HDMR) [2] method is employed here for approximating the responses of interest in terms of functions of lower dimensions. This method has been implemented in RAVEN [3] and all TREAT transient calculations are performed using the multiphysics code MAMMOTH. In addition, a code interface has also been developed in order to couple RAVEN with MAMMOTH. The surrogate model is then tested and compared for the MAMMOTH model of the TREAT startup testing program Transient Test #15 core [4], and its accuracy is measured by its ability to estimate the mean, coefficient of variance, skewness, excess kurtosis and probability density function of the outputs of Transient Test #15 model, as well as root mean square error metric.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Material Accountancy Performance Indicator Toolkit

The MAPIT - Material Accountancy Performance Indicator Toolkit includes statistical tests and calculations based on open literature that calculate safeguards metrics. The tool can be used with user supplied datasets or with the several included datasets which are generated from models that are maintained at Sandia National Laboratories. The models are not included with the software. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525. SAND2021-0467 O

Shoman, Nathan↗

Deep Energy Retrofit Performance Metric Comparison: Eight California Case Studies

In this paper we will present the results of monitored annual energy use data from eight residential Deep Energy Retrofit (DER) case studies using a variety of performance metrics. For each home, the details of the retrofits were analyzed, diagnostic tests to characterize the home were performed and the homes were monitored for total and individual end-use energy consumption for approximately one year. Annual performance in site and source energy, as well as carbon dioxide equivalent (CO 2 e) emissions were determined on a per house, per person and per square foot basis to examine the sensitivity to these different metrics. All eight DERs showed consistent success in achieving substantial site energy and CO 2 e reductions, but some projects achieved very little, if any source energy reduction. This problem emerged in those homes that switched from natural gas to electricity for heating and hot water, resulting in energy consumption dominated by electricity use. This demonstrates the crucial importance of selecting an appropriate metric to be used in guiding retrofit decisions. Also, due to the dynamic nature of DERs, with changes in occupancy, size, layout, and comfort, several performance metrics might be necessary to understand a project’s success.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

SECA Coal-Based Systems - LGFCS

LGFCS is developing an integrated planar (IP) SOFC technology for mega-watt scale power generation including the potential for use in highly efficient, economically competitive central generation power plant facilities fuel by coal synthesis gas. This Department of Energy Solid-State Energy Conversion Alliance (SECA) program has been aimed at achieving further cell and stack technical advancements and assessing the readiness of the LGFCS SOFC stack technology to be scaled to larger-scale demonstrations as a path to commercialization. Significant progress was achieved in reducing to practice a higher performance and lower cost cell technology, identifying and overcoming degradation mechanisms, confirming the structural capability of the porous substrate for reliability, maturing the strip design for improved flow to allow high fuel utilization operation while minimizing degradation mechanisms and obtaining full scale block testing at 19 kW under representative conditions for eventual product and meeting SECA degradation metrics. The SECA program has played a key role within the overall LGFCS development program in setting the foundation of the technology to justify the progression of the technology to the next level of technology readiness testing.

Goettler, Richard↗

Cy-Phy ADS: Cyber Physical Anomaly Detection Framework for EV Charging Systems

Today’s large-scale Electric Vehicle (EV) infrastructures are heavily dependent on information communication technologies to maintain their operation and to support communication within sub-system components as well as the outside world. These technologies are vulnerable to various cyber and physical threats. Timely identification and mitigation of these threats are critical for improving human safety, avoiding economic losses, and preventing catastrophic system failures. By addressing this, our work presents a ResNet Autoencoder (AE) based Cyber-Physical Anomaly Detection System (Cy-Phy ADS) for detecting anomalies in EV Controller Area Network (CAN) protocol communication. It consists of four main components: Cyber-Physical Feature Extractor, ResNet AE-based Anomaly Detection Framework, Cyber-Physical Health Metric (CPHM), and Visualization Dashboard. The presented framework was trained and tested using CAN data collected from the EV charging system testbed at the Idaho National Laboratory. The presented Cy-Phy ADS compared against six widely used unsupervised anomaly detection algorithms: One Class Support Vector Machine (OCSVM), Variational Autoencoder (VAE), LSTM Autoencoder (LSTM AE), Isolation Forest (IForest), Principle Component Analysis (PCA) and Local Outlier Factor (LOF). Here the presented approach showed the highest accuracy among the compared methods. Further, the proposed approach showed comparable performance in terms of precision, F1, and False positive rate. It also showed the lowest training and inference time compared to the neural network-based baseline algorithms compared against with. Additionally, the Cy-Phy ADS has advantages such as unsupervised training, the ability to provide a holistic metric for system health characterization, and non-linear feature extraction.

99 GENERAL AND MISCELLANEOUS↗

An empirical comparison of a dynamic software testability metric to static cyclomatic complexity

This paper compares the dynamic testability prediction technique termed 'sensitivity analysis' to the static testability technique termed cyclomatic complexity. The application that we chose in this empirical study is a CASE generated version of a B-737 autoland system. For the B-737 system we analyzed, we isolated those functions that we predict are more prone to hide errors during system/reliability testing. We also analyzed the code with several other well-known static metrics. This paper compares and contrasts the results of sensitivity analysis to the results of the static metrics.

Voas, Jeffrey M.↗

Staying Current: A Community Readiness Framework for Marine Energy Applied to River Current Energy in Alaska

Marine energy-including wave, tidal, and river current energy-can provide a local energy source for rural and remote communities. Marine energy has the potential to bolster self-sufficiency and create economic opportunities while preserving ecological integrity. Communities may be interested in deploying, testing, and advancing these early-stage technologies to meet their needs. However, limited capacity, workforce constraints, and other barriers can challenge development. To better understand a community's interest in and preparedness for marine energy, we developed a suite of 150 'metrics of readiness.' Organized across seven categories-technical, social, environmental, strategic, governance, economic, financial-and 29 subcategories, the metrics provide a holistic perspective beyond the technical aspects of an energy device. We conducted a desktop application of the metrics of readiness for Igiugig, Alaska. The metrics were applied retrospectively for two points in time: before (2009) and after (2018) in-stream testing of a river current energy device. By documenting changes among categories and subcategories of the metrics, our results show the evolving nature of community readiness for river current energy. They also illustrate how our interdisciplinary framework captures the investment in environmental effects research and commitment to strategic planning that occurred in Igiugig. In future applications, we envision the framework could be used to foster public engagement in marine energy, collaborate with communities in project development, shape capacity building activities, prioritize investments, and inform research needs. While our study focuses on enabling river current energy in Alaska, the metrics of readiness have the potential to inform implementation of other renewable technologies with communities in new geographies.

13 HYDRO ENERGY↗

Technology Development for Modular, Low-Cost, High-Temperature Recuperators for Supercritical CO 2 Power Cycles (Final Report)

The program’s focus was to evaluate, advance, and demonstrate recuperator design concepts, materials, and fabrication methods that facilitate the commercial availability of compact and low-cost recuperators. Emphasis was placed on recompression closed Brayton cycles having a turbine inlet temperature exceeding 700°C and differential pressures on the order of 200 bar and were scalable to accommodate plant sizes from 10 to 1,000 MWe. The primary objectives of the first budget period, which included conceptual design, preliminary development, and prototype fabrication and testing, were accomplished. A new recuperator design concept, stacked-sheet heat exchanger (SSHX), was realized during the project. The design was taken from paper to hardware and successfully tested. The SSHX prototype recuperator met or exceeded program thermal, hydraulic and performance/cost metrics goals. The first budget period was extended to retire risk in scaling the recuperator from the kWt to MWt thermal capacity scale. The fabrication and design requirements for the budget period one extension were met. Delays associated with the Covid-19 pandemic prevented the fabrication and testing of the industrial scale prototype.

14 SOLAR ENERGY↗

Phenomenological constraints on Lemaître-Tolman-Bondi cosmological inhomogeneities from solar system dynamics

We, first, analytically work out the long-term, i.e. averaged over one orbital revolution, perturbations on the orbit of a test particle moving in a local Fermi frame induced therein by the cosmological tidal effects of the inhomogeneous Lemaître-Tolman-Bondi (LTB) model. The LTB solution has recently attracted attention, among other things, as a possible explanation of the observed cosmic acceleration without resorting to dark energy. Then, we phenomenologically constrain both the parameters K1 doteq ddot frakR / frakR and K2 doteq ddot frakR' / frakR' of the LTB metric in the Fermi frame by using different kinds of solar system data. The corrections Δdot varpi to the standard Newtonian/Einsteinian precessions of the perihelia of the inner planets recently estimated with the EPM ephemerides, compared to our predictions for them, yield preliminarily K{sub 1} = (4±8) × 10{sup −26} s{sup −2}, K{sub 2} = (3±7) × 10{sup −23} s{sup −2}. The residuals of the Cassini-based Earth-Saturn range, compared with the numerically integrated LTB range signature, allow to preliminarily obtain K{sub 1} ≈ K{sub 2} ≈ 10{sup −27} s{sup −2}. Actually, the LTB effects should be explicitly modeled in the ephemerides softwares, so that the entire planetary and spacecraft data sets should be accordingly re-processed. The LTB-induced distortions of the orbit of a typical object of the Oort cloud with respect to the commonly accepted Newtonian picture, based on the observations of the comet showers from that remote region of the solar system, point towards K{sub 1} ≈ K{sub 2}∼<10{sup −30}−10{sup −32} s{sup −2}. Such figures have to be compared with those inferred from cosmological data which are of the order of K{sub 1} ≈ K{sub 2} = −4 × 10{sup −36} s{sup −2}.

79 ASTRONOMY AND ASTROPHYSICS↗

Comparison of Test Procedures and Energy Efficiency Criteria in Selected International Standards & Labeling Programs for Copy Machines, External Power Supplies, LED Displays, Residential Gas Cooktops and Televisions

This report presents a technical review of international minimum energy performance standards (MEPS), voluntary and mandatory energy efficiency labels and test procedures for five products being considered for new or revised MEPS in China: copy machines, external power supply, LED displays, residential gas cooktops and flat-screen televisions. For each product, an overview of the scope of existing international standards and labeling programs, energy values and energy performance metrics and description and detailed summary table of criteria and procedures in major test standards are presented.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Multi-Dimensional Calibration of Impact Dynamic Models

NASA Langley, under the Subsonic Rotary Wing Program, recently completed two helicopter tests in support of an in-house effort to study crashworthiness. As part of this effort, work is on-going to investigate model calibration approaches and calibration metrics for impact dynamics models. Model calibration of impact dynamics problems has traditionally assessed model adequacy by comparing time histories from analytical predictions to test at only a few critical locations. Although this approach provides for a direct measure of the model predictive capability, overall system behavior is only qualitatively assessed using full vehicle animations. In order to understand the spatial and temporal relationships of impact loads as they migrate throughout the structure, a more quantitative approach is needed. In this work impact shapes derived from simulated time history data are used to recommend sensor placement and to assess model adequacy using time based metrics and orthogonality multi-dimensional metrics. An approach for model calibration is presented that includes metric definitions, uncertainty bounds, parameter sensitivity, and numerical optimization to estimate parameters to reconcile test with analysis. The process is illustrated using simulated experiment data.

Horta, Lucas G.↗

Development of Biomimetic Membranes for Near Zero PC Power Plant Emissions

The first objective of this project was to develop, evaluate and compare two different CO2 separation (capture) systems. The second was to carry the preferred solution to pre-pilot development and testing. To achieve these objectives we undertook several infrastructure enabling elements: (1) development of a preferred catalyst coupled with its immobilization onto a microporous polymer membrane, (2) design and development of a microporous membrane-based, contained liquid membrane permeator and a membrane-based absorber/desorber apparatus, (3) development of a resin-wafer electrodialytic absorber/desorber apparatus, (4) development and demonstration of a pre-treatment process to condition the feed gas stream, (5) and development of computer modeling of the components and of the integrated system. The first technology was an enzyme catalyzed, membrane supported, contained liquid membrane apparatus whose gas capture was pressure/vacuum and temperature driven. A first embodiment was as a permeator, i.e. a combined absorber/desorber in a single housing. The second embodiment was as discrete absorber and desorber units. The second technology was an enzyme catalyzed, ion exchange resin wafer electrodialytically-based separation. For each of these technologies the objective was to design, manufacture, test and demonstrate the apparatus, first in the laboratory and then at pre-pilot scale, and to run it for sufficient time at the pre-pilot scale to demonstrate stability even in the face of upset. Tests would include several ranks of coal, which had been appropriately pre-treated to remove NOx, SOx and particles, to a pre-determined acceptance level, as a basis for demonstrating efficient CO{sub 2} capture. The pre-pilot tests would be run at the Energy and Environmental Research Center (EERC) in North Dakota. A larger scale test (400m{sup 2} test unit) would later be run also at EERC. An economic goal was to compare the cost of CO{sub 2} capture by each of these methods with values obtained when using MEA (monoethanolamine) as a baseline case. Other metrics included capital and operating expense, parasitic loss and cost of electricity. A final goal was to carry out an initial examination of market forces to understand what barriers to entry for installation of CO{sub 2} capture equipment might exist and their relative importance.

01 COAL, LIGNITE, AND PEAT↗

Advanced HVAC Humidity Control for Hot-Humid Climates

During this project we develop and validate a cost-effective, integrated control solution to improve humidity control and comfort for energy-efficient homes in hot-humid climates. This study focuses on developing a strategy that is effective, field tested, and practical for builders to install with minimal disruption to standard practices. A successful solution would simplify the transition to high-performance humidity control and be the basis for design and installation guidance. By relying on the central system as a starting point, the strategy employed minimizes system complexity and cost for builders, while improving comfort and operating cost for homeowners. The solution strategy was to coordinate the cooling, dehumidification, and ventilation functions of central, ducted HVAC systems to better control indoor humidity, improve occupant thermal comfort, and capture energy savings. The primary strategic goals were to: (1) optimize dehumidification by the central air-conditioning system, particularly during part-load conditions, using conventional equipment with modified control settings and lower system airflows; (2) maximize ventilation during heating/cooling on-cycles, to “bank” and condition outdoor ventilation air, and minimize ventilation during off-cycles; (3) quantify the effectiveness and energy impact of the dehumidification and ventilation strategies, while identifying a metric that would be useful to evaluate latent effectiveness. For the test houses in our study, located in Richmond Hill, Georgia; Houston, Texas; and Monroe, Louisiana we observed: (1) the indoor humidity did not exceed 60% RH during the monitored cooling season for 99% of the time in Richmond Hill, 96% of the time in Houston, and 90% of the time in Monroe; (2) the dehumidification strategy improved the steady-state latent capacity of the HVAC system at design conditions by 16% to 49% at the Houston test house and by 28% to 71% at the Monroe test house, depending on which mode the system was operating in; and (3) the good results at the test houses were primarily due to the amount of time the air-conditioning system operated in ramping or dehumidification modes, or both, particularly during the early cooling season. This study demonstrates that air conditioners or heat pumps with a single-stage compressor can provide good humidity control without the need for a two-stage or variable-stage compressor system. The airflow and control settings for ramping and dehumidification modes are critical to control indoor humidity in hot-humid climates, particularly during part-load and shoulder season conditions. The dehumidification strategy used in this study did not jeopardize the mechanical reliability of the cooling equipment. The strategies used in this study are applicable across various equipment brands, models, and efficiency levels, and also applicable to a broad range of homes in hot-humid climates. Results will vary by specific equipment, location, and house configuration and construction.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Constraining Cosmology with Simulation-based inference and Optical Galaxy Cluster Abundance

We test the robustness of simulation-based inference (SBI) in the context of cosmological parameter estimation from galaxy cluster counts and masses in simulated optical datasets. We construct ``simulations'' using analytical models for the galaxy cluster halo mass function (HMF) and for the observed richness (number of observed member galaxies) to train and test the SBI method. We compare the SBI parameter posterior samples to those from an MCMC analysis that uses the same analytical models to construct predictions of the observed data vector. The two methods exhibit comparable performance, with reliable constraints derived for the primary cosmological parameters, ($\Omega_m$ and $\sigma_8$), and richness-mass relation parameters. We also perform out-of-domain tests with observables constructed from galaxy cluster-sized halos in the Quijote simulations. Again, the SBI and MCMC results have comparable posteriors, with similar uncertainties and biases. Unsurprisingly, upon evaluating the SBI method on thousands of simulated data vectors that span the parameter space, SBI exhibits worsened posterior calibration metrics in the out-of-domain application. We note that such calibration tests with MCMC is less computationally feasible and highlight the potential use of SBI to stress-test limitations of analytical models, such as in the use for constructing models for inference with MCMC.

79 ASTRONOMY AND ASTROPHYSICS↗

Semantic Pattern Analysis for Verbal Fluency Based Assessment of Neurological Disorders

In this paper, we present preliminary results of semantic pattern analysis of verbal fluency tests used for assessing cognitive psychological and neuropsychological disorders. We posit that recent advances in semantic reasoning and artificial intelligence can be combined to create a standardized computer-aided diagnosis tool to automatically evaluate and interpret verbal fluency tests. Towards that goal, we derive novel semantic similarity (phonetic, phonemic and conceptual) metrics and present the predictive capability of these metrics on a de-identified dataset of participants with and without neurological disorders.

concept reasoning↗