SEARCH · Search NASA

Results for “Testing metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11

All sources NASA NTRS DOE OSTI

NASA NTRSDate not supplied

Cumulative Noise Metric Design Considerations for the NASA Quesst Community Test Campaign With the X-59 Aircraft

The National Aeronautics and Space Administration (NASA) plans to collect X-59 aircraft dose-response data to inform international regulators in their efforts to establish acceptable noise standards to replace the current ban on overland supersonic commercial flight. For this upcoming community test campaign, the sound level from a supersonic overflight of the X-59 constitutes a single-event dose, and an end-of-day survey after multiple flight events constitutes a cumulative response. This study considers how X-59 community tests could be designed to explore the relationship between multiple single events and cumulative annoyance response. Presently, a day-night level summation of single-event doses formulates the cumulative dose; however, a generalization method may improve the understanding of the linkage between multiple single events and cumulative response. This work utilizes an analysis method that employs a parameter to vary the cumulative dose metric such that it can represent the loudest single event, remain as the day-night level summation, or be more responsive to the number of events. The analysis is demonstrated on simulated dose-response datasets derived from previous NASA field studies and from expectations of X-59 community tests. Results demonstrate that certain dose designs are more efficient than others in determining the analysis parameter.

dose-response modeling↗

NASA NTRS2010

Optimization of Regression Models of Experimental Data Using Confirmation Points

A new search metric is discussed that may be used to better assess the predictive capability of different math term combinations during the optimization of a regression model of experimental data. The new search metric can be determined for each tested math term combination if the given experimental data set is split into two subsets. The first subset consists of data points that are only used to determine the coefficients of the regression model. The second subset consists of confirmation points that are exclusively used to test the regression model. The new search metric value is assigned after comparing two values that describe the quality of the fit of each subset. The first value is the standard deviation of the PRESS residuals of the data points. The second value is the standard deviation of the response residuals of the confirmation points. The greater of the two values is used as the new search metric value. This choice guarantees that both standard deviations are always less or equal to the value that is used during the optimization. Experimental data from the calibration of a wind tunnel strain-gage balance is used to illustrate the application of the new search metric. The new search metric ultimately generates an optimized regression model that was already tested at regression model independent confirmation points before it is ever used to predict an unknown response from a set of regressors.

Ulbrich, N.↗

DOE OSTI2018

An evaluation of low-cost heuristics for matrix bandwidth and profile reductions

Hundreds of heuristics have been proposed to resolve the problems of bandwidth and profile reductions since the 1960s. We found 132 heuristics that have been applied to these problems in reviews of the literature. Among them, 14 were selected for which no other simulation or comparison revealed that the heuristic could be superseded by any other algorithm in the analyzed articles with respect to bandwidth or profile reduction. We also considered the computational costs of the heuristics during this process. Therefore, these 14 heuristics were selected as potentially being the best low-cost methods to solve the bandwidth and/or profile reduction problems. Results of the 14 selected heuristics are evaluated in this work. For evaluation on the set of test problems, a metric based on the relative percentage distance to the best possible bandwidth or profile is proposed. The most promising heuristics for several application areas are identified. Moreover, it was found that the FNCHC and GPS heuristics showed the best overall results in reducing the bandwidth of symmetric and asymmetric matrices among the evaluated heuristics, respectively. In addition, the NSloan and MPG heuristics showed the best overall results in reducing the profile of symmetric and asymmetric matrices among the heuristics among the evaluated heuristics, respectively.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2015

Testing the Kerr nature of black hole candidates using iron line spectra in the CPR framework

The iron Kα line commonly observed in the X-ray spectrum of both stellar-mass and supermassive black hole (BH) candidates originates from X-ray fluorescence of the inner accretion disk. Accordingly, it can be used to map the spacetime geometry around these objects. In this paper, we extend previous work using the iron Kα line to test the Kerr BH hypothesis. We adopt the Cardoso–Pani–Rico parametrization and we test the possibility of constraining possible deviations from the Kerr solution that can be obtained from observations across the range of BH spins and inclination angles. We confirm previous claims that the iron Kα line is potentially a quite powerful probe for testing the Kerr metric given sufficiently high quality data and with systematics under control, especially in the case of fast-rotating BHs and high inclination angles since both conditions serve to maximize relativistic effects. We find that some geometric perturbations from Kerr geometry manifest more strongly in the iron line profile than others. While the perturbation parameter ϵ{sub 3}{sup t} can be well constrained by the iron line profile, an orthogonal data set is necessary to constrain departures from Kerr geometry in ϵ{sub 3}{sup r}.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2023

Data efficiency and extrapolation trends in neural network interatomic potentials

Abstract Recently, key architectural advances have been proposed for neural network interatomic potentials (NNIPs), such as incorporating message-passing networks, equivariance, or many-body expansion terms. Although modern NNIP models exhibit small differences in test accuracy, this metric is still considered the main target when developing new NNIP architectures. In this work, we show how architectural and optimization choices influence the generalization of NNIPs, revealing trends in molecular dynamics (MD) stability, data efficiency, and loss landscapes. Using the 3BPA dataset, we uncover trends in NNIP errors and robustness to noise, showing these metrics are insufficient to predict MD stability in the high-accuracy regime. With a large-scale study on NequIP, MACE, and their optimizers, we show that our metric of loss entropy predicts out-of-distribution error and data efficiency despite being computed only on the training set. This work provides a deep learning justification for probing extrapolation and can inform the development of next-generation NNIPs.

36 MATERIALS SCIENCE↗

DOE OSTI2023

The Effects of Gamma Ray Integrated Dose on a Commercial 65-nm SRAM Device

Here this work shows that the static random access memory (SRAM) error rate for a commercial 65-nm device in a dose rate environment can be highly dependent upon the integrated dose (dose rate × pulse duration). While the typical metric for such testing is dose rate upset (DRU) level in rad(Si)/s, a series of dose rate experiments at Little Mountain Test Facility (LMTF) shows dependence on the integrated dose. The error rate is also found to be dependent on the core voltage, and the preradiation value of the bits. We believe that these effects are explained by a well charge depletion caused by gamma ray photocurrent.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2015

Quantitative adaptation analytics for assessing dynamic systems of systems: LDRD Final Report

Our society is increasingly reliant on systems and interoperating collections of systems, known as systems of systems (SoS). These SoS are often subject to changing missions (e.g., nation- building, arms-control treaties), threats (e.g., asymmetric warfare, terrorism), natural environments (e.g., climate, weather, natural disasters) and budgets. How well can SoS adapt to these types of dynamic conditions? This report details the results of a three year Laboratory Directed Research and Development (LDRD) project aimed at developing metrics and methodologies for quantifying the adaptability of systems and SoS. Work products include: derivation of a set of adaptability metrics, a method for combining the metrics into a system of systems adaptability index (SoSAI) used to compare adaptability of SoS designs, development of a prototype dynamic SoS (proto-dSoS) simulation environment which provides the ability to investigate the validity of the adaptability metric set, and two test cases that evaluate the usefulness of a subset of the adaptability metrics and SoSAI for distinguishing good from poor adaptability in a SoS. Intellectual property results include three patents pending: A Method For Quantifying Relative System Adaptability, Method for Evaluating System Performance, and A Method for Determining Systems Re-Tasking.

99 GENERAL AND MISCELLANEOUS↗

DOE OSTI2022

Introduction to Special Section: Machine Learning for Image-based Geologic Interpretation

Image-based geological interpretation has been a labor-intensive and time-consuming process because it requires well-trained geoscientists to identify geological structures, features, and textures from various types of images. These images include scanning electron microscopic images, optical microscopic images, optical photos, resistivity images, seismic volumes, remote-sensing images, etc. With fast-evolving machine learning (ML) technology and computing power in recent decades, computers can achieve nearhuman-level to super-human-level performance with scalable high efficiency in the computer vision field. These technological revolutions facilitated image-based geological interpretation in petroleum exploration and production. For example, a fault picking method applied to 3-D seismic volume data using deep learning can achieve superior performance in comparison to conventional auto-picking methods. In addition, under the new normal of low oil prices, the petroleum industry seeks cost-effective strategies such as automating traditionally labor-intensive processes. Nevertheless, the potential of applying ML to geological image interpretation is still facing a few key challenges including data scarcity, data distribution, poor data and/or label quality, data leakage, learning algorithms, model architecture, training methodologies, testing and evaluation metrics, hyper-parameters optimization, model drift, production deployment, and the like.

58 GEOSCIENCES↗

DOE OSTI2020

Implementation Plan for Combined Heat and Power Systems VOLTTRON Controller: Performance Monitoring and Real-Time Commissioning Algorithm Verification

Building-integrated cooling, heating, and power (CHP) systems are more efficient than conventional systems at providing local power and thermal energy, and favorable fuel prices are bound to spur their increased adoption. However, to realize the full benefit of the CHP systems, we must ensure persistence of energy efficient operations. Much of the inefficiency in the current building operations can be eliminated by use of automated performance monitoring (PM), real-time commissioning verification (CxV) and automated fault detection and diagnostic (AFDD) tools. Automation can help system operators make intelligent decisions. Remote and continuous monitoring of system conditions and performance will enable better management and integration of CHP with existing building systems. Continuous PM, real-time CxV, and AFDD could alleviate burdens for operations staff, enhance operations and maintenance (O&M), and improve reliability of building and CHP systems. To address the O&M challenges and to provide a means to maximize the rate-of-return of building-integrated CHP systems, the Building Technologies Office (BTO) within the U.S. Department of Energy’s (DOE’s) Office of Energy Efficiency and Renewable Energy (EERE) initiated a project to design, develop, and field test a VOLTTRON™-based supervisory controller and associated open-source algorithms. These algorithms will ensure real-time optimal operation of a building-integrated CHP system, support electric grid reliability, and lead to achieving the goal of clean, efficient, reliable, and affordable next-generation integrated energy system. Previous report listed the components for which PM, real-time CxV, and AFDD algorithms will be developed, how the algorithms will be tested, and the metrics that will be used to validate the algorithms and their ease of deployment. Deployment of these algorithms in the field will result in a reduction in energy consumption of between 10% and 20% (for both CHP and conventional building systems). This report builds upon the previous report by detailing the process by which PNNL will implement performance monitoring and real-time commissioning algorithms for CHP systems in conjunction with the use of the VOLTTRON CHP economic dispatch agent in host facilities.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

NASA NTRS1980

Analytical methodology for determination of helicopter IFR precision approach requirements

A systematic analytical approach to the determination of helicopter IFR precision approach requirements is formulated. The approach is based upon the hypothesis that pilot acceptance level or opinion rating of a given system is inversely related to the degree of pilot involvement in the control task. A nonlinear simulation of the helicopter approach to landing task incorporating appropriate models for UH-1H aircraft, the environmental disturbances and the human pilot was developed as a tool for evaluating the pilot acceptance hypothesis. The simulated pilot model is generic in nature and includes analytical representation of the human information acquisition, processing, and control strategies. Simulation analyses in the flight director mode indicate that the pilot model used is reasonable. Results of the simulation are used to identify candidate pilot workload metrics and to test the well known performance-work-load relationship. A pilot acceptance analytical methodology is formulated as a basis for further investigation, development and validation.

Phatak, A. V.↗

NASA NTRS1992

The use of the wavelet cluster analysis for asteroid family determination

The asteroid family determination has been analysis method dependent for a longtime. A new cluster analysis based on the wavelet transform has allowed an automatic definition of families with a degree of significance versus randomness. Actually this method is rather general and can be applied to any kind of structural analysis. We will rather concentrate on the main features of the method. The analysis has been performed on the set of 4100 asteroid proper elements computed by Milani and Knezevic (see Milani and Knezevic 1990). Twenty one families have been found and influence of the chosen metric has been tested. The results have beem compared to Zappala et al.'s ones (see Zappala et al 1990) obtained by the use of a completely different method applied to the same set of data. For the first time, a good overlapping has been found between both method results, not only for the big well known families but also for the smallest ones.

Benjoya, Phillippe↗

NASA NTRS2004

Peer-to-Peer Human-Robot Interaction for Space Exploration

NASA has embarked on a long-term program to develop human-robot systems for sustained, affordable space exploration. To support this mission, we are working to improve human-robot interaction and performance on planetary surfaces. Rather than building robots that function as glorified tools, our focus is to enable humans and robots to work as partners and peers. In this paper. we describe our approach, which includes contextual dialogue, cognitive modeling, and metrics-based field testing.

Fong, Terrence↗

NASA NTRS2017

Machine Learning Approaches to Increasing Value of Spaceflight Omics Databases

The number of spaceflight bioscience mission opportunities is too small to allow all relevant biological and environmental parameters to be experimentally identified. Simulated spaceflight experiments in ground-based facilities (GBFs), such as clinostats, are each suitable only for particular investigations -- a rotating-wall vessel may be 'simulated microgravity' for cell differentiation (hours), but not DNA repair (seconds) -- and introduce confounding stimuli, such as motor vibration and fluid shear effects. This uncertainty over which biological mechanisms respond to a given form of simulated space radiation or gravity, as well as its side effects, limits our ability to baseline spaceflight data and validate mission science. Machine learning techniques autonomously identify relevant and interdependent factors in a data set given the set of desired metrics to be evaluated: to automatically identify related studies, compare data from related studies, or determine linkages between types of data in the same study. System-of-systems (SoS) machine learning models have the ability to deal with both sparse and heterogeneous data, such as that provided by the small and diverse number of space biosciences flight missions; however, they require appropriate user-defined metrics for any given data set. Although machine learning in bioinformatics is rapidly expanding, the need to combine spaceflight/GBF mission parameters with omics data is unique. This work characterizes the basic requirements for implementing the SoS approach through the System Map (SM) technique, a composite of a dynamic Bayesian network and Gaussian mixture model, in real-world repositories such as the GeneLab Data System and Life Sciences Data Archive. The three primary steps are metadata management for experimental description using open-source ontologies, defining similarity and consistency metrics, and generating testing and validation data sets. Such approaches to spaceflight and GBF omics data may soon enable unique insight into which measured phenomena correlate to biological mechanisms that are truly affected by spaceflight conditions; which are most likely to be confounded by other variables; and which are insufficiently characterized, significantly increasing existing and future science return from ISS and spaceflight missions.

Gentry, Diana↗

NASA NTRS2019

Damage Mechanisms in Tapered Composite Structures Under Static and Fatigue Loading

In this work an integrated computational/experimental approach was developed to validate the predictive capabilities of State-of-the-Art (SoA) Progressive Damage Analysis (PDA) methods and tools. Specifically, a tapered composite structure incorporating ply-drops typical in the aerospace industry to spatially vary structural thickness was tested under static tension and cyclic tension fatigue loads. The data acquired from these tests included quantitative metrics such as pre-peak stiffness, peak load, location of delamination damage onset, and growth of delaminations as functions of applied static and fatigue loads. It was shown that the PDA tools were able to predict the pre-peak stiffness and peak load within ±10% of experimental average, thereby meeting and exceeding the pre-defined success criteria. Additionally, it was shown that the PDA tools were able to accurately predict the location of delamination onset and satisfactorily predict delamination growth under static tension loading. Overall, good correlations were achieved between modeling and experiments.

Rao, Prabhakar M.↗

NASA NTRSDate not supplied

TPSAS-NF1676L-35855-DND

UAM Noise Exploratory Meeting (April ’18): there is positive interest in forming a focused working group to define and address noise goals for UAM vehicles. Participants should include stakeholders across industry, government agencies, academia, and community groups; focus efforts on reducing or eliminating the barriers associated with community noise. Key topics of interest include: Tools & Technologies Ground & Flight Testing Human Response & Metrics Regulation & Community Outreach

Stephen A Rizzi↗

NASA NTRS2004

Mars Exploration Rover (MER) project environmental assurance program

This paper presents a comprehensive summary of the environmental assurance program for the MER project. A series of test and analysis metrics are generated. Selections of the numerous lessons that have been learned from implementation of the MER environmental assurance program are documented in this paper. They include both technical and programmatic lessons that would be helpful in improving implementation of the environmental program for future projects.

Hoffman, Alan R↗

NASA NTRSDate not supplied

Commercial Supersonic Fan Noise Modeling

This presentation describes the development of an empirical noise model for a two-stage fan designed for a supersonic commercial aircraft. The existing fan noise models are reviewed and compared with noise data from a wind tunnel experiment where a model scale two-stage fan was tested. An error metric was developed to quantify how well the measurements and empirical model match. This error metric was then used to determine new coefficients and parameters for a version of the fan model that is a good match for the experimental data.

supersonics, acoustics, turbofan↗

NASA NTRSDate not supplied

The International Space Station, Optical Communications, and Delay Tolerant Networking: Towards A Solar System Internet Architecture

As Delay Tolerant Networking (DTN) matures as a software product its use cases have extended to infrastructural and architectural studies, bringing DTN closer to a widespread, operational technology. While it is known that a DTN can be configured, given complete information about a system of nodes, to provision a system with a complete network capability, what remains is to tackle practical considerations such as scalability, networking best practices, and how to establish service providers. In this paper, we document progress towards a scalable DTN architecture that was tested across multiple organizational and project boundaries; our approach was tested across nodes flying on the International Space Station (ISS) connected to a ground network. The network consists of four main network areas: the ISS, the White Sands Complex (WSC), the Mission Cloud Platform (MCP), and NASA’s Huntsville Operations Support Center (HOSC). The connections on the ground (WSC, MCP, and HOSC) are straightforward technically, but represent administrative and non-technical challenges that had to be overcome. The connection from the ISS to WSC was realized through a hybrid optical/RF means. In particular, NASA’s Integrated Laser Communications Relay Demonstration (LCRD) can form a relay from lowEarth-orbit (LEO) to the Earth, and was used in conjunction with the LCRD LEO User Modem and Amplifier Terminal (ILLUMA-T) on the ISS to form a link to one of three geographically diverse ground stations. A major goal was to achieve communications while keeping the ISS nodes unaware of ground station choices and scheduling. These disparate links were networked using High-rate DTN (HDTN), which enabled the desired architecture. The network architecture, tests, HDTN performance metrics, and general observations made are all discussed in detail, including a discussion on the successful utilization of the DTN security standard known as DTN Bundle Protocol Security (BPSec). We conclude with suggestions for next steps, and in particular focus on extending this architecture to the Near Space Network (NSN) and the upcoming LunaNet.

Alan Hylton↗