Search NASA⌕ Search

SEARCH · Search NASA

Results for “Testing metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8
All sourcesNASA NTRSDOE OSTI
NASA NTRS2002

Technical Interchange Meeting Guidelines Breakout

Along with concept developers, the Systems Evaluation and Assessment (SEA) sub-element of VAMS will develop those scenarios and metrics required for testing the new concepts that reside within the System-Level Integrated Concepts (SLIC) sub-element in the VAMS project. These concepts will come from the NRA process, space act agreements, a university group, and other NASA researchers. The emphasis of those concepts is to increase capacity while at least maintaining the current safety level. The concept providers will initially develop their own scenarios and metrics for self-evaluation. In about a year, the SEA sub-element will become responsible for conducting initial evaluations of the concepts using a common scenario and metric set. This set may derive many components from the scenarios and metrics used by the concept providers. Ultimately, the common scenario\metric set will be used to help determine the most feasible and beneficial concepts. A set of 15 questions and issues, discussed below, pertaining to the scenario and metric set, and its use for assessing concepts, was submitted by the SEA sub-element for consideration during the breakout session. The questions were divided among the three breakout groups. Each breakout group deliberated on its set of questions and provided a report on its discussion.

Fong, Rob↗
DOE OSTI2024

Scalable learning of potentials to predict time-dependent Hartree–Fock dynamics

We propose a framework to learn the time-dependent Hartree–Fock (TDHF) inter-electronic potential of a molecule from its electron density dynamics. Although the entire TDHF Hamiltonian, including the inter-electronic potential, can be computed from first principles, we use this problem as a testbed to develop strategies that can be applied to learn a priori unknown terms that arise in other methods/approaches to quantum dynamics, e.g., emerging problems such as learning exchange–correlation potentials for time-dependent density functional theory. We develop, train, and test three models of the TDHF inter-electronic potential, each parameterized by a four-index tensor of size up to 60 × 60 × 60 × 60. Two of the models preserve Hermitian symmetry, while one model preserves an eight-fold permutation symmetry that implies Hermitian symmetry. Across seven different molecular systems, we find that accounting for the deeper eight-fold symmetry leads to the best-performing model across three metrics: training efficiency, test set predictive power, and direct comparison of true and learned inter-electronic potentials. All three models, when trained on ensembles of field-free trajectories, generate accurate electron dynamics predictions even in a field-on regime that lies outside the training set. To enable our models to scale to large molecular systems, we derive expressions for Jacobian-vector products that enable iterative, matrix-free training.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2010

A Geometric Approach to Modeling Microstructurally Small Fatigue Crack Formation: Simulation and Prediction of Crack Nucleation in AA 7075-T651 - 2

The objective of this paper is to develop further a framework for computationally modeling microstructurally small fatigue crack growth in AA 7075-T651 [1]. The focus is on the nucleation event, when a crack extends from within a second-phase particle into a surrounding grain, since this has been observed to be an initiating mechanism for fatigue crack growth in this alloy. It is hypothesized that nucleation can be predicted by computing a non-local nucleation metric near the crack front. The hypothesis is tested by employing a combination of experimentation and nite element modeling in which various slip-based and energy-based nucleation metrics are tested for validity, where each metric is derived from a continuum crystal plasticity formulation. To investigate each metric, a non-local procedure is developed for the calculation of nucleation metrics in the neighborhood of a crack front. Initially, an idealized baseline model consisting of a single grain containing a semi-ellipsoidal surface particle is studied to investigate the dependence of each nucleation metric on lattice orientation, number of load cycles, and non-local regularization method. This is followed by a comparison of experimental observations and computational results for microstructural models constructed by replicating the observed microstructural geometry near second-phase particles in fatigue specimens. It is found that orientation strongly influences the direction of slip localization and, as a result, in uences the nucleation mechanism. Also, the baseline models, replication models, and past experimental observation consistently suggest that a set of particular grain orientations is most likely to nucleate fatigue cracks. It is found that a continuum crystal plasticity model and a non-local nucleation metric can be used to predict the nucleation event in AA 7075-T651. However, nucleation metric threshold values that correspond to various nucleation governing mechanisms must be calibrated.

Hochhalter, Jake D.↗
NASA NTRS2018

Practical Aspects of Real-Time Modeling for the Learn-To-Fly Concept

Practical aspects of autonomous onboard real-time modeling for the NASA Learn-to-Fly concept are examined using flight data from two subscale test aircraft. A practical real-time global nonlinear aerodynamic modeling method is developed and explained, along with the multiple-input excitation needed for effective real-time modeling. Critical issues for integrating real-time global nonlinear aerodynamic modeling and local linear modeling with the real-time learning adaptive control and guidance elements of the Learn-to-Fly algorithm are identified and discussed. Real-time modeling results from NASA Learn-to-Fly flight demonstrations are presented and evaluated using model fit metrics and prediction tests.

Morelli, Eugene A.↗
DOE OSTI2020

Data-Driven Understanding of Low-to-Moderate Income Customers’ Adoption and Financial Qualification in Community Solar

Through this project, we sought to gather a body of customer data that either proved or disproved our assumption that metrics other than FICO can and should be used to qualify customers for community solar. Using customer data on income, FICO scores, and utility/rent/cell phone repayment history, we developed an alternative metric in order to test in the market at a small scale whether new qualifying metrics could open the shared solar market up to millions of additional households. We demonstrate that the EnergyScore is both more accurate in qualifying customers that will not default, and more inclusive in qualifying low to moderate income households. We are now in the process of utilizing the EnergyScore in a number of pilot projects, building a robust dataset of customer churn and default, and subsequently disseminating this data to more traditional and larger scale financiers and solar developer partners to change the way potential customers are qualified.

14 SOLAR ENERGY↗
NASA NTRSDate not supplied

Cumulative Noise Metric Design Considerations for the Nasa Quesst Community Test Campaign With the X-59 Aircraft

The National Aeronautics and Space Administration (NASA) plans to collect X-59 aircraft dose-response data to inform international regulators in their efforts to establish acceptable noise standards to replace the current ban on overland supersonic commercial flight. For this upcoming community test campaign, the sound level from a supersonic overflight of the X-59 constitutes a single-event dose, and an end-of-day survey after multiple flight events constitutes a cumulative response. This study considers how X-59 community tests could be designed to explore the relationship between multiple single events and cumulative annoyance response. Presently, a day-night level summation of single-event doses formulates the cumulative dose; however, a generalization method may improve the understanding of the linkage between multiple single events and cumulative response. This work utilizes an analysis method that employs a parameter to vary the cumulative dose metric such that it can represent the loudest single event, remain as the day-night level summation, or be more responsive to the number of events. The analysis is demonstrated on simulated dose-response datasets derived from previous NASA field studies and from expectations of X-59 community tests. Results demonstrate that certain dose designs are more efficient than others in determining the analysis parameter.

dose-response modeling↗
NASA NTRSDate not supplied

Cumulative Noise Metric Design Considerations for the NASA Quesst Community Test Campaign With the X-59 Aircraft

The National Aeronautics and Space Administration (NASA) plans to collect X-59 aircraft dose-response data to inform international regulators in their efforts to establish acceptable noise standards to replace the current ban on overland supersonic commercial flight. For this upcoming community test campaign, the sound level from a supersonic overflight of the X-59 constitutes a single-event dose, and an end-of-day survey after multiple flight events constitutes a cumulative response. This study considers how X-59 community tests could be designed to explore the relationship between multiple single events and cumulative annoyance response. Presently, a day-night level summation of single-event doses formulates the cumulative dose; however, a generalization method may improve the understanding of the linkage between multiple single events and cumulative response. This work utilizes an analysis method that employs a parameter to vary the cumulative dose metric such that it can represent the loudest single event, remain as the day-night level summation, or be more responsive to the number of events. The analysis is demonstrated on simulated dose-response datasets derived from previous NASA field studies and from expectations of X-59 community tests. Results demonstrate that certain dose designs are more efficient than others in determining the analysis parameter.

dose-response modeling↗
NASA NTRS2010

Optimization of Regression Models of Experimental Data Using Confirmation Points

A new search metric is discussed that may be used to better assess the predictive capability of different math term combinations during the optimization of a regression model of experimental data. The new search metric can be determined for each tested math term combination if the given experimental data set is split into two subsets. The first subset consists of data points that are only used to determine the coefficients of the regression model. The second subset consists of confirmation points that are exclusively used to test the regression model. The new search metric value is assigned after comparing two values that describe the quality of the fit of each subset. The first value is the standard deviation of the PRESS residuals of the data points. The second value is the standard deviation of the response residuals of the confirmation points. The greater of the two values is used as the new search metric value. This choice guarantees that both standard deviations are always less or equal to the value that is used during the optimization. Experimental data from the calibration of a wind tunnel strain-gage balance is used to illustrate the application of the new search metric. The new search metric ultimately generates an optimized regression model that was already tested at regression model independent confirmation points before it is ever used to predict an unknown response from a set of regressors.

Ulbrich, N.↗
DOE OSTI2023

Data efficiency and extrapolation trends in neural network interatomic potentials

Abstract Recently, key architectural advances have been proposed for neural network interatomic potentials (NNIPs), such as incorporating message-passing networks, equivariance, or many-body expansion terms. Although modern NNIP models exhibit small differences in test accuracy, this metric is still considered the main target when developing new NNIP architectures. In this work, we show how architectural and optimization choices influence the generalization of NNIPs, revealing trends in molecular dynamics (MD) stability, data efficiency, and loss landscapes. Using the 3BPA dataset, we uncover trends in NNIP errors and robustness to noise, showing these metrics are insufficient to predict MD stability in the high-accuracy regime. With a large-scale study on NequIP, MACE, and their optimizers, we show that our metric of loss entropy predicts out-of-distribution error and data efficiency despite being computed only on the training set. This work provides a deep learning justification for probing extrapolation and can inform the development of next-generation NNIPs.

36 MATERIALS SCIENCE↗
DOE OSTI2023

The Effects of Gamma Ray Integrated Dose on a Commercial 65-nm SRAM Device

Here this work shows that the static random access memory (SRAM) error rate for a commercial 65-nm device in a dose rate environment can be highly dependent upon the integrated dose (dose rate × pulse duration). While the typical metric for such testing is dose rate upset (DRU) level in rad(Si)/s, a series of dose rate experiments at Little Mountain Test Facility (LMTF) shows dependence on the integrated dose. The error rate is also found to be dependent on the core voltage, and the preradiation value of the bits. We believe that these effects are explained by a well charge depletion caused by gamma ray photocurrent.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2022

Introduction to Special Section: Machine Learning for Image-based Geologic Interpretation

Image-based geological interpretation has been a labor-intensive and time-consuming process because it requires well-trained geoscientists to identify geological structures, features, and textures from various types of images. These images include scanning electron microscopic images, optical microscopic images, optical photos, resistivity images, seismic volumes, remote-sensing images, etc. With fast-evolving machine learning (ML) technology and computing power in recent decades, computers can achieve nearhuman-level to super-human-level performance with scalable high efficiency in the computer vision field. These technological revolutions facilitated image-based geological interpretation in petroleum exploration and production. For example, a fault picking method applied to 3-D seismic volume data using deep learning can achieve superior performance in comparison to conventional auto-picking methods. In addition, under the new normal of low oil prices, the petroleum industry seeks cost-effective strategies such as automating traditionally labor-intensive processes. Nevertheless, the potential of applying ML to geological image interpretation is still facing a few key challenges including data scarcity, data distribution, poor data and/or label quality, data leakage, learning algorithms, model architecture, training methodologies, testing and evaluation metrics, hyper-parameters optimization, model drift, production deployment, and the like.

58 GEOSCIENCES↗
DOE OSTI2020

Implementation Plan for Combined Heat and Power Systems VOLTTRON Controller: Performance Monitoring and Real-Time Commissioning Algorithm Verification

Building-integrated cooling, heating, and power (CHP) systems are more efficient than conventional systems at providing local power and thermal energy, and favorable fuel prices are bound to spur their increased adoption. However, to realize the full benefit of the CHP systems, we must ensure persistence of energy efficient operations. Much of the inefficiency in the current building operations can be eliminated by use of automated performance monitoring (PM), real-time commissioning verification (CxV) and automated fault detection and diagnostic (AFDD) tools. Automation can help system operators make intelligent decisions. Remote and continuous monitoring of system conditions and performance will enable better management and integration of CHP with existing building systems. Continuous PM, real-time CxV, and AFDD could alleviate burdens for operations staff, enhance operations and maintenance (O&M), and improve reliability of building and CHP systems. To address the O&M challenges and to provide a means to maximize the rate-of-return of building-integrated CHP systems, the Building Technologies Office (BTO) within the U.S. Department of Energy’s (DOE’s) Office of Energy Efficiency and Renewable Energy (EERE) initiated a project to design, develop, and field test a VOLTTRON™-based supervisory controller and associated open-source algorithms. These algorithms will ensure real-time optimal operation of a building-integrated CHP system, support electric grid reliability, and lead to achieving the goal of clean, efficient, reliable, and affordable next-generation integrated energy system. Previous report listed the components for which PM, real-time CxV, and AFDD algorithms will be developed, how the algorithms will be tested, and the metrics that will be used to validate the algorithms and their ease of deployment. Deployment of these algorithms in the field will result in a reduction in energy consumption of between 10% and 20% (for both CHP and conventional building systems). This report builds upon the previous report by detailing the process by which PNNL will implement performance monitoring and real-time commissioning algorithms for CHP systems in conjunction with the use of the VOLTTRON CHP economic dispatch agent in host facilities.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
NASA NTRS1980

Analytical methodology for determination of helicopter IFR precision approach requirements

A systematic analytical approach to the determination of helicopter IFR precision approach requirements is formulated. The approach is based upon the hypothesis that pilot acceptance level or opinion rating of a given system is inversely related to the degree of pilot involvement in the control task. A nonlinear simulation of the helicopter approach to landing task incorporating appropriate models for UH-1H aircraft, the environmental disturbances and the human pilot was developed as a tool for evaluating the pilot acceptance hypothesis. The simulated pilot model is generic in nature and includes analytical representation of the human information acquisition, processing, and control strategies. Simulation analyses in the flight director mode indicate that the pilot model used is reasonable. Results of the simulation are used to identify candidate pilot workload metrics and to test the well known performance-work-load relationship. A pilot acceptance analytical methodology is formulated as a basis for further investigation, development and validation.

Phatak, A. V.↗
NASA NTRS1992

The use of the wavelet cluster analysis for asteroid family determination

The asteroid family determination has been analysis method dependent for a longtime. A new cluster analysis based on the wavelet transform has allowed an automatic definition of families with a degree of significance versus randomness. Actually this method is rather general and can be applied to any kind of structural analysis. We will rather concentrate on the main features of the method. The analysis has been performed on the set of 4100 asteroid proper elements computed by Milani and Knezevic (see Milani and Knezevic 1990). Twenty one families have been found and influence of the chosen metric has been tested. The results have beem compared to Zappala et al.'s ones (see Zappala et al 1990) obtained by the use of a completely different method applied to the same set of data. For the first time, a good overlapping has been found between both method results, not only for the big well known families but also for the smallest ones.

Benjoya, Phillippe↗
NASA NTRS2004

Peer-to-Peer Human-Robot Interaction for Space Exploration

NASA has embarked on a long-term program to develop human-robot systems for sustained, affordable space exploration. To support this mission, we are working to improve human-robot interaction and performance on planetary surfaces. Rather than building robots that function as glorified tools, our focus is to enable humans and robots to work as partners and peers. In this paper. we describe our approach, which includes contextual dialogue, cognitive modeling, and metrics-based field testing.

Fong, Terrence↗
NASA NTRS2017

Machine Learning Approaches to Increasing Value of Spaceflight Omics Databases

The number of spaceflight bioscience mission opportunities is too small to allow all relevant biological and environmental parameters to be experimentally identified. Simulated spaceflight experiments in ground-based facilities (GBFs), such as clinostats, are each suitable only for particular investigations -- a rotating-wall vessel may be 'simulated microgravity' for cell differentiation (hours), but not DNA repair (seconds) -- and introduce confounding stimuli, such as motor vibration and fluid shear effects. This uncertainty over which biological mechanisms respond to a given form of simulated space radiation or gravity, as well as its side effects, limits our ability to baseline spaceflight data and validate mission science. Machine learning techniques autonomously identify relevant and interdependent factors in a data set given the set of desired metrics to be evaluated: to automatically identify related studies, compare data from related studies, or determine linkages between types of data in the same study. System-of-systems (SoS) machine learning models have the ability to deal with both sparse and heterogeneous data, such as that provided by the small and diverse number of space biosciences flight missions; however, they require appropriate user-defined metrics for any given data set. Although machine learning in bioinformatics is rapidly expanding, the need to combine spaceflight/GBF mission parameters with omics data is unique. This work characterizes the basic requirements for implementing the SoS approach through the System Map (SM) technique, a composite of a dynamic Bayesian network and Gaussian mixture model, in real-world repositories such as the GeneLab Data System and Life Sciences Data Archive. The three primary steps are metadata management for experimental description using open-source ontologies, defining similarity and consistency metrics, and generating testing and validation data sets. Such approaches to spaceflight and GBF omics data may soon enable unique insight into which measured phenomena correlate to biological mechanisms that are truly affected by spaceflight conditions; which are most likely to be confounded by other variables; and which are insufficiently characterized, significantly increasing existing and future science return from ISS and spaceflight missions.

Gentry, Diana↗
NASA NTRS2019

Damage Mechanisms in Tapered Composite Structures Under Static and Fatigue Loading

In this work an integrated computational/experimental approach was developed to validate the predictive capabilities of State-of-the-Art (SoA) Progressive Damage Analysis (PDA) methods and tools. Specifically, a tapered composite structure incorporating ply-drops typical in the aerospace industry to spatially vary structural thickness was tested under static tension and cyclic tension fatigue loads. The data acquired from these tests included quantitative metrics such as pre-peak stiffness, peak load, location of delamination damage onset, and growth of delaminations as functions of applied static and fatigue loads. It was shown that the PDA tools were able to predict the pre-peak stiffness and peak load within ±10% of experimental average, thereby meeting and exceeding the pre-defined success criteria. Additionally, it was shown that the PDA tools were able to accurately predict the location of delamination onset and satisfactorily predict delamination growth under static tension loading. Overall, good correlations were achieved between modeling and experiments.

Rao, Prabhakar M.↗
NASA NTRSDate not supplied

TPSAS-NF1676L-35855-DND

UAM Noise Exploratory Meeting (April ’18): there is positive interest in forming a focused working group to define and address noise goals for UAM vehicles. Participants should include stakeholders across industry, government agencies, academia, and community groups; focus efforts on reducing or eliminating the barriers associated with community noise. Key topics of interest include: Tools & Technologies Ground & Flight Testing Human Response & Metrics Regulation & Community Outreach

Stephen A Rizzi↗