Search NASA⌕ Search

SEARCH · Search NASA

Results for “Testing metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Integrated Mechanical System Pods (CRADA Final Report)

LBNL provided input to setting the requirements related to indoor air quality and contributed to the development of metrics and test plans to evaluate prototypes of the mechanical pods, which were developed by other members of the project team. The testing of prototypes was conducted by and at Syracuse University, another project partner. LBNL also collaborated with RMI to develop a basic plan for data collection when the pods are deployed for field testing in Phase 2.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2020

Design, Development and Testing Plan for Energy Efficiency Algorithms Related to Building-Integrated Cooling, Heating, and Power Systems

Building-integrated cooling, heating and power (CHP) systems are more efficient than conventional systems at providing local power and thermal energy, and favorable fuel prices are bound to spur their increased adoption. It is estimated that the total CHP technical potential in the commercial building sector is approximately 76 GW, but only a small fraction of it is actually deployed. Advanced control and monitoring systems that realize the full potential of these CHP systems can transform this capacity into an asset for both the owner and the grid and also accelerate the adoption of these systems in the building sector. However, to realize the full benefit of the CHP systems, we must ensure persistence of energy efficient operations. It has been demonstrated by several studies that as much as 30% of building energy use is wasted. Much of the waste results from the inability to anticipate load variations (design day vs. non-design day operations; diurnal variation of loads), identify improper operation, and efficiently operate, control and maintain building systems. Moreover, many buildings, even those with adequate control infrastructure, lack knowledgeable staff to operate and maintain energy systems. Operators of buildings with CHP systems will constantly need to make tradeoffs (economic dispatch) between onsite power generation and purchasing power, between onsite heat production from distributed generation versus heating/cooling from conventional systems, and different passive and active load management measures. Much of the inefficiency in the current building operations can be eliminated by use of automated performance monitoring (PM), real-time commissioning verification (CxV) and automated fault detection and diagnostic (AFDD) tools. Automation can help system operators make intelligent decisions. Remote and continuous monitoring of system conditions and performance will enable better management and integration of CHP with existing building systems. Continuous PM, real-time CxV and AFDD could alleviate burdens for operations staff, enhance operations and maintenance (O&M), and improve reliability of building and CHP systems. To address the O&M challenges and to provide a means to maximize the rate-of-return of building-integrated CHP systems, the Building Technologies Office (BTO) within the U.S. Department of Energy’s (DOE’s) Office of Energy Efficiency and Renewable Energy (EERE) initiated a project to design, develop, and field test a VOLTTRON™-based supervisory controller and associated open-source algorithms. These algorithms will ensure real-time optimal operation of a building-integrated CHP system, support electric grid reliability, and lead to achieving the goal of a clean, efficient, reliable and affordable next-generation integrated energy systems. This report lists the components for which PM, real-time CxV and AFDD algorithms will be developed, how the algorithms will be tested, and metrics that will be used to validate the algorithms and easy of deployment. Deployment of these algorithms in the field will result in savings reduction in energy consumption of between 10% and 20%.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2026

Modeling Framework for Data Center

This chapter highlights the critical need for advanced modeling of data centers due to their rapidly increasing energy consumption and impact on grid reliability. Driven by the demand for AI applications, data centers are projected to consume a significant portion of US energy by 2028, putting stress on an already challenged power grid. The chapter emphasizes the importance of "fast" time-scale models to understand the dynamic interactions between data centers and the grid, especially given the rapid power fluctuations of AI workloads. It outlines a modeling framework that includes both offline and real-time EMT domain simulations, detailing the necessary representations for various components like utility interfaces, transformers, IT loads, UPS, cooling loads, Battery Energy Storage Systems (BESS), generators, protection systems, and higher-level control systems. While standard simulation tools like PSCAD offer basic models, custom development is often required to accurately capture the unique and fast-changing behaviors of modern data centers. The chapter also discusses key metrics and test cases for validating these models, focusing on transient load responses, protection relay coordination, and demand flexibility. Finally, it addresses the challenges of modeling large-scale data centers, such as computational complexity and the trade-off between model fidelity and practicality, suggesting hybrid modeling approaches as a solution. The overarching goal is to create a robust framework that helps assess data center impacts on grid stability, identify vulnerabilities, and inform the development of standards for reliable integration of these large loads into the bulk power system.

25 ENERGY STORAGE↗
DOE OSTI2025

Foundational Dataset for Developing Large-Sample Stream Temperature Models in the Conterminous United States

This dataset provides inputs, evaluation results, and trained weights from a large-sample Long Short-Term Memory (LSTM) model designed to predict daily stream temperatures across unregulated river reaches in the conterminous United States (CONUS). It includes dynamic meteorological and hydrologic forcings, static physiographic attributes, and model outputs from cross-validation experiments spanning 300 basins. It supports reproducible modeling, direct application for new basins, and provides data suitable for integration with reservoir and river simulations under current and future climates. It contains two .zip files described below · RQ-AI_runs.zip: Model outputs from 10-fold cross-validation experiments, including observed and predicted daily stream temperatures, along with test performance metrics for water years 2017–2019. Two versions are included: 1. Model trained and validated using subbasin-area weighted dynamic features. 2. Model trained and validated using whole-basin area weighted dynamic features. · RQ-AI_inputs.zip: Collection of all formatted dynamic and static predictor datasets (meteorological, hydrologic, and physiographic features) used in model training and analysis. Detailed instructions and data structure is held at the following GitLab repository: https://code.ornl.gov/tempwise/training.

Gomez-Velez, Jesus [Oak Ridge National Laboratory ↗
DOE OSTI2022

Validation of Calibrated k–ε Model Parameters for Jet-in-Crossflow

Previous efforts determined a set of calibrated, optimal model parameter values for Reynolds-averaged Navier–Stokes (RANS) simulations of a compressible jet in crossflow (JIC) using a $k–ε$ turbulence model. These parameters were derived by comparing simulation results to particle image velocimetry (PIV) data of a complementary JIC experiment under a limited set of flow conditions. Here, a $k–ε$ model using both nominal and calibrated parameters is validated against PIV data acquired from a much wider variety of JIC cases, including a realistic flight vehicle. The results from the simulations using the calibrated model parameters showed considerable improvements over those using the nominal values, even for cases that were not used in the calibration procedure that defined the optimal parameters. This improvement is demonstrated using a number of quality metrics that test the spatial alignment of the jet core, the magnitudes of multiple flow variables, and the location and strengths of vortices in the counter-rotating vortex cores on the PIV planes. These results suggest that the calibrated parameters have applicability well outside the specific flow case used in defining them and that with the right model parameters, RANS solutions for the JIC can be improved significantly over those obtained from the nominal model.

42 ENGINEERING↗
DOE OSTI2024

TCR-H: explainable machine learning prediction of T-cell receptor epitope binding on unseen datasets

Artificial-intelligence and machine-learning (AI/ML) approaches to predicting T-cell receptor (TCR)-epitope specificity achieve high performance metrics on test datasets which include sequences that are also part of the training set but fail to generalize to test sets consisting of epitopes and TCRs that are absent from the training set, i.e., are ‘unseen’ during training of the ML model. We present TCR-H, a supervised classification Support Vector Machines model using physicochemical features trained on the largest dataset available to date using only experimentally validated non-binders as negative datapoints. TCR-H exhibits an area under the curve of the receiver-operator characteristic (AUC of ROC) of 0.87 for epitope ‘hard splitting’ (i.e., on test sets with all epitopes unseen during ML training), 0.92 for TCR hard splitting and 0.89 for ‘strict splitting’ in which neither the epitopes nor the TCRs in the test set are seen in the training data. Furthermore, we employ the SHAP (Shapley additive explanations) eXplainable AI (XAI) method for post hoc interrogation to interpret the models trained with different hard splits, shedding light on the key physiochemical features driving model predictions. TCR-H thus represents a significant step towards general applicability and explainability of epitope:TCR specificity prediction.

60 APPLIED LIFE SCIENCES↗
NASA NTRS1977

General relativity and satellite orbits - The motion of a test particle in the Schwarzschild metric

The motion of a satellite of negligible mass about a massive spherically symmetric object in a space of Schwarzschild geometry is calculated by using ordinary Newtonian methods for an inverse-square gravitational field acted upon by an inverse-cube disturbing function. The disturbing function is expressed in terms of the Keplerian elements of the orbit and substituted in the Lagrange planetary equations. The equations can be integrated to calculate the displacement in position of the satellite due to the relativistic potential. For example, the Beacon Explorer C satellite is found to be displaced by about 17.4 cm after one revolution. For Mercury the advance of the perihelion is about 85 km after one orbit, and the maximum periodic displacement is about 13 km.

Rubincam, D. P.↗
NASA NTRS1987

Methodology requirements for intelligent systems architecture

The methodology required for the development of the 'intelligent system architecture' of distributed computer systems which integrate standard data processing capabilities with symbolic processing to provide powerful and highly autonomous adaptive processing capabilities must encompass three elements: (1) a design knowledge capture system, (2) computer-aided engineering, and (3) verification and validation metrics and tests. Emphasis must be put on the earliest possible definition of system requirements and the realistic definition of allowable system uncertainties. Methodologies must also address human factor issues.

Grant, Terry↗
NASA NTRS1991

Performance analysis of LAN bridges and routers

Bridges and routers are used to interconnect Local Area Networks (LANs). The performance of these devices is important since they can become bottlenecks in large multi-segment networks. Performance metrics and test methodology for bridges and routers were not standardized. Performance data reported by vendors is not applicable to the actual scenarios encountered in an operational network. However, vendor-provided data can be used to calibrate models of bridges and routers that, along with other models, yield performance data for a network. Several tools are available for modeling bridges and routers - Network II.5 was used. The results of the analysis of some bridges and routers are presented.

Hajare, Ankur R.↗
NASA NTRS1993

The impact of flying qualities on helicopter operational agility

Flying qualities standards are formally set to ensure safe flight and therefore reflect minimum, rather than optimum, requirements. Agility is a flying quality but relates to operations at high, if not maximum, performance. While the quality metrics and test procedures for flying, as covered for example in ADS33C, may provide an adequate structure to encompass agility, they do not currently address flight at high performance. This is also true in the fixed-wing world and a current concern in both communities is the absence of substantiated agility criteria and possible conflicts between flying qualities and high performance. AGARD is sponsoring a working group (WG19) title 'Operational Agility' that deals with these and a range of related issues. This paper is condensed from contributions by the three authors to WG19, relating to flying qualities. Novel perspectives on the subject are presented including the agility factor, that quantifies performance margins in flying qualities terms; a new parameter, based on maneuver acceleration is introduced as a potential candidate for defining upper limits to flying qualities. Finally, a probabilistic analysis of pilot handling qualities ratings is presented that suggests a powerful relationship between inherent airframe flying qualities and operational agility.

Padfield, Gareth D.↗
NASA NTRS1994

Intelligent vehicle control: Opportunities for terrestrial-space system integration

For 11 years the Department of Defense has cooperated with a diverse array of other Federal agencies including the National Institute of Standards and Technology, the Jet Propulsion Laboratory, and the Department of Energy, to develop robotics technology for unmanned ground systems. These activities have addressed control system architectures supporting sharing of tasks between the system operator and various automated subsystems, man-machine interfaces to intelligent vehicles systems, video compression supporting vehicle driving in low data rate digital communication environments, multiple simultaneous vehicle control by a single operator, path planning and retrace, and automated obstacle detection and avoidance subsystem. Performance metrics and test facilities for robotic vehicles were developed permitting objective performance assessment of a variety of operator-automated vehicle control regimes. Progress in these areas will be described in the context of robotic vehicle testbeds specifically developed for automated vehicle research. These initiatives, particularly as regards the data compression, task sharing, and automated mobility topics, also have relevance in the space environment. The intersection of technology development interests between these two communities will be discussed in this paper.

Shoemaker, Charles↗
NASA NTRS1995

Automated spectral classification and the GAIA project

Two dimensional spectral types for each of the stars observed in the global astrometric interferometer for astrophysics (GAIA) mission would provide additional information for the galactic structure and stellar evolution studies, as well as helping in the identification of unusual objects and populations. The classification of the large quantity generated spectra requires that automated techniques are implemented. Approaches for the automatic classification are reviewed, and a metric-distance method is discussed. In tests, the metric-distance method produced spectral types with mean errors comparable to those of human classifiers working at similar resolution. Data and equipment requirements for an automated classification survey, are discussed. A program of auxiliary observations is proposed to yield spectral types and radial velocities for the GAIA-observed stars.

Lasala, Jerry↗
NASA NTRS1995

NASA/WVU Software Research Laboratory, 1995

In our second year, the NASA/WVU Software Research Lab has made significant strides toward analysis and solution of major software problems related to V&V activities. We have established working relationships with many ongoing efforts within NASA and continue to provide valuable input into policy and decision-making processes. Through our publications, technical reports, lecture series, newsletters, and resources on the World-Wide-Web, we provide information to many NASA and external parties daily. This report is a summary and overview of some of our activities for the past year. This report is divided into 6 chapters: Introduction, People, Support Activities, Process, Metrics, and Testing. The Introduction chapter (this chapter) gives an overview of our project beginnings and targets. The People chapter focuses on new people who have joined the Lab this year. The Support chapter briefly lists activities like our WWW pages, Technical Report Series, Technical Lecture Series, and Research Quarterly newsletter. Finally, the remaining four chapters discuss the major research areas that we have made significant progress towards producing meaningful task reports. These chapters can be regarded as portions of drafts of our task reports.

Sabolish, George J.↗
NASA NTRS2005

Lightweight Nonmetallic Thermal Protection Materials Technology (LNTPMT) Project

Contents include the following: Exploration systems research and technology program structure. Project objective. Overview of project. Candidate thermal protection system (PS) materials. Definition of reference missions and space environments. Technical performance metrics (TPMs).Testing (types of tests). Conclusion.

Flynn, Kevin↗
NASA NTRS2013

Aircraft Engine Gas Path Diagnostic Methods: Public Benchmarking Results

Recent technology reviews have identified the need for objective assessments of aircraft engine health management (EHM) technologies. To help address this issue, a gas path diagnostic benchmark problem has been created and made publicly available. This software tool, referred to as the Propulsion Diagnostic Method Evaluation Strategy (ProDiMES), has been constructed based on feedback provided by the aircraft EHM community. It provides a standard benchmark problem enabling users to develop, evaluate and compare diagnostic methods. This paper will present an overview of ProDiMES along with a description of four gas path diagnostic methods developed and applied to the problem. These methods, which include analytical and empirical diagnostic techniques, will be described and associated blind-test-case metric results will be presented and compared. Lessons learned along with recommendations for improving the public benchmarking processes will also be presented and discussed.

Simon, Donald L.↗
NASA NTRSDate not supplied

Error Localization Examples: Looking for a Needle in a Hay-stack

Finite element models (FEM) are routinely developed and used during fabrication of high dollar-value hardware. NASA as part of the pre-flight certification of launch vehicles routinely conducts vibration and static tests to calibrate models used for flight-risk assessments. As part of the calibration process, certain areas in the model are modified, using engineering judgment and sensitivity analysis, to match the test results. Unfortunately, tools to identify problem areas in the FEM using test data directly are scarce. Over the years, Error Localization Algorithms (ELA) have been proposed with very limited success. Recently, the Analytical Dynamics Model Improvement (ADMI) algorithm, which computes closed-form mass and stiffness corrections to match the test data exactly, have been shown to be effective for error localization. The paper will present several FEM example problems where ELA is used with simulated test data to determine FEM problem areas. For each example, the correct answer is shown along with ELA results. It is shown that the ELA process is able to identify general problem areas in the FEM, which are consistent with known model perturbations. However, in most cases the ELA identified area of improvement is larger than the true answer. Nonetheless, with proper optimization tools, calibration results using the ELA identified areas provide excellent results.

error localization↗