Search NASA⌕ Search

SEARCH · Search NASA

Results for “metric standardization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9
All sourcesNASA NTRSDOE OSTI
NASA NTRS2016

Up Periscope! Designing a New Perceptual Metric for Imaging System Performance

Modern electronic imaging systems include optics, sensors, sampling, noise, processing, compression, transmission and display elements, and are viewed by the human eye. Many of these elements cannot be assessed by traditional imaging system metrics such as the MTF. More complex metrics such as NVTherm do address these elements, but do so largely through parametric adjustment of an MTF-like metric. The parameters are adjusted through subjective testing of human observers identifying specific targets in a set of standard images. We have designed a new metric that is based on a model of human visual pattern classification. In contrast to previous metrics, ours simulates the human observer identifying the standard targets. One application of this metric is to quantify performance of modern electronic periscope systems on submarines.

imaging↗
DOE OSTI2020

Performance Portability Evaluation of OpenCL Benchmarks across Intel and NVIDIA Platforms

We evaluate the capabilities of vendor-provided OpenCL implementations for performance portability across multiple computing platforms. The Rodinia benchmark suite is used for this evaluation. We apply the metric defined by Pennycook et al., and we use roofline efficiency from the Roofline performance model as the "performance efficiency" in the metric's definition. We found that the delivered performance portability is similar for several benchmarks, even if the roofline-based performance efficiencies across platforms are very different among the benchmarks. To help distinguish between these instances, we extend the metric by adding the standard deviation of the performance efficiencies for each benchmark. We argue that the standard deviation gives additional insight into performance portability assessment since it adds the performance variability across platforms. Additionally, we discuss the challenges to measure performance portability associated with algorithms and system software. In terms of algorithms, we need to carefully construct the benchmarks and appropriately use the concurrency available on a platform. In terms of system software, we depend on the vendor performance tools to support the desired programming model and runtime to be able to measure the metrics of interest.

GPU↗
NASA NTRS1991

Metrics

A development status evaluation is presented for practical software performance measurement, or 'metrics', in which major innovations have recently occurred. Metrics address such aspects of software performance as whether a software project is on schedule, how many errors can be expected from it, whether the methodology being used is effective and the relative quality of the software employed. Metrics may be characterized as explicit, analytical, and subjective. Attention is given to the bases for standards and the conduct of metrics research.

Mcgarry, Frank↗
DOE OSTI2024

IDAES-PSE Software Tools for Optimizing Energy Systems and Market Interactions

Modern power grids coordinate electricity production and consumption via multi-scale wholesale energy markets. Historically, levelized cost metrics were the de facto standard for techno-eco-nomic analyses of energy systems and comparison of technology options. However, these metrics neglect the complexity of energy infrastructure including the time-varying value of electricity. An emerging alternative is multi-period optimization, which considers the locational marginal price of electricity as input data (parameters). In this work, we present a general interface for multi-period optimization with time-varying energy prices to facilitate rapid analysis and comparison of potential energy systems models. The PriceTakerModel class is written in the IDAES-PSE platform and allows users to generate a multi-period, price-taker model instance, as well as automatically generate common operational constraints for their model, such as start-up and shutdown. We show this interface successfully generates multi-period price-taker models, facilitates model discrimination, and aids in analyzing various technologies for deployment in unique energy markets.

Laky, Daniel↗
NASA NTRS2007

Capabilities of NASA's Space Physics Data Facility as Resources to Enable the Heliophysics Virtual discipline Observatories (VxOs)

SPDF now supports a broad range of data, user services and other activities. These include: CDAWeb current multi-mission data graphics, listings, file subsetting and supersetting by time and parameters; SSCWeb and 3-D Java client orbit graphics, listings and conjunction queries; OMNIWeb 1/5/60 minute interplanetary parameters at Earth; product-level SPASE descriptions of data including holdings of nssdcftp; VSPO SPASE-based heliophysics-wide product site finding and data use;, standard Data format Translation Webservices (DTWS); metrics software and others. These data and services are available through standard user and application webservices interfaces, so middleware services such as the Heliophysics VxOs, and externally-developed clients or services, can readily leverage our data and capabilities. Beyond a short summary of the above, we will then conduct the talk as a conversation to evolving VxO needs and planned approach to leverage such existing and ongoing services.

McGuire, Robert E.↗
DOE OSTI2023

Identifying strengths and weaknesses of methods for computational network inference from single-cell RNA-seq data

Single-cell RNA-sequencing (scRNA-seq) offers unparalleled insight into the transcriptional programs of different cellular states by measuring the transcriptome of thousands of individual cells. An emerging problem in the analysis of scRNA-seq is the inference of transcriptional gene regulatory networks and a number of methods with different learning frameworks have been developed to address this problem. Here, we present an expanded benchmarking study of eleven recent network inference methods on seven published scRNA-seq datasets in human, mouse, and yeast considering different types of gold standard networks and evaluation metrics. We evaluate methods based on their computing requirements as well as on their ability to recover the network structure. We find that, while most methods have a modest recovery of experimentally derived interactions based on global metrics such as Area Under the Precision Recall curve, methods are able to capture targets of regulators that are relevant to the system under study. Among the top performing methods that use only expression were SCENIC, PIDC, MERLIN or Correlation. Addition of prior biological knowledge and the estimation of transcription factor activities resulted in the best overall performance with the Inferelator and MERLIN methods that use prior knowledge outperforming methods that use expression alone. We found that imputation for network inference did not improve network inference accuracy and could be detrimental. Comparisons of inferred networks for comparable bulk conditions showed that the networks inferred from scRNA-seq datasets are often better or at par with the networks inferred from bulk datasets. Our analysis should be beneficial in selecting methods for network inference. At the same time, this highlights the need for improved methods and better gold standards for regulatory network inference from scRNAseq datasets.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Determining the profitability of energy storage over its life cycle using levelized cost of storage

Levelized cost of storage (LCOS) can be a simple, intuitive, and useful metric for determining whether a new energy storage plant would be profitable over its life cycle and to compare the cost of different energy storage technologies. However, researchers and industry decision makers still use conflicting definitions of LCOS. For example, some include charging cost, while others only include round trip efficiency (RTE) losses. Additionally, inputs to the existing formulations are not specific enough to generate repeatable results across studies, which reduces trust in the metric. To push for standardization in economic assessment of batteries and other energy storage devices, the authors review existing definitions of LCOS and identify the desired characteristics for a standard. They then propose a new definition and demonstrate that it fits these characteristics very well relative to other prominent options. Unit analysis is applied to this proposed definition to provide a deeper understanding of the equations and to demonstrate its effectiveness. Finally, the sensitivity of LCOS to different input parameters is investigated to help users understand how to compare analyses from literature to their own. The authors also provide a spreadsheet and a Python script to streamline adoption of the proposed definition.

25 ENERGY STORAGE↗
DOE OSTI2023

Laboratory testing methods to evaluate the reliability of occupancy sensors for commercial building applications

The energy performance of commercial buildings is greatly influenced by occupants which are highly variable and among the most unpredictable components of a building's operation. While most building control systems use fixed, predetermined occupancy schedules, these fixed occupancy levels can be quite different from actual occupancy. This can cause unnecessary energy consumption, particularly from heating, ventilation, and air conditioning (HVAC) and lighting systems which are responsible for approximately 60% of commercial buildings' energy use. The use of occupancy counting sensor systems integrated with building management system controls is one method that can be used to improve the energy-consuming performance of buildings. However, there is no standardized universal methodology and metrics to evaluate their reliability. The aim of this research is to develop a uniform evaluation methodology to assess the reliability of occupancy counting sensor systems in a controlled laboratory environment. The developed testing methodology includes both “typical” scenarios representing the occupancy scenarios of a typical commercial building, and “failure” testing scenarios which represent a range of potential scenarios that may impact a sensor system's reliability. These methods were then implemented in a case study to evaluate the performance of two novel occupancy counting sensor systems (i.e., door-centric, and camera-based). Results suggest that typical testing results can be used to compare the overall performance of the occupancy counting sensor systems; however, failure testing is also important to understand the weaknesses of the sensor system in order to select the suitable one for the intended use of the commercial building. In addition, the proposed methodology includes a modified confusion matrix which enables the ability to identify if failures are caused by over or under counting occupants and to what extent this occurs over the testing period.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2025

TPCpp-10M: Simulated proton-proton collisions in a time projection chamber for AI foundation models

Scientific foundation models hold great promise for advancing nuclear and particle physics by improving analysis precision and accelerating discovery. Yet, progress in this field is often limited by the lack of openly available large scale datasets, as well as standardized evaluation tasks and metrics. Furthermore, the specialized knowledge and software typically required to process particle physics data pose significant barriers to interdisciplinary collaboration with the broader machine learning community. This work introduces a large, openly accessible dataset of 10 million simulated proton-proton collisions, designed to support self-supervised training of foundation models. To facilitate ease of use, the dataset is provided in a common NumPy format. In addition, it includes 70,000 labeled examples spanning three well defined downstream tasks: track finding, particle identification, and noise tagging, to enable systematic evaluation of the foundation model's adaptability. The simulated data are generated using the Pythia Monte Carlo event generator at a center of mass energy of $\sqrt{s}$ = 200 GeV and processed with Geant4 to include realistic detector conditions and signal emulation in the sPHENIX Time Projection Chamber at the Relativistic Heavy Ion Collider, located at Brookhaven National Laboratory. This dataset resource establishes a common ground for interdisciplinary research, enabling machine learning scientists and physicists alike to explore scaling behaviors, assess transferability, and accelerate progress toward foundation models in nuclear and high energy physics. The complete simulation and reconstruction chain is reproducible with the sPHENIX software stack. All data and code locations are provided under Data Accessibility.

Data Analysis, Statistics and Probability (physics↗
DOE OSTI2025

Modeling Offshore Wind Farm Performance in Coastal Low-Level Jets Using Coupled Mesoscale-Microscale Large Eddy Simulations

Accurately predicting wind farm reliability under complex offshore atmospheric conditions remains a key challenge, particularly during noncanonical meteorological events such as coastal low-level jets (LLJs). LLJs, characterized by strong nonmonotonic vertical shear and directional veer, depart significantly from the simplified inflow assumptions embedded in conventional design standards, low-fidelity engineering models, and microscale large eddy simulations of the atmospheric boundary layer. In this work, we use the virtual wind farm framework—an exascale, graphics processing unit–accelerated large eddy simulation platform coupled with high-fidelity aeroservoelastic turbine models and advanced mesoscale-microscale coupling via the ExaWind software stack—to investigate turbine responses under realistic LLJ forcing. Simulations are performed over the U.S. North Atlantic offshore domain with the use of meteorological inputs from New York State Energy Research and Development Authority buoy data, focusing on a representative LLJ case impacting the International Energy Agency 15 MW reference turbine. Our results show that LLJs can cause up to 50% power deficits in downstream turbine rows and significantly amplify low-speed shaft and tower loads through nonlinear coupling between complex inflow characteristics and turbine structural dynamics. Two primary mechanisms drive these load amplifications: (1) unique LLJ inflow features—including veer and vertical/lateral shear—and (2) the downstream evolution of the flow under stable thermal stratification, which suppresses turbulence mixing and alters wake recovery. These mechanisms produce streamwise variations in turbine loading not captured by standard hub height–based metrics or existing design load case (DLC) definitions. This study highlights the critical role of rotor-scale flow gradients in driving fatigue and system-level aeroelastic responses, challenging current DLC and control strategies. We advocate the integration of full-flow field, environment-aware wind inputs into load modeling and control algorithms. By leveraging exascale computing to resolve mesoscale-microscale coupling, this work lays the groundwork for next-generation offshore wind turbine design and operation in meteorologically complex marine environments.

17 WIND ENERGY↗
DOE OSTI2024

2024 Photovoltaic Inverter Reliability Workshop Summary Report & Proceedings

The National Renewable Energy Laboratory (NREL) organized the 2024 Photovoltaic Inverter Reliability Workshop on April 11-12, 2024, hosted at NREL's South Table Mountain campus in Golden, Colorado. The workshop was organized around seven key topics, including the present state of inverter reliability; solutions for reliability challenges; life cycle cost and ownership issues; testing, standards, performance, and reliability metrics; data reporting, analytics, and sharing; and the future of PV inverter reliability research. Participants included inverter manufacturers, national laboratory researchers, academics, independent testing laboratories, and more. Over the course of the two-day workshop, attendees arrived at several key priorities and conclusions. This report summarizes these conclusions and then collects presentations from the workshop into a record of the workshop's proceedings.

14 SOLAR ENERGY↗
NASA NTRS1985

Communicating through deep space

NASA's Deep Space Network (DSN) consists of a worldwide set of communication stations and a central control facility in California, enabling communication with spacecraft thousands of millions of miles from earth. The stations have gone from 26 m diameter antennas operating at 960 MHz to 70 m diameter (by 1988) at 8400 MHz. The DSN provides exceptional performance in high gain steerable antennas, ultra-low noise receivers, high power transmitters, frequency and time standards, and precise radio metric data. Spacecraft missions envisaged in the 1990's for the continuing exploration of the Solar System include an array of increasingly complex visits to the inner planets, asteroids and comets and the outer planets. The Deep Space Network planned for the mid-1980s may not meet all the needs of these missions without substantial change. Deep space stations may require conversion to operation with beam waveguides, higher frequency and relative frequency stability of 10 to the -16th. A deep space relay station in earth orbit could permit operation at higher frequencies, with attendant higher performance. Long range planning to select the appropriate future network configurations and develop the technologies essential to their implementation is underway.

Smith, J. G.↗
NASA NTRS2002

NASA Technical Standards Program

The NASA Technical Standards Program was officially established in 1997 as result of a directive issued by the Administrator. It is responsible for Agency wide technical standards development, adoption (endorsement), and conversion of Center-unique standards for Agency wide use. One major element of the Program is the review of NSA technical standards products and replacement with non-Government Voluntary Consensus Standards in accordance with directions issued by the Office of Management and Budget. As part of the Program's function, it developed a NASA Integrated Technical Standards Initiative that consists of and Agency wide full-text system, standards update notification system, and lessons learned-standards integration system. The Program maintains a 'one stop-shop' Website for technical standards ad related information on aerospace materials, etc. This paper provides information on the development, current status, and plans for the NAS Technical Standards Program along with metrics on the utility of the products provided to both users within the nasa.gov Domain and the Public Domain.

Gill, Paul S.↗
NASA NTRS2003

Engineering Lessons Learned and Technical Standards Integration: Capturing Key Technologies for Future Space Missions

Capturing engineering lessons learned derived from past experiences and new technologies, then integrating them with technical standards, provides a viable process for enhancing engineering capabilities. The development of future space missions will require ready access, not only to the latest technical standards, but also to lessons learned derived from past experiences and new technologies. The integration of this information such that it is readily accessible by engineering and programmatic personnel is a key aspect of enabling technology. This paper addresses the development of a new and innovative Lessons Learned/Best Practices/Applications Notes--Standards Integration System, including experiences with its initial implementation as a pilot effort within the NASA Technical Standards Program. Included are metrics on the Program, feedbacks from users, future plans, and key issues that are being addressed to expand the System's utility. The objective is the enhancement of engineering capabilities on all aspects of systems development applicable to the success of future space missions.

Mellen, Daniele P.↗
NASA NTRS2003

NASA Technical Standards Program

The NASA Technical Standards Program was officially established in 1997 as result of a directive issued by the Administrator. It is responsible for Agency wide technical standards development, adoption (endorsement), and conversion of Center-unique standards for Agency wide use. One major element of the Program is the review of NSA technical standards products and replacement with non-Government Voluntary Consensus Standards in accordance with directions issued by the Office of Management and Budget. As part of the Program s function, it developed a NASA Integrated Technical Standards Initiative that consists of and Agency wide full-text system, standards update notification system, and lessons learned - standards integration system. The Program maintains a "one stop-shop" Website for technical standards ad related information on aerospace materials, etc. This paper provides information on the development, current status, and plans for the NAS Technical Standards Program along with metrics on the utility of the products provided to both users within the nasa.gov Domain and the Public Domain.

Gill, Paul S.↗
NASA NTRS2008

Software Simulates Sight: Flat Panel Mura Detection

In the increasingly sophisticated world of high-definition flat screen monitors and television screens, image clarity and the elimination of distortion are paramount concerns. As the devices that reproduce images become more and more sophisticated, so do the technologies that verify their accuracy. By simulating the manner in which a human eye perceives and interprets a visual stimulus, NASA scientists have found ways to automatically and accurately test new monitors and displays. The Spatial Standard Observer (SSO) software metric, developed by Dr. Andrew B. Watson at Ames Research Center, measures visibility and defects in screens, displays, and interfaces. In the design of such a software tool, a central challenge is determining which aspects of visual function to include while accuracy and generality are important, relative simplicity of the software module is also a key virtue. Based on data collected in ModelFest, a large cooperative multi-lab project hosted by the Optical Society of America, the SSO simulates a simplified model of human spatial vision, operating on a pair of images that are viewed at a specific viewing distance with pixels having a known relation to luminance. The SSO measures the visibility of foveal spatial patterns, or the discriminability of two patterns, by incorporating only a few essential components of vision. These components include local contrast transformation, a contrast sensitivity function, local masking, and local pooling. By this construction, the SSO provides output in units of "just noticeable differences" (JND) a unit of measure based on the assumed smallest difference of sensory input detectable by a human being. Herein is the truly amazing ability of the SSO, while conventional methods can manipulate images, the SSO models human perception. This set of equations actually defines a mathematical way of working with an image that accurately reflects the way in which the human eye and mind behold a stimulus. The SSO is intended for a wide variety of applications, such as evaluating vision from unmanned aerial vehicles, measuring visibility of damage to aircraft and to the space shuttles, predicting outcomes of corrective laser eye surgery, inspecting displays during the manufacturing process, estimating the quality of compressed digital video, evaluating legibility of text, and predicting discriminability of icons or symbols in a graphical user interface.

Source record↗
NASA NTRS2010

Hybrid Black-Hole Binary Initial Data

"Traditional black-hole binary puncture initial data is conformally flat. This unphysical assumption is coupled with a lack of radiation signature from the binary's past life. As a result, waveforms extracted from evolutions of this data display an abrupt jump. In Kelly et al. [Class. Quantum Grav. 27:114005 (2010)], a new binary black-hole initial data with radiation contents derived in the post-Newtonian (PN) calculations was adapted to puncture evolutions in numerical relativity. This data satisfies the constraint equations to the 2.5PN order, and contains a transverse-traceless "wavy" metric contribution, violating the standard assumption of conformal flatness. Although the evolution contained less spurious radiation, there were undesired features; the unphysical horizon mass loss and the large initial orbital eccentricity. Introducing a hybrid approach to the initial data evaluation, we significantly reduce these undesired features."

Mundim, Bruno C.↗
NASA NTRS2012

Validation and Verification of Operational Land Analysis Activities at the Air Force Weather Agency

The NASA developed Land Information System (LIS) is the Air Force Weather Agency's (AFWA) operational Land Data Assimilation System (LDAS) combining real time precipitation observations and analyses, global forecast model data, vegetation, terrain, and soil parameters with the community Noah land surface model, along with other hydrology module options, to generate profile analyses of global soil moisture, soil temperature, and other important land surface characteristics. (1) A range of satellite data products and surface observations used to generate the land analysis products (2) Global, 1/4 deg spatial resolution (3) Model analysis generated at 3 hours. AFWA recognizes the importance of operational benchmarking and uncertainty characterization for land surface modeling and is developing standard methods, software, and metrics to verify and/or validate LIS output products. To facilitate this and other needs for land analysis activities at AFWA, the Model Evaluation Toolkit (MET) -- a joint product of the National Center for Atmospheric Research Developmental Testbed Center (NCAR DTC), AFWA, and the user community -- and the Land surface Verification Toolkit (LVT), developed at the Goddard Space Flight Center (GSFC), have been adapted to operational benchmarking needs of AFWA's land characterization activities.

Shaw, Michael↗