Search NASA⌕ Search

SEARCH · Search NASA

Results for “metric standardization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Simulation of PV Variability as a Function of PV Generation and Plant Size

The deployment of photovoltaic (PV) systems continues to show significant expansion; however, this growth has brought added attention to issues around the variability of the solar resource. Both spatial and temporal variability exist. Temporal scales can range from the sub-second to multiyear, whereas spatial scales can range from a few meters to tens of kilometers. There are multiple methods described in the literature to quantify PV variability at various spatial and temporal scales. This study focuses on short-term temporal variability and uses similar approaches with the addition of PV plant size a parameter to quantify variability. The method employed here incorporates the normalization of clear-and cloudy-sky conditions and PV plant size to quantify nominal variability metrics. The distribution and fluctuations of these metrics provide relevant information that is useful for system operations. The National Solar Radiation Database (NSRDB) is used to simulate PV variability as a function of PV generation and plant size. Hypothetical but realistic system information at 33 locations is used to model PV generation by feeding NSRDB solar irradiance data to the National Renewable Energy Laboratory’s System Advisor Model (SAM). Over the selected region, it is found that the aggregated ramp rates for the 1-minute data are associated with standard deviations ranging from 0.002–0.055 on a daily basis; however, hourly intervals induce higher aggregated ramp rates than the other timescales. Even though minute-to-minute variations are significant for the 1-minute time-scale, the standard deviation aggregated into a daily metric is smaller because of the cancellation of values.

irradiance↗
DOE OSTI2021

Simulation of PV Variability as a Function of PV Generation and Plant Size: Preprint

The deployment of photovoltaic (PV) systems continues to show significant expansion; however, this growth has brought added attention to issues around the variability of the solar resource. Both spatial and temporal variability exist. Temporal scales can range from the sub-second to multiyear, whereas spatial scales can range from a few meters to tens of kilometers. There are multiple methods described in the literature to quantify PV variability at various spatial and temporal scales. This study focuses on short-term temporal variability and uses similar approaches with the addition of PV plant size a parameter to quantify variability. The method employed here incorporates the normalization of clear- and cloudy-sky conditions and PV plant size to quantify nominal variability metrics. The distribution and fluctuations of these metrics provide relevant information that is useful for system operations. The National Solar Radiation Database (NSRDB) is used to simulate PV variability as a function of PV generation and plant size. Hypothetical but realistic system information at 33 locations is used to model PV generation by feeding NSRDB solar irradiance data to the National Renewable Energy Laboratory’s System Advisor Model (SAM). Over the selected region, it is found that the aggregated ramp rates for the 1-minute data are associated with standard deviations ranging from 0.002–0.055 on a daily basis; however, hourly intervals induce higher aggregated ramp rates than the other timescales. Even though minute-to-minute variations are significant for the 1-minute timescale, the standard deviation aggregated into a daily metric is smaller because of the cancellation of values.

41 EE - Solar Energy Technologies Office (EE-4S)↗
DOE OSTI2023

Comparative Analysis of Confidence Metrics for Nuclear Criticality Safety

Nuclear criticality safety standards provide guidance on the requirements and recommendations to establish confidence in computerized model results used to support operation with fissionable materials. By design, the guidance is not prescriptive, leaving the analysts free to determine how various sources of uncertainties are to be statistically aggregated. This report compares the analyses and key assumptions behind four notable methodologies documented in the nuclear criticality safety literature: the parametric, nonparametric, Whisper, and TSURFER methodologies. Because of the involved use of statistics entangled with heuristic recipes, the results of these methodologies are often difficult to interpret. Also, they are augmented by additional large administrative margins, eliminating the incentive to understand their differences. With the new resurgent wave of advanced nuclear systems focused on economizing operation—including advanced reactors, fuel cycles, and fuel concepts—there is a strong need to develop a clear understanding of uncertainties and their fusion methodologies to reduce uncertainties in a scientifically defensible manner. This report offers a deep dive into the various assumptions of the four noted methodologies, their adequacy, and their limitations, to provide guidance on developing confidence for the emergent nuclear systems. These systems are expected to be challenged by the scarcity of experimental data.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2026

“One Table to Rule Them All”: How a Single Table can Enable Extensive Insights, Analytics and Assessment on Human Mobility Data

While much research has been conducted in Human Mobility Science, most studies on the analytics/insights part generally focus on one of the following: processing and analytics on human stop-trip behavior, design of individual mobility metrics (often in silos), calculation and characterization of only a handful (typically 5-6) of human mobility metrics on geospatial-temporal human mobility data of interest. Although human mobility research offers a vast and diverse array of available metrics, most individual studies typically compute only a small subset of five or six metrics at a time when analyzing trajectory datasets of human mobility across different areas of interest. This paper is motivated by the critical need to repeatedly compute an extensive array of human mobility metrics across several trajectory datasets and perform individual metric-level benchmarking to establish a new, standardized Test and Evaluation (T&E) suite for the field of Human Mobility Science. We first present our findings on the minimal yet sufficient pre-processing required to reliably and efficiently compute a wide range of human mobility metrics. The key findings are specifically related to the proposed Composite Stop Locations table, which serves as a core pre-processing data layer. Subsequently, we present a case study demonstrating how the Composite Stop Locations table facilitates computation of at least 14 distinct human mobility metrics (unlike 5-6 different set of metrics used for studies in the literature) using the popular and open-source OpenPFLOW dataset. Finally, we have also presented an example of our benchmarking methodology to evaluate the quality and performance of the trajectory dataset of interest, assessed across multiple human mobility metrics.

De, Debraj [ORNL] (ORCID:0000000233630020)↗
DOE OSTI2026

Demonstration of GaAs-Based Photovoltaics on Acoustically Spalled Surfaces Grown by MOCVD

This study demonstrates the potential of sonic lift-off (SLO) technology for repeatable substrate reuse in gallium arsenide (GaAs)–based photovoltaics (PVs), offering a pathway toward cost effective production for terrestrial PV applications. Single-junction GaAs solar cells fabricated on acoustically spalled substrates achieved performance metrics comparable with those grown on standard commercial substrates, exhibiting short-circuit current densities (J sc ) of approximately 26.2–26.4 mA/cm 2 , open-circuit voltages (V oc ) around 1001 mV, fill factors (FFs) of 84%, and AM1.5G efficiencies near 22%. These results confirm that one reuse cycle can be realized without a performance penalty, offering a direct pathway to reducing substrate cost in fabrication. Furthermore, subsequent reuse of the substrates highlighted critical challenges, as devices fabricated from reused substrates showed significant degradation in performance (J sc of 15.5 mA/cm 2 , V oc of 748 mV, FF of 72%, and efficiency of 8%), primarily due to degradation of the epitaxial structure resulting from repeated spalling cycles.

Acoustic Spalling↗
DOE OSTI2022

Development and testing of a performance evaluation methodology to assess the reliability of occupancy sensor systems in residential buildings

With the emergence of advanced occupancy sensor technologies to better detect occupancy in buildings, a universal methodology and metrics are required to evaluate and report sensor systems’ reliability and compare the performance across multiple sensor systems. Herein this research presents a methodology to assess the reliability of occupancy sensor systems in residential buildings in a controlled laboratory environment, including both “typical” and “failure” testing scenarios. The developed methodology was then implemented to evaluate a novel occupancy detection sensor system’s reliability. “Typical” testing evaluates the overall accuracy of the sensor system, which suggest how reliable the occupancy sensor system is over time. Results show that on average, the precision and recall are 0.75 and 0.70, indicating similar numbers of false positives and false negatives across the dataset. The overall accuracy of the tested sensor system was 62.4% to 76.4%. Failure testing results indicate whether there are influential variables impacting the sensor performance. For the tested sensor system, the number of occupants, presence of large objects, presence of interior light sources, and number of doors are not influential, while lighting level, location of occupants, additional door in the entry/exit area, and having the TV on are variables determined to impact the sensor system performance.

47 OTHER INSTRUMENTATION↗
DOE OSTI2022

An Alternative Timing and Synchronization Approach for Situational Awareness and Predictive Analytics

Accurate and synchronized timing information is required by power system operators for controlling the grid infrastructure (relays, Phasor Measurement Units (PMUs), etc.) and determining asset positions. Satellite-based global positioning system (GPS) is the primary source of timing information. However, GPS disruptions today (both intentional and unintentional) can significantly compromise the reliability and security of our electric grids. A robust alternate source for accurate timing is critical to serve both as a deterrent against malicious attacks and as a redundant system in enhancing the resilience against extreme events that could disrupt the GPS network. To achieve this, we rely on the highly accurate, terrestrial atomic clock-based network for alternative timing and synchronization. In this paper, we discuss an experimental setup for an alternative timing approach. The data obtained from this experimental setup is continuously monitored and analyzed using various time deviation metrics. We also use these metrics to compute deviations of our clock with respect to the National Institute of Standards and Technologys (NIST) GPS data. The results obtained from these metric computations are elaborately discussed. Finally, we discuss the integration of the procedures involved, like real-time data ingestion, metric computation, and result visualization, in a novel microservices-based architecture for situational awareness.

Chinthavali, Supriya↗
DOE OSTI2022

Improving energy performance metrics to maximize the benefits of disruptive technologies

Advances in appliance and equipment systems' technologies also cause changes in their technical characteristics that can affect energy consumption and associated greenhouse gas emissions. Energy performance evaluation methods that use testing and metrics can evolve to account for energy consumption contributed from new technological characteristics. However, there is no systematic effort to enable the advancement of test procedures and performance metrics to maximize the energy, climate, and financial benefits of disruptive technologies. Therefore, it is important for countries to regularly update their energy-efficiency policy programs, such as standards and labels, by improving test procedures and metrics to reflect the innovation in emerging technologies as well as to mitigate the risk of deploying obsolete technology. This paper shows how energy performance metrics have been improving in selected appliances and equipment and provides insights to design a systematic effort to improve metrics and test procedures at the speed of ongoing changes in technologies and markets.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2023

Process-Oriented Diagnostics: Principles, Practice, Community Development, and Common Standards

Process-oriented diagnostics (PODs) aim to provide feedback for model developers through model analysis based on physical hypotheses. However, the step from a diagnostic based on relationships among variables, even when hypothesis driven, to specific guidance for revising model formulation or parameterizations can be substantial. The POD may provide more information than a purely performance-based metric, but a gap between POD principles and providing actionable information for specific model revisions can remain. Furthermore, in coordinating diagnostics development, there is a trade-off between freedom for the developer, aiming to capture innovation, and near-term utility to the modeling center. Best practices that allow for the former, while conforming to specifications that aid the latter, are important for community diagnostics development that leads to tangible model improvements. Promising directions to close the gap between principles and practice include the interaction of PODs with perturbed physics experiments and with more quantitative process models as well as the inclusion of personnel from modeling centers in diagnostics development groups for immediate feedback during climate model revisions. Examples are provided, along with best-practice recommendations, based on practical experience from the NOAA Model Diagnostics Task Force (MDTF). Common standards for metrics and diagnostics that have arisen from a collaboration between the MDTF and the Department of Energy’s Coordinated Model Evaluation Capability are advocated as a means of uniting community diagnostics efforts.

58 GEOSCIENCES↗
DOE OSTI2021

Cambium Documentation: Version 2021

Cambium is a tool that assembles structured data sets of simulated hourly emission, cost, and operational data for modeled futures of the U.S. electric sector with metrics designed to be useful for long-term decision-making. It was built to expand the metrics reported in the National Renewable Energy Laboratory's (NREL's) Standard Scenarios - an annually released set of projections of how the U.S. electric sector could evolve across a suite of different potential futures, looking forward through 2050 (Cole et al. 2021) Information about Cambium and related publications can be found at https://nrel.gov/analysis/cambium.html, and the Cambium data sets for the Standard Scenarios can be viewed and downloaded at https://cambium.nrel.gov/. In this documentation, we define the metrics reported in Cambium databases (Section 4) and document the Cambium-specific methods for calculating those metrics (Section 5). As this document is intended to cover multiple Cambium data releases, we do not document specific scenarios here - for individual data releases, readers should look for an accompanying report that describes the assumptions and data that underlie that specific release. For Cambium databases that are built on NREL's Standard Scenarios, for example, reports describing the scenarios can be found on the Standard Scenarios web site (https://www.nrel.gov/analysis/standard-scenarios.html).

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2021

Quality Control Methods for Advanced Metering Infrastructure Data

While urban-scale building energy modeling is becoming increasingly common, it currently lacks standards, guidelines, or empirical validation against measured data. Empirical validation necessary to enable best practices is becoming increasingly tractable. The growing prevalence of advanced metering infrastructure has led to significant data regarding the energy consumption within individual buildings, but is something utilities and countries are still struggling to analyze and use wisely. In partnership with the Electric Power Board of Chattanooga, Tennessee, a crude OpenStudio/EnergyPlus model of over 178,000 buildings has been created and used to compare simulated energy against actual, 15-min, whole-building electrical consumption of each building. In this study, classifying building type is treated as a use case for quantifying performance associated with smart meter data. This article attempts to provide guidance for working with advanced metering infrastructure for buildings related to: quality control, pathological data classifications, statistical metrics on performance, a methodology for classifying building types, and assess accuracy. Advanced metering infrastructure was used to collect whole-building electricity consumption for 178,333 buildings, define equations for common data issues (missing values, zeros, and spiking), propose a new method for assigning building type, and empirically validate gaps between real buildings and existing prototypes using industry-standard accuracy metrics.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

A new algae technical standards focus group: Summarizing and guiding the algae state of the art

The value proposition of algae technical metrics was the subject of a recent, industry-led, Focus Group interactive discussion. The interactive platform allowed for real time capturing and priority ranking of the topics that are deemed critical challenges for a future successful algae-based bioeconomy. This work ultimately sets the stage for providing guidance, methods, and standard reference materials tailored to respective algae industry segments for the cultivation, processing and marketing of high-quality and safe algae-based products. The group solicits input from the broader algae community on the value proposition and presents a case here to build a consensus around algae-specific metrics. The value proposition of technical standards to the industry is that guidance obtained through these collaborations builds producers' and consumers' trust that algae and algae-derived products are safe, reliable, and may allow for expanded market access, and increased competitiveness and international trade.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

Assessment of Seasonal Energy Performance for Room Air Conditioners in Multiple Climates

This report provides a general approach for adopting a seasonal energy efficiency metric by examining the climate-specific temperature bin distribution for air conditioner use, the seasonal energy efficiency results according to International Standards Organization (ISO) 16358 by climate region, and the relationships in seasonal energy efficiency across regional metrics. This approach could make the seasonal efficiency metric values comparable with those from other countries with standards consistent with ISO 16358. Policy action and market transformation can be accelerated and effectively harmonized with other regional or international efforts by adopting a seasonal energy efficiency metric. The framework and methodology described in this report have been applied in the following publications: Shaffie et al. 2022, Park et al. 2021, Park et al. 2019, and UNEP 2019a.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2021

Species Extrapolation of Propyl Acetate Dose Metrics

We demonstrate the ability of the propyl PBPK model to predict dose metrics of propyl acetate, propanol, and propionic acid from a standard 90-day subchronic inhalation study of propyl acetate in male and female rats. The model was used to predict the same dose metrics in “reference” male and female humans using the same exposure conditions. Finally, we used reverse dosimetry with the model to predict what exposure conditions would lead to the same dose metrics measured in rats. These extrapolations of internal dose metrics based on known species differences in physiology and measured differences in metabolism offer a more scientific species extrapolation than conventional uncertainty approaches, potentially of interest for risk assessment.

63 RADIATION, THERMAL, AND OTHER ENVIRON. POLLUTAN↗
DOE OSTI2026

PRIME: An evaluation framework for protein representation inference and generalization in viral mutation space

Background Protein language models (PLMs) have revolutionized protein fitness prediction, yet their application to rapidly evolving viral pathogens is often confounded by extreme sequence homology. This homology leads to “data leakage” in standard random validation splits, yielding inflated performance metrics that fail to translate into real-world biosurveillance utility. Results We present Protein Representation Inference for Mutation Evaluation (PRIME), a framework that integrates domain-specific fine-tuning with a rigorous position-stratified validation protocol to evaluate viral threats. Using a dataset of 347,432 SARS-CoV-2 receptor binding domain (RBD) sequences, we demonstrate that while random training data split yields deceptive R 2 values (> 0.90), they fail to generalize to novel mutational sites. By benchmarking models up to 650 M parameters, we show that domain-specific fine-tuning of the ESM-C 600 M model with correctly stratified data provides an initial demonstration of predictive signal for binding affinity and expression at unseen mutational sites of binding affinity and expression on unseen sites (R 2 ~0.23), a significant advancement over base foundation models which exhibit no predictive power (R 2 <0). PRIME’s embedding-based clustering identified 3.03% of bat coronavirus sequences as candidates for further experimental prioritization based on their functional similarity to human-infective strains in embedding space, offering a perspective complementary to traditional phylogenetic methods. Conclusion PRIME establishes a new benchmark for the application of PLMs in pathogen surveillance. Our findings demonstrate that state-of-the-art models and fine-tuning, when paired with stratified validation, provide biologically meaningful insights into pathogen evolution and zoonotic risk.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2026

An improved dataset for predicting mammal infecting viruses from genetic sequence information

There have been several attempts to develop machine learning (ML) models to identify human infecting viruses from their genomic sequences, with varying degrees of success. Direct comparison between models is problematic, because these models are typically trained and evaluated on different datasets with alternative data splitting schemes, features, and model performance metrics. In this paper we present a standardized dataset of mammal infecting and non-infecting viral pathogens, refined from the previous work of Mollentze et al. to include the latest literature evidence, roughly doubling the number of curated host-virus records available to the community, and new host target labels, primate and mammal. The new host labels were included for several reasons, including previous reports that classification performance is better at broader taxonomic ranks and the idea that there may be more data for primate infection that might serve as a suitable proxy for zoonotic potential and avoidance of false positives for human infection due to absence of evidence. On this dataset, we report the performance of eight machine learning models for predicting mammal-infecting viruses from their genomic sequences. We find that randomly assigning cases in our improved dataset to training/testing sets, when compared to the original assignments into training/testing in Mollentze et al., increases the overall average ROC AUC of prediction of human infection from 0.663 ± 0.070 to 0.784 ± 0.013, consistent with the reduction in phylogenetic distance between train and test sets (relative entropy change from 3.00 to 0.08). The broadest host category of mammal infection can be predicted most reliably at 0.850 ± 0.020. We share our improved dataset and code to enable standardized comparisons of machine learning methods to predict human host infections. Overall, we have presented preliminary evidence that classification of virus host infection is more tractable at higher taxonomic ranks, that unsurprisingly reducing the phylogenetic distance between training and test sets can improve predictive performance, that peptide kmer features appear to be harmful to out of sample model performance, and we are left with the question of whether models for virus host prediction can reasonably be expected to perform well in out of sample scenarios given the likelihood that viruses do not share a common ancestor. Consistent with this concern, when the data is resampled such that there is no overlap between viral families in training and test sets (relative entropy > 24), models perform no better than random chance at prediction of human infection regardless of whether kmers are included (ROC AUC 0.50 ± 0.08) or not (ROC AUC 0.50 ± 0.04).

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

Performance Assessments of Demand Flexibility from Grid-Interactive Efficient Buildings: Issues and Considerations

This SEE Action Network report explains basic concepts and fundamental considerations for assessing the actual demand flexibility performance of buildings participating in demand flexibility programs and responding to time-varying retail rates. Demand flexibility is the capability of distributed energy resources (DERs) to adjust a building’s load profile across different timescales. Assessments determine the timing, location, quantity, and quality of grid services provided. The results can be used for financial settlements and to improve performance of demand flexibility, support its consideration in resource potential studies and electricity system planning, and contribute to cost-effectiveness evaluations. While practitioners and regulators regularly find opportunities to improve performance assessments of demand-related services, to a large degree current best practices are sufficient for basic service offerings as demand flexibility is implemented today. However, advances in assessment practices will be required in a future with grid-interactive efficient buildings (GEBs) that provide continuous demand flexibility by integrating multiple DERs and flexibility modes (load shed, load shift, modulate, and generate). Example practices include using new baseline constructs—including whether baselines are even required for certain building flexibility modes or configurations, deploying more advanced metering and analytics, further developing cybersecurity standards, improving communication standards for increased interoperability, and establishing performance metrics and assessment procedures for load modulation as it is more fully defined and implemented. This report provides information on prioritizing and designing performance assessment elements for demand flexibility programs and time-varying retail rates, assessment protocols, and research and development needs. Additional research sponsored by DOE’s Building Technologies Office offers more detailed technical information on assessing performance of demand flexibility, including definition of performance metrics.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

Livewire: Automatic Annotations

Diogenes processes datasets to provide data quality metrics for the Livewire platform and creates standardized data dictionaries from data annotations. Diogenes needs data annotations that clearly outline thenformat and organization of the data. It also relies on the type, class, and unit of each data piece for comprehensive analysis, which it cannot determine independently. The Annotation Tool significantly reduces the time needed to create annotations for Diogenes by generating data annotations with the correct formatting and content. It also employs machine learning and hard-coded models to automatically annotate data class, quality type, and data units.

33 - ADVANCED PROPULSION SYSTEMS↗