SEARCH · Search NASA

Results for “metric standardization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

All sources NASA NTRS DOE OSTI

NASA NTRS2022

File Specification for MERRA-2 Climate Statistics Products

The Modern Era Retrospective analysis for Research and Applications, Version 2 (MERRA-2) contains a wealth of information that can be used for weather and climate studies. By combining the assimilation of observations with a frozen version of the Goddard Earth Observing System (GEOS), a global analysis is produced at an hourly temporal resolution spanning from January 1980 through present (Gelaro et al., 2017). It can be difficult to parse through a multidecadal dataset such as MERRA-2 to evaluate the interannual variability of weather that occurs on a daily timescale, let alone determine the occurrence of an extreme weather event. Furthermore, it was recognized that standard metrics were needed to evaluate climate change among climate models and international research efforts. As a result of these concerns, the Expert Team on Climate Change Detection and Indices (ETCCDI) developed a set of indices that represent the frequency and intensity of extreme weather events using a daily time series of 2-m air temperature (T2m) and precipitation (Alexander et al., 2016). These indices were used as a basis to comprise a list of fields that represent daily extreme temperature and precipitation events, heatwaves, multi-day precipitation, as well monthly percentile statistics from the MERRA-2 dataset. Also included in this data product is a climatological long term mean and standard deviation representing the interannual variability on a monthly timescale.

MERRA-2↗

DOE OSTI2021

Quantitative Evaluation of Autonomous Driving in CARLA

There has been a great deal of recent advancements in end-to-end imitation and reinforcement learning for self-driving vehicles. Despite this, there is a severe lack of standardized metrics for evaluating the performance of autonomous self-driving agents. Existing metrics are generally lacking in their ability to capture a wide range of driving behaviors and compare the severity of different failure cases. In this work, we introduce the Quantitative Evaluation for Driving metric, or QED, which assigns a quantitative score from 0-100 that captures the quality of driving for any driving agent. Our QED metric assesses different aspects of driving behavior including the ability to stay in the center of the lane, avoid weaving and erratic behavior, follow the speed limit, and avoid collisions, and it can be used under a wide range of driving scenarios. To show the effectiveness of our QED metric, we compare the scores generated by QED against scores assigned by human evaluators on a total of 30 different drivers and 6 different towns in the CARLA driving simulator. In ``easy'' evaluation scenarios, where it is relatively straightforward to distinguish better drivers from worse drivers, QED attains an average Pearson correlation of 0.96 and average Spearman correlation of 0.97 when compared against human evaluators. In ``hard'' evaluation scenarios, where it is far more ambiguous how to rank/score different types of bad driving behavior, QED attains an average Pearson correlation of 0.82 and average Spearman correlation of 0.75 when compared against human evaluators, which are both slighter higher than when we compare human evaluators against each other. While QED may not capture every characteristic that defines good driving, we consider it an important foundation for reproducibility and standardization in the community.

Gao, Shang↗

DOE OSTI2020

Certifying the Performance of Fixative Technologies under Open Air Demolition Activities for D and D - 20096

Department of Energy (DOE) facilities undergoing deactivation and decommissioning (D and D) activities struggle with safety concerns of residual radioactive contamination leftover after gross decontamination efforts have concluded. The positive effects of implementing fixative technologies for decommissioning and maintenance is known across the DOE complex. However, there is no standardized metric in place that quantifies the performance of these technologies such as incombustible fixative platforms that immobilize the contamination in a solid polymer material under normal operating conditions or when exposed to stressors. The primary focus of this research is to provide the empirical data necessary to properly characterize the effects of implementing fixative technologies on mitigating the release of contamination. In addition, this effort has been used to identify and establish uniform testing protocols to quantify operational parameters of fixating platforms and properly credit such systems under a variety of operational conditions and stressors that can arise during open air demolition efforts (e.g. impact, water, thermal, etc.). The established protocols will be provided to the ASTM International E10.03 Subcommittee on Radiological Protection for Decontamination and Decommissioning of Nuclear Facilities and Components for the potential of formal standardization. Attributes to be addressed during testing are material compromise from impact stress, thermal stress for incombustible fixative materials, and immobilization factors (how well an immobilizing platform is capable of retaining fixated material during thermal and impact stressors). In developing these test methods, a greater understanding of the material's behavior under anticipated and unanticipated events, such as decommissioning activities and contingency events as outlined in the Basis of Interim Operations (BIO), can be successfully characterized. There are several key components of the experimental methodology that is essential in empirically certifying fixative technologies. A surrogate contaminant with a unique signature has been utilized with the implementation of a uniform contamination process of test coupons that is quantifiable and replicable for direct fixative comparisons. A modular test chamber has also been used to apply the various stressors of interest. Collection of released contamination that includes airborne and resettled contaminant particles and analyzed using mass spectrometry. This research directly supports DOE complex-wide concerns in terms of final disposition of nuclear facilities and the considerations made during their time between ceasing facility operation and final disposition. The results of this effort aims to provide a standardized road map and logical decision flow to better facilitate fixative technologies into D and D activities required to achieve the desired facility end state. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

DOE OSTI2024

Tau Positron Emission Tomography for Predicting Dementia in Individuals With Mild Cognitive Impairment

An accurate prognosis is especially pertinent in mild cognitive impairment (MCI), when individuals experience considerable uncertainty about future progression. To evaluate the prognostic value of tau positron emission tomography (PET) to predict clinical progression from MCI to dementia. This was a multicenter cohort study with external validation and a mean (SD) follow-up of 2.0 (1.1) years. Data were collected from centers in South Korea, Sweden, the US, and Switzerland from June 2014 to January 2024. Participant data were retrospectively collected and inclusion criteria were a baseline clinical diagnosis of MCI; longitudinal clinical follow-up; a Mini-Mental State Examination (MMSE) score greater than 22; and available tau PET, amyloid-β (Aβ) PET, and magnetic resonance imaging (MRI) scan less than 1 year from diagnosis. A total of 448 eligible individuals with MCI were included (331 in the discovery cohort and 117 in the validation cohort). None of these participants were excluded over the course of the study. Exposures included Tau PET, Aβ PET, and MRI. Positive results on tau PET (temporal meta–region of interest), Aβ PET (global; expressed in the standardized metric Centiloids), and MRI (Alzheimer disease [AD] signature region) was assessed using quantitative thresholds and visual reads. Clinical progression from MCI to all-cause dementia (regardless of suspected etiology) or to AD dementia (AD as suspected etiology) served as the primary outcomes. The primary analyses were receiver operating characteristics. In the discovery cohort, the mean (SD) age was 70.9 (8.5) years, 191 (58%) were male, the mean (SD) MMSE score was 27.1 (1.9), and 110 individuals with MCI (33%) converted to dementia (71 to AD dementia). Only the model with tau PET predicted all-cause dementia (area under the receiver operating characteristic curve [AUC], 0.75; 95% CI, 0.70-0.80) better than a base model including age, sex, education, and MMSE score (AUC, 0.71; 95% CI, 0.65-0.77; P = .02), while the models assessing the other neuroimaging markers did not improve prediction. In the validation cohort, tau PET replicated in predicting all-cause dementia. Compared to the base model (AUC, 0.75; 95% CI, 0.69-0.82), prediction of AD dementia in the discovery cohort was significantly improved by including tau PET (AUC, 0.84; 95% CI, 0.79-0.89; P < .001), tau PET visual read (AUC, 0.83; 95% CI, 0.78-0.88; P = .001), and Aβ PET Centiloids (AUC, 0.83; 95% CI, 0.78-0.88; P = .03). In the validation cohort, only the tau PET and the tau PET visual reads replicated in predicting AD dementia. In this study, tau-PET showed the best performance as a stand-alone marker to predict progression to dementia among individuals with MCI. This suggests that, for prognostic purposes in MCI, a tau PET scan may be the best currently available neuroimaging marker.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2021

Energy and power quality measurement for electrical distribution in AC and DC microgrid buildings

Today's selection of DC microgrid buildings features a diverse set of electrical topologies and turnkey solutions, each with specific design trade-offs and optimizations. Designers desperately need standardized metrics and procedures for measurement and verification (M&V) to analyze and compare the advantages of each DC solution to traditional AC building networks. This work develops M&V procedures for quantifying and comparing the energy efficiency and power quality in buildings. To calculate full-building efficiency, this work introduces the measurement-informed modeling method, a procedure that develops and refines a building's energy model with metered data. To quantify power quality, this work defines a new voltage quality index that applies to both AC and DC buildings. This article describes the equipment, instrumentation, and operation necessary to calculate the efficiency and power quality. It then demonstrates these methods with a set of field tests. We report these M&V procedures can ultimately be used to compare and improve the efficiency and power quality of various DC topologies.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2025

Leveraging large language models to automate the identification of healthcare access barriers for veterans

Objective: To develop and evaluate an automated system for identifying healthcare barriers focusing on transportation issues in veterans’ clinical notes using large language models (LLMs) and to assess the impact of different prompting strategies on classification performance and explanation consistency. Methods: We developed a hybrid system combining pattern matching for templated notes with LLM analysis for free-text notes. Using 2000 manually annotated clinical notes, we compared four prompting strategies (dual-role short, dual-role long, analysis-first, analysis-only) across Mistral-7B and Llama-3.1 models. We evaluated classification performance using standard metrics and assessed explanation consistency through embedding similarity analysis. Results: The analysis-first strategy achieved superior performance, with Mistral-7B reaching an F1 score of 0.914, outperforming traditional machine learning approaches (GBM: 0.786, BERT: 0.811). LLMs demonstrated higher explanation consistency within models (mean cosine similarity 0.887–0.908) compared to cross-model similarities (0.767–0.872). Pattern matching successfully handled 6.7% of templated notes deterministically. Mistral-7B showed greater internal consistency but higher abstention rates compared to Llama-3.1. Conclusion: Requiring LLMs to analyze evidence before classification improves both accuracy and explanation consistency for identifying transportation barriers in clinical notes. This approach enables automated barrier detection at scale while providing clinically relevant explanations, supporting both population-level healthcare planning and individual patient care decisions.

Healthcare access barriers↗

DOE OSTI2024

Temperature and Composition Dependence Modeling of Viscosity and Electrical Conductivity of Low-Activity Waste Glass Melts

The development of models that accurately relate the properties of a glass melt to its temperature and composition is important for glass formulation, melter control, and modeling the melt flow, refractory corrosion, and production rate. Using a database consisting of more than 4,000 data points measured between 900 °C and 1250 °C for over 600 unique low-activity waste glass compositions, we developed models for the melt viscosity and electrical conductivity. Models based on the Gaussian process regression approach outperformed models based on the Vogel–Fulcher–Tammann equation according to four standard metrics and yielded reliable prediction intervals. The models found primarily linear effects between properties and individual components, except for the effect of the Na 2 O mass fraction on the electrical conductivity. The effects were found to be consistent with current theories on physical processes involved with those properties.

36 MATERIALS SCIENCE↗

DOE OSTI2022

A Generalized Transformer-Based Pulse Detection Algorithm

Pulse-like signals are ubiquitous in the field of single molecule analysis, e.g., electrical or optical pulses caused by analyte translocations in nanopores. The primary challenge in processing pulse-like signals is to capture the pulses in noisy backgrounds, but current methods are subjectively based on a user-defined threshold for pulse recognition. Here, we propose a generalized machine-learning based method, named pulse detection transformer (PETR), for pulse detection. PETR determines the start and end time points of individual pulses, thereby singling out pulse segments in a time-sequential trace. It is objective without needing to specify any threshold. It provides a generalized interface for downstream algorithms for specific application scenarios. PETR is validated using both simulated and experimental nanopore translocation data. It returns a competitive performance in detecting pulses through assessing them with several standard metrics. Finally, the generalization nature of the PETR output is demonstrated using two representative algorithms for feature extraction.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2021

Deep learning-based segmentation of lithium-ion battery microstructures enhanced by artificially generated electrodes

Accurate 3D representations of lithium-ion battery electrodes, in which the active particles, binder and pore phases are distinguished and labeled, can assist in understanding and ultimately improving battery performance. Here, we demonstrate a methodology for using deep-learning tools to achieve reliable segmentations of volumetric images of electrodes on which standard segmentation approaches fail due to insufficient contrast. We implement the 3D U-Net architecture for segmentation, and, to overcome the limitations of training data obtained experimentally through imaging, we show how synthetic learning data, consisting of realistic artificial electrode structures and their tomographic reconstructions, can be generated and used to enhance network performance. We apply our method to segment x-ray tomographic microscopy images of graphite-silicon composite electrodes and show it is accurate across standard metrics. We then apply it to obtain a statistically meaningful analysis of the microstructural evolution of the carbon-black and binder domain during battery operation.

25 ENERGY STORAGE↗

DOE OSTI2021

Characterizing the acceleration time of laser-driven ion acceleration with data-informed neural networks

Peak ion energy is an important figure-of-merit in short-pulse, laser-driven ion acceleration and is dependent on an associated acceleration time. Standard metrics for these quantities depend on analytical results such as the self-similar fluid model or empirical models based on relatively small experimental and simulation datasets. In this work we attempt to use a data-informed neural network (NN) as a surrogate model for a large ensemble of PIC simulations to investigate an effective acceleration time. We explore the application of a stacked convolutional and recurrent NN architecture for improved regression by incorporating the time dependencies of the data into the training process. Of particular note is how pretraining a network on lower fidelity data, e.g. 1D analytical results, greatly improves the network's ability to learn more complex, higher fidelity data. Finally, the dependency of the acceleration time on various laser and plasma parameters is explored.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2021

Electrical Measurement and Verification of Energy in DC Buildings

Today's selection of DC buildings features a diverse set of electrical topologies and turnkey solutions, and each has specific design trade-offs and optimizations. Designers desperately need standardized metrics and procedures for measurement and verification (M&V) to analyze and compare the advantages of each DC solution to traditional AC building networks. This work develops the Measurement-Informed Modeling (MIM) method, which can be used to determine full-building efficiency and energy savings. The MIM M&V procedure develops a building model, and refines the model with metered data. This work demonstrates the MIM method by measuring the full-building efficiency of two DC buildings operated by the Institute of Building Research in Shenzhen, China. The MIM procedure can ultimately be used to compare and improve the efficiency of various DC topologies.

buildings↗

DOE OSTI2026

Object-Based Evaluation of Dynamical and Statistical Downscaled Precipitation Products over CONUS

High-resolution precipitation data, generated through dynamical downscaling (DD) or statistical downscaling (SD) of global climate model output, provide critical information for regional climate assessment and adaptation planning. Most downscaling development and validation have focused on accurate gridscale precipitation construction and ignored the spatial structure of precipitation across model grids and at the event scale. However, many applications, e.g., hydrologic modeling and the analysis using the downscaled precipitation, require a reasonable representation of the spatial structure of precipitation within watersheds. Therefore, a set of standard metrics to evaluate the representation of the spatial structure of individual storms across diverse downscaled precipitation products is desired. To address this need, we conducted an object-based evaluation of precipitation in decades-long DD and SD products over the contiguous United States (CONUS). Specifically, we evaluate their ability to reproduce various features of precipitation objects in the observations: total volume, precipitation area, peak intensity, and spatial structure. Multiple metrics (bias, Perkins score, and nonparametric statistical tests) are used to quantify model performance. Our evaluation reveals notable variations in performance among individual products across different climate zones and seasons, as well as between extreme and nonextreme events. In general, most DD products exhibit balanced performance across the four precipitation object features, while SD products vary more significantly in their performance across products. Based on this comprehensive evaluation, we provide guidance on choosing downscaled products for specific regions, seasons, and precipitation object features. These findings and recommendations can inform precipitation-relevant modeling and analysis over CONUS, guide future downscaling technique developments, and provide actionable information for climate impact assessment and adaptation.

Downscaling↗

DOE OSTI2026

Object-Based Evaluation of Dynamical and Statistical Downscaled Precipitation Products over CONUS

Environmental sciences↗

DOE OSTI2023

Factors Influencing Building Demand Flexibility

The U.S. Department of Energy’s National Roadmap for Grid-interactive Efficient Buildings (GEB) acknowledged that building demand flexibility (DF) is both an important strategy to decarbonizing the buildings sector and an important resource for meeting the changing needs of the electrical grid such as improving grid reliability. However, understanding the complexity and uncertainties in real building field performance of DF strategies is a large gap hindering stakeholders on both grid and buildings side to make investments on deploying such strategies. The research work in this report intended to advance understanding of the variability and influential factors in building demand flexibility. Adding such knowledge based on lab testing results and measured performance data from real buildings is an important contribution. The report uses standardized metrics and methods to quantify DF performance from field-measured DF datasets of two significant building groups of big-box retail and medium office buildings to present the challenge of building DF variability in multiple dimensions. The report presents findings related to how several key factors influence building demand flexibility from implementing a common, cost-effective DF control strategy (i.e., adjusting zone temperatures). The findings are supported by full-scale lab testing, field data analysis and simulation research. The authors also provided application-oriented recommendations to stakeholders such as building aggregators, utility program design professionals, sophisticated building portfolio owners, and more.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2020

Development of an Optimized Gasoline Surrogate Formulation for PACE Experiments and Simulations

New powertrain solutions are needed to address societal challenges stemming in part from a transportation sector which relies heavily on combustion of conventional hydrocarbon fuels like gasoline. One way to advance new solutions is through predictive simulations. Predictive simulations can potentially limit the extent of experimental validations, reduce time spent in engineering design cycles, unlock new strategies for high efficiency combustion with power density, and minimize tailpipe emissions. However, the simulation tools currently available are either too computationally expensive, inadequate in their accuracy, or a combination of the two. A DOE-funded consortium of national laboratories called the Partnership to Advance Combustion Engines (PACE) seeks to address this gap by rapidly delivering new knowledge and tools which enable market-competitive powertrain solutions for light-duty vehicles. Approaches to modeling combustion in powertrain systems typically incorporate computational fluid dynamics (CFD) simulations with chemical kinetic models. These CFD simulations generally use surrogate fuels featuring a limited number of existing components in a reduced kinetic model to limit computational costs. Such surrogate fuels often sacrifice matching several or more combustion and physical properties of the target gasoline fuel. Validation of surrogate fuels is also often only pursued for a small subset of standard metrics such as research or motor octane number (RON, MON). The purpose of this project is to develop an optimal E10 gasoline surrogate fuel suitable for tasks across PACE to facilitate rapid common analysis and progression toward consortium goals. This project received contributions from tasks led by eight PACE principal investigators. The project funding here reflects all tasks under Pitz, whose tasks also contributed to additional PACE projects. Based on the optimal gasoline surrogate fuel composition, a reduced chemical kinetic model will be shared with PACE researchers, industry, and the broader combustion community.

33 ADVANCED PROPULSION SYSTEMS↗

DOE OSTI2024

Marine Energy Commercialization Review: Evaluation of the Transition From Public to Private Capital

The mission of the U.S. Department of Energy's Water Power Technologies Office (WPTO) is to advance marine energy technologies through research, testing, and commercialization. This paper explores the barriers and potential solutions for marine energy commercialization by evaluating publicly available literature, feedback from public and private actors, and historical WPTO actions. A key finding is the absence of standardized metrics to measure marine energy commercialization progress and the lack of targeted success goals. This paper aims to define those metrics, informed by public and private goals and the challenges developers experience, and to further evaluate targets offered by public funders and private capital providers. Recommendations to address barriers in marine energy commercialization include enhancing public-private communication, refining commercialization requirements, leveraging technology transfer programs, and exploring novel funding mechanisms like green bonds and contracts for difference. Addressing these challenges through proposed adjustments could facilitate the transition of marine energy technologies from public funding to sustainable private investment, ultimately advancing their commercialization.

13 HYDRO ENERGY↗

DOE OSTI2025

State Spotlight on Resilience: The Michigan Public Service Commission and Data Informed Accountability

Resilience activities are an emerging area within regulatory utility policy. As resilience frameworks develop, many Public Utility Commissions (PUCs) are looking to initiate resilience activities and have identified enhancing grid reliability as a practical starting point. Reliability describes the grid’s ability to deliver electricity steadily and without interruptions under normal or “blue sky” conditions. Reliability is measured using industry standard metrics such as outage frequency (SAIFI) and outage duration (SAIDI). Resilience is broader, reflecting the grid’s ability to both withstand and recover from disruptions, particularly those caused by extreme events such as severe storms. A reliable grid minimizes routine disruptions; a resilient grid ensures the system can recover quickly when disruptions occur, preventing them from escalating into widespread or prolonged crises.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2025

Artificial Intelligence Benchmarking

AI benchmarking is a method for evaluating the effectiveness of an AI model using a set of standardized metrics, for example, high school-level math exams. These benchmarks and their results will enable ranking various AI models based on their effectiveness in performing a specific task.

Krishnan, Anjay [Fermilab]↗