Search NASA⌕ Search

SEARCH · Search NASA

Results for “metric standardization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

PV Validation Hub

The Validation Hub will be a clearinghouse for the transfer of novel algorithms and software from the PV research community to industry. Potential algorithms tested in the Hub could include the estimation of various PV loss factors and the detection of various operational issues. The primary function of the Hub will be for developers to submit executable code which will run on hosted data sets. Developers will receive private reports on the accuracy and performance (e.g., run-time) of the submitted algorithms, and public high level summaries will be hosted. These summaries will indicate the organization who submitted the algorithm (e.g., links to GitHub pages, documentation websites, etc.), high-level accuracy metrics, and standardized performance metrics. These results will be stored in a publicly available database, accessible through the Hub, with the ability for users to sort and filter the results. In short, the Hub will be presented to public users as a collection of interactive leaderboards, organized around specific analysis tasks pertinent to the PV data science community. These tasks include things such as the estimation of various PV loss factors and the detection of various operational issues. We will present progress on the development of this hub, including preliminary results of comparative validation of PV data science algorithms and progress towards building the platform itself.

algorithm↗
DOE OSTI2026

Scaling Demand Flexibility: Building on 30 Years of Energy Efficiency Success

With electricity consumption across the United States (US) and Canada anticipated to grow, energy efficiency program administrators have a key role to play in helping to ensure energy affordability and reliability in support of the broader economic systems utilities and grid support. Connected, demand side load balancing solutions, such as load shifting heating, ventilation and air conditioning (HVAC) systems and managed charging for electric vehicles (EVs), can dynamically manage energy, allowing for more volumetric electricity consumption without incurring the expense of upgraded transmission and distribution capabilities. When combined, or aggregated, many small loads can be managed to have meaningful impact on energy demand on the grid. Utilities and their partners have an opportunity to leverage decades of experience and the infrastructure needed to assess, design, implement, and measure programs to scale up the adoption of equipment with built-in load flexibility capabilities. Current efforts among a wide variety of electricity system service providers, utilities, standards agencies, regulators, national labs and private industry stakeholders aim to identify common standards, metrics, and methodologies for valuing grid services offered by demand side equipment. By combining those efforts with decades of proven energy efficiency resources, utilities are poised to effectuate a scaling up of equipment with energy management capabilities installed in homes and businesses across the US and Canada. This paper will provide an overview of how utilities are approaching this era of load growth and new peak demands across the United States and Canada. It will highlight the specific strategies that program administrators are employing to advance market transformation for grid-enabled products and devices that have the greatest potential to reduce energy use and increase load flexibility.

Grant, Peter↗
DOE OSTI2025

The Power of Many: An Ensemble Approach to Spectral Similarity

Quantifying the similarity between two mass spectra─a known reference mass spectrum and an unidentified sample mass spectrum─is at the heart of compound identification workflows in gas chromatography–mass spectrometry (GC-MS). The reference spectrum most like the sample is assigned as its identification (provided some quantitative similarity threshold is met, e.g., 80%) and thus accurately measuring similarity is essential. Significant research has gone toward developing metrics for this purpose, each of which has attempted to improve upon existing methods by incorporating GC-MS-specific information (e.g., peak ratios or retention times) or adopting various statistical and algorithmic frameworks. While this active development has led to a plethora of similarity metrics with demonstrated value across different contexts, the unfortunate consequence has been confusion surrounding which metric should be used as a global standard. No such metric is currently accepted as the standard method because different metrics have demonstrated optimal performance in different contexts. In this work, we propose an ensemble approach to spectral similarity scoring that combines the collective information from across existing similarity metrics to form an improved, globally representative similarity metric as a step toward establishing a global standard method. In conclusion, the resulting ensemble metrics are evaluated on over 88,000 spectra of varying complexity and demonstrate improved abilities to accurately rank the correct reference spectrum as the top-matching candidate for a sample relative to the rankings generated by individual similarity scores.

Carbohydrates↗
DOE OSTI2024

Circular Economy for Photovoltaics in Service of Energy Transition

The challenge of energy transition is immediate and immense; current projections target 75 TW of photovoltaics (PV) capacity by 2050. While any transition to renewable energy technology is preferable to the current fossil-based system, it is ideal to improve the sustainability of PV to minimize negative environmental and social impacts. Circular economy (CE) has been proposed as a method to improve the sustainability of PV, especially for emerging materials like perovskites. CE is a set of actions, principles, and systems which aim to design out waste and keep products and materials in use, to reduce environmental impacts and enable sustainable development. At the most basic level, CE is "reduce, reuse, recycle", the R-actions, in ranked order. CE of a PV technology can be metricized in a variety of ways, such as the Material Circularity Indicator (Smith and Jones, Ellen MacArthur Foundation, 2019) or recycling rates. Unfortunately, standard CE metrics have several shortcomings for measuring renewable energy technologies in the context of deployment for energy transition (Figge 2018, Saidani 2019): 1) Only measure mass flows; 2) De-prioritization of the use phase in favor of mass circularity when scoring; and 3) Tight focus on a single product scale The use phase and energy flows of PV are key to energy transition, and therefore need to be quantified. Additionally, correlating product-scale to system-scale is necessary for quantifying the environmental impacts of energy transition. Life Cycle Assessment (LCA) can address some of these concerns, but also focuses on a single product scale and has trouble capturing the dynamics of system-scale energy transition, such as the interaction of module lifetime with manufacturing demands for energy transition deployment schedules. Therefore, we developed an open-source Python-based system dynamics model to quantify the mass, energy and carbon impacts of CE R-actions for PV technologies in the energy transition; PV in the CE (PV ICE) (Ovaitt & Mirletz 2021). The tool captures supply chains from material extraction through end of life, incorporating 5 circular end of life pathways. PV ICE takes in any evolving bill of materials, module properties and deployment schedule to support researchers and decision makers with data-backed insights. In this work, we quantify and compare proposed CE sustainable PV module designs and lifecycle management strategies, spanning currently commercialized technologies, government and industry technology targets, and several low Technology Readiness Level (TRL) emerging PV technologies, including perovskites. Our analyses capture the projected evolutions of lifetime, efficiency and material circularity of these PV technologies, as well as their material supply chains. Our analyses emphasize the importance of examining a suite of metrics to identify priorities and tradeoffs, and inform design or lifecycle management decisions holistically. Previous analyses have demonstrated the central importance of PV module lifetime to support energy transition while minimizing impacts. High levels of material circularity (>90%) enable minimizing lifecycle wastes, can reduce virgin material demands if paired with improving efficiency, but demonstrate tradeoffs in energy return on investment. In the fervor of new material and technology development, it is important to remember that CE is not the end goal; decarbonization and energy transition are the end goal. CE should be used in service to improve the sustainability of PV, and R-actions evaluated for their usefulness and efficacy to this end.

carbon↗
DOE OSTI2025

MS25: Materials Science-Focused Benchmark Data Set for Machine Learning Interatomic Potentials

Here, we present MS25, a benchmark data set for evaluating machine learning interatomic potentials (MLIPs) across diverse materials-relevant systems including MgO surfaces, liquid water, zeolites, a catalytic Pt surface reaction, high-entropy alloys (HEAs), and disordered Zr-oxides. Five MLIP architectures (MACE, NequIP, Allegro, MTP, and Torch-ANI) are trained and tested, focusing not only on traditional metrics (energies, forces, and stresses) but also explicitly validating derived physical observables such as lattice constants, volumes, and reaction barriers. We find that most models reach comparable accuracy on standard error metrics across the simple systems, although equivariant MLIPs offer 1.5–2× improvements over nonequivariant MLIPs in energy and force error for structurally complex or compositionally disordered environments such as HEAs and Zr–O systems. Our analysis highlights that low errors in energy and force predictions do not guarantee reliable observables, emphasizing the necessity of explicit validation. We demonstrate limitations in cross-framework transferability, as models trained on one zeolite framework (CHA) fail to reliably generalize to predictions of structurally distinct frameworks (e.g., MFI). Size-extensive tests show some dependence on system size for MgO, resulting from forced periodicity. The HEA and Zr–O data sets are identified as challenging tests for future benchmarks and MLIP model architecture developments as they show significant differentiation in error between MLIP architectures and are still relatively difficult at 1000 training images. Moving forward, we recommend that benchmarking efforts shift their focus from marginal accuracy improvements in energy and force errors toward identifying and understanding model failure modes, rigorously assessing transferability, and evaluating how their errors affect observable predictions. For researchers looking to choose an MLIP architecture, we suggest selecting equivariant MLIP architectures if the complexity of the system is a challenge. For simple materials problems, auxiliary features such as integration with molecular dynamics engines, trade-offs between computational data set generation cost vs MLIP inference speed, and framework integration may play a more important decision factor than small differences in error metrics that are unlikely to matter for production-level research.

chemical structure↗
DOE OSTI2023

Visualization Quality Assessment

Understanding how inaccuracies in visualizations affect users’ perception and understanding of scientific data is hard. Inaccuracies in visualizations are quite common and could arise from a range of sources such as errors in the original dataset arising from compression artifacts, errors in the capturing device, noise during transmission of the data, effects due to the algorithm being used to convert data to visualization images, images generated from neural networks, and sources we have yet to discover. Many image quality assessment metrics have been developed to quantify image errors. However, these are usually focused on “natural images” rather than visualizations of scientific data. Common image quality assessment metrics (IQAs) include MSE, PSNR, perceptual metrics such SSIM, FSIM as well as perceptual metrics using deep learning approaches. However, a critical part of understanding how errors are perceived by humans, and subsequently developing more accurate quality assessment metrics, is through user evaluation studies. The goal of this software is to develop a visualization quality assessment (VQA) process that will enable the generation of VQAs that can be used to quantify errors in scientific data visualizations. The VQA development process will include software to support user evaluation experimental design, analysis of visualization differences against standard quality metrics, and the ability to develop additional VQA metrics specific to scientific visualization images.

Grosset, Andre↗
DOE OSTI2025

Synthetic Scientific Image Generation with VAE, GAN, and Diffusion Model Architectures

Generative AI (genAI) has emerged as a powerful tool for synthesizing diverse and complex image data, offering new possibilities for scientific imaging applications. This review presents a comprehensive comparative analysis of leading generative architectures, ranging from Variational Autoencoders (VAEs) to Generative Adversarial Networks (GANs) on through to Diffusion Models, in the context of scientific image synthesis. We examine each model's foundational principles, recent architectural advancements, and practical trade-offs. Our evaluation, conducted on domain-specific datasets including microCT scans of rocks and composite fibers, as well as high-resolution images of plant roots, integrates both quantitative metrics (SSIM, LPIPS, FID, CLIPScore) and expert-driven qualitative assessments. Results show that GANs, particularly StyleGAN, produce images with high perceptual quality and structural coherence. Diffusion-based models for inpainting and image variation, such as DALL-E 2, delivered high realism and semantic alignment but generally struggled in balancing visual fidelity with scientific accuracy. Importantly, our findings reveal limitations of standard quantitative metrics in capturing scientific relevance, underscoring the need for domain-expert validation. We conclude by discussing key challenges such as model interpretability, computational cost, and verification protocols, and discuss future directions where generative AI can drive innovation in data augmentation, simulation, and hypothesis generation in scientific research.

Generative Adversarial Networks↗
DOE OSTI2023

Prioritizing Off-Gas Metrics: A Guide for Comparable Off-Gas Capture Testing

The Material Recovery and Waste Form Development (MRWFD) off-gas team had a workshop, hosted by Idaho National Laboratory (INL), to align goals and expectations for off-gas research. The workshop included team members from four national laboratories. The workshop focused on defining distinct R&D phases with specific metrics, outlining standard test and measurement protocols for Iodine and Krypton/Xenon sorbents, brainstorming approaches to future disruptive technologies, and recognizing parameters with more inherent risk, requiring more rigorous evaluation. This report will serve as a guide for future off-gas work. Its purpose is to foster efficient collaboration across diverse research facilities and invite direct comparison of materials and results.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2026

Editorial: Towards the rapid and systematic assessment of vaccine technologies

The COVID-19 pandemic highlighted both the extraordinary potential of modern vaccinology and persistent challenges in how vaccine technologies are assessed. While vaccines can be developed and deployed at unprecedented speed, our ability to predict efficacy in a population is constrained by methodological difficulties, underreporting of negative results, and limited comparability across studies. This editorial introduces a Research Topic that brings together an interdisciplinary collection of experimental, computational, and theoretical contributions spanning multiple pathogens and vaccine platforms. Across these contributions, emerging themes emphasize the need for standardized immunogenicity metrics, transparent reporting including negative findings, and harmonized experimental protocols to support meaningful comparisons. This editorial highlights community practices and shared commitments – supported by researchers, funders, and journals – that could strengthen reproducibility, transparency, and cumulative learning in vaccine research.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Part distortion monitoring in additive manufacturing using machining

In additive manufacturing, accumulation of residual stresses can result in severe part distortion from the desired preform shape. Current methods for in-situ part distortion monitoring in additive manufacturing typically require expensive sensors, or capital equipment, and require time-consuming post-processing to understand the shape deviation. This paper presents an in-situ method, in the context of hybrid manufacturing, for part distortion detection using machining of additively manufactured parts. As a surrogate, three test artifacts were used to represent different distorted geometries. The tool axis positions from the machine tool controller and the cutting power were monitored during a facing operation. Cutting power data was used to detect the tool entry and exit in the workpiece using a novel approach with power standard deviation metric. The workpiece geometry and distorted configuration was subsequently predicted for positional and rotational deviations to within 2 mm accuracy using synchronized tool position data with cutting power. The proposed method can be used in a hybrid (additive and subtractive) machine tool to periodically check part distortion in the additive build. The method is applicable for any additive process and is low-cost and computationally inexpensive.

36 MATERIALS SCIENCE↗
DOE OSTI2026

Time Matters: A Survival Analysis of Public Electric Vehicle Charging Infrastructure Utilization

The rapid adoption of plug-in electric vehicles (PEVs) places significant demands on public charging infrastructure, making it critical to understand and optimize charger utilization. This study provides one of the most comprehensive analyses of charging behavior to date by applying a survival analysis to a dataset of nearly 16 million level 2 (L2) and direct current (DC) fast charger sessions across the United States from 2017 to 2022. Using Kaplan-Meier curves and log rank tests, our analysis reveals statistically significant and distinct duration patterns influenced by charger type, time of day, and day of the week. We find that L2 charging sessions exhibit high variability tied to venue type, whereas DC sessions are more uniform, typically lasting 30-45 min. This study introduces the operational efficiency score (OES), a metric for standardizing the performance evaluation of charging stations. Our findings offer actionable insights for optimizing charger deployment, developing dynamic pricing strategies to reduce vehicle dwell time, and improving load management for grid operators, ultimately enhancing the efficiency and availability of public charging infrastructure.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2026

Machine learning surrogates for ion energy–angle distributions in thermal and RF plasma sheaths

Ion energy–angle distributions (IEADs) at material surfaces are a critical input for plasma–material interaction (PMI) studies in fusion devices, yet they are computationally expensive to obtain using particle-in-cell (PIC) simulations. In this work, we develop a machine learning surrogate based on a deep deconvolutional neural network (DDeCNN) trained on large databases generated with the hPIC2 code. The surrogate is capable of reconstructing IEADs from sheath parameters for both thermal and radio-frequency (RF) plasmas, including cases with multiple ion species. Across thousands of test cases, the model achieves high accuracy, with over 97 % of predictions classified as good or average based on standard error metrics (MAE, MSE, L2). Even in the more challenging RF and multi-species regimes, the surrogate reliably captures the multi-peak structure of PIC results. Once trained, the surrogate produces IEADs in milliseconds on a common workstation, yielding speedups of six to seven orders of magnitude compared with running a full PIC simulation. This computational gain enables dense parameter scans and direct coupling of IEAD predictions with PMI and erosion models on whole-device scales in fusion-relevant conditions.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2025

Interactive Gas Chemistry for Enhanced Science Capabilities of the Energy Exascale Earth System Model Version 3

Atmospheric chemistry plays a crucial role in Earth system models (ESMs), controlling atmospheric composition and radiative balance; it is highly interactive with the physical climate, biogeochemical cycles, and human systems. However, it often imposes computational challenges in an ESM. Here we develop a full troposphere‐stratosphere interactive chemistry module for the US Department of Energy's Energy Exascale Earth System Model (E3SM). We intentionally build a streamlined module based on E3SM version 2 that interacts with other components and maintains all of major chemical and chemistry‐climate feedbacks. The module incorporates a new, highly efficient tracer advection scheme; linearization of stratospheric chemistry; and abridged tropospheric chemical mechanism with 28 reactive tracers. This new model, E3SM‐chem, can readily perform century‐long climate simulations of ozone, methane, and nitrous oxide based on emission scenarios as well as provide hourly budgets for the gas‐phase radicals that drive aerosol chemistry. We evaluate E3SM‐chem with an atmosphere‐only simulation as in the recent climate model intercomparison project (CMIP6) finding results similar to the other CMIP6 models. For the present‐day, E3SM‐chem matches the standard measurement metrics for stratospheric and tropospheric ozone, surface air quality, other key reactive gases like carbon monoxide, and the methane lifetime. Overall, E3SM‐chem maintains the climate fidelity of the baseline model while adding at most 20% to the computational cost of the atmosphere model. Hence, interactive chemistry can be a default configuration for long climate simulations at resolutions of 1° or finer, which is crucial for producing self‐consistent chemistry‐climate feedbacks that alter the climate system.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Measuring the capabilities of quantum computers

Quantum computers can now run interesting programs, but each processor’s capability—the set of programs that it can run successfully—is limited by hardware errors. These errors can be complicated, making it difficult to accurately predict a processor’s capability. Benchmarks can be used to measure capability directly, but current benchmarks have limited flexibility and scale poorly to many-qubit processors. We show how to construct scalable, efficiently verifiable benchmarks based on any program by using a technique that we call circuit mirroring. With it, we construct two flexible, scalable volumetric benchmarks based on randomized and periodically ordered programs. We use these benchmarks to map out the capabilities of twelve publicly available processors, and to measure the impact of program structure on each one. We find that standard error metrics are poor predictors of whether a program will run successfully on today’s hardware, and that current processors vary widely in their sensitivity to program structure.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

A comparison of machine learning methods to classify radioactive elements using prompt-gamma-ray neutron activation data

The detection of illicit radiological materials is critical to establishing a robust second line of defence in nuclear security. Neutron-capture prompt-gamma activation analysis (PGAA) can be used to detect multiple radioactive materials across the entire Periodic Table. However, long detection times and a high rate of false positives pose a significant hindrance in the deployment of PGAA-based systems to identify the presence of illicit substances in nuclear forensics. In the present work, six different machine-learning algorithms were developed to classify radioactive elements based on the PGAA energy spectra. The model performance was evaluated using standard classification metrics and trend curves with an emphasis on comparing the effectiveness of algorithms that are best suited for classifying imbalanced datasets. We analyse the classification performance based on Precision, Recall, F1-score, Specificity, Confusion matrix, ROC-AUC curves, and Geometric Mean Score (GMS) measures. The tree-based algorithms (Decision Trees, Random Forest and AdaBoost) have consistently outperformed Support Vector Machine and K-Nearest Neighbours. Based on the results presented, AdaBoost is the preferred classifier to analyse data containing PGAA spectral information due to the high recall and minimal false negatives reported in the minority class.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

On data set tensions and signatures of new cosmological physics

ABSTRACT Can new cosmic physics be uncovered through tensions amongst data sets? Tensions in parameter determinations amongst different types of cosmological observation, especially the ‘Hubble tension’ between probes of the expansion rate, have been invoked as possible indicators of new physics, requiring extension of the ΛCDM paradigm to resolve. Within a fully Bayesian framework, we show that the standard tension metric gives only part of the updating of model probabilities, supplying a data co-dependence term that must be combined with the Bayes factors of individual data sets. This shows that, on its own, a reduction of data set tension under an extension to ΛCDM is insufficient to demonstrate that the extended model is favoured. Any analysis that claims evidence for new physics solely on the basis of alleviating data set tensions should be considered incomplete and suspect. We describe the implications of our results for the interpretation of the Hubble tension.

Cortês, Marina (ORCID:0000000304853767)↗