Results for “metric categorization” — Page 3

DOE OSTI2025

Challenges in predicting protein-protein interactions of understudied viruses: Arenavirus-human interactions

Understanding protein-protein interactions (PPIs) between viruses and host organisms is crucial for uncovering infection mechanisms and identifying potential therapeutic targets. The ability to generalize PPI predictive models across understudied viruses presents a significant challenge. In this work, we use arenavirus-human PPIs to illustrate the difficulties associated with model generalization, which are compounded by a lack of both positive and negative data. We employ a Transfer Learning approach to investigate arenavirus-human PPIs by utilizing models trained on better-studied virus-human and human-human PPIs. Additionally, we curate and assess four types of negative sampling datasets to evaluate their impact on model performance. Despite the overall high accuracies (93–99 %) and AUPRC scores (0.8–0.9) appearing promising, further analysis indicates that these performance metrics can be misleading due to data leakage, data bias, and overfitting, especially concerning under-represented viral proteins. We reveal these gaps and assess the impact of data imbalance using standard k-fold cross-validation and Independent Blind Testing with a Balanced Dataset, resulting in a drop in accuracy below 50 %. We propose a viral protein-specific evaluation framework that categorizes viral proteins into majority and minority classes based on their representation in the dataset, enabling comparison of model performance across these groups using balanced accuracies. This framework offers a more robust evaluation of model generalizability, addressing biases inherent in standard evaluation techniques and paving the way for more reliable PPI prediction models for understudied viruses.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2024

Best practice guidelines for the use of economic and technical KPIs

Key Performance Indicators (KPIs) are an important set of metrics used to assess various aspects of photovoltaic (PV) systems, including their long-term performance, economic viability and carbon footprint. Technical KPIs support data-driven and informed decision-making when optimizing PV systems and provide a comprehensive overview of how PV systems operate across different conditions and climates. Different KPIs are commonly employed throughout the entire value chain of PV projects and can be categorized into technical, economic and sustainability aspects.

14 SOLAR ENERGY↗

DOE OSTI2023

Assessing feasible H2–CO2 sources in the US as Feedstocks for Sustainable Aviation Fuel Precursors: Acetic Acid and Ethanol Production via Hydrogenotrophic Pathways

The environmental impact of carbon dioxide emissions is significant, and research is focused on mitigating these emissions and developing eco-friendly technologies in line with green chemistry principles. Waste-to-energy technologies play a crucial role in converting waste into renewable energy and valuable biofuels and bioproducts. This study specifically explores the utilization of waste gas emissions, particularly carbon dioxide, from various sources in the United States for the production of sustainable aviation fuel (SAF) precursors, such as ethanol and acetic acid. The study categorizes and quantifies the volumes of carbon dioxide emissions into three types: non-biogenic, biogenic, and biogenic emissions from ethanol production facilities. Stoichiometric calculations are applied to compare the amounts of carbon dioxide from each category with the available hydrogen production capacity, determining if sufficient hydrogen is present for converting carbon dioxide into SAF precursors. The study reveals two key findings. Firstly, there is a significant reserve of carbon dioxide, approximately 1648 million metric tons per year (MMTy), combining all three categories, which would require a substantial increase of approximately 35–40 times in the existing hydrogen production capacity of 4.988 MMTy. This increased hydrogen production has the potential to yield approximately 1067.82 MMTy of acetic acid and 189.19 MMTy of ethanol annually. Secondly, upon analyzing the quality and application of the three sources of carbon dioxide with the currently available hydrogen production capacity, it is found that biogenic carbon dioxide from ethanol plants is the most suitable choice for immediate production of SAF precursors. This would theoretically result in an annual production of 1.36 MMTy of ethanol and 1.772 MMTy of acetic acid. The other two sources of carbon dioxide can be considered potential reserves for future utilization when additional hydrogen production facilities are established. The study provides a foundation for assessing the aggregation potential required for acetic acid and ethanol production. By optimizing the use of waste gases as raw materials, the study not only enables the production of SAF precursors but also contributes to the passive reduction of greenhouse gas emissions.

Carbon Dioxide↗

DOE OSTI2023

A systematic literature review on residential demand response with a focus on opportunities for low-income communities

Demand Response improves efficiency and grid stability, reducing peak load and total system costs. It can potentially contribute to other societal aspects, including mitigating climate change and improving air quality, health, stress levels, and comfort. This paper reviewed the literature on residential demand response programs in the United States, through the lens of the five energy justice principles, identifying and categorizing a set of 165 papers from 1960 to 2024. As a result, we observe significant progress in distributive justice, however, mostly focused on increasing participation rather than attending specific needs of historically marginalized communities. Concordantly, recognition and procedural justice dimensions are lagging, while cosmopolitan justice studies are scarce, underscoring a need to improve the measurement of impacts resulting from the implementation of demand response programs. Future research opportunities include the consolidation of standardized definitions and metrics, to better understand burdens and needs of diverse residential users. In terms of policy, collection and transparent access to standardized data is required as a condition to the design of flexible, customizable, transparent, and easy to navigate demand response programs. As a result, a better understanding of and engagement with customers has the potential to increase adoption rates, persistence in the program, and overall impacts for the system, households, and society.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

DOE OSTI2025

Exploring urban typologies using comprehensive analysis of transportation dynamics

Abstract As urban areas continue to expand and develop, categorizing cities into typologies offers a valuable framework for understanding metropolitan dynamics and fostering inter-city collaboration. However, existing typologies related to urban mobility have limitations, failing to consider cities within a single large urban region and often overlooking crucial dimensions such as trip demand and traffic flow. In this paper, we introduce a transportation-focused characterization for cities within a large urban region, specifically the San Francisco Bay Area, California. We incorporate over 40 metrics across five transportation dimensions: trip demand, road network, multi-modal network, traffic flow, and land use. Specifically, for the trip demand dimension, we include metrics capturing residents’ trip characteristics, such as mode share, intra-city trips, and inter-city trips. Additionally, we analyze the purpose of trips entering the city to gain a deeper understanding of incoming trip patterns. In the traffic flow dimension, we examine metrics like vehicle miles traveled, delay, and congestion to assess the traffic conditions on the street network. These, combined with other dimensions, provide a comprehensive view of a city’s transportation dynamics. Using unsupervised machine learning clustering methods, we identified eight distinct typologies for the Bay Area: Live Work Cities; Job and Activity Magnet Cities; Anchor Cities; Multi-modal Cities; Hyper-connected Cities; Low-density Residential Cities; Medium-density Residential Cities; and Mixed-use Residential Cities. Our findings show that many clusters are strongly influenced by trip demand and traffic flow metrics. Finally, we examine the practicality of this typology and its potential to guide collaborative transportation management strategies. The typologies provide a foundation for dialogue among Bay Area cities, focusing on evaluating shared characteristics and leveraging successes or challenges to develop unified strategies for transportation management.

Kuncheria, Anu↗

DOE OSTI2021

Developing Technology Performance Level Assessments for Early-Stage Wave Energy Converter Technologies: Preprint

The advantage of using Technology Performance Level (TPL) in conjunction with Technology Readiness Level (TRL) assessments in guiding technology development trajectories to successful outcomes in less time, at less overall cost, and with less encountered risk has been well articulated in the literature. In partnership with industry and international collaborators, a TPL assessment methodology for grid-connected applications has been developed through the application of the systems engineering approach. Metrics under seven different categories have been developed, weighted based on their relative relevance, and combined to yield a composite score. The methodology has been implemented in a spreadsheet tool plus a web application specifically aimed at assessing early stage (TRL 1-3) concepts. The target use cases are (a) technology developers improving their design, to find fatal flaws early, to get feedback on current design, to identify areas of improvement that will yield the highest return on investment, (b) reviewers assessing technologies in competitions or for making funding decisions, (c) investor or project developer doing due diligence, (d) policy makers landscaping the technology domain for formulating R&D strategy. The methodology and the tools are undergoing continuous improvement based on the experience and lessons learnt from applying it to internal and external marine energy technology development projects. The methodology is also being adapted for assessing WECs servicing markets outside the continental grid - broadly categorized as Powering the Blue Economy (PBE) applications. Such applications have vastly different functional requirements entailing a modification of the methodology to account for their higher risk tolerance, reduced price sensitivities, lower power needs, different permitting protocols, etc. This paper presents the latest status of the TPL assessment methodology and tools, describes its adaptation to select PBE markets, and explores its extension to other domains where it could provide a comprehensive and holistic measure of a nascent or disruptive technology's technoeconomic performance potential.

metrics↗

DOE OSTI2021

Developing Technology Performance Level Assessments for Early-Stage Wave Energy Converter Technologies

The advantage of using Technology Performance Level (TPL) in conjunction with Technology Readiness Level (TRL) assessments in guiding technology development trajectories to successful outcomes in less time, at less overall cost, and with less encountered risk has been well articulated in the literature. In partnership with industry and international collaborators, a TPL assessment methodology for grid-connected applications has been developed through the application of the systems engineering approach. Metrics under seven different categories have been developed, weighted based on their relative relevance, and combined to yield a composite score. The methodology has been implemented in a spreadsheet tool plus a web application specifically aimed at assessing early stage (TRL 1-3) concepts. The target use cases are (a) technology developers improving their design, to find fatal flaws early, to get feedback on current design, to identify areas of improvement that will yield the highest return on investment, (b) reviewers assessing technologies in competitions or for making funding decisions, (c) investor or project developer doing due diligence, (d) policy makers landscaping the technology domain for formulating R&D strategy. The methodology and the tools are undergoing continuous improvement based on the experience and lessons learnt from applying it to internal and external marine energy technology development projects. The methodology is also being adapted for assessing WECs servicing markets outside the continental grid - broadly categorized as Powering the Blue Economy (PBE) applications. Such applications have vastly different functional requirements entailing a modification of the methodology to account for their higher risk tolerance, reduced price sensitivities, lower power needs, different permitting protocols, etc. This paper presents the latest status of the TPL assessment methodology and tools, describes its adaptation to select PBE markets, and explores its extension to other domains where it could provide a comprehensive and holistic measure of a nascent or disruptive technology's technoeconomic performance potential.

metrics↗

DOE OSTI2024

Carbon- and Oxygen-rich stars in MaStar: identification and classification

ABSTRACT Carbon- and Oxygen-rich stars populating the Thermally Pulsing Asymptotic Giant Branch (TP-AGB) phase of stellar evolution are relevant contributors to the spectra of ∼1 Gyr old populations. Atmosphere models for these types are uncertain, due to complex molecules and mass-loss effects. Empirical spectra are then crucial, but samples are small due to the short (∼3 Myr) TP-AGB lifetime. Here we exploit the vastness of the MaNGA Stellar library MaStar (∼60 000 spectra) to identify C and O-rich type stars. We define an optical colour selection with cuts of (g − r) > 2 and (g − i) < 1.55(g − r) − 0.07, calibrated with known C- and O-rich spectra. This identifies C- and O-rich stars along clean, separated sequences. An analogue selection is found in V, R, I bands. Our equation identifies C-rich and O-rich spectra with predictive performance metric F1-scores of 0.72 and 0.74 (over 1), respectively. We finally identify 41 C- and 87 O-rich type AGB stars in MaStar, 5 and 49 of which do not have a SIMBAD counterpart. We also detect a sample of non-AGB, dwarf C-stars. We further design a fitting procedure to classify the spectra into broad spectral types, by using as fitting templates empirical C- and O-rich spectra. We find remarkably good fits for the majority of candidates and categorize them into C- and O-rich bins following existing classifications, which correlate to effective temperature. Our selection models can be applied to large photometric surveys (e.g. Euclid, Rubin). The classified spectra will facilitate future evolutionary population synthesis models.

Hill, Lewis (ORCID:0000000180528080)↗

DOE OSTI2022

Selecting Post-Processing Schemes for Accurate Detection of Small Objects in Low-Resolution Wide-Area Aerial Imagery

In low-resolution wide-area aerial imagery, object detection algorithms are categorized as feature extraction and machine learning approaches, where the former often requires a post-processing scheme to reduce false detections and the latter demands multi-stage learning followed by post-processing. In this paper, we present an approach on how to select post-processing schemes for aerial object detection. We evaluated combinations of each of ten vehicle detection algorithms with any of seven post-processing schemes, where the best three schemes for each algorithm were determined using average F-score metric. The performance improvement is quantified using basic information retrieval metrics as well as the classification of events, activities and relationships (CLEAR) metrics. We also implemented a two-stage learning algorithm using a hundred-layer densely connected convolutional neural network for small object detection and evaluated its degree of improvement when combined with the various post-processing schemes. The highest average F-scores after post-processing are 0.902, 0.704 and 0.891 for the Tucson, Phoenix and online VEDAI datasets, respectively. The combined results prove that our enhanced three-stage post-processing scheme achieves a mean average precision (mAP) of 63.9% for feature extraction methods and 82.8% for the machine learning approach.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2025

Impacts of Precipitation Events on Concentrations of Oxygenated Gas- and Particle-Phase Compounds Observed in the Amazon

Removal of gases and particles by precipitation (wet deposition) is a critical process that significantly influences the transport and chemical transformation of atmospheric compounds. However, there are few studies that directly measure or constrain the rates of this process under real-world conditions. This work quantifies the net change in ambient concentrations during precipitation events (removal rates) of gas- and particle-phase organic compounds at a surface site near Manaus, Brazil, during the GoAmazon2014/5 campaign. Removal rates of identified and unknown compounds that have been previously classified into source-based clusters are measured during rain events and categorized based on estimated properties of compounds and clusters. Highly oxygenated gases, such as isoprene oxidation products, are removed during precipitation events with a median removal rate of 0.09 h –1 and the fastest analyte is removed at a rate of 0.22 h –1 . Removal rates of particle-phase compounds are observed at roughly this median rate, while less soluble gases, such as terpenes, exhibit low removal rates. These results are roughly in agreement with prior theoretical estimates of wet deposition rates for comparable compounds, providing an empirical point of comparison while noting that our metric reflects the net influence of precipitation events rather than wet deposition alone.

atmospheric chemistry↗

DOE OSTI2023

EVs@Scale Next-Gen Profiles - EV Profile Capture 2023

As part of the U.S. DOE EVs@Scale consortium Next-Gen Profiles (NGP) project, the profile capture of production electric vehicles undergoing high power charging (HPC) is conducted over a wide range of conditions to explore variance and performance. Charge session parameters are collected from both the electric vehicle (EV) and electric vehicle supply equipment (EVSE) at a rate of 10Hz and entered into a time-series database for analysis. These charge profiles are captured under nominal and off-nominal conditions, exploring the impact of battery state of charge (SOC), battery temperature, vehicle condition, smart charge management (SCM), and EVSE limitations. Nominal conditions are defined to be ideal conditions that should transfer the maximum allowable energy in the minimum possible amount of time. Nominal condition profiles are compared across EVs to characterize state-of-the-art EV charging performance against one another. Off-nominal condition profiles are compared against its nominal condition profile counterpart to highlight the variance across less desirable starting conditions within a single EV. Under nominal conditions, most EVs can achieve the original equipment manufacturer (OEM) rated peak-power and charge times. Peak power across EVs has variance due to the vehicle design, battery topology, charging strategy, etc. These 10-100% nominal preconditioned charge profiles across 13 EVs (11 light-duty (LD), 2 heavy-duty (HD)) were used for analysis in power curve analysis, power distribution, SOC and range comparison, thermal impacts of current draw, battery pack size and energy charged, and ramp rates. Power curve analysis show the uniqueness in power vs time curves across all EVs, breaking down the features of a typical profile and where variation is typically seen. Power distribution results showed that over 50% of charge time is spent below 50kW and only 12.1% is spent above 200kW. SOC and range performance yielded different top performing EVs when exploring goals of performance from SOC and range gained after 10-min (EV2) and 20-min (EV8), and time-to-achieve 80% SOC (EV8) and 200 miles of range (EV1). Current draw from 400-volt EVs had a higher thermal impact to cable/connector temperatures when compared to 800-volt EVs, due to a higher current requirement to achieve similar power levels. There was high variance in C rating, a useful metric when comparing the relationship between peak/average charge session power and relative battery pack size, across EVs under test. Ramp rates during initial power transfer were examined, fastest and slowest speeds ranging from 192.5kW/second to 2.6kW/second respectively. A similar analysis of ramp rates was also conducted for OCPP curtailment testing, where EVs underwent a 2-minute 65A curtailment request before returning to full-power charge. Under off-nominal conditions, most EVs experienced variation from the nominal condition profiles. EVs that underwent the full set of NGP defined testing conditions were compared, analysis of which was categorized by 800-volt and 400-volt EVs. It should be noted that the 800-volt EVs had significantly higher peak power ratings than the 400-volt EVs, and thus were more prone to variance. Initial state of charge, battery temperature, and vehicle condition had a considerable impact on peak power levels and charge time for the 800-volt vehicles under test. EVSE limited tests lowered achievable power down to 200kW, greatly impacted 800-volt charging power, and had little to no effect on 400-volt charging power. Adapter testing was performed on a single 400-volt EV, where power and current limitations were found but overall charge time was not significantly impacted. Adapter and boost converter testing was performed on a single 800-volt EV, where both charge power and charge time were greatly impacted. Charge profiles are unique, and comparing such requires analysis that examines starting conditions, vehicle and battery topologies,

Charging↗

DOE OSTI2023

The Current State of Light-Duty Electric Vehicle Supply Equipment Costs: An Assessment of Contemporary Understanding

This study uses a hybrid meta-analysis and literature-review approach to understand the current state of knowledge regarding the costs of electric vehicle supply equipment (EVSE). We present a novel way to consider, categorize, and label measures of cost and show cost measure estimates from a sample of 13 recent studies. We find that in general, there is too much variation and too few commonly represented EVSE cost measures to reasonably provide aggregate figures for these measures. We propose a convention for presenting EVSE cost measures that includes the application (commercial or residential), the power level (Level 1, Level 2, DCFC [with further distinction based on rated power capacity]), and the type of cost measure (hardware, installation, operation, and total cost). We contend that providing researchers with standard cost measures will help to advance our knowledge of EVSE costs by ensuring that future work will use common metrics. Establishing common metrics will enable conventional meta-analyses that will make assessments of EVSE costs even more accessible. Additionally, common metrics will make tracking costs more reliable as the technology continues to evolve and become more ubiquitous.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

DOE OSTI2023

Medium-temperature phase change material integration in domestic heat pump water heaters for improved thermal energy storage

In this review, we examine state-of-the-art developments in integrating phase change materials (PCMs) for thermal energy storage (TES) in domestic heat pump water heaters (HPWHs). The component design optimization and control optimization of HPWHs and TES are reviewed for insight into improving the thermal capacity and efficiency of a PCM-integrated HPWH. The state-of-the-art review is categorized by the stage of development of the PCM for deployment in HPWHs. To select appropriate PCMs for HPWHs, a six-factor down-selection process is used to determine the best material(s) for integration in HPWHs with appropriate heat exchanger design. Ultimately, food-grade PCMs appear to be the best candidate for integration of TES in domestic HPWHs because they are nontoxic, highly cyclable, and have heat transfer properties accommodable to water heating when integrated in a manner to overcome the thermal conductivity limitations of the material. A key parameter of water heating performance is thermal heating power of the PCM component, which is not often reported. Many studies report significant improvement in capacity and efficiency. Furthermore, many performance metrics are identified from the literature to quantify the system performance, but agreement across studies is not found. Unified energy factor and first hour rating performance tests are required for commercially available HPWHs, and these performance metrics could homogenize the literature. Ultimately, we find that select PCMs are ready for deployment with HPWHs for performance improvement, and component design and control optimizations are new avenues of research and development required for a commercially viable PCM-integrated HPWH system.

25 ENERGY STORAGE↗

DOE OSTI2021

Revisiting matrix-based inversion of scanning mobility particle sizer (SMPS) and humidified tandem differential mobility analyzer (HTDMA) data

Abstract. Tikhonov regularization is a tool for reducing noise amplification during data inversion. This work introduces RegularizationTools.jl, a general-purpose software package for applying Tikhonov regularization to data. The package implements well-established numerical algorithms and is suitable for systems of up to ∼ 1000 equations. Included is an abstraction to systematically categorize specific inversion configurations and their associated hyperparameters. A generic interface translates arbitrary linear forward models defined by a computer function into the corresponding design matrix. This obviates the need to explicitly write out and discretize the Fredholm integral equation, thus facilitating fast prototyping of new regularization schemes associated with measurement techniques. Example applications include the inversion involving data from scanning mobility particle sizers (SMPSs) and humidified tandem differential mobility analyzers (HTDMAs). Inversion of SMPS size distributions reported in this work builds upon the freely available software DifferentialMobilityAnalyzers.jl. The speed of inversion is improved by a factor of ∼ 200, now requiring between 2 and 5 ms per SMPS scan when using 120 size bins. Previously reported occasional failure to converge to a valid solution is reduced by switching from the L-curve method to generalized cross-validation as the metric to search for the optimal regularization parameter. Higher-order inversions resulting in smooth, denoised reconstructions of size distributions are now included in DifferentialMobilityAnalyzers.jl. This work also demonstrates that an SMPS-style matrix-based inversion can be applied to find the growth factor frequency distribution from raw HTDMA data while also accounting for multiply charged particles. The outcome of the aerosol-related inversion methods is showcased by inverting multi-week SMPS and HTDMA datasets from ground-based observations, including SMPS data obtained at Bodega Marine Laboratory during the CalWater 2/ACAPEX campaign and co-located SMPS and HTDMA data collected at the US Department of Energy observatory located at the Southern Great Plains site in Oklahoma, USA. Results show that the proposed approaches are suitable for unsupervised, nonparametric inversion of large-scale datasets as well as inversion in real time during data acquisition on low-cost reduced-instruction-set architectures used in single-board computers. The included software implementation of Tikhonov regularization is freely available, general, and domain-independent and thus can be applied to many other inverse problems arising in atmospheric measurement techniques and beyond.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2024

Predicting Drug Effects from High-dimensional Asymmetric Drug Data Sets using Graph Neural Networks: A Comprehensive Analysis of Multi-target Drug Effect Prediction

Graph neural networks (GNNs) have emerged as one of the most effective Machine learning (ML) techniques for drug effect prediction from drug molecular graphs. Despite having immense potential, GNN models lack performance when using data sets that contain high dimensional asymmetrically co-occurrent drug effects as targets with complex correlations between them. Training individual learning models for each drug effect and incorporating every prediction result for a wide spectrum of drug effects is beyond practicality. Such an implication provides a testbed to address this challenge as multi-target prediction problems, aiming to predict all drug effects at a time. We develop standard and hybrid graph neural networks (GNNs)to perform two separate tasks that are multi-regression for continuous values and multi-label classification for categorical values contained in our data sets. Since this step makes the target data even more sparse and introduces asymmetric label co-occurrence, the learning of multi-label classification models becomes difficult and heavily impacts the GNN's performance. To address these challenges, we propose a new data oversampling technique to improve multi-label classification performances on all the given imbalanced molecular graph data sets. Using the technique, we improve the data imbalance ratio of the drug effects better than before while protecting the data set's integrity. Finally, we evaluate multi-label classification performance using the best-performant hybrid GNN model on all the oversampled data sets obtained from the proposed oversampling technique. These results outperform those of other ML models including GNN models when they are trained on the original data sets or oversampled data sets using MLSMOTE (a well-known oversampling technique) in all evaluation metrics precision, recall, and F1 score by a significant margin.

Bose, Avishek [ORNL]↗

DOE OSTI2023

Explainable Artificial Intelligence Technology for Predictive Maintenance

The domestic nuclear power plant fleet has relied on labor-intensive and time-consuming preventive maintenance programs, thus driving up operation and maintenance costs to achieve high-capacity factors. Artificial intelligence and machine learning can help simplify complex problems, such as diagnosing equipment degradation, to enable more effective decision-making. Benefits will be felt not only within existing analog and digital instrumentation and control, but also work processes, the integration of people with technology, and most importantly, the business case. Together, these hold promise to make nuclear power more efficient and reduce costs associated with operation and maintenance. While the artificial intelligence and machine learning technologies hold significant promise in the nuclear industry, there are challenges or barriers to their adoption. This report outlines the those different machine learning adoption barriers (categorized as historical, technical, economic, regulatory, and user) that the industry must overcome to realize the full benefits of artificial intelligence and machine learning capabilities for long-term economic sustainability. This report also provides solutions for some of these barriers by focusing on improving the explainability of machine learning to encourage trust from the end-user. Trust and explainability are essential to machine learning adoption. This report focuses on research-developed solutions to some of these barriers while analyzing a non-safety-related system, namely the circulating water system. This system frequently experiences waterbox fouling which our models preemptively diagnoses then explains to the operator how those conclusions were reached. This report presents and discusses the inherent trade-off between machine learning performance (in terms of accuracy) and explainability, where highly accurate machine learning methods (such as deep-learning) are the least explainable, and the most explainable methods (such as decision trees) are the least accurate. In addition, explainability of artificial intelligence techniques in terms of transparency and post-hoc metrics are discussed. This report outlines the importance of data novelty and value of new information in evaluating both the explainability and trustworthiness. Novelty detection helps to establish consistency or inconsistency of the new data with respect to the training data. On the other hand, value of information could be a part of the user-centric visualization recommendation system that request additional information to be collected, thereby strengthening the machine learning outcomes. During this project, a copyrighted user-centric visualization that aligns with a human-in-the-loop approach was developed. The user-centric visualization presents different levels of information and can be tailored as per user credentials to gain user confidence. One of the salient features of the user-centric visualization is it presents machine learning methods with explainability metrics. A simplified version of the user-centric visualization was presented to 32 users with varying levels of machine learning expertise. Feedback was solicited to test the hypothesis that the app contained sufficient explainability and that the users would trust the algorithm. Overall, the app was positively received, and the hypothesis was supported. This report discusses the trust-but-verify framework – a potential approach to build user trust artificial intelligence. The framework discusses trust from the human level to artificial intelligence level. The fundamental premise of the trust but verify framework is derived from an observation of nuclear safety culture (i.e., nuclear power plant personnel do not rely on a singular source of data to make a decision). This also ties back to the user-centric visualization that presents different levels of information to achieve both explainability and trustworthiness of artificial intelligence. Even so, the adoption of artificial intelligence and machine learning in the nuclear industry faces additional barriers, namely regulatory and stakeholder readiness. To overcome these challenges, new solutions must gain regulatory approval and cater to stakeholder needs. The Nuclear Regulatory Committee has a 5-year strategic plan which prepares them for reviewing artificial intelligence technologies in licensee submissions. Early and frequent engagement with the regulator is encouraged. Additionally, artificial intelligence solutions should incorporate human-in-the-loop considerations and offer explainability. Stakeholders must prepare by hiring or training staff to adapt to advancing technology in everyday plant tasks.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2024

Data-Informed Evaluation Framework for Integrated Energy Systems: Insights from Power, Process Heat, and Hydrogen Production Applications

The multi-criteria decision analysis (MCDA) framework provides a systematic evaluation of the diverse preferences and performance metrics associated with alternative solutions. This approach is advantageous over a single-criterion methodology, which are only valid under conditions that assume ceteris paribus or an "apples-to-apples" comparison. However, selecting suitable technologies for integrated energy systems (IES) can be likened to an "apples-to-oranges" comparison, given the heterogeneous factors at stake. These factors include economics and performance parameters, geological compatibility, and environmental impacts. Consequently, past research has often employed a mixture of qualitative and quantitative criteria tailored to the specific interests of each study. While the method proves effective in handling the intricate interplay of criteria, the resulting rankings and scores can vary from study to study. This inconsistency is introduced from the use of subjectively defined thresholds and weights. As a result, decision-makers frequently find it challenging to establish clear connections between specific criteria and the resulting scores, as the transformation of criteria into ordinal scores results in a substantial loss of information. To address this challenge, we introduce a data-informed IES evaluation framework that offers comprehensive, interpretable, and traceable evaluations backed by quantifiable rationale. First, we identified key IES evaluation criteria from a decade of literature, focusing on relevant IES applications in power, process heat, and hydrogen production. We leveraged state-of-the-art cost estimates from the Idaho National Laboratory (INL) and technical data from 78 reactor designs from the International Atomic Energy Agency (IAEA) and the Organization for Economic Co-operation and Development - Nuclear Energy Agency (OECD-NEA). Lastly, we established thresholds by analyzing the mean, variance, root mean square, and slope of values across alternatives, categorizing the preferences of decision-makers into distinct utility functions, such as linear, saturating, exponential, and stepwise. Our approach yielded two main outcomes: (1) it provided consistent assessments across different stakeholder groups and (2) it visualized uncertainties in the decision-making context via comprehensive sensitivity analysis. To demonstrate the impact of our framework, we conducted case studies on 6 reactor designs (AP1000, NuScale, BWRX-300, Xe-100, eVinci, iMSR) for the three applications. Our data-driven framework proved to be highly effective in addressing heterogenous uncertainties faced by varied decision-makers? preferences and IES applications, as well as cost and technical estimates of advanced reactors.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

DOE OSTI2024

Data-Informed Evaluation Framework for Integrated Energy Systems: Insights from Power, Process Heat, and Hydrogen Production Applications

The multi-criteria decision analysis (MCDA) framework provides a systematic evaluation of the diverse preferences and performance metrics associated with alternative solutions. This approach is advantageous over a single-criterion methodology, which are only valid under conditions that assume ceteris paribus or an "apples-to-apples" comparison. However, selecting suitable technologies for integrated energy systems (IES) can be likened to an "apples-to-oranges" comparison, given the heterogeneous factors at stake. These factors include economics and performance parameters, geological compatibility, and environmental impacts. Consequently, past research has often employed a mixture of qualitative and quantitative criteria tailored to the specific interests of each study. While the method proves effective in handling the intricate interplay of criteria, the resulting rankings and scores can vary from study to study. This inconsistency is introduced from the use of subjectively defined thresholds and weights. As a result, decision-makers frequently find it challenging to establish clear connections between specific criteria and the resulting scores, as the transformation of criteria into ordinal scores results in a substantial loss of information. To address this challenge, we introduce a data-informed IES evaluation framework that offers comprehensive, interpretable, and traceable evaluations backed by quantifiable rationale. First, we identified key IES evaluation criteria from a decade of literature, focusing on relevant IES applications in power, process heat, and hydrogen production. We leveraged state-of-the-art cost estimates from the Idaho National Laboratory (INL) and technical data from 78 reactor designs from the International Atomic Energy Agency (IAEA) and the Organization for Economic Co-operation and Development - Nuclear Energy Agency (OECD-NEA). Lastly, we established thresholds by analyzing the mean, variance, root mean square, and slope of values across alternatives, categorizing the preferences of decision-makers into distinct utility functions, such as linear, saturating, exponential, and stepwise. Our approach yielded two main outcomes: (1) it provided consistent assessments across different stakeholder groups and (2) it visualized uncertainties in the decision-making context via comprehensive sensitivity analysis. To demonstrate the impact of our framework, we conducted case studies on 6 reactor designs (AP1000, NuScale, BWRX-300, Xe-100, eVinci, iMSR) for the three applications. Our data-driven framework proved to be highly effective in addressing heterogenous uncertainties faced by varied decision-makers? preferences and IES applications, as well as cost and technical estimates of advanced reactors.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗