Evaluating the reliability of machine-learning-based predictions used in nuclear power plant instrumentation and control systems
Not Available
Chen, Edward (ORCID:0000000180088097)↗
SEARCH · Search NASA
Results for “Reliability Prediction”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Application of machine learning to evaluating and remediating models for energy and environmental engineering
Machine learning (ML) algorithms have been increasingly successful in their applications to solve energy and environmental engineering problems. ML algorithms have the advantage of being able to solve highly nonlinear issues effectively. Furthermore, considering the limited sample size of data collected in energy and environmental engineering, obtaining a ML model with reasonable accuracy is simple. Unfortunately, the vast majority of the current applications of ML algorithms lack effective screening of dominant factors and comprehensive model validation, which weakens the predictive ability of the models. The present study takes the minimum miscible pressure (MMP) of CO 2 - oil systems as an example. It establishes a systematic and robust predictive model to address this issue. Based on 147 sets of slim tube tests, the predictive models of the MMPs are investigated by application of eight ML algorithms. The paper concludes that most of the published ML models in the field of energy and environmental engineering prediction are not reliable. Furthermore, it addresses the main reasons for the poor performance of some predictive models built by ML and provides guidelines on how to make such models robust. Further, to the best of our knowledge, this is the first study to point out the defects of current ML modeling methods and propose countermeasures for their application in energy and environmental engineering problems.
29 ENERGY PLANNING, POLICY, AND ECONOMY↗
Physically Informed Machine Learning Prediction of Electronic Density of States
The electronic structure of a material, such as its density of states (DOS), provides key insights into its physical and functional properties and serves as a valuable source of high-quality features for many materials screening and discovery workflows. Still, the computational cost of calculating the DOS, most commonly with density functional theory (DFT), becomes prohibitive for meeting high-fidelity or high-throughput requirements, necessitating a cheaper but sufficiently accurate surrogate. To fulfill this demand, we develop a general machine learning method based on graph neural networks for predicting the DOS purely from atomic positions, six orders of magnitude faster than DFT. This approach can effectively use large materials databases and be applied generally across the entire periodic table to materials classes of arbitrary compositional and structural diversity. We furthermore devise a highly adaptable scheme for physically informed learning which encourages the DOS prediction to favor physically reasonable solutions defined by any set of desired constraints. This functionality provides a means for ensuring that the predicted DOS is reliable enough to be used as an input to downstream materials screening workflows to predict more complex functional properties, which rely on accurate physical features.
36 MATERIALS SCIENCE↗
Limiting Current in Nanostructured Block Copolymer Electrolytes
The next-generation electrolytes for lithium batteries must be able to conduct ions at sufficiently high current densities; yet this regime is rarely studied directly. The limiting current density of an electrolyte quantifies the highest possible rate of ion transport under an applied dc potential. Herein, we report on the limiting current density in twelve nanostructured polystyrene-block-poly(ethylene oxide) (PS-b-PEO, or SEO) copolymer electrolytes. We find that the limiting current at a given salt concentration increases systematically with increasing volume fraction of the PEO block (φ EO ). In contrast, the effective-medium theory, commonly used to analyze conductivity in block copolymer electrolytes, predicts that limiting current is independent of φ EO . To resolve this conundrum, the ionic conductivity, the mutual diffusion coefficient of the salt, and the steady-state current fraction of the block copolymer electrolytes were measured. These measurements enable predictions of limiting current with no adjustable parameters using the concentrated solution theory. We found quantitative agreement between experimentally measured limiting current densities and predictions based on the concentrated solution theory. This work sheds light on how to reliably measure and predict limiting current density in composite electrolytes.
25 ENERGY STORAGE↗
HYPPO: A Surrogate-Based Multi-Level Parallelism Tool for Hyperparameter Optimization
We present a new software, HYPPO, that enables the automatic tuning of hyperparameters of various deep learning (DL) models. Unlike other hyperparameter optimization (HPO) methods, HYPPO uses adaptive surrogate models and directly accounts for uncertainty in model predictions to find accurate and reliable models that make robust predictions. Using asynchronous nested parallelism, we are able to significantly alleviate the computational burden of training complex architectures and quantifying the uncertainty. HYPPO is implemented in Python and can be used with both TensorFlow and PyTorch libraries. We demonstrate various software features on time-series prediction and image classification problems as well as a scientific application in computed tomography image reconstruction. Finally, we show that (1) we can reduce by an order of magnitude the number of evaluations necessary to find the most optimal region in the hyperparameter space and (2) we can reduce by two orders of magnitude the throughput for such HPO process to complete.
adaptation models↗
Virtual refrigerant charge sensor for variable-speed heat pumps based on feature selection
The refrigerant charge level in heat pump systems significantly impacts their energy efficiency. Virtual refrigerant charge (VRC) sensing technology has been comprehensively investigated and well-established due to its lower cost compared to physical sensors. However, the previous VRC research often relied on expert judgment and physical reasoning for their variable selection, which can potentially select redundant (or highly correlated) or insignificant features, and it is also primarily focused on single-speed systems. To address these challenges, this study proposes a VRC algorithm for variable-speed heat pumps that selects features through a rigorous feature selection method in combination with physical insights. We also propose a piecewise linear model structure segmented by subcooling temperature to accurately predict charge levels, particularly when subcooling temperatures are substantially low. The proposed algorithm was evaluated using experimental data of a residential R410A heat pump, and the performance was compared with two baseline VRC algorithms. The results are: (1) The proposed algorithm outperforms for the case with subcooling temperature less than 1 °C. (2) The proposed algorithm achieves a tested mean absolute percentage error (MAPE) of 4.23%, and improves the overall accuracy for cooling conditions by approximately 60%, compared with the two baseline algorithms. (3) The proposed algorithm uses two fewer features and improves the accuracy for undercharge cooling conditions by 68.0%, compared with baseline algorithm 2. These improvements enhance prediction accuracy and prevent overfitting, providing a more reliable refrigerant charge level prediction and helping improve the heat pump energy efficiency.
Liang, Chenjiyu↗
Software Validation Work With The ZPPR-15 Data
The analysis activities for fast reactors involve using many different pieces of software that are relied upon for their predictive capabilities. For this software to be considered reliable, documented proof that the predictions of the software are accurate is required. In this manuscript, the validation work that covers some of the Argonne software used in fast reactor design activities is discussed and displayed. This validation work includes neutron and gamma flux distributions, reaction rate distributions, and reactivity worth. In an ideal world, a reactor development program would have access to a comprehensive set of experimental facilities to help inform the design aspects of the reactor itself. While thermal-hydraulics experiments, and to a limited degree mechanical experiments, can be carried out today for validation needs, neutronics related experimental facilities are rather impractical because of the lack of experimental facilities. Given the desired time table for construction of new reactors, the reconstitution or creation of new neutronic experimental facilities is untenable and thus those reactor development programs must rely upon any available experimental measurements that are qualitatively similar to the design. While a methodology has been proposed to assess the similarity between the past experimental measurements and the reactor itself, that aspect is beyond the scope of this manuscript. In this manuscript, the focus is entirely placed on the analysis results for a series of experiments carried out at the ZPPR facility in Idaho in the mid-1980s. In this regard, this manuscript only shows the validation of the stated neutronics software for specific loadings of the ZPPR reactor. Because of the fuel form, its proposed enrichment, and the material content of the reactor core, the ZPPR-15 experiments were identified as potential validation data for the reactor. The ZPPR-15 experiments were intended as mockups of a 330 MWe Integral Fast Reactor program which was a follow on program to the Clinch River Breeder Reactor. In the ZPPR-15 series of experiments, measurements of the neutron spectrum, control rod worth, sodium void worth, foil reaction rate distributions, Doppler worth of heated samples, gamma dose, and axial expansion worth were all carried out and published. In many cases, these reactivity coefficients are good candidates to validate the reactivity coefficient calculation scheme used by the analysis software and included in the safety analysis activities of fast reactor development projects today. This manuscript discusses the modeling methodology and accuracy of the calculated experimental results using the LANL software MCNP and the ANL software package ARC (Argonne Reactor Codes). As will be shown, for many of the experimental measurements, the two software packages are found to be good predictive analysis tools for those experiments. In other cases, problems with the analysis methodology or underlying cross section data are exposed which indicates where predictive analysis is not as reliable. Finally, in some of the measurements the conclusion is reached that the experimental measurement cannot be reproduced with the analysis software as it is simply too difficult.
22 GENERAL STUDIES OF NUCLEAR REACTORS↗
Software Validation Work With The ZPPR-15 Data
The analysis activities for fast reactors involve using many different pieces of software that are relied upon for their predictive capabilities. For this software to be considered reliable, documented proof that the predictions of the software are accurate is required. In this manuscript, the validation work that covers some of the Argonne software used in fast reactor design activities is discussed and displayed. This validation work includes neutron and gamma flux distributions, reaction rate distributions, and reactivity worth. In an ideal world, a reactor development program would have access to a comprehensive set of experimental facilities to help inform the design aspects of the reactor itself. While thermal-hydraulics experiments, and to a limited degree mechanical experiments, can be carried out today for validation needs, neutronics related experimental facilities are rather impractical because of the lack of experimental facilities. Given the desired time table for construction of new reactors, the reconstitution or creation of new neutronic experimental facilities is untenable and thus those reactor development programs must rely upon any available experimental measurements that are qualitatively similar to the design. While a methodology has been proposed to assess the similarity between the past experimental measurements and the reactor itself, that aspect is beyond the scope of this manuscript. In this manuscript, the focus is entirely placed on the analysis results for a series of experiments carried out at the ZPPR facility in Idaho in the mid-1980s. In this regard, this manuscript only shows the validation of the stated neutronics software for specific loadings of the ZPPR reactor. Because of the fuel form, its proposed enrichment, and the material content of the reactor core, the ZPPR-15 experiments were identified as potential validation data for the reactor. The ZPPR-15 experiments were intended as mockups of a 330 MWe Integral Fast Reactor program which was a follow on program to the Clinch River Breeder Reactor. In the ZPPR-15 series of experiments, measurements of the neutron spectrum, control rod worth, sodium void worth, foil reaction rate distributions, Doppler worth of heated samples, gamma dose, and axial expansion worth were all carried out and published. In many cases, these reactivity coefficients are good candidates to validate the reactivity coefficient calculation scheme used by the analysis software and included in the safety analysis activities of fast reactor development projects today. This manuscript discusses the modeling methodology and accuracy of the calculated experimental results using the LANL software MCNP and the ANL software package ARC (Argonne Reactor Codes). As will be shown, for many of the experimental measurements, the two software packages are found to be good predictive analysis tools for those experiments. In other cases, problems with the analysis methodology or underlying cross section data are exposed which indicates where predictive analysis is not as reliable. Finally, in some of the measurements the conclusion is reached that the experimental measurement cannot be reproduced with the analysis software as it is simply too difficult.
Aliberti, Gerardo↗
All-purpose, all-sky photometric redshifts for the Legacy Imaging Surveys Data Release 8
ABSTRACT In this paper, we present photometric redshift (photo-z) estimates for the Dark Energy Spectroscopic Instrument (DESI) Legacy Imaging Surveys, currently the most sensitive optical survey covering the majority of the extragalactic sky. Our photo-z methodology is based on a machine-learning approach, using sparse Gaussian processes augmented with Gaussian mixture models (GMMs) that allow regions of parameter space to be identified and trained separately in a purely data-driven way. The same GMMs are also used to calculate cost-sensitive learning weights that mitigate biases in the spectroscopic training sample. By design, this approach aims to produce reliable and unbiased predictions for all parts of the parameter space present in wide area surveys. Compared to previous literature estimates using the same underlying photometry, our photo-zs are significantly less biased and more accurate at z > 1, with negligible loss in precision or reliability for resolved galaxies at z < 1. Our photo-z estimates offer accurate predictions for rare high-value populations within the parent sample, including optically selected quasars at the highest redshifts (z > 6), as well as X-ray or radio continuum selected populations across a broad range of flux (densities) and redshift. Deriving photo-z estimates for the full Legacy Imaging Surveys Data Release 8, the catalogues provided in this work offer photo-z estimates predicted to be of high quality for ≳9 × 108 galaxies over ${\sim}19\, 400\, \text{deg}^{2}$ and spanning 0 < z ≲ 7, offering one of the most extensive samples of redshift estimates ever produced.
79 ASTRONOMY AND ASTROPHYSICS↗
Machine Learning-Based Process Control for Injection Molding of Recycled Polypropylene
The increased interest in artificial intelligence in manufacturing has driven the adoption of machine learning to optimize processes and improve efficiency. A key challenge in injection molding is the variability of recycled materials, which affects part quality and processing stability. This study presents a novel closed-loop process control approach for injection molding, leveraging machine learning to adaptively predict processing inputs and quality outcomes. The methodology was tested on five blends of recycled polypropylene (rPP), using artificial neural networks (ANNs), linear regression, and polynomial regression to model the relationships between material properties and process parameters. The dataset was split 80/20 into training and testing sets. The ANN model was implemented using TensorFlow and Keras, with six hidden layers of 32 neurons per layer, ReLU activation, and an Adam optimizer. Empirical tuning and early stopping were used to optimize performance and prevent overfitting. Predictions were evaluated based on mean absolute error (MAE), mean squared error (MSE), and percentage error. The results showed that yield stress, ultimate elongation, and part weight were accurately predicted within a 5% error for linear and polynomial regression models and within a 10% error for the ANN. However, modulus predictions were less reliable, with errors of ~11% for ANN and linear regression and ~40% for polynomial regression, reflecting the inherent variability of this property in rPP blends. Predictions of processing inputs had errors ranging from 3% to 25%, depending on the model and response variable. No single modeling approach was consistently superior across all responses, highlighting the complexity of the relationship between material properties, process parameters, and quality metrics. Overall, the work demonstrates that closed-loop process control, powered by machine learning, can effectively predict key quality parameters in injection molding of recycled materials. The proposed approach can improve process stability and material utilization, facilitating increased adoption of sustainable materials.
Krantz, Joshua↗
Bayesian Estimation of the D(p,γ) 3 He Thermonuclear Reaction Rate
Big bang nucleosynthesis (BBN) is the standard model theory for the production of the light nuclides during the early stages of the universe, taking place for a period of about 20 minutes after the big bang. Deuterium production, in particular, is highly sensitive to the primordial baryon density and the number of neutrino species, and its abundance serves as a sensitive test for the conditions in the early universe. The comparison of observed deuterium abundances with predicted ones requires reliable knowledge of the relevant thermonuclear reaction rates, and their corresponding uncertainties. Recent observations reported the primordial deuterium abundance with percent accuracy, but some theoretical predictions based on BBN are at tension with the measured values because of uncertainties in the cross section of the deuterium-burning reactions. In this work, we analyze the S-factor of the D(p,γ) 3 He reaction using a hierarchical Bayesian model. We take into account the results of eleven experiments, spanning the period of 1955--2021; more than any other study. We also present results for two different fitting functions, a two-parameter function based on microscopic nuclear theory and a four-parameter polynomial. Furthermore, our recommended reaction rates have a 2.2\% uncertainty at 0.8~GK, which is the temperature most important for deuterium BBN. Differences between our rates and previous results are discussed.
79 ASTRONOMY AND ASTROPHYSICS↗
Forest fire threatens global carbon sinks and population centres under rising atmospheric water demand
Levels of fire activity and severity that are unprecedented in the instrumental record have recently been observed in forested regions around the world. Using a large sample of daily fire events and hourly climate data, here we show that fire activity in all global forest biomes responds strongly and predictably to exceedance of thresholds in atmospheric water demand, as measured by maximum daily vapour pressure deficit. The climatology of vapour pressure deficit can therefore be reliably used to predict forest fire risk under projected future climates. We find that climate change is projected to lead to widespread increases in risk, with at least 30 additional days above critical thresholds for fire activity in forest biomes on every continent by 2100 under rising emissions scenarios. Escalating forest fire risk threatens catastrophic carbon losses in the Amazon and major population health impacts from wildfire smoke in south Asia and east Africa.
54 ENVIRONMENTAL SCIENCES↗
Light Water Reactor Sustainability Program: Use of Time Distributions to Predict Operator Procedure Performance in Dynamic Human Reliability Analysis
The Human Unimodel for Nuclear Technology to Enhance Reliability (HUNTER) framework affords software capable of conducting human reliability analysis (HRA) using a dynamic approach built around operating procedures (OPs) from nuclear power plants (NPPs). Previous HUNTER reports document the development of this software tool, the coupling of HUNTER to the simulator code, the collection of operator performance data by using simulators to calibrate HUNTER models, and linking HUNTER to probabilistic risk assessment (PRA) software. The present report largely addresses two topics. The first is a new function in HUNTER called the HUNTER Procedure Performance Predictor (P3). HUNTER P3 uses HUNTER’s built in Monte Carlo tools featuring human performance variability to identify potential error traps in procedures. The second topic is time distribution analysis to generate time inputs for dynamic HRA. The current analysis was performed to investigate time distributions for task primitives, which are the minimum task unit of analysis used in dynamic HRA modeling. Using the time distribution data, the elapsed time for human actions in an extended loss of AC power (ELAP) scenario is then investigated. Time data and prediction are essential for modeling procedure performance.
99 GENERAL AND MISCELLANEOUS↗
Reliability of emergency and standby diesel generators: Impact on energy resiliency solutions
Emergency diesel generators are the most common form of backup power for critical loads when the grid fails and are most often deployed as stand-alone generators (<2000 kW) tied to individual buildings for hospitals, emergency services, military bases, ports, airports, industries, and commercial facilities. Understanding the finite reliability of emergency diesel generators during continuous operation is critical for energy planners, managers, and end-users. A new analysis of two large non-public emergency diesel generator operational data sets shows that commonly used reliability metrics are inadequate to predict the performance during a grid outage. Additional metrics addressing the likelihood of an emergency diesel generator to be available at the time an outage starts, successfully start and carry load, and constantly run for the duration of the outage are needed to predict the continuous run-time reliability of emergency diesel generators during a grid outage. This is the first analysis of the reliability of modern generators that follow standard maintenance protocols commonly used for backup power and these new metrics are shown to be dependent on the level of emergency diesel generator maintenance with values that can vary by more than an order of magnitude. Even using the new metrics for well-maintained emergency diesel generators, the analysis shows that the single emergency diesel generators configurations are only 80% likely to provide power for the duration of a two-week grid outage.
ENERGY PLANNING, POLICY, AND ECONOMY,POWER TRANSMI↗
A Hybrid Reliability Model Using Generalized Renewal Processes for Predictive Maintenance in Nuclear Power Plant Circulating Water Systems
Not Available
21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
Mitigative Strategies for Recovering From Large Language Model Trust Violations
In this study, we investigated strategies to address trust issues arising from errors in large language models (LLMs). The study examined the impact of confidence scores, system capability explanations, and user feedback on trust restoration post-error. 68 participants viewed the responses of an LLM to 20 general trivia questions, with an error introduced on the third trial. Each participant was presented with one mitigation strategy. Participants rated their overall trust in the model and the reliability of the answer. Results showed an immediate drop in trust after the error; however, there were no differences across the three strategies in trust recovery. All conditions had a logarithmic trend in trust recovery following error. Differences in overall trust were predicted by perceived reliability of the answer, suggesting that participants were evaluating results critically and using that to inform their trust in the model. Qualitative data supported this finding; participants expressed lasting distrust despite the LLM’s later accuracy. Results showcase the need to prioritize accuracy in LLM deployment, because early errors may irrevocably damage user trust calibration and later adoption.
97 MATHEMATICS AND COMPUTING↗
Controls on Barite (BaSO 4 ) Precipitation in Unconventional Reservoirs
Barite (BaSO 4 ) precipitation is one of the most ubiquitous examples of secondary sulfate mineral scaling in shale oil and gas reservoirs. Often, a suite of chemical additives is used during fracturing operations to inhibit the accumulation of mineral scales, though their efficacy is widely varied and poorly understood. This study combines experimental data and multi-component numerical reactive transport modeling to offer a more comprehensive understanding of the geochemical behavior of barite accumulation in shale matrices under conditions typical of fracturing operations. A variety of additives and conditions are individually tested in batch reactor experiments to identify the factors controlling barite precipitation. Our experimental results demonstrate a pH dependence in the rate of barite precipitation, which we use to develop a predictive model including a pH-dependent term that satisfactorily reproduces our observations. Further, this model is then extended to consider the behavior of three major shale samples of highly variable mineralogy (Eagle Ford, Marcellus, and Barnett). This data-validated model offers a reliable tool to predict and ultimately mitigate against secondary mineral accumulation in unconventional shale reservoirs.
54 ENVIRONMENTAL SCIENCES↗
Multireference diffusion Monte Carlo reaches 2D materials
Abstract Quantum confinement in 2D materials strongly enhances electronic correlation effects. Therefore, predicting the properties of these unique materials, with both a high level of accuracy and computational efficiency, without relying on adjustable parameters or functionals, remains an outstanding theoretical challenge. The majority of theoretical studies are based on the approximations of density functional theory (DFT). The reliability of DFT predictions are heavily dependent on the choice of an approximated exchange-correlation functional. Here, we estimate the magnitude of impact of correlation on the total energy for the quintessential 2D material, graphene, by performing and comparing state-of-the-art selected CI and quantum Monte Carlo extrapolated calculations for a single unit cell at the$$\Gamma$$point. We demonstrate that Self-Healing Diffusion Monte Carlo (SHDMC) obtains a very compact, but high-quality wavefunction for this system that lacks the strong basis set dependence displayed by state of the art quantum chemistry methods. The SHDMC wavefunction is of higher quality compared to that obtained from sCI, in the same orbital basis, while being$$\sim$$ 1000 times smaller in terms of determinant count compared to sCI. We also demonstrate that extrapolating SHDMC results to the infinite determinant limit compares extremely well with complete basis set extrapolated sCI. Our work paves the way for future validation of SHDMC applied to challenging 2D materials.
Science & Technology - Other Topics↗