Engineering topics
Dinh, Nam
Publications and source records attributed to Dinh, Nam.
Knowledge representation to support EMDAP implementation in advanced reactor licensing applications
Not Available
Evaluating the reliability of machine-learning-based predictions used in nuclear power plant instrumentation and control systems
Not Available
An Integrated Framework for Risk Assessment of Safety-related Digital Instrumentation and Control Systems in Nuclear Power Plants: Methodology Advancement and Application
This report documents activities performed by Idaho National Laboratory (INL) during fiscal year (FY) 2024 for the U.S. Department of Energy (DOE) Light Water Reactor Sustainability (LWRS) Program, Risk Informed Systems Analysis (RISA) Pathway, Digital Instrumentation and Control (DI&C) Risk Assessment project. The goal of the RISA Pathway is to optimize safety margins and minimize uncertainties to achieve economic efficiencies while maintaining high levels of safety. This is accomplished by providing scientific basis to better represent safety margins and factors that contribute to cost and safety, and by developing new technologies that reduce operating costs. The research efforts for FY 2024 encompass methodology refinement and exploration. The efforts include: (1) The implementation of a natural language processing tool to expedite key aspects of the reliability analysis methods developed by INL; (2) advances to support intersystem CCF analysis by providing guidance for and identification of coupling mechanisms that may contribute to CCF; (3) the investigation of how generative artificial intelligence tools can aid in hazard analysis and diversity and defense in depth (i.e., D3) assessments; (4) Industry collaboration, allowing the demonstration of and INL's risk assessment tools to support risk assessment of DI&C systems at early and late stages of development; (4) a roadmap for the development of a software for each of INL's risk assessment tools; (5) The development of a theory and methodology manual for a risk quantification methodology; (6) the development of a reliability analysis for machine learning (ML)-integrated control systems.
Using Generative AI to implement the discrepancy checker for a Nearly Autonomous Management and Control System for Advanced Reactors
Developments related to generative artificial intelligence (AI) have brought a major breakthrough in AI. These developments are rapidly accelerating developments in different science and engineering applications. Nearly Autonomous Management and Control (NAMAC) system provides recommendations to the operator for maintaining the safety and performance of the reactor. The discrepancy checker (DC) is an important component of the NAMAC) system, whose goal is to determine if the plant is moving towards the expected system state after the control actions are injected. In this work, we explore generative AI methods, particularly, a generative pretrained transformer (GPT) for implementing the DC function in NAMAC. The GPT-based DC aims to alert the operator in situations outside NAMAC’s scope and act as a chatbot the operator can use to retrieve relevant information. This study involves two versions of GPT developed by OpenAI: GPT-3.5 and GPT-4. These GPTs are trained on huge amounts of undisclosed general domain datasets. We explored two methods to adapt GPTs for DC implementation in NAMAC: fine-tuning and retrieval augmented generation. A small knowledge base (information file) that encompasses rules for DC implementation and some general information related to NAMAC has been created to support DC implementation using GPT. In this work, the GPT-based DC implementations have been tested for their reasoning abilities, comprehension, information retrieval, and extraction abilities. It should be noted that this paper only presents a preliminary study to test the feasibility of DC implementation using generative AI technology. Given the potential risks and severe consequences associated with nuclear reactor applications, combined with the black-box nature of AI, extensive offline and online testing and reliability analyses of GPT-based DCs are needed for further developing such capabilities.
Trustworthiness modeling and evaluation for a nearly autonomous management and control system
The Nearly Autonomous Management and Control (NAMAC) system supports the advanced reactor operation by recommending control actions to operators based on real-time measurements and digital twins (DTs) learning from the knowledge base. To enable the safe and reliable use of autonomous technologies, NAMAC and its recommendations should be trustworthy to operators and regulators at both the design and operation stages. This study proposes a NAMAC trustworthiness modeling and evaluation framework supported by trustworthiness ontologies and evidence-based approaches. The development-time and run-time ontologies are separately constructed and then converted to Bayesian networks to quantitatively evaluate the NAMAC trustworthiness. This evaluation is demonstrated by collecting and characterizing evidence from NAMAC practices, such as the development and assessment of the NAMAC system, data coverage assessment, and the training and optimizations of neural-network-based DTs. Our proposed approach can aggregate various trustworthiness attributes of complex artificial-intelligence-supported systems for safety-critical applications. It also considers the interaction between different DTs and extends beyond the trustworthiness evaluation of a single DT. In conclusion, the evidence-based method enhances the transparency of the trustworthiness modeling and evaluation processes and helps identify uncertainties and subjectivity involved in the processes.
Risk Analysis of Various Design Architectures for High Safety-significant Safety-related Digital Instrumentation and Control Systems of Nuclear Power Plants during Accident Scenarios
This report documents the plus-up activities performed by Idaho National Laboratory (INL) during Fiscal Year (FY) 2022 for the U.S. Department of Energy (DOE) Light Water Reactor Sustainability (LWRS) Program, Risk Informed Systems Analysis (RISA) Pathway, digital instrumentation and control (DI&C) risk assessment project. In FY 2019, the RISA Pathway initiated a project to develop a risk assessment strategy for delivering a strong technical basis to support effective, licensable, and secure DI&C technologies for digital upgrades/designs. An integrated risk assessment technology for the DI&C systems was proposed for this strategy, which aims to (1) provide a best-estimate, risk-informed capability to quantitatively and accurately estimate the safety margin obtained from plant modernization, especially for the high safety-significant safety-related (HSSSR) DI&C systems, (2) support and supplement existing advanced risk-informed DI&C design guides by providing quantitative risk information and evidence, (3) offer a capability of design architecture evaluation of various DI&C systems to support system design decisions and diversity and redundancy applications, (4) assure the long-term safety and reliability of HSSSR DI&C systems, and (5) reduce uncertainty in costs and support integration of DI&C systems in the plant. To achieve these technical goals and deal with the expensive licensing justifications from regulatory insights, the LWRS-developed framework instructs nuclear vendors and utilities on how to effectively lower the costs associated with digital compliance and speed industry advances by: (1) defining an integrated risk-informed analysis process for DI&C upgrade, including hazard analysis, reliability analysis, and consequence analysis, (2) applying systematic and risk-informed tools to address common cause failures (CCFs) and quantify corresponding failure probabilities for DI&C technologies, particularly software CCFs, (3) evaluating the impact of digital failures at the component level, system level, and plant level, and (4) providing insights and suggestions on designs to manage the risks, thus to support the development, licensing, and deployment of advanced DI&C technologies on nuclear power plant (NPPs). Adding diversity within system or components is the main means to eliminate and mitigate CCFs, but diversity also increases plant complexity and errors and may not address all sources of systematic failures. How to optimize the diversity and redundancy applications for the safety-critical DI&C systems remains a challenge. To deal with the technical issues in addressing potential software CCFs in HSSSR DI&C systems of NPPs and supporting relevant design optimization, the framework provides: ? An integrated best-estimate, risk-informed capability to address new technical digital issues quantitatively, accurately, and efficiently in plan modernization progress, such as software CCFs in HSSSR DI&C systems of NPPs ? A common and a modularized platform for DI&C designers, software developers, cybersecurity analysts, and plant engineers to efficiently predict and prevent risk in the early design stage of DI&C systems ? Technical bases and risk-informed insights to assist U.S. Nuclear Regulatory Commission (NRC) and industry to address and fulfill the risk-informed alternatives for evaluation of CCFs in HSSSR DI&C systems of NPPs ? An integrated risk-informed tool that offers a capability of design architecture evaluation of various DI&C systems to support system design decisions in diversity and redundancy applications. The plus-up research and development efforts of this project in FY 2022 are focused on methodology improvement of software CCF modeling and estimation, prevention analysis, importance analysis and risk analysis of various design architectures of HSSSR DI&C systems. This work greatly enhances the capability of the LWRS-developed framework for the risk assessment and design optimization of safety-critical DI&C systems. It should be noted that all the analyses are performed for the demonstration of the LWRS-developed framework, not for the evaluation of relevant systems. Results are obtained based on very limited design information and testing data.
An Integrated Framework for Risk Assessment of Safety-related Digital Instrumentation and Control Systems in Nuclear Power Plants: Methodology Refinement and Exploration
This report documents activities performed by Idaho National Laboratory (INL) during Fiscal Year (FY) 2023 for the U.S. Department of Energy (DOE) Light Water Reactor Sustainability (LWRS) Program, Risk Informed Systems Analysis (RISA) Pathway, digital instrumentation and control (DI&C) risk assessment project. In FY 2019, the RISA Pathway initiated a project to develop a risk assessment strategy for delivering a technical basis to support effective, and secure DI&C technologies for digital upgrades/designs. A risk assessment-informed framework was proposed for this strategy, which aims to (1) provide a best-estimate, risk informed capability to quantitatively estimate the safety margin obtained from plant modernization, especially for safety-related DI&C systems, (2) support and supplement existing risk informed DI&C design guides by providing quantitative risk information and evidence, (3) offer a capability of design architecture evaluation of various DI&C systems, (4) assure the long-term safety and reliability of safety-related DI&C systems, and (5) reduce uncertainty in costs and support integration of DI&C systems in the plant. To achieve these technical goals, the LWRS-developed framework provides a means to address relevant technical issues by: (1) defining a risk informed analysis process for DI&C upgrade that integrates hazard analysis, reliability analysis, and consequence analysis, (2) applying risk informed tools to address common cause failures (CCFs) and quantify corresponding failure probabilities for DI&C technologies, particularly software CCFs, (3) evaluating the impact of digital failures at the component level, system level, and plant level, and (4) providing insights and suggestions on designs to manage the risks, thus to support the development and deployment of advanced DI&C technologies in nuclear power plants (NPPs). Adding diversity within a system or components is the primary means to eliminate and mitigate CCFs, but diversity also increases system complexity and may not address all sources of systematic failures. Optimization of diversity and redundancy applications for the safety-critical DI&C systems remains a challenge. To deal with the technical issues in addressing potential software CCFs in safety-related DI&C systems of NPPs and supporting relevant design optimization, the proposed framework provides: (a) A best-estimate, risk informed capability to address new technical digital issues quantitatively, focusing on software CCFs in safety-related DI&C systems of NPPs; (b) A common and a modularized platform for DI&C designers, software developers, cybersecurity analysts, and plant engineers to predict and prevent risk in the early design stage of DI&C systems; (c) Technical bases and risk informed insights to assist users address the risk informed alternatives for evaluation of CCFs in safety-related DI&C systems of NPPs; and (d) A risk informed tool that offers a capability of design architecture evaluation of various DI&C systems to support system design decisions in diversity and redundancy applications. The research and development efforts of this project in FY 2023 are focused on refining current methods on software CCF modeling and estimation and exploring additional innovative approaches to risk assessment of DI&C systems to enable a more comprehensive and complete assessment of various safety-related DI&C design architectures. The primary audience of this report are DI&C designers, engineers, and probabilistic risk assessment (PRA) practitioners. This includes stakeholders, such as the nuclear utilities and regulators who consider the deployment and upgrade of DI&C systems, DI&C software developers and reviewers, and cybersecurity specialists. It should be noted that all the analyses are performed for the demonstration of the methodology, not for the evaluation of an actual digital control system. Results are obtained based on limited design information and testing data.
Dynamic Model Agnostic Reliability Evaluation of Machine-Learning Models Integrated in Instrumentation & Control Systems
In recent years, the field of machine learning (ML), specifically neural networks, has grown significantly and has spurred research in its applicability to digital instrumentation and control systems (DI&C). While ML models have shown promise in operational contexts, the trustworthiness of using such algorithms has not been adequately assessed. Failures of ML integrated systems are not well understood, and the lack of comprehensive risk modeling can degrade the trustworthiness in these systems. In recent reports by the National Institute for Standards and Technology (NIST) [1] and the Nuclear Regulatory Commission (NRC) [2], they indicate that trustworthiness in ML is a critical barrier and will play a vital role in the safe, accountable, and secure operation of intelligent systems. Thus, in this work, we demonstrate a dynamic model-agnostic method to quantify the relative reliability of AI/ML predictions by incorporating out-of-distribution (OOD) detection on the training dataset. It is well documented that most ML algorithms excel at interpolation (or near-interpolation) tasks but experience significant performance degradation at extrapolation. The method, referenced as the Laplacian distributed decay for reliability (LADDR), determines the difference between the operational and training datasets which can used to the relative reliability of AI/ML predictions. LADDR is then demonstrated on a feedforward neural network based digital twin used for the prediction of safety significant factors during a loss-of-flow transient. LADDR is used to demonstrate how training data can be used as evidence to support the relative reliability of ML/AI predictions enhancing the overall trustworthiness of the system.
Dynamic Model Agnostic Reliability Evaluation of Machine-Learning Models Integrated in Instrumentation & Control Systems
In recent years, the field of data-driven neural network-based machine learning (ML) algorithms has grown significantly and spurred research in its applicability to instrumentation and control systems. While they are promising in operational contexts, the trustworthiness of such algorithms is not adequately assessed. Failures of ML-integrated systems are poorly understood; the lack of comprehensive risk modeling can degrade the trustworthiness of these systems. In recent reports by the National Institute for Standards and Technology, trustworthiness in ML is a critical barrier to adoption and will play a vital role in intelligent systems' safe and accountable operation. Thus, in this work, we demonstrate a real-time model-agnostic method to evaluate the relative reliability of ML predictions by incorporating out-of-distribution detection on the training dataset. It is well documented that ML algorithms excel at interpolation (or near-interpolation) tasks but significantly degrade at extrapolation. This occurs when new samples are "far" from training samples. The method, referred to as the Laplacian distributed decay for reliability (LADDR), determines the difference between the operational and training datasets, which is used to calculate a prediction's relative reliability. LADDR is demonstrated on a feedforward neural network-based model used to predict safety significant factors during different loss-of-flow transients. LADDR is intended as a "data supervisor" and determines the appropriateness of well-trained ML models in the context of operational conditions. Ultimately, LADDR illustrates how training data can be used as evidence to support the trustworthiness of ML predictions when utilized for conventional interpolation tasks.
Systems-Theoretic Hazard Analysis of Digital Human-System Interface Relevant to Reactor Trip
Slides for 2021 ANS-NPIC-HMIT conference
Uncertainty Quantification for Multiphase Computational Fluid Dynamics Closure Relations with a Physics-Informed Bayesian Approach
Multiphase Computational Fluid Dynamics (MCFD) based on the two-fluid model is considered a promising tool to model complex two-phase flow systems. MCFD simulation can predict local flow features without resolving interfacial information. As a result, the MCFD solver relies on closure relations to describe the interaction between the two phases. Those empirical or semi-mechanistic closure relations constitute a major source of uncertainty for MCFD predictions. In this paper, we leverage a physics-informed uncertainty quantification (UQ) approach to inversely quantify the closure relations’ model form uncertainty in a physically consistent manner. This proposed approach considers the model form uncertainty terms as stochastic fields that are additive to the closure relation outputs. Combining dimensionality reduction and Gaussian processes, the posterior distribution of the stochastic fields can be effectively quantified within the Bayesian framework with the support of experimental measurements. As this UQ approach is fully integrated into the MCFD solving process, the physical constraints of the system can be naturally preserved in the UQ results. Here, in a case study of adiabatic bubbly flow, we demonstrate that this UQ approach can quantify the model form uncertainty of the MCFD interfacial force closure relations, thus effectively improving the simulation results with relatively sparse data support.
Data coverage assessment on neural network based digital twins for autonomous control system
We report in a recently developed Nearly Autonomous Management and Control (NAMAC) system, neural networks (NNs) are used to develop digital twins for diagnosis (DT-Ds). However, NNs are not usually considered extrapolation models and may result in large errors if they are applied to unseen data outside the training data (uncovered). In this study, we propose a data coverage assessment (DCA) to determine if the NN-based DT-Ds are extrapolated based on their epistemic uncertainty. The uncertainty quantification algorithms and uncertainty thresholds are selected based on the confusion matrix of classifying evaluation data into covered or uncovered data. To demonstrate the adaptability of the proposed framework, we applied it to a basic feedforward neural network and a more advanced recurrent neural network based on a more nonlinear database. Case studies show that the proposed framework can distinguish unseen data for both basic and advanced applications with proper uncertainty quantification algorithms and thresholds.
Development and assessment of prognosis digital twin in a NAMAC system
The nearly autonomous management and control (NAMAC) system is a comprehensive control system to assist plant operations by furnishing control recommendations to operators. Prognosis digital twin (DT-P) is a critical component in NAMAC for predicting action effects and supporting NAMAC decision-making during normal and accident scenarios. To quantifying and reducing uncertainty of machine-learning-based DT-Ps in multi-step predictions, this work investigates and derives insights from the application of three techniques for optimizing the performance of DT-P by long short-term memory recurrent neural networks, including manual search, sequential model-based optimization, and physics-guided machine learning. Finally, sequential model-based optimization and physics-guide machine learning result in smallest errors when the predicting transients are similar to the training data.
An Integrated Framework for Risk Assessment of High Safety Significant Safety-related Digital Instrumentation and Control Systems in Nuclear Power Plants: Methodology and Demonstration
This report documents the activities performed by Idaho National Laboratory (INL) during Fiscal Year (FY) 2022 for the U.S. Department of Energy (DOE) Light Water Reactor Sustainability (LWRS) Program, Risk Informed Systems Analysis (RISA) Pathway, digital instrumentation and control (DI&C) risk assessment project. In FY 2019, the RISA Pathway initiated a project to develop a risk assessment strategy for delivering a technical basis to support effective and secure DI&C technologies for digital upgrades/designs. A framework was proposed for this strategy, which aims to (1) provide a best-estimate, risk-informed capability to quantitatively and accurately estimate the risk impact of plant modernization, considering the introduction of high safety-significant safety-related (HSSSR) DI&C systems, (2) support and supplement existing risk-informed DI&C design guides by providing quantitative risk information and evidence, (3) offer a capability of design architecture evaluation of various DI&C systems, (4) assure the long-term safety and reliability of HSSSR DI&C systems, and (5) reduce uncertainty in costs and support integration of DI&C systems in the plant. To achieve these technical goals, the framework provides a means to address relevant technical issues by: (1) defining a risk-informed analysis process for DI&C upgrade, that integrates hazard analysis, reliability analysis, and consequence analysis, (2) applying risk-informed tools to address common cause failures (CCFs) and quantify corresponding failure probabilities for DI&C technologies, particularly software CCFs, (3) evaluating the impact of digital failures at the component level, system level, and plant level, and (4) providing insights and suggestions on designs to manage the risks, thus to support the development and deployment of advanced DI&C technologies on nuclear power plant (NPPs).
Application of Orthogonal Defect Classification for Software Reliability Analysis
The modernization of existing and new nuclear power plants with digital instrumentation and control systems (DI&C) is a recent and highly trending topic. However, there lacks strong consensus on best-estimate reliability methodologies by both the United States (U.S.) Nuclear Regulatory Commission (NRC) and the industry. This has resulted in hesitation for further modernization projects until a more unified methodology is realized. In this work, we develop an approach called Orthogonal-defect Classification for Assessing Software Reliability (ORCAS) to quantify probabilities of various software failure modes in a DI&C system. The method utilizes accepted industry methodologies for software quality assurance that are also verified by experimental or mathematical formulations. In essence, the approach combines a semantic failure classification model with a reliability growth model to predict (and quantify) the potential failure modes of a DI&C software system. The semantic classification model is used to address the question: How do latent defects in software contribute to different software failure root causes? The use of reliability growth models is then used to address the question: Given the connection between latent defects and software failure root causes, how can we quantify the reliability of the software? A case study was conducted on a representative I&C platform (ChibiOS) running a smart sensor acquisition software developed by Virginia Commonwealth University (VCU). The testing and evidence collection guidance in ORCAS was applied, and defects were uncovered in the software. Qualitative evidence, such as condition coverage, was used to gauge the completeness and trustworthiness of the assessment while quantitative evidence was used to determine the software failure probabilities. The reliability of the software was then estimated and compared to existing operational data of the sensor device. It is demonstrated that by using ORCAS, a semantic reasoning framework can be developed to justify if the software is reliable (or unreliable) while still leveraging the strength of the existing methods.
Failure Mechanism Traceability and Application in Human System Interface of Nuclear Power Plants using RESHA
In recent years, there has been considerable effort to modernize existing and new nuclear power plants with digital instrumentation and control systems (DI&C). However, there has also been considerable concern both by industry and regulatory bodies for the risk and consequence analysis of these systems. Of particular concern are digital common cause failures (CCFs) specifically related to software defects. These “misbehaviors” by the software can occur in both the control and monitoring of a system. While many new methods have been proposed to identify potential software failure modes, such as Systems-theoretic Process Analysis (STPA), Hazard and Consequence Analysis for Digital Systems (HAZCADS), etc., these methods are focused primarily on the control action pathway of a system. In contrast, the information feedback pathway lacks unsafe control actions (UCAs), which are typically related to software basic events; thus, assessment of software basic events in such systems is unclear. In this work, we present the idea of intermediate processors and unsafe information flow (UIF) to help safety analysts trace failure mechanisms in the feedback pathway and how they can be integrated into a fault tree for improved assessment capability. The concepts presented are demonstrated in two comprehensive case studies, a smart sensor integrated platform for unmanned autonomous vehicles and another on a representative advanced human system interface (HSI) for safety critical plant monitoring. The qualitative software basic events are identified, and a fault tree analysis is conducted based on a modified Redundancy-guided Systems-theoretic Hazard Analysis (RESHA) methodology. The case studies demonstrate the use of UIF and intermediate processors in the fault tree to improve traceability of software failures in highly complex digital instrumentation feedback. The improved method can also clarify fault tree construction when multiple component dependencies are present in the system.
Application of Orthogonal Defect Classification for Software Reliability Analysis
The modernization of existing and new nuclear power plants with digital instrumentation and control systems (DI&C) is a recent and highly trending topic. However, there lacks strong consensus on best-estimate reliability methodologies by both the United States (U.S.) Nuclear Regulatory Commission (NRC) and the industry. This has resulted in hesitation for further modernization projects until a more unified methodology is realized. In this work, we develop an approach called Orthogonal-defect Classification for Assessing Software Reliability (ORCAS) to quantify probabilities of various software failure modes in a DI&C system. The method utilizes accepted industry methodologies for software quality assurance that are also verified by experimental or mathematical formulations. In essence, the approach combines a semantic failure classification model with a reliability growth model to predict (and quantify) the potential failure modes of a DI&C software system. The semantic classification model is used to address the question: How do latent defects in software contribute to different software failure root causes? The use of reliability growth models is then used to address the question: Given the connection between latent defects and software failure root causes, how can we quantify the reliability of the software? A case study was conducted on a representative I&C platform (ChibiOS) running a smart sensor acquisition software developed by Virginia Commonwealth University (VCU). The testing and evidence collection guidance in ORCAS was applied, and defects were uncovered in the software. Qualitative evidence, such as condition coverage, was used to gauge the completeness and trustworthiness of the assessment while quantitative evidence was used to determine the software failure probabilities. The reliability of the software was then estimated and compared to existing operational data of the sensor device. It is demonstrated that by using ORCAS, a semantic reasoning framework can be developed to justify software reliability (or unreliability) while still leveraging the strength of the existing methods.