Search NASA⌕ Search

SEARCH · Search NASA

Results for “software test efficiency software risk assessment”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

5 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Improving and Automating Building Model Data Exchange

There are many instances throughout a project’s lifecycle where there arises a need for quick and accurate risk assessment of building designs. For example, an unexpected design change during construction may necessitate structural engineers to perform a seismic risk assessment on analytical models of the updated building design using high fidelity structural analysis software, such as ANSYS or Abaqus. However, the efficiency of such workflows often depends upon the interoperability of architectural design software and structural analysis software. When the quality of this interoperability is lacking or even non-existent, the efficiency of virtual engineering workflows is hampered, which increases project costs. A McGraw Hill industry survey of professional users of Building Information Modeling (BIM) technologies found that there is high demand for BIM interoperability for structural analysis, but that the value/difficulty ratio is currently too low for practical use. There have been efforts by the academic community to facilitate model data exchange between the architectural design and structural analysis domains, but such solutions have not been widely adopted by industry, face technical challenges, and oftentimes are limited in applicability for users of various BIM software. Therefore, INL is developing capabilities to improve, automate, and generalize model data exchange between architectural BIM software (e.g., Revit) and structural analysis software (e.g., SAP2000, ANSYS). The goal is to help expedite and automate as much of the pre-processing step for creating analytical models in finite element analysis software as reasonably as possible. Such a "BIM-to-FEA" conversion tool should provide direct benefit to end-users through accuracy, automation, quick turn-around, and wide applicability. To generalize the application of this BIM-to-FEA conversion tool and increase its useability among the many different commercial BIM software currently used by industry, the program is being developed with the concept of openBIM. OpenBIM is the application of non-proprietary, open data standards that allow for BIM model data exchange in a format that is accessible, retainable, and useable for all users. The most widely used open, non-proprietary data exchange format for BIM is the Industry Foundation Classes (IFC) schema. IFC is developed by buildingSMART international and is ISO certified (ISO 16739-1:2018). The BIM-to-FEA conversion tool is being developed for compatibility with typical commercial building designs of steel framed structures. The tool is currently capable of importing architectural BIM data of framed building structures, recognizing and extracting the aspects of the model that are required for structural analysis, adjusting the connectivity of frame members, and finally exporting to an analytical model stored in the IFC format. The exported IFC analytical model can then be imported into various openBIM compliant software, such as SAP2000. Such capabilities have already been tested on commercial software, as shown above, and continue to be improved. Work is underway to test the conversion on various commercial BIM software, develop a user-friendly interface, incorporate the program into the broader DeepLynx data warehouse project being developed by INL, and to eventually open-source the tool for the benefit of the community. Future development of the tool envisions the ability for efficient iterative risk assessment of generative building designs, all within a workflow utilizing open-source tools. One such open-source tool will be MOOSE, an advanced finite element analysis tool developed at INL. The conversion tool will also branch out from typical commercial building designs and will aim to incorporate nuclear construction. The aim will be to convert both structural and non-structural components of nuclear facilities, such as curved concrete containment structures and piping systems, respectively.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Cyber risk assessment and investment optimization using game theory and ML-based anomaly detection and mitigation for wide-area control in smart grids

The electric power grid is increasingly becoming susceptible to cyber attacks that exploit vulnerabilities in the smart grid control, information, and physical layers. Successful cyber attacks can have catastrophic impacts on the social and economic well-being of any nation all over the globe. It has, thus, become imperative to secure the smart grid against such adversarial actions to ensure stable, secure, and reliable operation of the grid. The existing research and industry practices prove to be inadequate in terms of providing pragmatic and effective defense methodologies and measures for long-term cybersecurity planning and real-time cybersecurity for grid operation. For example, existing works lack models that incorporate uncertain behavior of cyber-attackers and pragmatic defense measures for cyber risk assessment and cybersecurity investment optimization which often provide unreliable and strictly qualitative solutions to these problems. At the same time, with the growing number of cyber incidents in the grid, there still exists a need to develop attack-resilient algorithms for wide-area monitoring, protection, and control (WAMPAC) applications like the wide-area voltage control systems (WAVCS) for Flexible AC Transmissions Systems (FACTS) that lack in scalable and feasible solutions from the cybersecurity perspective. This dissertation proposes novel models and methodologies for: (1) Cybersecurity planning, and (2) Cybersecurity for system operation. The cybersecurity planning is achieved through cyber risk assessment and cybersecurity resource investment optimization for long-term cybersecurity of the grid using game theory and attack-defense trees. Cybersecurity for system operation consists of development of cyber anomaly detection and mitigation algorithms for flexible AC transmission system (FACTS) controller-based wide-area voltage control systems (WAVCS) using machine learning (ML), and software defined networking-based moving target defense network routing for achieving real-time cyber-physical security for grid operations. This is followed by hardware-in-the-loop (HIL) implementation and evaluation of these attack prevention, detection, and mitigation algorithms and methodologies showcasing their feasibility in a close to real-world environment. For cybersecurity planning, a novel approach involving a combination of game theory and attack defense trees (ADT) for optimal cybersecurity resource allocation in the smart grid is proposed. This methodology involves modeling of the cyber-physical smart grid substations as ADTs, defining attacker costs, defense costs, and attack probabilities for attack access points. Using game theoretical formulation, optimal defense strategies for the defender of the system to invest cybersecurity resources in the grid are obtained. Additionally, a game-theoretic framework is developed for quantitative cyber-physical risk assessment of the grid under a dynamically changing cyber threat space and uncertain behavior of cyber attackers which is further used to optimize investments in the smart grid's cybersecurity resources. The attacker, defender, and the smart grid system are modeled while incorporating attacker-stochasticity and federal guidelines for smart grid cybersecurity. This allows quantification of threat, vulnerabilities, and attack impact of the grid for quantitative risk assessment. The defender's budget to invest in the security resources in the grid is optimized based on the strategies leading to minimum system risk. The evaluation of the proposed solutions highlight the feasibility for practical implementation of these methodologies and algorithms in the smart grid, while taking the federal requirements and guidelines for smart grid security into consideration. For achieving cybersecurity for system operation, attack prevention, detection, and mitigation algorithms and methodologies are developed specifically for FACTS-based WAVCS. Anomaly detection and mitigation in the WAVCS are achieved using algorithms based on machine learning which involves offline training and testing of ML models with CPS datasets incorporating physics-based features that allow accurate distinction between system faults and cyber attacks. For attack prevention, a methodology based on software defined network (SDN)-based moving target defense (MTD) network routing is proposed that enables prevention of Denial of Service (DoS) type attacks on the smart grid communication system. Subsequently, these methodologies and algorithms are implemented and evaluated on an HIL testbed that allows for real-time attack prevention, detection, and mitigation of emulated cyber attacks on the WAVCS in a close to real-world environment. The results show highly accurate and efficient performance of the implemented algorithms and methodologies with the smart grid system operating within the NERC's system operation limits even in the presence of DoS and data integrity cyber attacks. This work opens up future research opportunities in other directions such as (1) Expanding cybersecurity planning methodologies to real-time cyber contingency analysis with different game formulations; and (2) Applying the cybersecurity for system operation algorithms to broader categories of wide-area control applications.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Regional-scale fault-to-structure earthquake simulations with the EQSIM framework: Workflow maturation and computational performance on GPU-accelerated exascale platforms

Continuous advancements in scientific and engineering understanding of earthquake phenomena, combined with the associated development of representative physics-based models, is providing a foundation for high-performance, fault-to-structure earthquake simulations. However, regional-scale applications of high-performance models have been challenged by the computational requirements at the resolutions required for engineering risk assessments. The EarthQuake SIMulation (EQSIM) framework, a software application development under the US Department of Energy (DOE) Exascale Computing Project, is focused on overcoming the existing computational barriers and enabling routine regional-scale simulations at resolutions relevant to a breadth of engineered systems. This multidisciplinary software development—drawing upon expertise in geophysics, engineering, applied math and computer science—is preparing the advanced computational workflow necessary to fully exploit the DOE’s exaflop computer platforms coming online in the 2023 to 2024 timeframe. Achievement of the computational performance required for high-resolution regional models containing upward of hundreds of billions to trillions of model grid points requires numerical efficiency in every phase of a regional simulation. This includes run time start-up and regional model generation, effective distribution of the computational workload across thousands of computer nodes, efficient coupling of regional geophysics and local engineering models, and application-tailored highly efficient transfer, storage, and interrogation of very large volumes of simulation data. This article summarizes the most recent advancements and refinements incorporated in the workflow design for the EQSIM integrated fault-to-structure framework, which are based on extensive numerical testing across multiple graphics processing unit (GPU)-accelerated platforms, and demonstrates the computational performance achieved on the world’s first exaflop computer platform through representative regional-scale earthquake simulations for the San Francisco Bay Area in California, USA.

58 GEOSCIENCES↗
DOE OSTI2023

Risk Analysis of Various Design Architectures for High Safety-significant Safety-related Digital Instrumentation and Control Systems of Nuclear Power Plants during Accident Scenarios

This report documents the plus-up activities performed by Idaho National Laboratory (INL) during Fiscal Year (FY) 2022 for the U.S. Department of Energy (DOE) Light Water Reactor Sustainability (LWRS) Program, Risk Informed Systems Analysis (RISA) Pathway, digital instrumentation and control (DI&C) risk assessment project. In FY 2019, the RISA Pathway initiated a project to develop a risk assessment strategy for delivering a strong technical basis to support effective, licensable, and secure DI&C technologies for digital upgrades/designs. An integrated risk assessment technology for the DI&C systems was proposed for this strategy, which aims to (1) provide a best-estimate, risk-informed capability to quantitatively and accurately estimate the safety margin obtained from plant modernization, especially for the high safety-significant safety-related (HSSSR) DI&C systems, (2) support and supplement existing advanced risk-informed DI&C design guides by providing quantitative risk information and evidence, (3) offer a capability of design architecture evaluation of various DI&C systems to support system design decisions and diversity and redundancy applications, (4) assure the long-term safety and reliability of HSSSR DI&C systems, and (5) reduce uncertainty in costs and support integration of DI&C systems in the plant. To achieve these technical goals and deal with the expensive licensing justifications from regulatory insights, the LWRS-developed framework instructs nuclear vendors and utilities on how to effectively lower the costs associated with digital compliance and speed industry advances by: (1) defining an integrated risk-informed analysis process for DI&C upgrade, including hazard analysis, reliability analysis, and consequence analysis, (2) applying systematic and risk-informed tools to address common cause failures (CCFs) and quantify corresponding failure probabilities for DI&C technologies, particularly software CCFs, (3) evaluating the impact of digital failures at the component level, system level, and plant level, and (4) providing insights and suggestions on designs to manage the risks, thus to support the development, licensing, and deployment of advanced DI&C technologies on nuclear power plant (NPPs). Adding diversity within system or components is the main means to eliminate and mitigate CCFs, but diversity also increases plant complexity and errors and may not address all sources of systematic failures. How to optimize the diversity and redundancy applications for the safety-critical DI&C systems remains a challenge. To deal with the technical issues in addressing potential software CCFs in HSSSR DI&C systems of NPPs and supporting relevant design optimization, the framework provides: ? An integrated best-estimate, risk-informed capability to address new technical digital issues quantitatively, accurately, and efficiently in plan modernization progress, such as software CCFs in HSSSR DI&C systems of NPPs ? A common and a modularized platform for DI&C designers, software developers, cybersecurity analysts, and plant engineers to efficiently predict and prevent risk in the early design stage of DI&C systems ? Technical bases and risk-informed insights to assist U.S. Nuclear Regulatory Commission (NRC) and industry to address and fulfill the risk-informed alternatives for evaluation of CCFs in HSSSR DI&C systems of NPPs ? An integrated risk-informed tool that offers a capability of design architecture evaluation of various DI&C systems to support system design decisions in diversity and redundancy applications. The plus-up research and development efforts of this project in FY 2022 are focused on methodology improvement of software CCF modeling and estimation, prevention analysis, importance analysis and risk analysis of various design architectures of HSSSR DI&C systems. This work greatly enhances the capability of the LWRS-developed framework for the risk assessment and design optimization of safety-critical DI&C systems. It should be noted that all the analyses are performed for the demonstration of the LWRS-developed framework, not for the evaluation of relevant systems. Results are obtained based on very limited design information and testing data.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2019

Comparative Assessment of Experimental Testing of Instrument with an Embedded Digital Device Using Model-Based and Conventional Methods

The use of digital equipment has not been widespread in U.S. nuclear facilities. A major contributing factor to this situation arises from regulatory concerns about common cause failures (CCFs) stemming from the use of identical software in redundant systems, questions as to how to quantify the reliability of software-based products, validity of tools that are used for software verification and validation (V&V), and the effectiveness of software QA procedures. Compounding the concerns are sporadic incidents of digital equipment malfunctions in nuclear power plants, including lockups resulting from software memory leaks, feedwater control malfunctions due to excessive traffic on Ethernet networks, software logic errors, and other problems leading to plant trips. Additionally, the limited amount of demonstrated research and a lack of industry consensus on quantitative software reliability methodologies has hindered the widespread adoption of digital I&C in the nuclear industry. In a recent Regulatory Issue Summary (RIS), the U.S. Nuclear Regulatory Commission (NRC) identified concerns about the impact of greater use of instrumentation and control (I&C) equipment with an embedded digital device (EDD) [1]. Specifically, the NRC staff states that increased use of such devices “may increase a facility’s vulnerability to a CCF” (i.e., common-cause failure). The prevailing NRC guidance regarding software CCF (SCCF) identifies two design attributes that are acceptable for eliminating CCF concerns: (1) diversity or (2) testability (specifically, 100% testability). Either solution can result in high costs and remaining licensing uncertainty (i.e., how much diversity is enough? how to ensure test coverage of every possible sequence of device states?). Consequently, many utilities and reactor designers have limited or avoided more extensive use of digital technology to minimize licensing, scheduling, and financial risk. Without development of cost effective qualification methods to satisfy regulatory requirements and address the potential for CCF vulnerability associated with EDDs, the nuclear power industry may not be able to realize the benefits of digital technology achieved by other industries. In order to address these challenges, the U.S. Department of Energy (DOE) is sponsoring this research to assess the prospect of CCF in digital I&C equipment. The purpose of the current research is to develop an effective approach employing science-based methods to resolve concerns about CCF vulnerability that serve to inhibit deployment of advanced instrumentation (e.g., sensors, actuators, microcontrollers) with EDDs in nuclear power applications. The research objectives address the challenge of establishing high levels of safety and reliability assurance needed for the qualification of EDDs (e.g., microprocessors, programmable logic devices) that are subject to software design faults, complex failure modes, and CCF vulnerability. Specific objectives are: (1) assess the regulatory context for treatment of CCF vulnerability a graded approach to their qualification, (3) develop and extend model-based testing methods to enable effective demonstration of whether devices are subject to CCF, which may arise from vulnerabilities introduced at any stage of the design lifecycle, (4) establish a cost-effective testing framework that incorporates automation and test scenario prioritization, and (5) demonstrate the qualification approach through selection and testing of candidate digital device(s). This report documents an assessment of the results from experimental testing of a representative instrument with an embedded digital device. The experiment serves as a proof-of-concept demonstration of a model-based testing (MBT) framework. The objective of MBT is to ensure that an EDD undergoes suitably comprehensive testing to establish that it is not subject to SCCF. The approach by which MBT accomplishes this objective is through development and application of an effective test suite that provides the appropriate coverage to detect the full range of postulated faults that may arise at the requirements, design, and code (implementation) level of the EDD software. The findings of the experiment involve the application of the MBT framework by Ohio State University (OSU) through testing of the selected demonstration instrument (i.e., the prototype smart pressure sensor developed by Virginia Commonwealth University, VCU). In addition, the experiment findings also involve application of a more conventional black-box testing approach that serves as a baseline against which the MBT methodology can be compared. The primary outcome documented in this report is the comparative analysis of the capabilities demonstrated for each method. The results of the analysis demonstrate clear benefits that arise from the MBT methodology. The comparison indicates greater effectiveness from the more comprehensive evidence available through MBT and enhanced efficiency achieved through automation within the MBT framework. The structure of the report addresses the approach and application of each test method as well as the comparative analysis of the experimental results. Chapter 2 gives a summary overview of the demonstration approach. Chapter 3 covers the baseline testing and Chapter 4 documents the model-based testing. Chapter 5 describes the findings of the comparative analysis and presents the conclusions that can be drawn about the effectiveness and efficiency of the MBT methodology.

47 OTHER INSTRUMENTATION↗