Proof or Consequences the Verification of Curiosity's Software
No abstract available
logic model checking↗
SEARCH · Search NASA
Results for “Software Verification”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Program Verification for Extreme-Scale Applications (Final Scientific/Technical Report)
The project seeks to develop tools and techniques to help software developers verify the correctness and accuracy of their code. The target domain is scientific software of the kind widely used and developed in the Department of Energy research community, with a particular focus on "extreme scale" programs - those that are expected to involve possibly millions of parallel threads of execution. The report covers the University of Delaware contribution to the collaborative project. The project had a number of successful outcomes, especially regarding the development and extension of the CIVL software verification framework. CIVL is a verification tool for C or Fortran programs that use MPI, OpenMP, CUDA, and/or Pthreads for parallelization. CIVL went through vast improvements and extensions, and was successfully applied to a number of challenging codes. It found a subtle bug in the Devito PDE framework. It was able to verify the functional correctness of a conjugate gradient solver using a novel probabilistic technique with vanishingly small chance of error. CIVL was used very successfully in verification competitions, and the CIVL solutions to the competition challenges were presented and published. CIVL was also used successfully by other researchers on a computational chemistry kernel.
97 MATHEMATICS AND COMPUTING↗
Comparative Assessment of Experimental Testing of Instrument with an Embedded Digital Device Using Model-Based and Conventional Methods
The use of digital equipment has not been widespread in U.S. nuclear facilities. A major contributing factor to this situation arises from regulatory concerns about common cause failures (CCFs) stemming from the use of identical software in redundant systems, questions as to how to quantify the reliability of software-based products, validity of tools that are used for software verification and validation (V&V), and the effectiveness of software QA procedures. Compounding the concerns are sporadic incidents of digital equipment malfunctions in nuclear power plants, including lockups resulting from software memory leaks, feedwater control malfunctions due to excessive traffic on Ethernet networks, software logic errors, and other problems leading to plant trips. Additionally, the limited amount of demonstrated research and a lack of industry consensus on quantitative software reliability methodologies has hindered the widespread adoption of digital I&C in the nuclear industry. In a recent Regulatory Issue Summary (RIS), the U.S. Nuclear Regulatory Commission (NRC) identified concerns about the impact of greater use of instrumentation and control (I&C) equipment with an embedded digital device (EDD) [1]. Specifically, the NRC staff states that increased use of such devices “may increase a facility’s vulnerability to a CCF” (i.e., common-cause failure). The prevailing NRC guidance regarding software CCF (SCCF) identifies two design attributes that are acceptable for eliminating CCF concerns: (1) diversity or (2) testability (specifically, 100% testability). Either solution can result in high costs and remaining licensing uncertainty (i.e., how much diversity is enough? how to ensure test coverage of every possible sequence of device states?). Consequently, many utilities and reactor designers have limited or avoided more extensive use of digital technology to minimize licensing, scheduling, and financial risk. Without development of cost effective qualification methods to satisfy regulatory requirements and address the potential for CCF vulnerability associated with EDDs, the nuclear power industry may not be able to realize the benefits of digital technology achieved by other industries. In order to address these challenges, the U.S. Department of Energy (DOE) is sponsoring this research to assess the prospect of CCF in digital I&C equipment. The purpose of the current research is to develop an effective approach employing science-based methods to resolve concerns about CCF vulnerability that serve to inhibit deployment of advanced instrumentation (e.g., sensors, actuators, microcontrollers) with EDDs in nuclear power applications. The research objectives address the challenge of establishing high levels of safety and reliability assurance needed for the qualification of EDDs (e.g., microprocessors, programmable logic devices) that are subject to software design faults, complex failure modes, and CCF vulnerability. Specific objectives are: (1) assess the regulatory context for treatment of CCF vulnerability a graded approach to their qualification, (3) develop and extend model-based testing methods to enable effective demonstration of whether devices are subject to CCF, which may arise from vulnerabilities introduced at any stage of the design lifecycle, (4) establish a cost-effective testing framework that incorporates automation and test scenario prioritization, and (5) demonstrate the qualification approach through selection and testing of candidate digital device(s). This report documents an assessment of the results from experimental testing of a representative instrument with an embedded digital device. The experiment serves as a proof-of-concept demonstration of a model-based testing (MBT) framework. The objective of MBT is to ensure that an EDD undergoes suitably comprehensive testing to establish that it is not subject to SCCF. The approach by which MBT accomplishes this objective is through development and application of an effective test suite that provides the appropriate coverage to detect the full range of postulated faults that may arise at the requirements, design, and code (implementation) level of the EDD software. The findings of the experiment involve the application of the MBT framework by Ohio State University (OSU) through testing of the selected demonstration instrument (i.e., the prototype smart pressure sensor developed by Virginia Commonwealth University, VCU). In addition, the experiment findings also involve application of a more conventional black-box testing approach that serves as a baseline against which the MBT methodology can be compared. The primary outcome documented in this report is the comparative analysis of the capabilities demonstrated for each method. The results of the analysis demonstrate clear benefits that arise from the MBT methodology. The comparison indicates greater effectiveness from the more comprehensive evidence available through MBT and enhanced efficiency achieved through automation within the MBT framework. The structure of the report addresses the approach and application of each test method as well as the comparative analysis of the experimental results. Chapter 2 gives a summary overview of the demonstration approach. Chapter 3 covers the baseline testing and Chapter 4 documents the model-based testing. Chapter 5 describes the findings of the comparative analysis and presents the conclusions that can be drawn about the effectiveness and efficiency of the MBT methodology.
47 OTHER INSTRUMENTATION↗
Electric power system test and verification program
Space Station Freedom's (SSF's) electric power system (EPS) hardware and software verification is performed at all levels of integration, from components to assembly and system level tests. Careful planning is essential to ensure the EPS is tested properly on the ground prior to launch. The results of the test performed on breadboard model hardware and analyses completed to date have been evaluated and used to plan for design qualification and flight acceptance test phases. These results and plans indicate the verification program for SSF's 75-kW EPS would have been successful and completed in time to support the scheduled first element launch.
Rylicki, Daniel S.↗
Uncertainty Propagation from Flux to α and from Time to α in Reaction History
This report documents measurement uncertainty propagation from flux to α and from time to α in gamma reaction history. Analytical formulas are derived, and on the basis of them, their discrete versions are provided to serve for current reaction history software verification, and for development of the future versions of the software. The discrete version is provided for the general case: sparse data with varying time intervals between the data points.
73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
7 Processes that Enable NASA Software Engineering Technologies: Value-Added Process Engineering
The presentation reviews Agency process requirements and the purpose, benefits, and experiences or seven software engineering processes. The processes include: product integration, configuration management, verification, software assurance, measurement and analysis, requirements management, and planning and monitoring.
Housch, Helen↗
Verification of the REBUS Software
Ongoing design activities at Argonne National Laboratory are requiring a thorough verification of the Argonne Reactor Computation codes be performed. REBUS is central to this system. The driver for this effort requires the Triangular-Z and hexagonal-Z core geometry options of REBUS to be verified. Previous work identified the REBUS features required to be verified to support current design activities, features of which are generally applicable to hexagonal-Z fast reactor designs. The scope of this verification effort includes verifying REBUS’s ability to correctly intepret the user input model, verifying that the features identified yield the intended results, and verifying the correctness of the REBUS output tables. The REBUS software verification relies heavily upon the accuracy of the embedded DIF3D software, the verification of which was completed and documented elsewhere. Given that DIF3D produces an accurate solution, the primary focus of the verification in the REBUS software is to ensure that it properly uses the DIF3D solution and that the depletion system (Bateman equations) are correctly implemented. This manuscript reiterates the verification tasks and displays results with respect to the features needed for current design activities. Analytic solutions of the Batemen equations are displayed and the results calculated with REBUS are displayed demonstrating the accuracy. Since coupled Bateman and neutron diffusion/transport solutions are extremely difficult to obtain, much of the focus is placed on how REBUS uses a given DIF3D solution assuming the accuracy of the DIF3D solution. The verification effort identified no issues that are debilitating or otherwise impactful to the design usage of REBUS, and thus REBUS version 11.0, release 3012 is considered verified. It is important to note that several outputs of REBUS are identified to be inaccurate, such as burnup in MWD/MT. Most of the relevant ones for VTR are generally accurate with 10-20% errors which is not impactful as all regular REBUS users are aware of this issue and know how to hand calculate the results. The REBUS manual further makes it clear that these values are consistent with the methodology being used by REBUS and thus the “errors” are more of an inconsistent definition with respect to what a user would expect given a definition in literature. Other issues that were identified included unclear documentation and software bugs all of which were inconsequential to the final results.
22 GENERAL STUDIES OF NUCLEAR REACTORS↗
Expert system verification and validation study. Delivery 3A and 3B: Trip summaries
Key results are documented from attending the 4th workshop on verification, validation, and testing. The most interesting part of the workshop was when representatives from the U.S., Japan, and Europe presented surveys of VV&T within their respective regions. Another interesting part focused on current efforts to define industry standards for artificial intelligence and how that might affect approaches to VV&T of expert systems. The next part of the workshop focused on VV&T methods of applying mathematical techniques to verification of rule bases and techniques for capturing information relating to the process of developing software. The final part focused on software tools. A summary is also presented of the EPRI conference on 'Methodologies, Tools, and Standards for Cost Effective Reliable Software Verification and Validation. The conference was divided into discussion sessions on the following issues: development process, automated tools, software reliability, methods, standards, and cost/benefit considerations.
French, Scott↗
A digital flight control system verification laboratory
A NASA/FAA program has been established for the verification and validation of digital flight control systems (DFCS), with the primary objective being the development and analysis of automated verification tools. In order to enhance the capabilities, effectiveness, and ease of using the test environment, software verification tools can be applied. Tool design includes a static analyzer, an assertion generator, a symbolic executor, a dynamic analysis instrument, and an automated documentation generator. Static and dynamic tools are integrated with error detection capabilities, resulting in a facility which analyzes a representative testbed of DFCS software. Future investigations will ensue particularly in the areas of increase in the number of software test tools, and a cost effectiveness assessment.
De Feo, P.↗
From Verified Models to Verifiable Code
Declarative specifications of digital systems often contain parts that can be automatically translated into executable code. Automated code generation may reduce or eliminate the kinds of errors typically introduced through manual code writing. For this approach to be effective, the generated code should be reasonably efficient and, more importantly, verifiable. This paper presents a prototype code generator for the Prototype Verification System (PVS) that translates a subset of PVS functional specifications into an intermediate language and subsequently to multiple target programming languages. Several case studies are presented to illustrate the tool's functionality. The generated code can be analyzed by software verification tools such as verification condition generators, static analyzers, and software model-checkers to increase the confidence that the generated code is correct.
Lensink, Leonard↗
Software Quality Assurance Plan ANSYS LSDYNA Version 2023R1
ANSYS Inc. develops and markets engineering simulation software and services used in the aerospace, automotive, manufacturing, electronics, biomedical, energy, defense, and many other industries. ANSYS is dedicated to engineering simulation and is the world’s leading software provider. ANSYS was founded in 1970 and is headquartered in Canonsburg, Pennsylvania. ANSYS provides an engineering analysis tool combining structural, thermal, computational fluid dynamics, acoustic and electromagnetic simulation capabilities. ANSYS LS-DYNA is the most used explicit simulation program capable of simulating the response of materials to short periods of severe loading. Its many elements, contact formulations, material models, and other controls can be used to simulate complex models with control over all the details of the problem. ANSYS LS-DYNA has a vast array of capabilities to simulate extreme deformation problems using its explicit solver. Engineers can tackle simulations involving material failure and look at how the failure progresses through a part or through a system. Models with large amounts of parts or surfaces interacting with each other are also easily handled, and the interactions and load passing between complex behaviors are modeled accurately. Using computers with higher numbers of CPU cores can drastically reduce solution times. In addition, many consulting firms and hundreds of universities use ANSYS for analysis, research, and educational purposes. ANSYS is recognized worldwide as one of the most widely used and capable programs of its type. ANSYS has successfully passed over 100 customer quality system audits against American Society of Mechanical Engineers (ASME) NQA-1 and 10 CFR Part 50, Appendix B, since the company was founded, over 60 of which have been since 1997. ANSYS has successfully passed over 100 International Organization for Standardization (ISO) 9001 assessments. ANSYS design analysis software is the first created within a quality system with ISO 9001 certification, which is the internationally accepted quality standard. Product development, testing, maintenance, and support processes also meet the US Nuclear Regulatory Commission’s (NRC’s) quality requirements, as they have for nearly four decades. ANSYS staff perform more than 60,000 software verification tests before releasing each new product. ASME NQA-1-2012 (Subpart 2.7 is specific to software) is the industry- and NRC-accepted approach (consensus standard) for meeting 10 CFR Part 50, Appendix B, requirements.
11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
High-Rate Delay Tolerant Networking (HDTN) Software Requirements Analysis
This document serves as a detailed analysis of the main networking protocols implemented by HDTN. Sources of the protocol specifications include Internet Engineering Task Force (IETF) Request for Comments (RFC) and Consultative Committee for Space Data Systems (CCSDS) standards. The focus of this report is to derive software requirements suitable for NASA Procedural Requirements 7150.2D Class B compliance, including requirements traceability and software verification and validation, from the source specifications. This analysis will be incorporated into the finalized HDTN Software Requirements Specification (SRS) but does not encompass the full scope of the HDTN SRS. Requirements in this document are considered draft. The complete requirements will include bundle application requirements, interface requirements, computer resource requirements, software quality factors, and additional requirements as determined by the project. This document is publicly released to the greater community to receive feedback and foster collaboration opportunities.
Rachel Dudukovich↗
Formal Methods Specification and Verification Guidebook for Software and Computer Systems: Planning and Technology Insertion - Volume 1
The Formal Methods Specification and Verification Guidebook for Software and Computer Systems describes a set of techniques called Formal Methods (FM), and outlines their use in the specification and verification of computer systems and software. Development of increasingly complex systems has created a need for improved specification and verification techniques. NASA's Safety and Mission Quality Office has supported the investigation of techniques such as FM, which are now an accepted method for enhancing the quality of aerospace applications. The guidebook provides information for managers and practitioners who are interested in integrating FM into an existing systems development process. Information includes technical and administrative considerations that must be addressed when establishing the use of FM on a specific project. The guidebook is intended to aid decision makers in the successful application of FM to the development of high-quality systems at reasonable cost. This is the first volume of a planned two-volume set. The current volume focuses on administrative and planning considerations for the successful application of FM.
Source record↗
System diagnostic builder
The System Diagnostic Builder (SDB) is an automated software verification and validation tool using state-of-the-art Artificial Intelligence (AI) technologies. The SDB is used extensively by project BURKE at NASA-JSC as one component of a software re-engineering toolkit. The SDB is applicable to any government or commercial organization which performs verification and validation tasks. The SDB has an X-window interface, which allows the user to 'train' a set of rules for use in a rule-based evaluator. The interface has a window that allows the user to plot up to five data parameters (attributes) at a time. Using these plots and a mouse, the user can identify and classify a particular behavior of the subject software. Once the user has identified the general behavior patterns of the software, he can train a set of rules to represent his knowledge of that behavior. The training process builds rules and fuzzy sets to use in the evaluator. The fuzzy sets classify those data points not clearly identified as a particular classification. Once an initial set of rules is trained, each additional data set given to the SDB will be used by a machine learning mechanism to refine the rules and fuzzy sets. This is a passive process and, therefore, it does not require any additional operator time. The evaluation component of the SDB can be used to validate a single software system using some number of different data sets, such as a simulator. Moreover, it can be used to validate software systems which have been re-engineered from one language and design methodology to a totally new implementation.
Nieten, Joseph L.↗
Hosted Services for Advanced V and V Technologies: An Approach to Achieving Adoption without the Woes of Usage
Attempts to achieve widespread use of software verification tools have been notably unsuccessful. Even 'straightforward', classic, and potentially effective verification tools such as lint-like tools face limits on their acceptance. These limits are imposed by the expertise required applying the tools and interpreting the results, the high false positive rate of many verification tools, and the need to integrate the tools into development environments. The barriers are even greater for more complex advanced technologies such as model checking. Web-hosted services for advanced verification technologies may mitigate these problems by centralizing tool expertise. The possible benefits of this approach include eliminating the need for software developer expertise in tool application and results filtering, and improving integration with other development tools.
Koga, Dennis↗
Development of Advanced Verification and Validation Procedures and Tools for the Certification of Learning Systems in Aerospace Applications
Adaptive control technologies that incorporate learning algorithms have been proposed to enable automatic flight control and vehicle recovery, autonomous flight, and to maintain vehicle performance in the face of unknown, changing, or poorly defined operating environments. In order for adaptive control systems to be used in safety-critical aerospace applications, they must be proven to be highly safe and reliable. Rigorous methods for adaptive software verification and validation must be developed to ensure that control system software failures will not occur. Of central importance in this regard is the need to establish reliable methods that guarantee convergent learning, rapid convergence (learning) rate, and algorithm stability. This paper presents the major problems of adaptive control systems that use learning to improve performance. The paper then presents the major procedures and tools presently developed or currently being developed to enable the verification, validation, and ultimate certification of these adaptive control systems. These technologies include the application of automated program analysis methods, techniques to improve the learning process, analytical methods to verify stability, methods to automatically synthesize code, simulation and test methods, and tools to provide on-line software assurance.
Jacklin, Stephen↗
Formal Verification of Large Software Systems
We introduce a scalable proof structure to facilitate formal verification of large software systems. In our approach, we mechanically synthesize an abstract specification from the software implementation, match its static operational structure to that of the original specification, and organize the proof as the conjunction of a series of lemmas about the specification structure. By setting up a different lemma for each distinct element and proving each lemma independently, we obtain the important benefit that the proof scales easily for large systems. We present details of the approach and an illustration of its application on a challenge problem from the security domain
Yin, Xiang↗
Verification of the Space Shuttle entry GN&C system
The certification procedures for the initial Shuttle flight are discussed. Particular attention is paid to the entry guidance, navigation, and control (GNC) verification, comprising tests, analysis, demonstration, inspection, and simulation. Flow diagrams for the verification and operational flight sequences are provided, along with a block diagram of the GNC circuitry interfaces. The development of the test matrix software for the GNC is outlined, noting the constant interplay between software verification and spacecraft reconfiguration to meet simulated performance requirements. Comparison of GNC performance predictions with actual entry flight data showed a good match in all performance areas except for sideslip excursions, bank overshoots, an area of transonic buffet, and an increased lift/drag ratio in the preflare to landing flight phase.
Van Hoften, J. D. A.↗