Search NASA⌕ Search

SEARCH · Search NASA

Results for “Verification”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Verification and Validation Tests of Gamma Library of MC2-3 for Coupled Neutron and Gamma Heating Calculation

For the accurate assessment of the heat generation rate in fast reactors, the gamma library of MC2 -3 and the MC2 -3 + GAMSOR procedure employing the coupled neutron and gamma heating calculation has been thoroughly verified against Monte Carlo results and validated using the ZPPR-15D gamma dose measurement data. NJOY outputs are post-processed in a consistent way with the NJOY procedure to avoid any missing data or double counting of data. Prompt heating for 379 out of 391 isotopes in the gamma library was verified against MCNP6.2 to within 1% relative error in total heating for most isotopes. For both a simple one-dimensional slab problem representing a sodium cooled fast reactor and the Experimental Breeder Reactor II (EBR-II) benchmark problem, the root-mean-square values of assembly power error were less than 0.5% for fuel assemblies, ~1% in blankets and ~1 to ~3% in reflectors compared to MCNP6.2 results. The most plausible cause for the 3% error in a reflector assembly is believed to be the error in the multigroup neutron cross sections for the reflector assembly. For validation, gamma doses measured with thermoluminiscent dosimeters (TLDs) in the ZPPR-15D experiment were calculated using GAMSOR. Due to the uncertainty in the TLD measurement with regards to the energy deposition of photons and neutrons, the validation data leads to a 12.7% uncertainty on the experimental measurement. With this uncertainty bound, the calculated doses all fell within one standard deviation of the measured value. Combined with the accurate calculation of reaction rate distributions and neutron spectrum measurements, these results indicate good agreement for the neutron and gamma heating calculations that were performed.

coupled neutron and gamma heating↗
DOE OSTI2024

High-Fidelity Numerical Wave Tank Verification & Validation Study: Wave Generation Through Paddle Motion: Preprint

This paper presents a numerical benchmark study of wave propagation due to a paddle motion using different high-fidelity numerical models, which are capable of replicating the nearly actual physical wave tank testing. A full time series of the measured wave generation paddle motion which was used to generate wave propagation in the physical wave tank will be utilized in each of the models contributed by IEA OES Task 10's participants, which includes both computational fluid dynamics (CFD) and smooth hydrodynamic particle (SPH). The high-fidelity simulations of the physical wave testcase will allow for the evaluation of the initial transient effects from wave ramp-up and its evolution in the wave tank over time for two representative regular waves with varying levels of nonlinearity. A couple of interesting metrics like the predicted wave surface elevation at select wave probes, wave period, and phase-shift in time will be assessed to evaluate the relative accuracy of numerical models versus experimental data within specified time intervals. These models will serve as a guide for modelers in the wave energy community and provide a base case to allow further and more detailed numerical modeling of the fixed Kramer Sphere Cases under wave excitation force wave tank testing.

HYDRO ENERGY,TIDAL AND WAVE POWER↗
NASA NTRS2000

Computer Simulations to Study Diffraction Effects of Stacking Faults in Beta-SiC: II. Experimental Verification: Experimental Verification - 2

Earlier results from computer simulation studies suggest a correlation between the spatial distribution of stacking errors in the Beta-SiC structure and features observed in X-ray diffraction patterns of the material. Reported here are experimental results obtained from two types of nominally Beta-SiC specimens, which yield distinct XRD data. These samples were analyzed using high resolution transmission electron microscopy (HRTEM) and the stacking error distribution was directly determined. The HRTEM results compare well to those deduced by matching the XRD data with simulated spectra, confirming the hypothesis that the XRD data is indicative not only of the presence and density of stacking errors, but also that it can yield information regarding their distribution. In addition, the stacking error population in both specimens is related to their synthesis conditions and it appears that it is similar to the relation developed by others to explain the formation of the corresponding polytypes.

Pujar, Vijay V.↗
NASA NTRSDate not supplied

An Evaluation of Extended Reality Technologies for Use in Verification Testing at NASA 2024 HRP IWS Abstract

BACKGROUND At NASA, verification testing is the formal process of ensuring that a product conforms to requirements set by a project or program. Some verification methods, such as Demonstrations and Test, require either the end product or a mockup of the product with sufficient fidelity to stand-in for the product during the test. Traditionally, these mockups have been physical (e.g., foam-core and wood) but there is growing interest in exploring new methods for testing with these mockups. These methods include virtual reality (VR), mixed reality, and augmented reality which are collectively referred to as eXtended Reality (XR) technologies. VR has already been adopted and used by many in the aerospace industry as a tool for use in early design phases (e.g., developmental testing) and may have the most potential for use in verification tests. Benefits of using VR mockups offer cost effectiveness, ease of iteration, simulation of hazardous conditions (e.g., an egress through a hatch with smoke obscuring vision), and the ability to simulate microgravity conditions, which are challenging to do with physical mockups. However, the validity of test results obtained from VR mockup demonstrations or testing, compared to the current gold standard of physical mockups, remains uncertain. It is unlikely that there is one clean answer as there are many different types of verification outcomes and each XR technology must be evaluated on its own merits. This is not an issue during developmental testing as the design is still in flux and the total success of the design is not dependent upon the results of a developmental test. Verification tests, however, only happen once, assuming no change to the design, and the results are used to certify the product. Therefore, establishing the validity of XR mockup-based verification outcomes is essential before considering them for any use in verification tests. OBJECTIVE AND METHOD To address this concern, the Human Research Program has funded a project to explore and qualify how XR technologies might be used in verification demonstration and testing at NASA. Currently, we are conducting a review of the literature on the utilization of XR mockups for design activities, prototyping, and user testing. We are employing the Strengths, Weaknesses, Opportunities, and Threats (SWOT) analysis method to identify the pros, cons, and barriers to adoption of XR technologies for verification testing at NASA. Additionally, we are developing a framework to guide the deployment of XR mockups for verification tests. Building upon available evidence from the literature and subject-matter expert feedback, our goal for the framework is to provide guidelines for which forms of XR mockups are suitable for a given verification test, when only physical mockups should be employed and to highlight areas for which more evidence is needed. To further refine our framework and to contribute to the body of evidence, we are planning a lab-based experiment comparing a VR mockup to a physical twin for a set of select verification outcomes. ANTICIPATED RESULTS In this presentation, we will present the work we conducted to evaluate XR technologies for use in verification tests at NASA. We will summarize and report our findings from the SWOT analysis and our lab-based study, and we will present the current state of the XR Technologies for Verification Testing framework. We will conclude by summarizing remaining work and future directions for the project. Technologies for Verification Testing framework. We will conclude by summarizing remaining work and future directions for the project.

Extended Reality↗
NASA NTRS2013

Regression Verification Using Impact Summaries

Regression verification techniques are used to prove equivalence of syntactically similar programs. Checking equivalence of large programs, however, can be computationally expensive. Existing regression verification techniques rely on abstraction and decomposition techniques to reduce the computational effort of checking equivalence of the entire program. These techniques are sound but not complete. In this work, we propose a novel approach to improve scalability of regression verification by classifying the program behaviors generated during symbolic execution as either impacted or unimpacted. Our technique uses a combination of static analysis and symbolic execution to generate summaries of impacted program behaviors. The impact summaries are then checked for equivalence using an o-the-shelf decision procedure. We prove that our approach is both sound and complete for sequential programs, with respect to the depth bound of symbolic execution. Our evaluation on a set of sequential C artifacts shows that reducing the size of the summaries can help reduce the cost of software equivalence checking. Various reduction, abstraction, and compositional techniques have been developed to help scale software verification techniques to industrial-sized systems. Although such techniques have greatly increased the size and complexity of systems that can be checked, analysis of large software systems remains costly. Regression analysis techniques, e.g., regression testing [16], regression model checking [22], and regression verification [19], restrict the scope of the analysis by leveraging the differences between program versions. These techniques are based on the idea that if code is checked early in development, then subsequent versions can be checked against a prior (checked) version, leveraging the results of the previous analysis to reduce analysis cost of the current version. Regression verification addresses the problem of proving equivalence of closely related program versions [19]. These techniques compare two programs with a large degree of syntactic similarity to prove that portions of one program version are equivalent to the other. Regression verification can be used for guaranteeing backward compatibility, and for showing behavioral equivalence in programs with syntactic differences, e.g., when a program is refactored to improve its performance, maintainability, or readability. Existing regression verification techniques leverage similarities between program versions by using abstraction and decomposition techniques to improve scalability of the analysis [10, 12, 19]. The abstractions and decomposition in the these techniques, e.g., summaries of unchanged code [12] or semantically equivalent methods [19], compute an over-approximation of the program behaviors. The equivalence checking results of these techniques are sound but not complete-they may characterize programs as not functionally equivalent when, in fact, they are equivalent. In this work we describe a novel approach that leverages the impact of the differences between two programs for scaling regression verification. We partition program behaviors of each version into (a) behaviors impacted by the changes and (b) behaviors not impacted (unimpacted) by the changes. Only the impacted program behaviors are used during equivalence checking. We then prove that checking equivalence of the impacted program behaviors is equivalent to checking equivalence of all program behaviors for a given depth bound. In this work we use symbolic execution to generate the program behaviors and leverage control- and data-dependence information to facilitate the partitioning of program behaviors. The impacted program behaviors are termed as impact summaries. The dependence analyses that facilitate the generation of the impact summaries, we believe, could be used in conjunction with other abstraction and decomposition based approaches, [10, 12], as a complementary reduction technique. An evaluation of our regression verification technique shows that our approach is capable of leveraging similarities between program versions to reduce the size of the queries and the time required to check for logical equivalence. The main contributions of this work are: - A regression verification technique to generate impact summaries that can be checked for functional equivalence using an off-the-shelf decision procedure. - A proof that our approach is sound and complete with respect to the depth bound of symbolic execution. - An implementation of our technique using the LLVMcompiler infrastructure, the klee Symbolic Virtual Machine [4], and a variety of Satisfiability Modulo Theory (SMT) solvers, e.g., STP [7] and Z3 [6]. - An empirical evaluation on a set of C artifacts which shows that the use of impact summaries can reduce the cost of regression verification.

Backes, John↗
NASA NTRSDate not supplied

Guiding Integration of Formal Verification in Assurance Cases

Assurance cases are being increasingly acknowledged as away to build trust in complex systems with autonomous capabilities. An assurance case is a comprehensive, defensible, and valid justification that a system will function as intended for a specific mission and operating environment. Formal verification is often reserved for the most critical components of such systems. However, formal verification tools are often complex, and their usage is subject to many constraints and contextual dependencies. This can raise challenges both for performing the verification as well as reflecting the verification results appropriately in the assurance case, especially for non-expert users of the verification tool. To address these challenges, we present a tool-supported methodology for integrating formal verification results in an assurance case by capturing key verification method information in a rigorously constructed assurance case. In particular, we capture the tool specification in terms of its inputs, outputs, and assurance constraints as assumptions over inputs and guarantees provided over its outputs. The tool specification is parametrized over the inputs and outputs to both guide the intended application of the tool, as well as to check that the tool has been applied following the stated assumptions and that the guarantees hold. We define a generic tool assurance argument pattern that enables integration of the verification results in the assurance case by allowing custom refinement and automated instantiation for each tool use. We demonstrate our methodology on two formal verification tools and their applications to the verification of neural network properties for the aircraft domain.

Assurance Cases↗
NASA NTRS2005

Structural Verification of the Redesigned Space Shuttle Bipod Foam Closeout

This document outlines the structural verification approach for the Space Shuttle External Tank Forward Bipod Foam Closeout. Due to the Space Shuttle Columbia accident, debris has become a major concern. The intent of the structural verification is to ensure that any debris shed from the bipod is within acceptable limits. Since cohesive failure due to internal defects was identified as the most likely cause of the STS-107 bipod ramp foam failure, verification for this failure mode receives particular emphasis. However, all failure modes for TPS are considered and appropriate verification rationale is developed for each failure mode. Figure 1 depicts the structural verification of a production design where analysis and test are the primary methods of verification. It can be seen that the successful completion of structural verification is dependent on three main areas: 1. Production process control and quality assurance must ensure that test articles and/or analytical models are representative of (or conservatively envelope) production hardware in terms of geometry, materials and processing. Variability and defects must be considered. 2. Flight environments must be sufficiently characterized to bound driving environments for all failure modes. Applied environments, either test or analytical, must be representative of flight environments and have a load factor that satisfies design requirements. 3. Structural verification must include all failure modes. A comprehensive list of failure modes and the underlying failure mechanisms has been generated based on flight and test experience. Verification tests and / or analyses must address each failure mode. ET TPS Verification is accomplished by a combination of analysis, test, and similarity.

Poole, Eric L.↗
NASA NTRS2015

The SeaHorn Verification Framework

In this paper, we present SeaHorn, a software verification framework. The key distinguishing feature of SeaHorn is its modular design that separates the concerns of the syntax of the programming language, its operational semantics, and the verification semantics. SeaHorn encompasses several novelties: it (a) encodes verification conditions using an efficient yet precise inter-procedural technique, (b) provides flexibility in the verification semantics to allow different levels of precision, (c) leverages the state-of-the-art in software model checking and abstract interpretation for verification, and (d) uses Horn-clauses as an intermediate language to represent verification conditions which simplifies interfacing with multiple verification tools based on Horn-clauses. SeaHorn provides users with a powerful verification tool and researchers with an extensible and customizable framework for experimenting with new software verification techniques. The effectiveness and scalability of SeaHorn are demonstrated by an extensive experimental evaluation using benchmarks from SV-COMP 2015 and real avionics code.

Model Checking↗
DOE OSTI2019

REBUS-PC v1.5 Verification Test Summary

Motivated in part by its usage for analysis of the University of Missouri Research Reactor (MURR), the verification of the REBUS-PC software for θ-R-Z geometries is discussed and displayed in this manuscript. Of particular importance is to show that the eigenvalue, flux, power, and burnup outputs of the software given a user problem is correct. REBUS-PC contains many features and MURR only makes use of some of those aspects. As an example, the use of region specific cross section sets, interpolation of the cross section data with respect to burnup, and automatic movement of control rods during the fuel cycle is discussed and displayed. To demonstrate the accuracy, 22 verification tests based upon 7 physical geometries are discussed. Geometry descriptions are provided in Section 2 and the REBUS-PC results for each verification test are discussed in Section 3. These verification tests consist of both analytic problems with known solutions and a more complicated input consistent with usage of REBUS-PC on the MURR reactor itself called STRZ. In this verification test summary report, the REBUS-PC software is repeatedly demonstrated to reproduce, or closely match, the eigenvalue, flux, and power distributions from analytic test problem solutions, demonstrating that the implemented finite difference approximation in θ-R-Z is working as intended. Several additional REBUS-PC features are tested, such as control rod movement by detailed comparisons of the results against reference solutions obtained using an auxiliary implemented MatLab program. As expected, the REBUS-PC software produces results that are 100% consistent with the stated capabilities and thus the verification exercise is completed. These verification tests should be executed with each re-compilation of the REBUS-PC source and after any source code changes. All output results from any such new work should be rigorously compared against the stored output of the 22 verification tests cited in this report that forms the basis of the verification process.

97 MATHEMATICS AND COMPUTING↗
NASA NTRSDate not supplied

An Analysis of Extended Reality Mockups for Use in Verification: Phase 2

NASA commercial providers are increasing their use of Virtual Reality (VR) and Hybrid Reality (HR) technologies in their system development process. The use of these technologies, collectively referred to as Extended Reality (XR) technologies, has been limited to the development phase, but there have been requests to integrate VR and HR mockups into verifications. Verifications require, at minimum, a high-fidelity mockup for activities involving human participation (tests, demonstrations, inspections). HR technology shares many of the strengths but may not share some of the critical shortcomings of VR. Specifically, HR allows for integrating physical objects, such as a suit, which may be critical to evaluating a system. Adopting HR technology still has many of the same challenges as VR. Namely, there is little to no data about the validity of evaluations conducted with HR mockups, an established process, or criteria for evaluating the appropriateness of HR mockups. Because verifications are final and only need to happen once, HR mockups must be adequately vetted before being approved for use in verifications. Further hindering the adoption of these technologies is the lack of consensus on what constitutes a high-fidelity XR mockup. Even for physical mockups, there are guidelines and common criteria, but nothing has been formalized. Discussion about when an XR mockup can be used would be greatly helped by clearly defining what a high-fidelity XR mockup is and how it might be measured. The team will engage with relevant stakeholders to learn more about the benefits and challenges of adopting hybrid reality mockups for verification. The team will build upon work from Phase 1 by maturing a framework to guide decisions for evaluating XR mockups for use in verifications. The team will also conduct experimental studies to evaluate advantages and tradeoffs of hybrid/mixed reality relative to physical and virtual reality mockups. Finally, data from Phase 1 and Phase 2 will be synthesized into a set of best practices for developing and validating XR mockups for verification. We will highlight the work conducted to date in Phase 2. We will present the status of the XR for Verification Framework, a summary of best practices identified thus far, and give a synopsis of future work.

verification testing↗
DOE OSTI2024

Verification of RESRAD-OFFSITE Code (V.4)

This report documents the verification of RESRAD-OFFSITE Version 4.0 and describes, where necessary, the verification of the following: • The data comprising the standard dose and risk coefficient libraries in the RESRAD database files Master_dcf_ICRP07.mdb and Master_dcf_2k.mdb. • The extraction and transfer of the data from the selected database file to the computational code by the RESRAD-OFFSITE 4.0 interface, ResOWin.exe. • The different processes that are modeled by the main computational code in RESRAD OFFSITE 4.0, ResOMain.exe. • The data displayed in the graphical and text reports. Many verifications were performed as part of the quality assurance quality control program associated with the development and release of RESRAD-OFFSITE 4.0, namely: • developer testing, • internal independent testing, and • release testing. Some were also performed in response to questions from users regarding the performance of the code. The main text of the report focuses on summarizing a subset of those tests, both independent and developer tests that verified the computations performed by the code. The verifications included in this report served as the basis for the development of the release tests of the computational executables and provided the quantitative results to be compared with the code output. The input and output interfaces and the data transfers between the various executables of the code were tested while performing the verification testing. They were tested intentionally during release testing. This report also provides some basic information to help in understanding the activities that were verified. The report: • outlines the components of RESRAD-OFFSITE 4.0 and the interconnections between these components, • outlines the processes modeled by the computational code, • provides summary figures and tables to offer confirmation of the verification of the computational components of the code, • reproduces the verifiers’ reports, if available, in individual appendices, • refers to the previous verification report (Yu et al. 2011) for more details about some of the verifications, and • reproduces the test cases and the testers’ reports from the release testing in individual appendices, when possible.

54 ENVIRONMENTAL SCIENCES↗