On the Interpretation and Scope of the V&V 20 Standard for Verification and Validation in Computational Fluid Dynamics and Heat Transfer.
Abstract not provided.
Source record↗
SEARCH · Search NASA
Results for “Verification and Validation (V&V)”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Nuclear Materials Process Modeling at the Y-12 National Security Complex
The Y-12 National Security Complex (Y-12) has implemented process modeling for various accountable nuclear materials operations that are performed throughout the plant. Using a discrete, event-based dynamic simulation program, key nuclear material streams are modeled, allowing Y-12 to effectively manage numerous points of interest within the plant’s production operations. Integration of the various material processes into a single, interdependent supply and demand model is one of the ongoing focuses within Y-12’s process modeling effort. The primary purpose of using dynamic simulation modeling is to allow for analysis of the nuclear materials inventories and forecasted supplies based on future demands. Analysis of these inventories includes capacity evaluation, bottleneck mitigation, and assessments of individual pieces of equipment to inform future facility investment decisions and associated project schedules. Modeling of the nuclear materials processes throughout the complex also allows for incorporation of changes relevant to production capabilities such as the upcoming transition of specific operations to the new Uranium Processing Facility. Prior to implementation of process modeling, Y-12 forecasted supply and demand of accountable nuclear materials streams using Microsoft Excel. With deterministic models such as Microsoft Excel, the annual forecasts, generated within data input condition parameters, can only provide a fixed point of data. Fixed data cannot simulate integrated material streams and account for the possibility of occurrences and other changes that dynamic simulations take into consideration. Y-12’s dynamic process modeling allows integrated simulations of multiple accountable nuclear materials processes, including supply and demand forecasting and analysis, and is a coordinated effort involving many steps of verification and validation (V&V), site briefings, testing, reporting, data mining, planning, and documentation that spans various programs throughout the Y-12 complex.
12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
Improved Verification and Validation Testing and Tools including Nuclear Criticality Safety Applications with the MCNP6.3® Code [Abstract]
A new Python-based framework has been developed to enable a more consistent layout with automatable setup, execution, and documentation of all verification and validation (V&V) test suites previously established for use with the MCNP code. In this paper, the new general framework for the V&V test suites is discussed, including information on all of the current capabilities and plans for future capabilities. For nuclear criticality safety applications, the existing V&V benchmark problems within the criticality, extended criticality, and analytic k-effective test suites have been ported into this new framework. In addition, the status and updates to the Rossi-alpha and subcritical multiplicationtest suites will be discussed. Some V&V results exercising new MCNP6.3 capabilities will be demonstrated.
97 MATHEMATICS AND COMPUTING↗
UPDATES FROM THE INVOLUTE WORKING GROUP
The HFIR, RHF, and FRM II reactors represent a particular class of Research and Test Reactors that provide some of the most intense and continuous neutron fluxes for science, industry, and medical applications. These high-performance reactors have achieved compact cores by operating with Highly Enriched Uranium fuel (HEU, 235U/U ≥ 20 wt. %) and utilizing fuel plates curved as an involute. Due to the proliferation risks, the international community aims to reduce or eliminate, when possible, the use of HEU fuel in civilian facilities by converting them to a Low-Enriched Uranium fuel (LEU, 235U/U < 20 wt. %). Conversion of these reactors without significantly compromising their performance or safety is a challenging endeavor that can tremendously benefit from advanced computational tools and thus, eliminate unnecessary conservatism to ensure sufficient thermal margins. Therefore, models are being developed using modern Computational Fluid Dynamics (CFD) and Computational Structural Mechanics (CSM) software to evaluate the steady-state safety margins of various LEU designs instead of being reliant on the more traditional, conservative methods. To gain the confidence and acceptance of high-fidelity modeling by the nuclear regulators, Argonne National Laboratory (ANL) and the involute reactors have formed an informal scientific group, the Involute Working Group (IWG). The IWG facilitates inter-organizational collaboration on experimental benchmarking, code-to-code comparisons, and Verification and Validation (V&V). This paper describes some of the recent IWG efforts in validating software against the existing experimental data, as well as code-to-code comparisons of different software used by the IWG members.
Bergeron, Aurelien↗
Verification and Validation of START: A Spent Nuclear Fuel Routing and Decision Support Tool
The Stakeholder Tool for Assessing Radioactive Transportation (START) is a web-based geospatial decision-support tool being developed by the US Department of Energy’s Office of Integrated Waste Management (IWM) to support federal interim storage for spent nuclear fuel (SNF) and associated transportation. START provides many functions for the IWM program including: serving as a communications tool for conveying geospatial data and information, an options analysis tool for exploring potential transport modes and routes for transporting SNF from nuclear power plants to future federal interim storage facilities, an emergency response planning tool for Tribes and States to identify training needs along potential SNF transport corridors, an environmental analysis tool for estimating potential radiation dose exposure from incident-free and incident-case SNF transport conditions, and a systems analysis support tool providing route-related inputs for system throughput analysis. As part of the START development process, a verification and validation (V&V) effort is being undertaken. In the initial V&V phase, several outputs of the START tool were checked such as the total distance, population and population densities within the buffer zone, and incident free dose. The V&V process is fluid as it will be utilized after each version change to ensure that the core functionalities of the tool are maintained and the results are consistent with the previous versions. Efforts have also been put into developing scripts to aid in the process of automating certain sections of the V&V work. As some of the V&V efforts use Environmental Systems Research Institute’s (ESRI) tools upon which the START framework is built, the START data were compared against outputs from tools like Quantum Geographic Information System (QGIS) for buffer zone populations and route lengths to ensure independence of the V&V process. Good agreement was observed between the START results and the independent V&V studies with the majority of the differences falling between 1% and 5% for populations within the buffer zone and route distance. This presentation describes the development and design of the START tool, V&V methods employed for various metrics of interest and their respective results, and future plans.
START, transportation, GIS, V&V↗
Verification and Validation of the Stakeholder Tool for Assessing Radioactive Transportation (START) – 22323
The U.S. Department of Energy (DOE) Office of Integrated Waste Management is planning for the eventual transportation, storage, and disposal of spent nuclear fuel (SNF) and high-level radioactive waste (HLW) from nuclear power plant and DOE sites. The Stakeholder Tool for Assessing Radioactive Transportation (START) is a web-based, geospatial decision-support tool developed for evaluating routing options and other aspects of transporting SNF and HLW, covering rail, truck, barge, and intermodal infrastructure and operations in the continental United States. The verification and validation (V&V) process is intended to independently assess START to provide confidence in the ability of START to accurately provide intended results. The V&V process checks the START tool using a variety of methods, ranging from independent hand calculations to comparison of START performance and results to those of other codes. The V&V activity was conducted independently from the START development team with opportunities to provide feedback and collaborate throughout the process. The V&V analyzed attributes of transportation routes produced by START, including route distance and both population and population density captured within buffer zones around routes. Population in the buffer zone, population density in the buffer zone, and route distance were all identified as crucial outputs of the START code and were subject to V&V tasks. Some of the improvements identified through the V&V process were standardizing the underlying population data in START, changing the projection of the population raster data, and changes to the methodology used for population density to improve its applicability for expected users. This collaboration also led to suggested improvements to some of the underlying shape file segments within START. A formal series of test routes went through the V&V process from START version 3.2.1 following the implementation of the recommended improvements to START. The V&V of the buffer zone population and distance reported in START were achieved by utilizing a custom workflow developed in QGIS. This platform was chosen because it is fully independent of START. The buffer zone population and route distances in START showed excellent agreement to independent V&V test results. Over 200 route test cases were run; in all cases, the percent difference in the population within the buffer zone was less than +/- 5%, and the majority of cases were below +/- 1%. These cases were also duplicated in ArcMap, which is Environmental Systems Research Institute’s (ESRI) GIS desktop application. The ArcMap results also show good agreement; however, they cannot be considered independent of START because START also uses ESRI software. The V&V tasks are currently addressing population density and estimated radiation dose in START as well as implementing further automation of the V&V process to allow larger suites of test routes to be run more easily and quickly by less experienced GIS users. These improvements to the V&V process will make it a valuable tool for the development team and the stakeholders so that each new iteration of START can easily be checked for quality and consistency. This paper and presentation will describe the V&V process, findings and implications, and ongoing V&V activities.
START, DOE, Transporation, dose, GIS↗
ECAR-6654 Rev 0 Verification and Validation of RELAP5-3D for the Design and Safety Analysis of MARVEL Microreactor
The objective of this document is to provide relevant information about the verification and validation (V&V) status of RELAP5-3D for the MARVEL microreactor design and safety analysis and to provide indications about needed V&V activities before MARVEL microreactor operation.
21 - SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLAN↗
ECAR-7300 Rev 1 Verification and Validation of MCNP6.2 for MARVEL Neutronic Analysis
This report documents the verification and validation (V&V) efforts of the Monte Carlo N-Particle transport code (MCNP) version 6.2 on the Sawtooth supercomputer for the Microreactor Applications Research Validation and Evaluation (MARVEL) microreactor required for the preliminary documented safety analysis. This document records V&V for a safety, hazards, analysis, and design software used for design and analysis of safety class structures, system and components (SSC)s.
21 - SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLAN↗
ECAR-8055 Rev 0 Verification and Validation of Star-CCM+ for Computational Fluid Dynamics Analyses for the MARVEL Microreactor
The objective of this Engineering Calculations and Analysis Report (ECAR) is to provide documentation and highlight relevant information regarding the verification and validation (V&V) of the commercial computational fluid dynamics (CFD) code STAR-CCM+ for the thermal and fluids analyses performed for the MARVEL microreactor.
21 - SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLAN↗
NASA IV&V Program NASA's Independent Verification and Validation Program
General overview of NASA's IVV Program organizational structure and office functions.
Organization↗
V&V of Fault Management: Challenges and Successes
This paper describes the results of a special breakout session of the NASA Independent Verification and Validation (IV&V) Workshop held in the fall of 2012 entitled "V&V of Fault Management: Challenges and Successes." The NASA IV&V Program is in a unique position to interact with projects across all of the NASA development domains. Using this unique opportunity, the IV&V program convened a breakout session to enable IV&V teams to share their challenges and successes with respect to the V&V of Fault Management (FM) architectures and software. The presentations and discussions provided practical examples of pitfalls encountered while performing V&V of FM including the lack of consistent designs for implementing faults monitors and the fact that FM information is not centralized but scattered among many diverse project artifacts. The discussions also solidified the need for an early commitment to developing FM in parallel with the spacecraft systems as well as clearly defining FM terminology within a project.
Verification↗
Formal Methods of V&V of Partial Specifications: An Experience Report
This paper describes our work exploring the suitability of formal specification methods for independent verification and validation (IV&V) of software specifications for large, safety critical systems. An IV&V contractor often has to perform rapid analysis on incomplete specifications, with no control over how those specifications are represented. Lightweight formal methods show significant promise in this context, as they offer a way of uncovering major errors, without the burden of full proofs of correctness. We describe an experiment in the application of the method SCR. to testing for consistency properties of a partial model of requirements for Fault Detection Isolation and Recovery on the space station. We conclude that the insights gained from formalizing a specification is valuable, and it is the process of formalization, rather than the end product that is important. It was only necessary to build enough of the formal model to test the properties in which we were interested. Maintenance of fidelity between multiple representations of the same requirements (as they evolve) is still a problem, and deserves further study.
Easterbrook, Steve↗
Robust Requirements Tracing via Internet Search Technology: Improving an IV and V Technique
There are three major objectives to this phase of the work. (1) Improvement of Information Retrieval (IR) methods for Independent Verification and Validation (IV&V) requirements tracing. Information Retrieval methods are typically developed for very large (order of millions - tens of millions and more documents) document collections and therefore, most successfully used methods somewhat sacrifice precision and recall in order to achieve efficiency. At the same time typical IR systems treat all user queries as independent of each other and assume that relevance of documents to queries is subjective for each user. The IV&V requirements tracing problem has a much smaller data set to operate on, even for large software development projects; the set of queries is predetermined by the high-level specification document and individual requirements considered as query input to IR methods are not necessarily independent from each other. Namely, knowledge about the links for one requirement may be helpful in determining the links of another requirement. Finally, while the final decision on the exact form of the traceability matrix still belongs to the IV&V analyst, his/her decisions are much less arbitrary than those of an Internet search engine user. All this suggests that the information available to us in the framework of the IV&V tracing problem can be successfully leveraged to enhance standard IR techniques, which in turn would lead to increased recall and precision. We developed several new methods during Phase II; (2) IV&V requirements tracing IR toolkit. Based on the methods developed in Phase I and their improvements developed in Phase II, we built a toolkit of IR methods for IV&V requirements tracing. The toolkit has been integrated, at the data level, with SAIC's SuperTracePlus (STP) tool; (3) Toolkit testing. We tested the methods included in the IV&V requirements tracing IR toolkit on a number of projects.
Hayes, Jane↗
Validation (not just verification) of Deep Space Missions
ion & Validation (V&V) is a widely recognized and critical systems engineering function. However, the often used definition 'Verification proves the design is right; validation proves it is the right design' is rather vague. And while Verification is a reasonably well standardized systems engineering process, Validation is a far more abstract concept and the rigor and scope applied to it varies widely between organizations and individuals. This is reflected in the findings in recent Mishap Reports for several NASA missions, in which shortfalls in Validation (not just Verification) were cited as root- or contributing-factors in catastrophic mission loss. Furthermore, although there is strong agreement in the community that Test is the preferred method for V&V, many people equate 'V&V' with 'Test', such that Analysis and Modeling aren't given comparable attention. Another strong motivator is a realization that the rapid growth in complexity of deep-space missions (particularly Planetary Landers and Space Observatories given their inherent unknowns) is placing greater demands on systems engineers to 'get it right' with Validation.
verification↗
IV&V Assurance Case Design for Artemis II
As human-rated missions like those in NASA’s Artemis program continue to grow in both size and complexity, and the role of software in achieving mission objectives expands dramatically, NASA’s Independent Verification and Validation (IV&V) Teams face evolving challenges in assuring the safety and performance of the safety- and mission-critical embedded software that is essential to landing astronauts on the surface of the Moon by 2024. Key among these challenges is IV&V’s desire to present a cohesive, integrated assurance statement to its stakeholders that encapsulates and summarizes our assurance positions across the integrated Artemis systems and their combined role in support of a safe and successful flight. In order to meet this challenge, the IV&V Teams have begun a transition to using formal assurance case concepts and documentation in the Goal Structuring Notation (GSN) to build an argument in support of software assurance. IV&V recognizes significant benefits to the logical argumentation structure provided by assurance cases and GSN over our current practices for documenting and managing assurance claims. In order to reap these benefits, IV&V is integrating the use of assurance case concepts with our paradigm of follow-the-risk capability based assurance. Because of this, assurance cases created and used by IV&V are distinct from the sort of assurance case created by a development project or embedded software assurance organization. IV&V’s assurance cases depend much less upon standards and regulations, and more on evidence captured by IV&V regarding the environment, requirements, design, and implementation. IV&V constructs an independent network of claims based on an independent decomposition of arguments. Based upon the risk posture of these claims and their associated software and software artifacts, IV&V then develops and executes engineering analyses and testing, which provide evidence to either support or refute the claim. This emerging risk-informed assurance case methodology is being put into practice as IV&V plans for support of the Artemis II mission, the first flight of the Orion capsule and Space Launch System with astronauts on board.
Gerek Whitman↗
IV&V Assurance Case Design for Artemis II
As human-rated missions like those in NASA's Artemis program continue to grow in both size and complexity, and the role of software in achieving mission objectives expands dramatically, NASA's Independent Verification and Validation (IV&V) Teams face evolving challenges in assuring the safety and performance of the safety- and mission-critical embedded software that is essential to landing astronauts on the surface of the Moon by 2024. Key among these challenges is IV&V's desire to present a cohesive, integrated assurance statement to its stakeholders that encapsulates and summarizes our assurance positions across the integrated Artemis systems and their combined role in support of a safe and successful flight. In order to meet this challenge, the IV&V Teams have begun a transition to using formal assurance case concepts and documentation in the Goal Structuring Notation (GSN) to build an argument in support of software assurance. IV&V recognizes significant benefits to the logical argumentation structure provided by assurance cases and GSN over our current practices for documenting and managing assurance claims. In order to reap these benefits, IV&V is integrating the use of assurance case concepts with our paradigm of follow-the-risk capability based assurance. Because of this, assurance cases created and used by IV&V are distinct from the sort of assurance case created by a development project or embedded software assurance organization. IV&V's assurance cases depend much less upon standards and regulations, and more on evidence captured by IV&V regarding the environment, requirements, design, and implementation. IV&V constructs an independent network of claims based on an independent decomposition of arguments. Based upon the risk posture of these claims and their associated software and software artifacts, IV&V then develops and executes engineering analyses and testing, which provide evidence to either support or refute the claim. This emerging risk-informed assurance case methodology is being put into practice as IV&V plans for support of the Artemis II mission, the first flight of the Orion capsule and Space Launch System with astronauts on board.
Whitman, Gerek↗
Ares I-X Range Safety Simulation Verification and Analysis IV and V
NASA s ARES I-X vehicle launched on a suborbital test flight from the Eastern Range in Florida on October 28, 2009. NASA generated a Range Safety (RS) flight data package to meet the RS trajectory data requirements defined in the Air Force Space Command Manual 91-710. Some products included in the flight data package were a nominal ascent trajectory, ascent flight envelope trajectories, and malfunction turn trajectories. These data are used by the Air Force s 45th Space Wing (45SW) to ensure Eastern Range public safety and to make flight termination decisions on launch day. Due to the criticality of the RS data in regards to public safety and mission success, an independent validation and verification (IV&V) effort was undertaken to accompany the data generation analyses to ensure utmost data quality and correct adherence to requirements. Multiple NASA centers and contractor organizations were assigned specific products to IV&V. The data generation and IV&V work was coordinated through the Launch Constellation Range Safety Panel s Trajectory Working Group, which included members from the prime and IV&V organizations as well as the 45SW. As a result of the IV&V efforts, the RS product package was delivered with confidence that two independent organizations using separate simulation software generated data to meet the range requirements and yielded similar results. This document captures ARES I-X RS product IV&V analysis, including the methodology used to verify inputs, simulation, and output data for an RS product. Additionally a discussion of lessons learned is presented to capture advantages and disadvantages to the IV&V processes used.
Tarpley, Ashley↗
Development of Methodologies for IV and V of Neural Networks
Non-deterministic systems often rely upon neural network (NN) technology to "lean" to manage flight systems under controlled conditions using carefully chosen training sets. How can these adaptive systems be certified to ensure that they will become increasingly efficient and behave appropriately in real-time situations? The bulk of Independent Verification and Validation (IV&V) research of non-deterministic software control systems such as Adaptive Flight Controllers (AFC's) addresses NNs in well-behaved and constrained environments such as simulations and strict process control. However, neither substantive research, nor effective IV&V techniques have been found to address AFC's learning in real-time and adapting to live flight conditions. Adaptive flight control systems offer good extensibility into commercial aviation as well as military aviation and transportation. Consequently, this area of IV&V represents an area of growing interest and urgency. ISR proposes to further the current body of knowledge to meet two objectives: Research the current IV&V methods and assess where these methods may be applied toward a methodology for the V&V of Neural Network; and identify effective methods for IV&V of NNs that learn in real-time, including developing a prototype test bed for IV&V of AFC's. Currently. no practical method exists. lSR will meet these objectives through the tasks identified and described below. First, ISR will conduct a literature review of current IV&V technology. TO do this, ISR will collect the existing body of research on IV&V of non-deterministic systems and neural network. ISR will also develop the framework for disseminating this information through specialized training. This effort will focus on developing NASA's capability to conduct IV&V of neural network systems and to provide training to meet the increasing need for IV&V expertise in such systems.
Taylor, Brian↗