Search NASA⌕ Search

SEARCH · Search NASA

Results for “verification & validation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 343 records · Page 19

Expert system verification and validation study. Phase 2: Requirements identification. Delivery 1: Updated survey report

The purpose is to report the state-of-the-practice in Verification and Validation (V and V) of Expert Systems (ESs) on current NASA and Industry applications. This is the first task of a series which has the ultimate purpose of ensuring that adequate ES V and V tools and techniques are available for Space Station Knowledge Based Systems development. The strategy for determining the state-of-the-practice is to check how well each of the known ES V and V issues are being addressed and to what extent they have impacted the development of Expert Systems.

Source record↗

Verification and Validation of Neutronic/Thermalhydraulic 3D-Time Dependent Model for Treatment of Super-critical States of Light water Research Reactors Accidents

This work presents the Verification and testing both the neutronic and thermal-hydraulics response of the positive reactivity-initiated power excursion accidents in small light water research reactors. Some research reactors have to build its own severe accidents code system. In this sense, a 3D space-time-dependent neutron diffusion models with thermal hydraulic feedback have been introduced, compared and tested both experimentally at criticality 14-cent and theoretically up to 1.5 $ with a number of similar codes. The results shows that no expected core failure or moderator boiling. (author)

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Validation and Verification of MCNP6 Against High-Energy Experimental Data and Calculations by Other Codes. III. The MPI Testing Primer

This primer presents a set of Validation and Verification (V&V) MCNP6 results calculated in parallel, with MPI, obtained mostly using its event generators at intermediate and high-energies, but also several examples of results using data libraries at energies below 150 MeV, against various recent experimental data. It is the third part of a set of MCNP6 Testing Primers, after its first, LA-UR-11-05129, and second, LA-UR-11-05627, publications. The MCNP6 test-problems discussed here are presented in the /VALIDATION CEM/, /VALIDATION LAQGSM/, and /DELAYED PARTICLES/ subdirectories in the MCNP6 /Testing/ directory. README files that contain short descriptions of every input file, the experiment, the quantity of interest that the experiment measures and its description in the MCNP6 output files, and the publication reference of that experiment are presented for every test-problem. Templates for plotting the corresponding results with xmgrace as well as pdf files with figures representing the final results of our V&V efforts are presented. Several technical “bugs” in MCNP6 while running it in parallel with MPI using its event generators were discovered during our current V&V of MCNP6 and were fixed mostly by Dr. Jeffrey S. Bull. Our results show that MCNP6 using its CEM03.03, LAQGSM03.03, Bertini, INCL+ABLA, and ISABEL event generators describes, as a rule, reasonably well different intermediate- and high-energy measured data and agrees very well with calculations by other codes. Just as expected, the MCNP6 results obtained with MPI are practically the same as the ones calculated with MCNP6 in a sequential mode. This primer isn’t meant to be read from cover to cover. Readers may skip some sections and go directly to a test-problem they are interested in.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Managing Complexity in the MSL/Curiosity Entry, Descent, and Landing Flight Software and Avionics Verification and Validation Campaign

The complexity of the Mars Science Laboratory (MSL) mission presented the Entry, Descent, and Landing systems engineering team with many challenges in its Verification and Validation (V&V) campaign. This paper describes some of the logistical hurdles related to managing a complex set of requirements, test venues, test objectives, and analysis products in the implementation of a specific portion of the overall V&V program to test the interaction of flight software with the MSL avionics suite. Application-specific solutions to these problems are presented herein, which can be generalized to other space missions and to similar formidable systems engineering problems.

Stehura, Aaron↗

Verification and Validation of Flight-Critical Systems

For the first time in many years, the NASA budget presented to congress calls for a focused effort on the verification and validation (V&V) of complex systems. This is mostly motivated by the results of the VVFCS (V&V of Flight-Critical Systems) study, which should materialize as a a concrete effort under the Aviation Safety program. This talk will present the results of the study, from requirements coming out of discussions with the FAA and the Joint Planning and Development Office (JPDO) to technical plan addressing the issue, and its proposed current and future V&V research agenda, which will be addressed by NASA Ames, Langley, and Dryden as well as external partners through NASA Research Announcements (NRA) calls. This agenda calls for pushing V&V earlier in the life cycle and take advantage of formal methods to increase safety and reduce cost of V&V. I will present the on-going research work (especially the four main technical areas: Safety Assurance, Distributed Systems, Authority and Autonomy, and Software-Intensive Systems), possible extensions, and how VVFCS plans on grounding the research in realistic examples, including an intended V&V test-bench based on an Integrated Modular Avionics (IMA) architecture and hosted by Dryden.

Brat, Guillaume↗

Validation and Verification of TEDS Facility HYBRID Modeling

The HYBRID modeling repository is an in-house developed library of models for selected integrated energy systems (IES) modelling. HYBRID models have been developed since 2015 to describe the physical operation of tightly coupled thermal systems including power generators, thermal transport systems, thermal storage, thermal-to-electric conversion systems, and other thermal applications. Here, validation and verification (V&V) capabilities are demonstrated using the Thermal Energy Distribution System (TEDS) at INL. Building upon prior work, the TEDS model has been updated and verified so that it better represents the installed system configuration and the operating control system. The model control system was changed to allow replication of actual experimental procedures. Experimental operations focusing primarily on thermocline tank performance were devised and performed. Several anomalies were found in the operation data of the experiment facility. V&V activities calibrating a selected input parameter are demonstrated on a single component as well as with a single parameter within the thermocline. Calibrating is then demonstrated on multiple components and a multi-parameter metric for the entire system. The validation methodology is successfully applied to validate the model with experimental data. It is also used to confirm a hypothesis behind one of the anomalies in experimental performance.

25 ENERGY STORAGE↗

Robotics Verification and Validation Strategies for Perseverance Rover Sampling and Caching

The Mars 2020 Sampling and Caching Subsystem(SCS) is the most complex robotic system ever fielded on a MarsRover. It includes a 5 degree-of-freedom Robotic Arm, coringdrill, gas Dust Removal Tool, interfaces for two turret-mountedinstruments, and an Adaptive Caching Assembly (ACA). TheACA is itself a complex robotic system, containing hardware tosupport docking and bit exchange, a 3 degree-of-freedomSample Handling Assembly for manipulating sample tubes,storage for several drill bits and sample tubes, and mechanismsto support observing and sealing samples collected by the drill.To successfully verify and validate the SCS hardware andsoftware and its integration with the Mars 2020 flight systemseveral key strategies were employed.The SCS Verification and Validation (V&V) program utilizedmultiple test venues with tiered levels of fidelity. These includedsimulation and visualization software environments, low fidelitydevelopment testbeds, testbeds with high fidelity SCS hardwareand commercial off-the shelf avionics, integrated systemtestbeds with flight-like avionics, and environmental testbedscapable of simulating Martian surface temperature andpressure. Multiple units of each SCS hardware componentmoved fluidly between test venues to accomplish myriadstandalone and coordinated test objectives. Test preparationand executions were performed by a diverse team of engineerswith training and technical ownership tailored for individualexperience and role. Despite significant differences between testvenues, the SCS V&V team established efficient and consistentprocesses and tools for procedure development, test execution,and data review that enabled personnel, as well as technicalproducts such as sequences and parameter configurations, toflow between venues effectively. A series of benchmark testsprovided evidence of performance consistency as elements weretransferred between venues and as system capability evolved.This paper provides an overview of the SCS V&V program andexplores several overarching strategies that enabled successfuloperation in the face of unprecedented complexity. Keyoutcomes of the SCS validation effort are summarized, alongwith lessons learned and beneficial integrations of validationtool and process innovations into Mars surface operations.

Brooks, Sawyer↗

Verification and validation of the open-source plasma fluid code: Zapdos

Zapdos is an open-source finite element plasma fluid solver based on the MOOSE multiphysics framework. This paper outlines Zapdos verification, benchmarking, and validation efforts for 1D and 2D RF capacitively coupled plasma discharge models for mid-range pressures (0.1 - 1 Torr). The verification process involved using the method of manufactured solutions to assess Zapdos spatial and temporal error convergence. L2 errors ranged from 10 -2 to 10 -4 , while the convergence’s slope were in agreement with the predicted slopes for the tested variable and time integration orders. The benchmarking process involved comparisons to previously results from the validated finite element code, LSODI. These works included 1D and 2D simulations for a range of plasma parameters (densities, temperatures, voltage, etc.). For the 1D cases, Zapdos and LSODI results were in very good agreement. In the 2D cases, variable behaviors matched, with slight discrepancies in peak values. The validation process involved comparisons to experimental works including electron density measurements by microwave interferometry and metastable density measurements by planar laser-induced fluorescence imaging. Results shown reasonable agreement at higher pressure, with results starting to diverge at low pressures. Furthermore, probable causes for this diverges are the limitation of the fluid assumption for plasmas at low pressure, or the need for more robust boundary conditions. Overall, Zapdos shown reasonable results for the verification, benchmarking, and validation efforts, and Zapdos can be downloaded at https://github.com/shannon-lab/zapdos.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Ocean Energy Systems Wave Energy Modelling Task: Modelling, Verification and Validation of Wave Energy Converters

The International Energy Agency Technology Collaboration Programme for Ocean Energy Systems (OES) initiated the OES Wave Energy Conversion Modelling Task, which focused on the verification and validation of numerical models for simulating wave energy converters (WECs). The long-term goal is to assess the accuracy of and establish confidence in the use of numerical models used in design as well as power performance assessment of WECs. To establish this confidence, the authors used different existing computational modelling tools to simulate given tasks to identify uncertainties related to simulation methodologies: (i) linear potential flow methods; (ii) weakly nonlinear Froude–Krylov methods; and (iii) fully nonlinear methods (fully nonlinear potential flow and Navier–Stokes models). This article summarizes the code-to-code task and code-to-experiment task that have been performed so far in this project, with a focus on investigating the impact of different levels of nonlinearities in the numerical models. Two different WECs were studied and simulated. The first was a heaving semi-submerged sphere, where free-decay tests and both regular and irregular wave cases were investigated in a code-to-code comparison. The second case was a heaving float corresponding to a physical model tested in a wave tank. We considered radiation, diffraction, and regular wave cases and compared quantities, such as the WEC motion, power output and hydrodynamic loading.

16 TIDAL AND WAVE POWER↗

The Verification and Validation of a Magnetic Plasma Fluid Model Utilizing the MOOSE (Multiphysics Object Oriented Simulation Environment) Framework

As the goal of achieving fusion power on the grid comes closer to fruition, fully coupled multiphysics models of fusion devices will be crucial. Currently, there are two main approaches to developing these platforms: (1) loosely coupled, where one couples existing codes and solvers together through input and output parameters and data, and (2) tightly coupled, where one develops the necessary models within a singular, integrated framework. This work focuses on the latter approach for magnetically confined fusion devices by developing a fluid-based plasma-edge model within the Multiphysics Object Oriented Simulation Environment (MOOSE) Framework. This effort is coordinated with other efforts to develop, test, demonstrate, and deploy fusion relevant multiphysics capabilities including electromagnetics, particle-in-cell plasma, tritium transport, and fusion blanket design. This new model is an expansion of the MOOSE-based plasma application, Zapdos, which was originally formulated to model low-temperature, non-magnetized plasma processes. Verification, benchmarking, and validation studies have been conducted. Verification studies involved utilizing the method of manufactured solutions and comparing the convergence slope of a known solution to the theoretical slope. Benchmarking consists of comparisons to existing edge codes, namely BOUT++ and SOLEDGE3X. Validation efforts focused on comparisons against open-source data from the TCV tokamak.

70 - PLASMA PHYSICS AND FUSION TECHNOLOGY↗

STAR-CCM+ Verification and Validation Plan

The commercial Computational Fluid Dynamics (CFD) code STAR-CCM+ provides general purpose finite volume method solutions for fluid dynamics and energy transport. This document defines plans for verification and validation (V&V) of the base code and models implemented within the code by the Consortium for Advanced Simulation of Light water reactors (CASL). The software quality assurance activities described herein are port of the overall software life cycle defined in the CASL Software Quality Assurance (SQA) Plan [Sieger, 2015]. STAR-CCM+ serves as the principal foundation for development of an advanced predictive multi-phase boiling simulation capability within CASL. The CASL Thermal Hydraulics Methods (THM) team develops advanced closure models required to describe the subgrid-resolution behavior of secondary fluids or fluid phases in multiphase boiling flows within the Eulerian-Eulerian framework of the code. These include wall heat partitioning models that describe the formation of vapor on the surface and the forces the define bubble/droplet dynamic motion. The CASL models are implemented as user coding or field functions within the general framework of the code. This report defines procedures and requirements for V&V of the multi-phase CFD capability developed by CASL THM. Results of V&V evaluations will be documented in a separate STAR-CCM+ V&V assessment report. This report is expected to be a living document and will be updated as additional validation cases are identified and adopted as part of the CASL THM V&V suite.

97 MATHEMATICS AND COMPUTING↗

GOES-R Spacecraft Verification and Validation Compared with Flight Results

The Geostationary Operational Environmental Satellite, R-Series (GOES-R) represents a dramatic improvement in GEO (Geostationary Earth Orbit) weather observation capabilities over the previous generation. To provide these new capabilities, GOES-R incorporates a number of new technologies flying for the first time. As with any new spacecraft design, extensive ground testing was performed to validate the vehicle performance. In this paper, we present several successes and several lessons-learned from the GOES-R verification and validation (V&V) efforts. Included are the Dynamic Interaction Test (DIT) results for jitter assessment, and comparison to flight results. Also included are the effects of thermally-induced alignment perturbations, along with post-launch mitigations. Finally, we discuss unexpected GOES-17 gyro performance, which caused a Safe Mode entry shortly after launch. V&V mitigations are presented, which will be used for the next two GOES-R vehicles.

GOES-17↗

Validation and verification of MCNP6 as a new simulation tool useful for medical applications

MCNP6, the latest and most advanced LANL transport code, representing a merger of MCNP5 and MCNPX has been Validated and Verified (V&V) against different experimental data and results by other codes relevant to medical applications. In the present work, we V&V MCNP6 using mainly the latest modifications of the Cascade-Exciton Model (CEM) and of the Los Alamos version of the Quark-Gluon String Model (LAQGSM) event generators CEM03.02 and LAQGSM03.03. We found that MCNP6 describes well data of interest for medical applications measured on both thin and thick targets and agrees very well with similar results obtained with other codes; MCNP6 may be a very useful tool for medical applications We plan to make MCNP6 available to the public via RSICC at Oak Ridge in the middle of 2011 but we are allowed to provide it to friendly US Beta-users outside LANL already now.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

An expert system development methodology which supports verification and validation

Expert systems have demonstrated commercial viability in a wide range of applications, but still face some obstacles to widespread use. A major stumbling block is the lack of well defined verification and validation (V and V) techniques. The primary difficulty with expert system V and V is the use of development methodologies which do not support V and V. As with conventional code, the key to effective V and V is the development methodology. An expert system development methodology is described which is based upon a panel review approach, that allows input from all parties concerned with the expert system.

Culbert, Chris↗

Expert system verification and validation study. Phase 2: Requirements Identification. Delivery 2: Current requirements applicability

The second phase of a task is described which has the ultimate purpose of ensuring that adequate Expert Systems (ESs) Verification and Validation (V and V) tools and techniques are available for Space Station Freedom Program Knowledge Based Systems development. The purpose of this phase is to recommend modifications to current software V and V requirements which will extend the applicability of the requirements to NASA ESs.

Source record↗

OpenMP 4.5 Validation and Verification Suite for Device Offload

OpenMP has been widely adopted for shared memory systems for over a decade. With the heterogeneity trend in architectures rapidly growing, the programming model needed to evolve such that applications could not only be ported to traditional CPUs but also to accelerators often acting as discrete or integrated devices to CPUs. To that end, OpenMP started to provide support for heterogeneous systems since 2013 when the version 4.0 of the specification was ratified. OpenMP 4.5 is being enhanced to cover major requirements of Exascale Computing Project (ECP) applications. As a result it is time-critical to ensure that the implementations of the 4.5 features are correct and conforming to the specification. This paper focuses on building a Validation and Verification testsuite that will test and present results for several offloading features implemented in compilers such as Clang, IBM XL C/C++, CCE, and GCC. We have results for our testsuite on TITAN, Summitdev and Summit at the Oak Ridge National Lab. We will highlight some of the ambiguities we encountered in the process of validating and verifying feature implementations. We also make the testsuite available for anyone to use and will walk the readers through the infrastructure and the workflow of the testsuite. A website has been built to capture our efforts narrated in this paper https://crpl.cis.udel.edu/ompvvsollve.

99 GENERAL AND MISCELLANEOUS↗