SEARCH · Search NASA

Results for “VERIFICATION”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

All sources NASA NTRS DOE OSTI

DOE OSTI2024

Sierra/SD – Verification Test Manual – 5.22

Verification and validation (V&V) of scientific computing programs are important at Sandia National Labs due to the expanding role of computational simulation in managing the United States nuclear stockpile. The complexities of structural response calculations used to analyze physical problems, the varieties of codes applied to the calculations, and the importance of accurate predictions when assessing field conditions demand confidence in the consistency and accuracy of computer codes. Confidence in the accuracy of the predictions arising from computer simulations must ultimately be gained through verification and validation. The Sierra salinas structural dynamics analysis code, Sierra/SD, is used at the DOE Laboratories, and in several DOD projects. The roles of Sierra/SD in the qualification of weapon systems and components for normal and hostile environments throughout the Stockpile-to-Target Sequence include to, • Redesign weapon components. • Certify weapon components and systems for target environments such as hypersonic vehicles. • Certify that components will survive the thermal mechanical shock loads associated with hostile environments. • Evaluate current stockpile issues, including issues associated with uncertainty quantification. • Address many other problems that are encountered in stockpile management. The Sierra/SD verification plan is described, and an evolving set of key verification tests are described in detail. The verification tests ensure the correctness of the mathematics and numerical algorithms associated with functionality describing engineering phenomena. Development is in accordance with a set of tailored Software Quality Engineering (SQE) practices. SQE practices guide the overall verification and validation effort.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS1975

Simulation verification techniques study

Results are summarized of the simulation verification techniques study which consisted of two tasks: to develop techniques for simulator hardware checkout and to develop techniques for simulation performance verification (validation). The hardware verification task involved definition of simulation hardware (hardware units and integrated simulator configurations), survey of current hardware self-test techniques, and definition of hardware and software techniques for checkout of simulator subsystems. The performance verification task included definition of simulation performance parameters (and critical performance parameters), definition of methods for establishing standards of performance (sources of reference data or validation), and definition of methods for validating performance. Both major tasks included definition of verification software and assessment of verification data base impact. An annotated bibliography of all documents generated during this study is provided.

Schoonmaker, P. B.↗

NASA NTRS1986

Verification of post permanently manned configuration Space Station elements

An account is given of the techniques and ground systems designed to fulfill post permanently manned configuration (PMC) Space Station verification tasks. Consideration is given to analysis using computer math models and computer-aided interface verification systems, testing using simulators and interface mixtures, and special inspection. It is noted that an initial Space Station design that accommodates and facilitates verification is crucial to an effective verification program as well as proper instrumentation, built-in test capability, and a precise configuration management, control and record system. It is concluded that post PMC verification should be accounted for both in the initial Space Station design and in the subsequent development of initial assembly flight verification techniques and capabilities.

Scully, E. J.↗

NASA NTRS1993

Towards the formal verification of the requirements and design of a processor interface unit

The formal verification of the design and partial requirements for a Processor Interface Unit (PIU) using the Higher Order Logic (HOL) theorem-proving system is described. The processor interface unit is a single-chip subsystem within a fault-tolerant embedded system under development within the Boeing Defense and Space Group. It provides the opportunity to investigate the specification and verification of a real-world subsystem within a commercially-developed fault-tolerant computer. An overview of the PIU verification effort is given. The actual HOL listing from the verification effort are documented in a companion NASA contractor report entitled 'Towards the Formal Verification of the Requirements and Design of a Processor Interface Unit - HOL Listings' including the general-purpose HOL theories and definitions that support the PIU verification as well as tactics used in the proofs.

Fura, David A.↗

NASA NTRS2006

Evaluation of Mesoscale Model Phenomenological Verification Techniques

Forecasters at the Spaceflight Meteorology Group, 45th Weather Squadron, and National Weather Service in Melbourne, FL use mesoscale numerical weather prediction model output in creating their operational forecasts. These models aid in forecasting weather phenomena that could compromise the safety of launch, landing, and daily ground operations and must produce reasonable weather forecasts in order for their output to be useful in operations. Considering the importance of model forecasts to operations, their accuracy in forecasting critical weather phenomena must be verified to determine their usefulness. The currently-used traditional verification techniques involve an objective point-by-point comparison of model output and observations valid at the same time and location. The resulting statistics can unfairly penalize high-resolution models that make realistic forecasts of a certain phenomena, but are offset from the observations in small time and/or space increments. Manual subjective verification can provide a more valid representation of model performance, but is time-consuming and prone to personal biases. An objective technique that verifies specific meteorological phenomena, much in the way a human would in a subjective evaluation, would likely produce a more realistic assessment of model performance. Such techniques are being developed in the research community. The Applied Meteorology Unit (AMU) was tasked to conduct a literature search to identify phenomenological verification techniques being developed, determine if any are ready to use operationally, and outline the steps needed to implement any operationally-ready techniques into the Advanced Weather Information Processing System (AWIPS). The AMU conducted a search of all literature on the topic of phenomenological-based mesoscale model verification techniques and found 10 different techniques in various stages of development. Six of the techniques were developed to verify precipitation forecasts, one to verify sea breeze forecasts, and three were capable of verifying several phenomena. The AMU also determined the feasibility of transitioning each technique into operations and rated the operational capability of each technique on a subjective 1-10 scale: (1) 1 indicates that the technique is only in the initial stages of development, (2) 2-5 indicates that the technique is still undergoing modifications and is not ready for operations, (3) 6-8 indicates a higher probability of integrating the technique into AWIPS with code modifications, and (4) 9-10 indicates that the technique was created for AWIPS and is ready for implementation. Eight of the techniques were assigned a rating of 5 or below. The other two received ratings of 6 and 7, and none of the techniques a rating of 9-10. At the current time, there are no phenomenological model verification techniques ready for operational use. However, several of the techniques described in this report may become viable techniques in the future and should be monitored for updates in the literature. The desire to use a phenomenological verification technique is widespread in the modeling community, and it is likely that other techniques besides those described herein are being developed, but the work has not yet been published. Therefore, the AMIU recommends that the literature continue to be monitored for updates to the techniques described in this report and for new techniques being developed whose results have not yet been published. 111

Lambert, Winifred↗

NASA NTRS2014

Turbulence Modeling Verification and Validation

Computational fluid dynamics (CFD) software that solves the Reynolds-averaged Navier-Stokes (RANS) equations has been in routine use for more than a quarter of a century. It is currently employed not only for basic research in fluid dynamics, but also for the analysis and design processes in many industries worldwide, including aerospace, automotive, power generation, chemical manufacturing, polymer processing, and petroleum exploration. A key feature of RANS CFD is the turbulence model. Because the RANS equations are unclosed, a model is necessary to describe the effects of the turbulence on the mean flow, through the Reynolds stress terms. The turbulence model is one of the largest sources of uncertainty in RANS CFD, and most models are known to be flawed in one way or another. Alternative methods such as direct numerical simulations (DNS) and large eddy simulations (LES) rely less on modeling and hence include more physics than RANS. In DNS all turbulent scales are resolved, and in LES the large scales are resolved and the effects of the smallest turbulence scales are modeled. However, both DNS and LES are too expensive for most routine industrial usage on today's computers. Hybrid RANS-LES, which blends RANS near walls with LES away from walls, helps to moderate the cost while still retaining some of the scale-resolving capability of LES, but for some applications it can still be too expensive. Even considering its associated uncertainties, RANS turbulence modeling has proved to be very useful for a wide variety of applications. For example, in the aerospace field, many RANS models are considered to be reliable for computing attached flows. However, existing turbulence models are known to be inaccurate for many flows involving separation. Research has been ongoing for decades in an attempt to improve turbulence models for separated and other nonequilibrium flows. When developing or improving turbulence models, both verification and validation are important steps in the process. Verification insures that the CFD code is solving the equations as intended (no errors in the implementation). This is typically done either through the method of manufactured solutions (MMS) or through careful step-by-step comparisons with other verified codes. After the verification step is concluded, validation is performed to document the ability of the turbulence model to represent different types of flow physics. Validation can involve a large number of test case comparisons with experiments, theory, or DNS. Organized workshops have proved to be valuable resources for the turbulence modeling community in its pursuit of turbulence modeling verification and validation. Workshop contributors using different CFD codes run the same cases, often according to strict guidelines, and compare results. Through these comparisons, it is often possible to (1) identify codes that have likely implementation errors, and (2) gain insight into the capabilities and shortcomings of different turbulence models to predict the flow physics associated with particular types of flows. These are valuable lessons because they help bring consistency to CFD codes by encouraging the correction of faulty programming and facilitating the adoption of better models. They also sometimes point to specific areas needed for improvement in the models. In this paper, several recent workshops are summarized primarily from the point of view of turbulence modeling verification and validation. Furthermore, the NASA Langley Turbulence Modeling Resource website is described. The purpose of this site is to provide a central location where RANS turbulence models are documented, and test cases, grids, and data are provided. The goal of this paper is to provide an abbreviated survey of turbulence modeling verification and validation efforts, summarize some of the outcomes, and give some ideas for future endeavors in this area.

Rumsey, Christopher L.↗

NASA NTRS2012

Cognitive Bias in the Verification and Validation of Space Flight Systems

Cognitive bias is generally recognized as playing a significant role in virtually all domains of human decision making. Insight into this role is informally built into many of the system engineering practices employed in the aerospace industry. The review process, for example, typically has features that help to counteract the effect of bias. This paper presents a discussion of how commonly recognized biases may affect the verification and validation process. Verifying and validating a system is arguably more challenging than development, both technically and cognitively. Whereas there may be a relatively limited number of options available for the design of a particular aspect of a system, there is a virtually unlimited number of potential verification scenarios that may be explored. The probability of any particular scenario occurring in operations is typically very difficult to estimate, which increases reliance on judgment that may be affected by bias. Implementing a verification activity often presents technical challenges that, if they can be overcome at all, often result in a departure from actual flight conditions (e.g., 1-g testing, simulation, time compression, artificial fault injection) that may raise additional questions about the meaningfulness of the results, and create opportunities for the introduction of additional biases. In addition to mitigating the biases it can introduce directly, the verification and validation process must also overcome the cumulative effect of biases introduced during all previous stages of development. A variety of cognitive biases will be described, with research results for illustration. A handful of case studies will be presented that show how cognitive bias may have affected the verification and validation process on recent JPL flight projects, identify areas of strength and weakness, and identify potential changes or additions to commonly used techniques that could provide a more robust verification and validation of future systems.

Larson, Steve↗

NASA NTRS2016

Z-2 Architecture Description and Requirements Verification Results

The Z-2 Prototype Planetary Extravehicular Space Suit Assembly is a continuation of NASA's Z series of spacesuits. The Z-2 is another step in NASA's technology development roadmap leading to human exploration of the Martian surface. The suit was designed for maximum mobility at 8.3 psid, reduced mass, and to have high fidelity life support interfaces. As Z-2 will be man-tested at full vacuum in NASA JSC's Chamber B, it was manufactured as Class II, making it the most flight-like planetary walking suit produced to date. The Z-2 suit architecture is an evolution of previous EVA suits, namely the ISS EMU, Mark III, Rear Entry I-Suit and Z-1 spacesuits. The suit is a hybrid hard and soft multi-bearing, rear entry spacesuit. The hard upper torso (HUT) is an all-composite structure and includes a 2-bearing rolling convolute shoulder with Vernier sizing mechanism, removable suit port interface plate (SIP), elliptical hemispherical helmet and self-don/doff shoulder harness. The hatch is a hybrid aluminum and composite construction with Apollo style gas connectors, custom water pass-thru, removable hatch cage and interfaces to primary and auxiliary life support feed water bags. The suit includes Z-1 style lower arms with cam brackets for Vernier sizing and government furnished equipment (GFE) Phase VI gloves. The lower torso includes a telescopic waist sizing system, waist bearing, rolling convolute waist joint, hard brief, 2 bearing soft hip thigh, Z-1 style legs with ISS EMU style cam brackets for sizing, and conformal walking boots with ankle bearings. The Z-2 Requirements Verification Plan includes the verification of more than 200 individual requirements. The verification methods include test, analysis, inspection, demonstration or a combination of methods. Examples of unmanned requirements include suit leakage, proof pressure testing, operational life, mass, isometric man-loads, sizing adjustment ranges, internal and external interfaces such as in-suit drink bag, partial pressure relief valve, purge valve, donning stand and ISS Body Restraint Tether (BRT). Examples of manned requirements include verification of anthropometric range, suit self-don/doff, secondary suit exit method, donning stand self-ingress/egress and manned mobility covering eight functional tasks. The eight functional tasks include kneeling with object pick-up, standing toe touch, cross-body reach, walking, reach to the SIP and helmet visor. This paper will provide an overview of the Z-2 design. Z-2 requirements verification testing was performed with NASA at the ILC Houston test facility. This paper will also discuss pre-delivery manned and unmanned test results as well as analysis performed in support of requirements verification.

Graziosi, Dave↗

NASA NTRS2018

Space System Verification Approach Based on MEAL and Mission Risk Posture

There is no "one size fits all" solution for verifying space avionics systems to ensure safety and mission success. This paper presents a verification approach based on MEAL and risk posture for space systems. MEAL refers to Mission, mission Environment, Application, and Lifetime of the application. In addition to the description of the verification approach, the paper also provides the awareness of the different levels of risks associated with verification tests and inspections when performed at part-, board- and box-level, and discusses the applications of the approach for flight heritage verification, commercial off the shelf (COTS) verification and radiation-effects verification.

Mission Environment Application and Lifetime (MEAL↗

NASA NTRS2018

Space System Verification Approach Based on MEAL and Mission Risk Posture

Mission Environment Application and Lifetime (MEAL↗

NASA NTRSDate not supplied

Adding a Verification View for Autonomous Real-Time Architecture

Spacecraft software systems continue to increase in complexity and must frequently operate autonomously for extended periods. Thus, the consequences of software flaws are increasing and it is necessary to improve development methods to minimize the risk of latent flaws in the operational software. Traditional software architectures place verification as one element in the development architectural view and treat verification as a later lifecycle activity that is considered complete when the system is certified for flight. The Gateway Vehicle System Manager team, recognizing the importance of verification throughout the system lifecycle, is treating verification as an additional architectural view that receives continuous attention from the requirements analysis phase and continuing for the life of the operational system. The verification view consists of two viewpoints: development and operational. We present background on the verification view and discuss the approaches the Vehicle System Manager team is taking.

James B Dabney↗

NASA NTRS2017

Runtime Verification Logics A Language Design Perspective

Runtime Verification is a light-weight approach to systems verification, where actual executions of a system are processed and analyzed using rigorous techniques. In this paper we shall narrow the term’s definition to represent the commonly studied variant consisting of verifying that a single system execution conforms to a specification written in a formal specification language. Runtime verification (in this sense) can be used for writing test oracles during testing when the system is too complex for full formal verification, or it can be used during deployment of the system as part of a fault protection strategy, where corrective actions may be taken in case the specification is violated. Specification languages for runtime verification appear to differ from for example temporal logics applied in model checking, in part due to the focus on monitoring of events that carry data, and specifically due to the desire to relate data values existing at different time points, resulting in new challenges in both the complexity of the monitoring approach and the expressiveness of languages. Over the recent years, numerous runtime verification specification languages have emerged, each with its different features and levels of expressiveness and usability. This paper presents an overview and a discussion of this design space.

Reger, Giles↗

DOE OSTI2024

Reusable Verification Components for High-Energy Physics readout ASICs

Verification is a critical aspect of designing front-end (FE) readout ASICs for High-Energy Physics (HEP) experiments. These ASICs share several similar functional features, resulting in similar verification objectives, which can be addressed using comparable verification strategies. This contribution presents a set of re-usable verification components for addressing common verification tasks, such as clock generation, reset handling, configuration, as well as hit and fault injections. The components were developed as part of the CHIPS initiative and they have been successfully used in the verification of multiple HEP ASICs.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2023

Verification and benchmarking relativistic electron beam transport through a background gas

It is necessary to establish confidence in high-consequence codes containing an extensive suite of physics algorithms in the regimes of interest. Verification problems allow code developers to assess numerical accuracy and increase confidence that specific sets of model physics were implemented correctly in the code. The two main verification techniques are code verification and solution verification. In this work, we present verification problems that can be used in other codes to increase confidence in simulations of relativistic beam transport. Specifically, we use the general plasma code EMPIRE to model and compare with the analytical solution to the evolution of the outer radial envelope of a relativistic charged particle beam. Additionally, we also outline a benchmark test of a relativistic beam propagating through a vacuum and pressurized gas cell, and present the results between EMPIRE and the hybrid code GAZEL. Further, we discuss the subtle errors that were caught with these problems and detail lessons learned.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

DOE OSTI2022

Verification of MOOSE/Bison's Heat Conduction Solver Using Combined Spatiotemporal Convergence Analysis

Bison is a computational physics code that uses the finite element method to model the thermo-mechanical response of nuclear fuel. Since Bison is used to inform high-consequence decisions, it is important that its computational results are reliable and predictive. One important step in assessing the reliability and predictive capabilities of a simulation tool is the verification process, which quantifies numerical errors in a discrete solution relative to the exact solution of the mathematical model. One step in the verification process—called code verification—ensures that the implemented numerical algorithm is a faithful representation of the underlying mathematical model, including partial differential or integral equations, initial and boundary conditions, and auxiliary relationships. In this paper, the code verification process is applied to spatiotemporal heat conduction problems in Bison. Simultaneous refinement of the discretization in space and time is employed to reveal any potential mistakes in the numerical algorithms for the interactions between the spatial and temporal components of the solution. For each verification problem, the correct spatial and temporal order of accuracy is demonstrated for both first- and second-order accurate finite elements and a variety of time-integration schemes. Furthermore, these results provide strong evidence that the Bison numerical algorithm for solving spatiotemporal problems reliably represents the underlying mathematical model in MOOSE. The selected test problems can also be used in other simulation tools that numerically solve for conduction or diffusion.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

DOE OSTI2020

ZND Verification Tests for Reactive Burn Models in FLAG

The ZND theory, named after Zeldovich, von Neumann, and Doering, provides a simple model for one-dimensional ideal steady-state detonation. It assumes that the detonation wave front starts with a shock that is a discontinuous jump and is followed by a finite-length reaction zone. Reactive burn (also called reactive flow) models are based on ZND theory, as they model the shock initiation and detonation process with a finite reaction rate. The ZND wave propagation test is essentially the only available test case where an analytic solution exists for verification of reactive burn models in numerical codes. However, there are extensions and variants of the ZND test that have been devised for verification of multidimensional flows. The objective of this work is to provide verification of the reactive burn models currently implemented in the Lagrangian hydrocode FLAG and investigate the influence of mesh resolution, artificial viscosity models, and the Arbitrary Lagrangian-Eulerian (ALE) Euler relaxer on the simulation results. The burn models of interest are the Wescott-Stewart-Davis (WSD) model, the Scaled Uniform Reactive Flow (SURF) model (specifically with the SURFplus model extension), and the Arrhenius shock temperature state dependent WSD (AWSD) model. Previously, Ralph Menikoff has used ZND tests for verification of the SURF and SURFplus models in the Eulerian hydrocode xRAGE. The ZND tests here are somewhat different than the approach by Menikoff. In particular, we use a piston-driven ZND detonation wave (via a prescribed constant velocity boundary condition) in a Lagrangian framework whereas Menikoff had a ZND wave followed by a invariant rarefaction wave. The xRAGE simulations were carried out on uniform grids and adaptive mesh refinement (AMR) grids. Although AMR was recently implemented in FLAG and now fully functional for 2D simulations, it will not be evaluated with ZND tests at this time. Some work has been done previously for validating the reactive burn models in FLAG. For example, the validation studies of SURF with shock-to-detonation (SDT) tests, cylinder tests, and gap-stick tests. Further validation of the AWSD, WSD, and SURF models is described in for SDT, multi-shock, cylinder, and corner-turning tests. Recently, a large parameter study with approximately twenty SDT tests was performed to validate the AWSD, WSD, SURF, and SURFplus models while also investigating mesh resolution and artificial viscosity settings. To the best of our knowledge, the current work represents the first documented verification of these burn models in FLAG. The reader should note that many of the tables and figures in this report use units of cm/g/µs, which are the default units for FLAG. However, some lengths and velocities are expressed in µm and mm/µs, respectively, which are typical units for detonation analyses.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

NASA NTRS2017

Cleared for Launch - Lessons Learned from the OSIRIS-REx System Requirements Verification Program

Requirements verification of a large flight system is a challenge. It is especially challenging for engineers taking on their first role in space systems engineering. This paper describes our approach to verification of the Origins, Spectral Interpretation, Resource Identification, Security-Regolith Explorer (OSIRIS-REx) system requirements. It also captures lessons learned along the way from developing systems engineers embroiled in this process. We begin with an overview of the mission and science objectives as well as the project requirements verification program strategy. A description of the requirements flow down is presented including our implementation for managing the thousands of program and element level requirements and associated verification data. We discuss both successes and methods to improve the managing of this data across multiple organizational interfaces. Our approach to verifying system requirements at multiple levels of assembly is presented using examples from our work at instrument, spacecraft, and ground segment levels. We include a discussion of system end-to-end testing limitations and their impacts to the verification program. Finally, we describe lessons learned that are applicable to all emerging space systems engineers using our unique perspectives across multiple organizations of a large NASA program.

Requirements Verification↗

NASA NTRS2018

Investigation of a Verification and Validation Tool with a Turbofan Aircraft Engine Application

The development of more advanced control architectures for turbofan aircraft engines can yield gains in performance and efficiency over the lifetime of an engine. However, the implementation of these increasingly complex controllers is contingent on their ability to provide safe, reliable engine operation. Therefore, having the means to verify the safety of new control algorithms is crucial. As a step towards this goal, CoCoSim, a publicly available verification tool for Simulink, is used to analyze C-MAPSS40k, a 40,000 lbf class turbo-fan engine model developed at NASA for testing new control algorithms. Due to current limitations of the verification software, several modifications are made to C-MAPSS40k to achieve compatibility with CoCoSim. Some of these modifications sacrifice fidelity to the original model. Several safety and performance requirements typical for turbofan engines are identified and constructed into a verification framework. Preliminary results using an industry standard baseline controller for these requirements are presented. While verification capabilities are demonstrated, a truly comprehensive analysis will require further development of the verification tool.

V&↗