Search NASA⌕ Search

SEARCH · Search NASA

Results for “Verification and Validation methods”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3

Validation of Test Methods for Air Leak Rate Verification of Spaceflight Hardware

As deep space exploration continues to be the goal of NASAs human spaceflight program, verification of the performance of spaceflight hardware becomes increasingly critical. Suitable test methods for verifying the leak rate of sealing systems are identified in program qualification testing requirements. One acceptable method for verifying the air leak rate of gas pressure seals is the tracer gas leak detector method. In this method, a tracer gas (commonly helium) leaks past the test seal and is transported to the leak detector where the leak rate is quantified. To predict the air leak rate, a conversion factor of helium-to-air is applied depending on the magnitude of the helium flow rate. The conversion factor is based on either the molecular mass ratio or the ratio of the dynamic viscosities. The current work was aimed at validating this approach for permeation-level leak rates using a series of tests with a silicone elastomer O-ring. An established pressure decay method with constant differential pressure was used to evaluate both the air and helium leak rates of the O-ring under similar temperature and pressure conditions. The results from the pressure decay tests showed, for the elastomer O-ring, that neither the molecular flow nor the viscous flow helium-to-air conversion factors were applicable. Leak rate tests were also performed using nitrogen and argon as the test gas. Molecular mass and viscosity based helium-to-test gas conversion factors were applied, but did not correctly predict the measured leak rates of either gas. To further this study, the effect of pressure boundary conditions was investigated. Often, pressure decay leak rate tests are performed at a differential pressure of 101.3 kPa with atmospheric pressure on the downstream side of the test seal. In space applications, the differential pressure is similar, but with vacuum as the downstream pressure. The same O-ring was tested at four unique differential pressures ranging from 34.5 to 137.9 kPa. Up to six combinations of upstream and downstream pressures for each differential pressure were compared. For a given differential pressure, the various combinations of upstream and downstream dry air pressures did not significantly affect the leak rate. As expected, the leak rate of the O-ring increased with increasing differential pressure. The results suggested that the current leak test pressure conditions, used to verify spacecraft sealing systems with elastomer seals, produce accurate values even though the boundary conditions do not model the space application.

helium leak detector↗

Properties of a Formal Method for Prediction of Emergent Behaviors in Swarm-based Systems

Autonomous intelligent swarms of satellites are being proposed for NASA missions that have complex behaviors and interactions. The emergent properties of swarms make these missions powerful, but at the same time more difficult to design and assure that proper behaviors will emerge. This paper gives the results of research into formal methods techniques for verification and validation of NASA swarm-based missions. Multiple formal methods were evaluated to determine their effectiveness in modeling and assuring the behavior of swarms of spacecraft. The NASA ANTS mission was used as an example of swarm intelligence for which to apply the formal methods. This paper will give the evaluation of these formal methods and give partial specifications of the ANTS mission using four selected methods. We then give an evaluation of the methods and the needed properties of a formal method for effective specification and prediction of emergent behavior in swarm-based systems.

Rouff, Christopher↗

Exploration of Uncertainty in Glacier Modelling

There are procedures and methods for verification of coding algebra and for validations of models and calculations that are in use in the aerospace computational fluid dynamics (CFD) community. These methods would be efficacious if used by the glacier dynamics modelling community. This paper is a presentation of some of those methods, and how they might be applied to uncertainty management supporting code verification and model validation for glacier dynamics. The similarities and differences between their use in CFD analysis and the proposed application of these methods to glacier modelling are discussed. After establishing sources of uncertainty and methods for code verification, the paper looks at a representative sampling of verification and validation efforts that are underway in the glacier modelling community, and establishes a context for these within overall solution quality assessment. Finally, an information architecture and interactive interface is introduced and advocated. This Integrated Cryospheric Exploration (ICE) Environment is proposed for exploring and managing sources of uncertainty in glacier modelling codes and methods, and for supporting scientific numerical exploration and verification. The details and functionality of this Environment are described based on modifications of a system already developed for CFD modelling and analysis.

Thompson, David E.↗

Contact pressure analysis to allow improved design for the clearing plate in a biomass comminution system

For biomass processing systems, feedstocks with small but equal sizes can improve flowability and thermochemical outputs. The Crumbler rotary shear system was designed to achieve such ideal feedstocks to reduce costs and energy throughout the biomass supply chain. Improving the wear resistance of major components in this system is important to decrease the cost. Accelerating the feedstock flow through the rotary shear machine plays an equivalent role of improving its efficiency. An analytical analysis and a finite-element analysis of the stationary clearing plate are presented in this work. Here, these tools are used to investigate the effect of contact surface curvature on the impact pressure. The optimized new clearing plate design is expected to improve the component’s lifetime and wood chips flowability. The field tests confirm that the trend of the new clearing plate design guided by FEA. The optimization method, model verification, and validation experience applied in this work can also be applied to other static components with similar simple contact wear problems.

09 BIOMASS FUELS↗

Summary of the 5th IAEA technical meeting on fusion data processing, validation and analysis (FDPVA)

The purpose of the 5th International Atomic Energy Agency technical meeting on fusion data processing, validation and analysis (FDPVA) (Ghent University, Ghent, Belgium, 12–15 June 2023) was to provide a platform during which a set of topics relevant to FDPVA were discussed with the view of meeting the needs of next step fusion devices such as ITER. The validation and analysis of experimental data obtained from diagnostics used to characterize fusion plasmas are crucial for a knowledge-based understanding of the physical processes governing the dynamics of these plasmas. This paper presents the recent progress and achievements in the domain of plasma diagnostics data analysis and synthetic diagnostics reported at the meeting, including concept description of new devices; fusion databases; integrated data analysis; inverse problems; uncertainty propagation, verification and validation; probabilistic methods and machine learning. The relevant results underline trends observed in the current major fusion confinement devices.

fusion databases↗

Flight test experience and controlled impact of a large, four-engine, remotely piloted airplane

A controlled impact demonstration (CID) program using a large, four engine, remotely piloted transport airplane was conducted. Closed loop primary flight control was performed from a ground based cockpit and digital computer in conjunction with an up/down telemetry link. Uplink commands were received aboard the airplane and transferred through uplink interface systems to a highly modified Bendix PB-20D autopilot. Both proportional and discrete commands were generated by the ground pilot. Prior to flight tests, extensive simulation was conducted during the development of ground based digital control laws. The control laws included primary control, secondary control, and racetrack and final approach guidance. Extensive ground checks were performed on all remotely piloted systems. However, manned flight tests were the primary method of verification and validation of control law concepts developed from simulation. The design, development, and flight testing of control laws and the systems required to accomplish the remotely piloted mission are discussed.

Kempel, R. W.↗

Flight test experience and controlled impact of a large, four-engine remotely piloted airplane

A controlled impact demonstration (CID) program using a large, four engine, remotely piloted transport airplane was conducted. Closed loop primary flight control was performed from a ground based cockpit and digital computer in conjunction with an up/down telemetry link. Uplink commands were received aboard the airplane and transferred through uplink interface systems to a highly modified Bendix PB-20D autopilot. Both proportional and discrete commands were generated by the ground pilot. Prior to flight tests, extensive simulation was conducted during the development of ground based digital control laws. The control laws included primary control, secondary control, and racetrack and final approach guidance. Extensive ground checks were performed on all remotely piloted systems. However, manned flight tests were the primary method of verification and validation of control law concepts developed from simulation. The design development, and flight testing of control laws and the systems required to accomplish the remotely piloted mission are discussed.

Kempel, R. W.↗

Standardized Radiation Shield Design Methods: 2005 HZETRN

Research committed by the Langley Research Center through 1995 resulting in the HZETRN code provides the current basis for shield design methods according to NASA STD-3000 (2005). With this new prominence, the database, basic numerical procedures, and algorithms are being re-examined with new methods of verification and validation being implemented to capture a well defined algorithm for engineering design processes to be used in this early development phase of the Bush initiative. This process provides the methodology to transform the 1995 HZETRN research code into the 2005 HZETRN engineering code to be available for these early design processes. In this paper, we will review the basic derivations including new corrections to the codes to insure improved numerical stability and provide benchmarks for code verification.

Wilson, John W.↗

Defining Handling Qualities of Unmanned Aerial Systems: Phase II Final Report

Unmanned Air Systems (UAS) are no longer coming, they are here, and operators from first responders to Google and Amazon are demanding access to the National Airspace System (NAS) for a wide variety of missions. This includes a proliferation of small UAS or sUAS that will operate beyond line of sight at altitudes of 500 ft and below. A myriad of issues continues to slow the development of verification, validation, and certification methods that will enable the safe introduction of UAS to the NAS. These issues include the lack of both a consensus in UAS categorization process and quantitative certification requirements, including the definition of handling qualities. Because of the wide variety of UAS types (fixed wing, rotary wing from traditional helicopters to multirotor configurations, ducted fans, airships, etc.) and vehicle size from micro vehicles to the Global Hawk with a wing span similar to that of a Boeing 737, there cannot be a one-size-fits-all set of requirements. To address these issues, Systems Technology Inc. (STI) has developed the UAS Handling Qualities Assessment (UAS-HQ) process and corresponding draft specification that will guide UAS stakeholders through a systematic evaluation process. The work described herein builds on the existing, highly successful, military rotorcraft handling qualities specifications that features a mission-oriented approach, a concept that originated at STI. The vehicle is first identified by a simple weight-based classification and then the associated vehicle mission task elements are considered. These missions have specific tasks inclusive to them that then dictate the criteria and demonstration maneuvers necessary to evaluate the UAS handling qualities. An assessment of both modeled responses and flight test data can then be conducted to examine the predicted versus actual handling qualities and, if required, design modifications can then be made. Mr. David Klyde, Vice President and Technical Director, Engineering Services, served as Principal Investigator, while Dr. Natalia Alexandrov served as the NASA LaRC technical representative. In the Phase II program, STI was joined by David Mitchell of Mitchell Aerospace Research and the University of Minnesota. Mr. Mitchell led the draft specification development effort, while the University of Minnesota UAV Lab conducted sUAS flight tests under the direction of Dr. Peter Seiler.

Klyde, David H.↗

L(sub 1) Adaptive Flight Control System: Flight Evaluation and Technology Transition

Certification of adaptive control technologies for both manned and unmanned aircraft represent a major challenge for current Verification and Validation techniques. A (missing) key step towards flight certification of adaptive flight control systems is the definition and development of analysis tools and methods to support Verification and Validation for nonlinear systems, similar to the procedures currently used for linear systems. In this paper, we describe and demonstrate the advantages of L(sub l) adaptive control architectures for closing some of the gaps in certification of adaptive flight control systems, which may facilitate the transition of adaptive control into military and commercial aerospace applications. As illustrative examples, we present the results of a piloted simulation evaluation on the NASA AirSTAR flight test vehicle, and results of an extensive flight test program conducted by the Naval Postgraduate School to demonstrate the advantages of L(sub l) adaptive control as a verifiable robust adaptive flight control system.

Xargay, Enric↗

Verification, Validation, and Solution Quality in Computational Physics: CFD Methods Applied to Ice Sheet Physics

Procedures and methods for veri.cation of coding algebra and for validations of models and calculations used in the aerospace computational fluid dynamics (CFD) community would be ef.cacious if used by the glacier dynamics modeling community. This paper presents some of those methods, and how they might be applied to uncertainty management supporting code veri.cation and model validation for glacier dynamics. The similarities and differences between their use in CFD analysis and the proposed application of these methods to glacier modeling are discussed. After establishing sources of uncertainty and methods for code veri.cation, the paper looks at a representative sampling of veri.cation and validation efforts that are underway in the glacier modeling community, and establishes a context for these within an overall solution quality assessment. Finally, a vision of a new information architecture and interactive scienti.c interface is introduced and advocated.

Thompson, David E.↗

Translating expert system rules into Ada code with validation and verification

The purpose of this ongoing research and development program is to develop software tools which enable the rapid development, upgrading, and maintenance of embedded real-time artificial intelligence systems. The goals of this phase of the research were to investigate the feasibility of developing software tools which automatically translate expert system rules into Ada code and develop methods for performing validation and verification testing of the resultant expert system. A prototype system was demonstrated which automatically translated rules from an Air Force expert system was demonstrated which detected errors in the execution of the resultant system. The method and prototype tools for converting AI representations into Ada code by converting the rules into Ada code modules and then linking them with an Activation Framework based run-time environment to form an executable load module are discussed. This method is based upon the use of Evidence Flow Graphs which are a data flow representation for intelligent systems. The development of prototype test generation and evaluation software which was used to test the resultant code is discussed. This testing was performed automatically using Monte-Carlo techniques based upon a constraint based description of the required performance for the system.

Becker, Lee↗

Software Verification Toolkit (SVT): Survey on Available Software Verification Tools and Future Direction

Writing software is difficult. However, writing complex, well tested and designed, and functionally correct software is incredibly difficult. An entire field of study is devoted to the validation and verification of software to address this problem, and in this paper we analyze the landscape of currently available third party software. We have divided our analyses into three separate subsections with regards to software validation: formal methods, static analysis, and test generation. Formal verification is the most complex method in which to validate software correctness, but also the most thorough as it truly validates the mathematical validity of the source code. Static analysis generally is relegated to abstract syntax tree traversal techniques to find errors related to faulty software such as memory leaks or stack overflow issues. Automatic test generation is similar in implementation to static analysis, but pushes a bit further in verifying the boundedness of function inputs and outputs with regards to annotated or parsed criteria. The crux of this report is to analyze and describe the software tools that implement these techniques to validate and verify software. Pros and cons related to installation, utilization, and capabilities of the frameworks are described, and reproducible examples are provided with a focus on usability. The initial survey concluded that the most interesting tools of note are Z3, Isabelle/HOL, and TLA+ with regards to formal verification; and Infer, Frama-C, and SonarQube with regards to static analysis. With these tools in mind, a final conjecture is provided that describes future avenues of utilizing these tools for developing a verification framework to assist in validating existing software at Sandia National Laboratories.

97 MATHEMATICS AND COMPUTING↗

Formal Methods of V&V of Partial Specifications: An Experience Report

This paper describes our work exploring the suitability of formal specification methods for independent verification and validation (IV&V) of software specifications for large, safety critical systems. An IV&V contractor often has to perform rapid analysis on incomplete specifications, with no control over how those specifications are represented. Lightweight formal methods show significant promise in this context, as they offer a way of uncovering major errors, without the burden of full proofs of correctness. We describe an experiment in the application of the method SCR. to testing for consistency properties of a partial model of requirements for Fault Detection Isolation and Recovery on the space station. We conclude that the insights gained from formalizing a specification is valuable, and it is the process of formalization, rather than the end product that is important. It was only necessary to build enough of the formal model to test the properties in which we were interested. Maintenance of fidelity between multiple representations of the same requirements (as they evolve) is still a problem, and deserves further study.

Easterbrook, Steve↗

Verification of NASA Emergent Systems

NASA is studying advanced technologies for a future robotic exploration mission to the asteroid belt. This mission, the prospective ANTS (Autonomous Nano Technology Swarm) mission, will comprise of 1,000 autonomous robotic agents designed to cooperate in asteroid exploration. The emergent properties of swarm type missions make them powerful, but at the same time are more difficult to design and assure that the proper behaviors will emerge. We are currently investigating formal methods and techniques for verification and validation of future swarm-based missions. The advantage of using formal methods is their ability to mathematically assure the behavior of a swarm, emergent or otherwise. The ANT mission is being used as an example and case study for swarm-based missions for which to experiment and test current formal methods with intelligent swam. Using the ANTS mission, we have evaluated multiple formal methods to determine their effectiveness in modeling and assuring swarm behavior.

Rouff, Christopher↗

NASA Space Observatory Precision Pointing Benchmark Problem Development

An interagency workshop in May 2021 on guidance, navigation, and control (GNC) verification and validation (V&V) methods and techniques identified needs for the formulation and public release of relevant V&V benchmark problems as a practical way forward for the discipline. A decision was made by the NASA Technical Fellow for GNC and the NASA Engineering and Safety Center (NESC) GNC Technical Discipline Team (TDT) to produce a benchmark problem focusing on the problem of space observatory precision pointing. This report contains the results of the NESC assessment.

NASA Engineering and Safety Center↗

Development of Advanced Verification and Validation Procedures and Tools for the Certification of Learning Systems in Aerospace Applications

Adaptive control technologies that incorporate learning algorithms have been proposed to enable automatic flight control and vehicle recovery, autonomous flight, and to maintain vehicle performance in the face of unknown, changing, or poorly defined operating environments. In order for adaptive control systems to be used in safety-critical aerospace applications, they must be proven to be highly safe and reliable. Rigorous methods for adaptive software verification and validation must be developed to ensure that control system software failures will not occur. Of central importance in this regard is the need to establish reliable methods that guarantee convergent learning, rapid convergence (learning) rate, and algorithm stability. This paper presents the major problems of adaptive control systems that use learning to improve performance. The paper then presents the major procedures and tools presently developed or currently being developed to enable the verification, validation, and ultimate certification of these adaptive control systems. These technologies include the application of automated program analysis methods, techniques to improve the learning process, analytical methods to verify stability, methods to automatically synthesize code, simulation and test methods, and tools to provide on-line software assurance.

Jacklin, Stephen↗

Progress Towards a Cartesian Cut-Cell Method for Viscous Compressible Flow

The proposed paper reports advances in developing a method for high Reynolds number compressible viscous flow simulations using a Cartesian cut-cell method with embedded boundaries. This preliminary work focuses on accuracy of the discretization near solid wall boundaries. A model problem is used to investigate the accuracy of various difference stencils for second derivatives and to guide development of the discretization of the viscous terms in the Navier-Stokes equations. Near walls, quadratic reconstruction in the wall-normal direction is used to mitigate mesh irregularity and yields smooth skin friction distributions along the body. Multigrid performance is demonstrated using second-order coarse grid operators combined with second-order restriction and prolongation operators. Preliminary verification and validation for the method is demonstrated using flat-plate and airfoil examples at compressible Mach numbers. Simulations of flow on laminar and turbulent flat plates show skin friction and velocity profiles compared with those from boundary-layer theory. Airfoil simulations are performed at laminar and turbulent Reynolds numbers with results compared to both other simulations and experimental data

Berger, Marsha↗