SEARCH · Search NASA

Results for “Verification and Validation methods”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

All sources NASA NTRS DOE OSTI

NASA NTRS2005

IEEE/NASA Workshop on Leveraging Applications of Formal Methods, Verification, and Validation

This volume contains the Preliminary Proceedings of the 2005 IEEE ISoLA Workshop on Leveraging Applications of Formal Methods, Verification, and Validation, with a special track on the theme of Formal Methods in Human and Robotic Space Exploration. The workshop was held on 23-24 September 2005 at the Loyola College Graduate Center, Columbia, MD, USA. The idea behind the Workshop arose from the experience and feedback of ISoLA 2004, the 1st International Symposium on Leveraging Applications of Formal Methods held in Paphos (Cyprus) last October-November. ISoLA 2004 served the need of providing a forum for developers, users, and researchers to discuss issues related to the adoption and use of rigorous tools and methods for the specification, analysis, verification, certification, construction, test, and maintenance of systems from the point of view of their different application domains.

Margaria, Tiziana↗

NASA NTRS1997

Formal Methods for Verification and Validation of Partial Specifications: A Case Study

This paper describes our work exploring the suitability of formal specification methods for independent verification and validation (IV&V) of software specifications for large, safety critical systems. An IV&V contractor often has to perform rapid analysis on incomplete specifications, with no control over how those specifications are represented. Lightweight formal methods show significant promise in this context, as they offer a way of uncovering major errors, without the burden of full proofs of correctness. We describe a case study of the use of partial formal models for V&V of the requirements for Fault Detection Isolation and Recovery on the space station. We conclude that the insights gained from formalizing a specification are valuable, and it is the process of formalization, rather than the end product that is important. It was only necessary to build enough of the formal model to test the properties in which we were interested. Maintenance of fidelity between multiple representations of the same requirements (as they evolve) is still a problem, and deserves further study.

Easterbrook, Steve↗

NASA NTRS1989

Evidence flow graph methods for validation and verification of expert systems

The results of an investigation into the use of evidence flow graph techniques for performing validation and verification of expert systems are given. A translator to convert horn-clause rule bases into evidence flow graphs, a simulation program, and methods of analysis were developed. These tools were then applied to a simple rule base which contained errors. It was found that the method was capable of identifying a variety of problems, for example that the order of presentation of input data or small changes in critical parameters could affect the output from a set of rules.

Becker, Lee A.↗

NASA NTRS1988

Evidence flow graph methods for validation and verification of expert systems

This final report describes the results of an investigation into the use of evidence flow graph techniques for performing validation and verification of expert systems. This was approached by developing a translator to convert horn-clause rule bases into evidence flow graphs, a simulation program, and methods of analysis. These tools were then applied to a simple rule base which contained errors. It was found that the method was capable of identifying a variety of problems, for example that the order of presentation of input data or small changes in critical parameters could effect the output from a set of rules.

Becker, Lee A.↗

NASA NTRS2017

Fuzzy Logic Controller Stability Analysis Using a Satisfiability Modulo Theories Approach

While many widely accepted methods and techniques exist for validation and verification of traditional controllers, at this time no solutions have been accepted for Fuzzy Logic Controllers (FLCs). Due to the highly nonlinear nature of such systems, and the fact that developing a valid FLC does not require a mathematical model of the system, it is quite difficult to use conventional techniques to prove controller stability. Since safety-critical systems must be tested and verified to work as expected for all possible circumstances, the fact that FLC controllers cannot be tested to achieve such requirements poses limitations on the applications for such technology. Therefore, alternative methods for verification and validation of FLCs needs to be explored. In this study, a novel approach using formal verification methods to ensure the stability of a FLC is proposed. Main research challenges include specification of requirements for a complex system, conversion of a traditional FLC to a piecewise polynomial representation, and using a formal verification tool in a nonlinear solution space. Using the proposed architecture, the Fuzzy Logic Controller was found to always generate negative feedback, but inconclusive for Lyapunov stability.

Fuzzy Logic Controller↗

NASA NTRS2017

Fuzzy Logic Controller Stability Analysis Using a Satisfiability Modulo Theories Approach

Fuzzy Logic Controller↗

NASA NTRS2009

Baseline Assessment and Prioritization Framework for IVHM Integrity Assurance Enabling Capabilities

Fundamental to vehicle health management is the deployment of systems incorporating advanced technologies for predicting and detecting anomalous conditions in highly complex and integrated environments. Integrated structural integrity health monitoring, statistical algorithms for detection, estimation, prediction, and fusion, and diagnosis supporting adaptive control are examples of advanced technologies that present considerable verification and validation challenges. These systems necessitate interactions between physical and software-based systems that are highly networked with sensing and actuation subsystems, and incorporate technologies that are, in many respects, different from those employed in civil aviation today. A formidable barrier to deploying these advanced technologies in civil aviation is the lack of enabling verification and validation tools, methods, and technologies. The development of new verification and validation capabilities will not only enable the fielding of advanced vehicle health management systems, but will also provide new assurance capabilities for verification and validation of current generation aviation software which has been implicated in anomalous in-flight behavior. This paper describes the research focused on enabling capabilities for verification and validation underway within NASA s Integrated Vehicle Health Management project, discusses the state of the art of these capabilities, and includes a framework for prioritizing activities.

Cooper, Eric G.↗

NASA NTRS2019

Benchmarking Mixed Mode Failure in Progressive Damage and Failure Analysis Methods

The verification and validation of progressive-damage-analysis finite element methods are difficult but critical tasks to undertake during their development. Verification exercises assess whether a predictive analysis tool produces results that are consistent with the fundamental concepts and assumptions of the tool under evaluation. Ideally, closed-form analytical solutions can be derived for which method verification results can be compared. Problems selected for computational tool verification are often simple and isolate individual features of the tool. In the case of progressive damage finite element methods, verifications should be performed to evaluate the ability of the model to predict the initiation of damage and its growth through the finite element mesh under a variety of conditions. Mabson et al. proposed a test case of a unidirectional, fiber-reinforced plate with a center crack subjected to tensile loads to evaluate matrix crack propagation predictions. The problem was modeled using the Abaqus Hashin continuum damage mechanics (CDM) model for fiber-reinforced composites. Different combinations of matrix strengths and element sizes were used in the simulations, and the results were compared to a closed-form solution based on linear elastic fracture mechanics (LEFM). It was determined that the Abaqus CDM model could predict the LEFM solution of Mode I cracks only when the finite element mesh density met specific requirements based on the material properties. This paper presents closed-form LEFM solutions for a center notch mixed mode (CNMM) verification problem. Parametric finite element analyses were developed using progressive damage analysis methods of both the Discrete Damage Mechanics (DDM) and CDM classes. The progressive damage analysis methods applied in the analyses of the CNMM problem include CompDam and the Floating Node Method. Analyses were conducted with various mode mixities and element sizes to verify that the damage models were working as intended and to identify any limits of applicability.

Leone, Frank↗

NASA NTRSDate not supplied

Optimizing Urban Air Mobility Research Through Bi-Directional Integration Processes for Effective Verification and Validation

Urban Air Mobility (UAM), a subset of Advanced Air Mobility (AAM), aims to revolutionize metropolitan transportation. This paper explores an engineering process framework within National Aeronautics and Space Administration (NASA)’s Air Mobility Pathfinders (AMP) project, which is focused on the safe scaling and seamless integration of UAM operations within the National Airspace System (NAS). Progressing through three phases — Initial, Midterm, and Mature — the Federal Aviation Administration (FAA)’s UAM Concept of Operations (ConOps) describes a proposed path for the evolution of UAM operations, presenting multifaceted challenges in technology, regulation, and stakeholder engagement. With this ConOps as a basis, the AMP project is executing research, development, test, and evaluation activities aimed at delivering validated reference architectures for the Midterm phase. This paper illuminates the pivotal role of the systems engineering process in managing these technical efforts. Robust verification and validation methods, along with systematic bi-directional integration process, enhance the ability to conduct research into the design and evolution of safe, efficient, and reliable UAM systems. In this paper a systematic framework will be presented and shown to facilitate interactions between UAM stakeholders and to effectively manage the complex architecture of UAM operations in the NAS.

National Airspace System↗

NASA NTRS2009

Antenna reconfiguration verification and validation

A method of testing the electrical functionality of an optically controlled switch in a reconfigurable antenna is provided. The method includes configuring one or more conductive paths between one or more feed points and one or more test point with switches in the reconfigurable antenna. Applying one or more test signals to the one or more feed points. Monitoring the one or more test points in response to the one or more test signals and determining the functionality of the switch based upon the monitoring of the one or more test points.

Becker, Robert C.↗

NASA NTRS2008

Validation of High-Fidelity CFD Simulations for Rocket Injector Design

Computational fluid dynamics (CFD) has the potential to improve the historical rocket injector design process by evaluating the sensitivity of performance and injector-driven thermal environments to the details of the injector geometry and key operational parameters. Methodical verification and validation efforts on a range of coaxial injector elements have shown the current production CFD capability must be improved in order to quantitatively impact the injector design process. This paper documents the status of a focused effort to compare and understand the predictive capabilities and computational requirements of a range of CFD methodologies on a set of single element injector model problems. The steady Reynolds-Average Navier-Stokes (RANS), unsteady Reynolds-Average Navier-Stokes (URANS) and three different approaches using the Large Eddy Simulation (LES) technique were used to simulate the initial model problem, a single element coaxial injector using gaseous oxygen and gaseous hydrogen propellants. While one high-fidelity LES result matches the experimental combustion chamber wall heat flux very well, there is no monotonic convergence to the data with increasing computational tool fidelity. Systematic evaluation of key flow field regions such as the flame zone, the head end recirculation zone and the downstream near wall zone has shed significant, though as of yet incomplete, light on the complex, underlying causes for the performance level of each technique. 1 Aerospace Engineer and Combustion CFD Team Leader, MS ER42, NASA MSFC, AL 35812, Senior Member, AIAA. 2 Professor and Director, Computational Combustion Laboratory, School of Aerospace Engineering, 270 Ferst Dr., Atlanta, GA 30332, Associate Fellow, AIAA. 3 Reilly Professor of Engineering, School of Mechanical Engineering, 585 Purdue Mall, West Lafayette, IN 47907, Fellow, AIAA. 4 Principal Member of Technical Staff, Combustion Research Facility, 7011 East Avenue, MS9051, Livermore, CA 94550, Associate Fellow, AIAA. 5 J. L. and G. H. McCain Endowed Chair, Mechanical Engineering, 104 Research Building East, University Park, PA 16802, Fellow, AIAA. American Institute of Aeronautics and Astronautics 1

Tucker, P. Kevin↗

NASA NTRS2021

Programming - What is Next?

The paper provides an introduction to the track: “Program- ming - What is Next?”, organized by the authors as part of ISoLA 2021: the 9th International Symposium On Leveraging Applications of Formal Methods, Verification and Validation. A total of 14 papers were pre- sented in the track, with responses to the question: what are the trends in current more recent programming languages, and what can be expected of future languages?. The track covers such topics as general-purpose programming languages, domain-specific languages, formal methods and modeling languages, textual versus graphical languages, and application programming versus embedded programming.

Steffen, Bernhard↗

NASA NTRS1982

Climate studies from satellite observations - Special problems in the verification of earth radiation balance, cloud climatology, and related climate experiments

A body of techniques that have been developed and planned for use during the Earth Radiation Budget Experiment (ERBE), the International Satellite Cloud Climatology Project (ISCCP), and related climate experiments of the 1980's are reviewed. Validation and verification methods must apply for systems of satellites. They include: (1) use of a normalization or intercalibration satellite, (2) special intensive observation areas located over ground-truth sites, and (3) monitoring of sun and earth by several satellites and/or several instruments at the same time. Since each climate application area has a hierarchy of user communities, validation techniques vary from very detailed methods to those that simply assure high relative accuracy in detecting space and time variations for climate studies. It is shown that climate experiments generally require more emphasis on long-term stability and internal consistency of satellite data sets than high absolute accuracy.

Vonder Haar, T. H.↗

NASA NTRS1989

Real-time application of knowledge-based systems

The Rapid Prototyping Facility (RPF) was developed to meet a need for a facility which allows flight systems concepts to be prototyped in a manner which allows for real-time flight test experience with a prototype system. This need was focused during the development and demonstration of the expert system flight status monitor (ESFSM). The ESFSM was a prototype system developed on a LISP machine, but lack of a method for progressive testing and problem identification led to an impractical system. The RPF concept was developed, and the ATMS designed to exercise its capabilities. The ATMS Phase 1 demonstration provided a practical vehicle for testing the RPF, as well as a useful tool. ATMS Phase 2 development continues. A dedicated F-18 is expected to be assigned for facility use in late 1988, with RAV modifications. A knowledge-based autopilot is being developed using the RPF. This is a system which provides elementary autopilot functions and is intended as a vehicle for testing expert system verification and validation methods. An expert system propulsion monitor is being prototyped. This system provides real-time assistance to an engineer monitoring a propulsion system during a flight.

Brumbaugh, Randal W.↗

NASA NTRS2000

The NPARC Alliance Verification and Validation Archive

The NPARC Alliance (National Project for Applications oriented Research in CFD) maintains a publicly-available, web-based verification and validation archive as part of the development and support of the WIND CFD code. The verification and validation methods used for the cases attempt to follow the policies and guidelines of the ASME and AIAA. The emphasis is on air-breathing propulsion flow fields with Mach numbers ranging from low-subsonic to hypersonic.

Slater, John W.↗

NASA NTRS2007

An Approach to Improved Credibility of CFD Simulations for Rocket Injector Design

Computational fluid dynamics (CFD) has the potential to improve the historical rocket injector design process by simulating the sensitivity of performance and injector-driven thermal environments to. the details of the injector geometry and key operational parameters. Methodical verification and validation efforts on a range of coaxial injector elements have shown the current production CFD capability must be improved in order to quantitatively impact the injector design process.. This paper documents the status of an effort to understand and compare the predictive capabilities and resource requirements of a range of CFD methodologies on a set of model problem injectors. Preliminary results from a steady Reynolds-Average Navier-Stokes (RANS), an unsteady Reynolds-Average Navier Stokes (URANS) and three different Large Eddy Simulation (LES) techniques used to model a single element coaxial injector using gaseous oxygen and gaseous hydrogen propellants are presented. Initial observations are made comparing instantaneous results, corresponding time-averaged and steady-state solutions in the near -injector flow field. Significant differences in the flow fields exist, as expected, and are discussed. An important preliminary result is the identification of a fundamental mixing mechanism, accounted for by URANS and LES, but missing in the steady BANS methodology. Since propellant mixing is the core injector function, this mixing process may prove to have a profound effect on the ability to more correctly simulate injector performance and resulting thermal environments. Issues important to unifying the basis for future comparison such as solution initialization, required run time and grid resolution are addressed.

Tucker, Paul K.↗

NASA NTRSDate not supplied

Runtime Monitoring for Unmanned Aerospace Systems with Neural Network Components

AI components (e.g., Deep Neural Networks) are increasingly used in unmanned Aerospace systems for safety-relevant applications. Rigorous Verification and Validation methods for such components are still in their infancy and thus, monitoring of the AI's behavior during runtime is essential. In this paper, we will present a runtime-monitoring architecture, which combines the advanced statistical analysis framework SYSAI (System Analysis using Statistical AI) with temporal and probabilistic runtime monitoring carried out by R2U2 (Realizable, Responsive, and Unobtrusive Unit). Learned statistical models of complex systems with AI components are produced by the SYSAI framework and provide detailed information to enable the R2U2 runtime monitor to efficiently perform advanced safety and performance checks in nominal and off-nominal conditions. We will present initial results of our tool set and architecture on a case study, a DNN-based autonomous centerline tracking system (ACT).

Yuning He↗

NASA NTRSDate not supplied

Discovery and Analysis of Rare High-Impact Failure Modes using Adversarial RL-Informed Sampling

Adaptive learning agents have tremendous potential to handle critical tasks currently performed by humans. Unfortunately, due to their complexity, it can be difficult to verify that these learning agents do not have critical failure modes. Standard verification and validation methods often do not apply directly to learning agents and Monte Carlo methods have difficulty covering even a small fraction of the state space, especially in multiagent systems or over long time horizons. To overcome this difficulty, we demonstrate an adaptive stress-testing method based on reinforcement learning of correlations that raise the probability of failure. This approach has three key properties: (1) it is able to find rare failure modes with far greater sample efficiency than Monte Carlo methods, (2) it can estimate the true probability of a failure mode despite the inherent bias in the learning method, and (3) it is capable of learning and resampling compact representations of multimodal failure spaces. These properties are important in practice as we need to find disparate failure modes while accounting for their actual relevance. This is a significant advantage over traditional adaptive stress testing methods that give abstract likelihoods of particular failure instances, but cannot estimate the probability of a broader failure mode. We test our algorithm on a simple problem from the aviation domain where an autonomous aircraft lands in gusty wind conditions. The results suggest that we can find failure modes with far fewer samples than the Monte Carlo approach and simultaneously estimate the probability of failure.

reinforcement learning↗