SEARCH · Search NASA

Results for “VERIFICATION”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 541 records · Page 30

All sources NASA NTRS DOE OSTI

NASA NTRS2017

Land surface Verification Toolkit (LVT)

LVT is a framework developed to provide an automated, consolidated environment for systematic land surface model evaluation Includes support for a range of in-situ, remote-sensing and other model and reanalysis products. Supports the analysis of outputs from various LIS subsystems, including LIS-DA, LIS-OPT, LIS-UE. Note: The Land Information System Verification Toolkit (LVT) is a NASA software tool designed to enable the evaluation, analysis and comparison of outputs generated by the Land Information System (LIS). The LVT software is released under the terms and conditions of the NASA Open Source Agreement (NOSA) Version 1.1 or later. Land Information System Verification Toolkit (LVT) NOSA.

surface model↗

NASA NTRS2017

Challenges in the Verification of Reinforcement Learning Algorithms

Machine learning (ML) is increasingly being applied to a wide array of domains from search engines to autonomous vehicles. These algorithms, however, are notoriously complex and hard to verify. This work looks at the assumptions underlying machine learning algorithms as well as some of the challenges in trying to verify ML algorithms. Furthermore, we focus on the specific challenges of verifying reinforcement learning algorithms. These are highlighted using a specific example. Ultimately, we do not offer a solution to the complex problem of ML verification, but point out possible approaches for verification and interesting research opportunities.

Van Wesel, Perry↗

NASA NTRS2017

Verification and Validation Process for Progressive Damage and Failure Analysis Methods in the NASA Advanced Composites Consortium

The Advanced Composites Consortium is a US Government/Industry partnership supporting technologies to enable timeline and cost reduction in the development of certified composite aerospace structures. A key component of the consortium's approach is the development and validation of improved progressive damage and failure analysis methods for composite structures. These methods will enable increased use of simulations in design trade studies and detailed design development, and thereby enable more targeted physical test programs to validate designs. To accomplish this goal with confidence, a rigorous verification and validation process was developed. The process was used to evaluate analysis methods and associated implementation requirements to ensure calculation accuracy and to gage predictability for composite failure modes of interest. This paper introduces the verification and validation process developed by the consortium during the Phase I effort of the Advanced Composites Project. Specific structural failure modes of interest are first identified, and a subset of standard composite test articles are proposed to interrogate a progressive damage analysis method's ability to predict each failure mode of interest. Test articles are designed to capture the underlying composite material constitutive response as well as the interaction of failure modes representing typical failure patterns observed in aerospace structures.

Wanthal, Steven↗

NASA NTRS2017

Towards Verification and Validation for Increased Autonomy

This presentation goes over the work we have performed over the last few years on verification and validation of the next generation onboard collision avoidance system, ACAS X, for commercial aircraft. It describes our work on probabilistic verification and synthesis of the model that ACAS X is based on, and goes on to the validation of that model with respect to actual simulation and flight data. The presentation then moves on to identify the characteristics of ACAS X that are related to autonomy and to discuss the challenges that autonomy pauses on VV. All work presented has already been published.

Giannakopoulou, Dimitra↗

NASA NTRS2018

Methodologies for Verification and Validation of Space Launch System (SLS) Structural Dynamic Models

Responses to challenges associated with verification and validation (V&V) of Space Launch System (SLS) structural dynamics models are presented in this paper. Four methodologies addressing specific requirements for V&V are discussed. (1) Residual Mode Augmentation (RMA), which has gained acceptance by various principals in the NASA community, defines efficient and accurate FEM modal sensitivity models that are useful in test-analysis correlation and reconciliation and parametric uncertainty studies. (2) Modified Guyan Reduction (MGR) and Harmonic Reduction (HR, introduced in 1976), developed to remedy difficulties encountered with the widely used Classical Guyan Reduction (CGR) method, are presented. MGR and HR are particularly relevant for estimation of "body dominant" target modes of shell-type SLS assemblies that have numerous "body", "breathing" and local component constituents. Realities associated with configuration features and "imperfections" cause "body" and "breathing" mode characteristics to mix resulting in a lack of clarity in the understanding and correlation of FEM- and test-derived modal data. (3) Mode Consolidation (MC) is a newly introduced procedure designed to effectively "de-feature" FEM and experimental modes of detailed structural shell assemblies for unambiguous estimation of "body" dominant target modes. Finally, (4) Experimental Mode Verification (EMV) is a procedure that addresses ambiguities associated with experimental modal analysis of complex structural systems. Specifically, EMV directly separates well-defined modal data from spurious and poorly excited modal data employing newly introduced graphical and coherence metrics.

Coppolino, Robert N.↗

NASA NTRS2018

Methodologies for Verification and Validation of Space Launch System (SLS) Structural Dynamic Models: Appendices

Verification and validation (V&V) is a highly challenging undertaking for SLS structural dynamics models due to the magnitude and complexity of SLS subassemblies and subassemblies. Responses to challenges associated with V&V of Space Launch System (SLS) structural dynamics models are presented in Volume I of this paper. Four methodologies addressing specific requirements for V&V are discussed. (1) Residual Mode Augmentation (RMA). (2) Modified Guyan Reduction (MGR) and Harmonic Reduction (HR, introduced in 1976). (3) Mode Consolidation (MC). Finally, (4) Experimental Mode Verification (EMV). This document contains the appendices to Volume I.

Coppolino, Robert N.↗

NASA NTRS2018

Validation & Verification of Electrical Components B2 Test Facility

The research focus of this project is to assist in the closure of Measurement, Monitoring and Control System (MMCS) and other electrical requirements in support of the B2 Space Launch System (SLS) Core Stage Green Run Test Project. Alongside this goal I am to understand project management tools to analyze and control activities/tasks associated with those items. The project I am working on will assist electrical engineers in developing and or identifying closure rationale for the MMCS and electrical requirements. Also as serving the role of a training project manager, I am to develop and maintain a method for tracking progress with estimate completion dates specifically identifying those items during this performance window. The methods I used in order to perform the research consisted of worddocuments, excel sheets, and pdf documents needed for review. For example, documents consisted of: a SLS Core Stage Green Run Facility Requirement Document (FRD) identifying the SSC MMCS and electrical requirements, a Stage Controller to SSC Interface Control Document (ICD) identifying the SSC requirements, and the B2 SLS Core Stage Green Run Test project System Requirements Document (SRD) which identifies the MMCS and electrical requirements. The files were sent to me by both mentors, Mr. Barry Robinson and Ms. Dawn Davis. Overall, the review process for the word documents and excel spreadsheets proved to be successful. The data for the electrical components were accurate and were consistent with the original recorded data. However, the consistency with the pdf documents did not follow up all the way. Attention to this mistake was made and further revisions were done in order for the data to agree with each other. Furthermore, an itinerary was designed using Microsoft Outlook in order to track progress with estimated completion dates. This project contributes to NASA/Center Missions and Goals through the Waterfall Model. The Waterfall Model is a linear system used for engineering design. In the Waterfall Model, the fourth step is Verification and Validation. This involves installation, testing, and debugging of the B2 SLS Core Stage electrical components. For the SLS Core Stage Green Run Test Project, each requirement contained in this approved requirements set will have at least one Verification Item (VI) assigned to it. During the design phase, VIs will mostly consist of Analysis or Inspection types. Design phase VIs and closure information will be documented in Dynamics Data Management System (DDMS) in the form of analysis reports and design documentation.

Bastian, Tyler↗

NASA NTRS2019

Verification Benchmarks and Validation of MAT261 for High Energy Dynamic Impact Applications

Advanced computational methods for composite structures can enable smarter testing and potentially certification by analysis. These methods, known as progressive damage and failure analysis (PDFA) methods, can predict lamina level mechanics including failure and subsequent propagation. When introduced into a commercial off the shelf finite element solver such as ABAQUS, LS-DYNA, or ANSYS, it enables end users to effectively exercise PDFA methods; however, it requires an understanding of the assumed mechanics and underlying coded response. In this paper, verification benchmarks developed under the NASA Advanced Composite Consortium for a class of PDFA methods known as continuum damage mechanics approaches are applied to the native LS-DYNA material model MAT261. As a result of the verification, an instability was discovered and subsequently corrected with an engineering solution. The results of the preliminary MAT261 simulations for high energy dynamic impact and the engineering solution are compared and a discussion about the significance is presented.

Molitor, Matthew↗

NASA NTRS2021

Constraining the Neutron Star Mass–Radius Relation and Dense Matter Equation of State with NICER. III. Model Description and Verification of Parameter Estimation Codes

We describe the X-ray pulse profile models we use and how we use them to analyze Neutron Star Interior Composition Explorer(NICER)observations of rotation-powered millisecond pulsars to obtain information about the mass–radius relation of neutron stars and the equation of state of the dense matter in their cores. Here we detail our modeling of the observed profile of PSR J0030+0451 that we analyzed in Miller et al. and Riley et al. and describe a cross-verification of computations of the pulse profiles of a star with R/M 3, in case stars this compact need to be considered in future analyses. We also present our early cross-verification efforts of the parameter estimation procedures used by Miller et al. and Riley et al. by analyzing two distinct synthetic data sets. Both codes yielded credible regions in the mass–radius plane that are statistically consistent with one another, and both gave posterior distributions for model parameter values consistent with the values that were used to generate the data. We also summarize the additional tests of the parameter estimation procedure of Miller et al. that used synthetic pulse profiles and the NICER pulse profile of PSR J0030+0451. We then illustrate how the precision of mass and radius estimates depends on the pulsar’s spin rate and the size of its hot spot by analyzing four different synthetic pulse profiles. Finally, we assess possible sources of systematic error in the estimates made using this technique, some of which may warrant further investigation.

Slavko Bogdanov↗

NASA NTRS2020

First-Order Runtime Verification using BDDs

Runtime Verification (RV) expedites the analyses of execution traces for detecting system errors and for statistical and quality analysis. Having started modestly, with checking temporal properties that are based on propositional (yes/no) values, the current practice of RV often involves properties that are parametrized by the data observed in the input trace. The specifications are based on various formalisms, such as automata, temporal logics, rule systems, and stream processing. Checking execution traces that are data intensive against a specification that imposes strong dependencies between the data, poses a nontrivial challenges; in particular if runtime verification has to be performed online, while many events that carry data appear within small time proximities. Towards achieving this goal, it was recently suggested to represent relations over the observed data values, based on BDDs, where data elements are enumerated and then converted into bit vectors. This representation provided a very simple and natural extension of an RV algorithm from propositional to first-order LTL, but more importantly, was shown to contribute to the memory compactness and to the speed, as was demonstrated using a corresponding implementation. We extend here the capabilities of BDD-based RV with the ability to express timing constraints, where the monitored events include (integer) clock values. We show how to efficiently operate on BDDs that represent both relations on (enumerations of) values and time dependencies, as required by the addition of the time constraints. We demonstrate our algorithm with an efficient implementation and provide experimental results.

Peled, Doron↗

NASA NTRSDate not supplied

Toward Verification of the γ-Reθt Transition Model in OVERFLOW and FUN3D

The results of an ongoing assessment of the transition modeling capability in NASA's OVERFLOW and FUN3D programs are presented, with a focus on the Langtry-Menter γ-Reθt transition model in combination with the shear-stress transport (SST) turbulence model. While the effect of numerics and boundary conditions on the accuracy and iterative convergence of RANS solutions for fully turbulent flows has been well documented, especially in the context of canonical flow configurations, the same cannot be said for transport-equation-based transition models coupled with RANS-based turbulence models. Given the criticality of transition modeling for new aircraft design and optimization, there has been a renewed focus on the accuracy of such transition models and their inconsistent implementation across different flow solvers as seen from the AIAA and NATO-AVT workshops. In this work, we aim to establish the verification of such models via these two well established CFD codes with different numerics. The goal is to produce high-quality data, such as grids, solutions, and other auxiliary data, that may be utilized for code verification by others in the CFD community. As a first step, the work reported here is focused on the Langtry-Menter transition model as applied to two simple 2D configurations, namely the flat plate, and the NLF-0416, respectively. The paper also highlights how the boundary conditions and baseline turbulence model can affect the solutions from the SST-based Langtry-Menter transition model. A preliminary evaluation of the automatic mesh adaption capabilities of these solvers and potential benefits for flow configurations involving a mix of laminar, transitional, and fully turbulent flows is also reported.

CFD Modeling↗

NASA NTRS2023

From Natural Language Requirements to the Verification of Programmable Logic Controllers: Integrating FRET into PLCverif

PLCverif is an actively developed project at CERN, enabling the formal verification of Programmable Logic Controller (PLC) programs in critical systems. In this paper, we present our work on improving the formal requirements specification experience in PLCverif through the use of natural language. To this end, we integrate NASA’s FRET, a formal requirement elicitation and authoring tool, into PLCverif. FRET is used to specify formal requirements in structured natural language, which automatically translates into temporal logic formulae. FRET’s output is then directly used by PLCverif for verification purposes. We discuss practical challenges that PLCverif users face when authoring requirements and the FRET features that help alleviate these problems. We present the new requirement formalization workflow and report our experience using it on two critical CERN case studies.

FRET↗

NASA NTRS2023

Autonomy Verification & Validation Roadmap and Vision 2045

Advanced capabilities planned for the next generation of autonomous and increasingly autonomous air vehicles will include non-traditional components based on artificial intelligence, machine learning, and complex optimization and planning algorithms. These complex components will be used to provide enhanced safety and high-level decision-making functions. However, there are serious barriers to the deployment of autonomous aircraft in the National Airspace System (NAS). Current civil aviation certification processes are based on the concept that the correct behavior of a system or a component must be completely specified and verified prior to operation. This report from the Autonomy Verification and Validation (V&V) Roadmap and Vision 2045 project presents the most recent effort to build a comprehensive list of verification challenges and needs for autonomous aircraft, a roadmap to meet those autonomy V&V needs, the services they can enable, and point to the certification gaps they fill. To accomplish these goals, we assembled a team of world-class researchers from the aerospace industry (Boeing, Collins Aerospace, and GeneralElectric) and academia (University of Michigan, University of Texas, and Massachusetts Institute of Technology) with deep expertise in autonomy, aerospace systems, and assurance of Artificial Intelligence/machine learning systems.

Software Assurance↗

NASA NTRSDate not supplied

From Natural Language Requirements to the Verification of Programmable Logic Controllers: Integrating FRET into PLCverif

FRET↗

NASA NTRS2022

Europa Clipper Payload Verification and Validation: Test and Analysis Program Design

NASA’s Europa Clipper mission – which is planned to launch in the mid-2020s – is designed to investigate the habitability of the jovian moon Europa with a state-of-the-art suite of over 10 sensors and instruments. Now past its project critical design review, Europa Clipper is starting the execution of its verification and validation (V&V) program. The payload V&V function in particular is responsible for designing and coordinating a thorough test and analysis program that will confirm the proper functioning and performance of our payload instruments. Given the large number of instruments to manage, we have developed a suite of standardized tests and analyses that are designed to cover all of the common interfaces across instruments, while allowing for flexibility to support the unique needs of individual instruments. This suite of activities, coordinated and/or performed by the payload team, forms the bulk of our payload verification and validation campaign. This paper provides a detailed summary of the full set of V&V activities in that suite, covering the four categories of activities: analysis of lower-level evidence, analysis via simulation, testing in the system testbeds, and testing in assembly, test, and launch operations (ATLO). We will describe the purpose of each activity type, how the activities are developed, and the review process for each type of activity. We then define the complete list of activities written for each instrument and how subtleties specific to certain instruments are managed. The discussion will provide a comprehensive understanding of the activities in the payload V&V purview and how various interfaces with different levels of the project are managed. This paper also describes key metrics the payload team uses to track V&V progress and some insights into the teaming design and division of labor we use to ensure consistency and completeness across the entire payload suite. Finally, the paper will offer a snapshot of the status of the Payload V&V program prior to instrument delivery and offer insights into the upcoming phase of instrument delivery and the start of ATLO.

Lee, Dennis↗

NASA NTRSDate not supplied

Verification Testing and Veg-05 Tomato Crop Production on the International Space Station

VEG-05 is an experiment to better define best practices for crop production and handling in space. Light can impact the growth habit, yield, nutritional composition, microbial levels, and flavor within crops and VEG-05 used the Veggie facilities on the International Space Station to grow ‘Red Robin’ dwarf tomatoes under different spectral compositions to assess these characteristics. Prior to launch, both a science verification test (SVT), and an experiment verification test (EVT) were conducted. SVT, and a previous fertilizer test, grew plants in both plant pillows and PONDS units and tested different fertilizer treatments, with each test under one of the light conditions. EVT used only plant pillows with the highest fertilizer composition tested, and two Veggie units were utilized. One Veggie had light settings consisting of equal levels of red: blue light while the second Veggie had a 90:10 ratio of red: blue light, both with the addition of ~10% green. For SVT, under the equal red: blue light treatment, fruit ripening in plant pillows was delayed. For EVT we saw fruit ripening earlier, especially in the red-rich treatment. In SVT we had mostly daily watering, and this led to excess water in plant pillows which leaked and caused fungus to grow. We reduced this excess in EVT, however, wilting events occurred. Regardless, more than 10 fruit were produced from each plant on average, with the high red treatment producing slightly heavier fruit. Microbial testing from fruit indicated that fruit were safe for consumption. VEG-05 flight operations ran December 2022 and March 2023, with considerable challenges in watering and unexpectedly low humidity on ISS during critical periods. Only 12 ripe fruit formed, and all were returned. Fruit and other samples are currently being analyzed. This research was co-funded by NASA’s Human Research Program and Space Biology.

Gioia Donna Massa↗

NASA NTRSDate not supplied

Verification Testing and VEG-05 Tomato Crop Production on the International Space Station

Gioia D. Massa↗

NASA NTRSDate not supplied

A Formal Verification Framework for Runtime Assurance

The simplex architecture is an instance of Runtime Assurance (RTA) where a trusted component takes control of a safety-critical system when an untrusted component violates a safety property. This paper presents a formalization of the simplex RTA framework in the language of hybrid programs. A feature of this formal verification framework is that, for a given system, a specific instantiation can be created and its safety properties are guaranteed by construction. Instantiations may be kept at varying levels of generality, allowing for black box components, such as ML/AI-based controllers, to be modeled. The framework is written in the Prototype Verification System (PVS) using Plaidypvs, an embedding of differential dynamic logic in PVS. As a proof of concept, the framework is illustrated on an automatic vehicle braking system.

Runtime assurance↗