Search NASA⌕ Search

SEARCH · Search NASA

Results for “Verification & validation (V&V)”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12
All sourcesNASA NTRSDOE OSTI
NASA NTRS2006

A Predictive Approach to Eliminating Errors in Software Code

NASA s Metrics Data Program Data Repository is a database that stores problem, product, and metrics data. The primary goal of this data repository is to provide project data to the software community. In doing so, the Metrics Data Program collects artifacts from a large NASA dataset, generates metrics on the artifacts, and then generates reports that are made available to the public at no cost. The data that are made available to general users have been sanitized and authorized for publication through the Metrics Data Program Web site by officials representing the projects from which the data originated. The data repository is operated by NASA s Independent Verification and Validation (IV&V) Facility, which is located in Fairmont, West Virginia, a high-tech hub for emerging innovation in the Mountain State. The IV&V Facility was founded in 1993, under the NASA Office of Safety and Mission Assurance, as a direct result of recommendations made by the National Research Council and the Report of the Presidential Commission on the Space Shuttle Challenger Accident. Today, under the direction of Goddard Space Flight Center, the IV&V Facility continues its mission to provide the highest achievable levels of safety and cost-effectiveness for mission-critical software. By extending its data to public users, the facility has helped improve the safety, reliability, and quality of complex software systems throughout private industry and other government agencies. Integrated Software Metrics, Inc., is one of the organizations that has benefited from studying the metrics data. As a result, the company has evolved into a leading developer of innovative software-error prediction tools that help organizations deliver better software, on time and on budget.

Source record↗
NASA NTRS2009

IT Project Success w\7120 and 7123 NPRs to Achieve Project Success

This slide presentation reviews management techniques to assure information technology development project success. Details include the work products, the work breakdown structure (WBS), system integration, verification and validation (IV&V), and deployment and operations. An example, the NASA Consolidated Active Directory (NCAD), is reviewed.

Walley, Tina L.↗
NASA NTRS2010

Using Web 2.0 Techniques in NASA's Ares Engineering Operations Network (AEON) Environment - First Impressions

The Mission Operations Laboratory (MOL) at Marshall Space Flight Center (MSFC) is responsible for Engineering Support capability for NASA s Ares rocket development and operations. In pursuit of this, MOL is building the Ares Engineering and Operations Network (AEON), a web-based portal to support and simplify two critical activities: Access and analyze Ares manufacturing, test, and flight performance data, with access to Shuttle data for comparison Establish and maintain collaborative communities within the Ares teams/subteams and with other projects, e.g., Space Shuttle, International Space Station (ISS). AEON seeks to provide a seamless interface to a) locally developed engineering applications and b) a Commercial-Off-The-Shelf (COTS) collaborative environment that includes Web 2.0 capabilities, e.g., blogging, wikis, and social networking. This paper discusses how Web 2.0 might be applied to the typically conservative engineering support arena, based on feedback from Integration, Verification, and Validation (IV&V) testing and on searching for their use in similar environments.

Scott, David W.↗
NASA NTRS2016

Technical Reference Suite Addressing Challenges of Providing Assurance for Fault Management Architectural Design

Research into complexities of software systems Fault Management (FM) and how architectural design decisions affect safety, preservation of assets, and maintenance of desired system functionality has coalesced into a technical reference (TR) suite that advances the provision of safety and mission assurance. The NASA Independent Verification and Validation (IV&V) Program, with Software Assurance Research Program support, extracted FM architectures across the IV&V portfolio to evaluate robustness, assess visibility for validation and test, and define software assurance methods applied to the architectures and designs. This investigation spanned IV&V projects with seven different primary developers, a wide range of sizes and complexities, and encompassed Deep Space Robotic, Human Spaceflight, and Earth Orbiter mission FM architectures. The initiative continues with an expansion of the TR suite to include Launch Vehicles, adding the benefit of investigating differences intrinsic to model-based FM architectures and insight into complexities of FM within an Agile software development environment, in order to improve awareness of how nontraditional processes affect FM architectural design and system health management. The identification of particular FM architectures, visibility, and associated IV&V techniques provides a TR suite that enables greater assurance that critical software systems will adequately protect against faults and respond to adverse conditions. Additionally, the role FM has with regard to strengthened security requirements, with potential to advance overall asset protection of flight software systems, is being addressed with the development of an adverse conditions database encompassing flight software vulnerabilities. Capitalizing on the established framework, this TR suite provides assurance capability for a variety of FM architectures and varied development approaches. Research results are being disseminated across NASA, other agencies, and the software community. This paper discusses the findings and TR suite informing the FM domain in best practices for FM architectural design, visibility observations, and methods employed for IV&V and mission assurance.

Fitz, Rhonda↗
NASA NTRS2019

Agile Approach to Assuring the Safety-Critical Embedded Software for NASA's Orion Spacecraft

Human-rated missions like those in NASA's Orion Program continue to grow in complexity. The role of software in achieving ambitious mission objectives has expanded dramatically in the last few decades. Assuring the safety and performance of the embedded flight software is quickly growing beyond the reach of traditional methods and resource levels. The methods used to build these software-dominant systems evolve in an on-going attempt to keep pace with the scope of our ambitions. Agile software development is now commonplace. The long timelines and large batches of work associated with traditional methods are being replaced by rapid delivery of small increments _ as system capabilities are realized in waves. Assurance of these critical software capabilities must therefore conquer an ever-expanding frontier of challenges, and do so with an approach matched to the evolving development methods. This paper recounts the journey of the Orion Independent Verification and Validation (IV&V) team as we addressed this dynamic environment. Widening our aperture to encompass a dramatically larger mission scope, while adjusting our cadence to synchronize with the rapid pace of agile software development, a new approach to IV&V is emerging. This approach is characterized by a sharper focus on mission capabilities, matched with a method to dynamically _follow the risk' as the IV&V team delivers more compelling assurance data in waves. Traditional methods prevalent in IV&V tend to scope the work using artifacts of the development process as they evolve from preliminary to final versions, and the pace of delivery was synchronized with the development timelines prevalent in the waterfall lifecycle. That more static approach is out of phase with the demands of the new environment. Scoping work according to the critical capabilities of the system (rather than artifacts of development) and synchronizing with the rapid pace of agile development, we are moving toward more effective parity with the demands of the environment. We explain the concrete steps we took, the principles that motivated our choices, and the results we have achieved to date.

Capability based assurance↗
NASA NTRS2019

Agile Approach to Assuring the Safety-Critical Embedded Software for NASA's Orion Spacecraft

Human-rated missions like NASA's Exploration Mission - 1 (EM-1) and the Orion Multi-Purpose Crew Vehicle are becoming exceedingly complex in terms of software's contribution to achieving mission objectives. The increasing complexity and inherent safety critical nature of the embedded flight software imposes a unique resource challenge to assurance providers responsible for affirming that the mission is going to fly safely. Another challenge NASA and other Government agencies are facing is that more and more software is being developed using an agile development methodology, which is divergent from the typical waterfall, iterative, and incremental development methodologies assurance providers generally observe in the development of safety-critical embedded software. Orion Independent Verification and Validation (IV&V) has addressed these challenges by providing focused assurance results of critical mission capabilities prioritized by a dynamic assessment of risk level. Prior to this approach, Orion IV&V evaluated areas of risk in much broader, and more static, terms. Due to the agile software development life cycle that Orion follows, IV&V findings were often reported months out of phase with the developer thereby imposing increased rework costs. As a result of evolving the approach to adding assurance on Orion, IV&V is able to incrementally deliver high-priority assurance conclusions and more impactful issues more in phase with the developer activities, thereby increasing the value of the findings to the project. The agile IV&V approach employed by the Orion IV&V team strives to achieve a cadence of delivery that matches the pace of development. This agile approach provides increased flexibility for the assurance provider to become more efficient in reporting assurance conclusions and issues. This paper and presentation will discuss the principles which drive the design of our approach, results to date, and stimulate thinking for groups looking to add assurance to software being developed using an agile methodology.

Justin Smith↗
NASA NTRSDate not supplied

Adaptive Stress Testing of Collision Avoidance Systems for Small UASs with Deep Reinforcement Learning

The next-generation Airborne Collision Avoidance System for smaller UASs (ACAS sXu) is currently being developed and tested by the Federal Aviation Administration (FAA) to provide detect-and-avoid capability for small unmanned aircraft operating beyond line-of-sight. Due to the complexity and safety-critical nature of the system, safety validation is important not only for the certification of the final system, but also for informing changes during the iterative development process. In this paper, we analyze a prototype of ACAS sXu in simulated aircraft encounters to discover scenarios of small near mid-air collisions (sNMACs), an important safety event in which two aircraft come closer than 50 feet horizontally and 15 feet vertically. Due to the size and complexity of the system as well as rarity of sNMAC events, traditional methods such as Monte Carlo testing often require informed setup and targeting to elicit failures. However, such a dependence on domain knowledge can be incompatible with the independent verification and validation (IV&V) process, the aim of which is to discover unforeseen issues. To address these challenges, we apply an accelerated validation method called adaptive stress testing (AST) to find the most likely sNMAC scenarios without reliance on system introspection. AST uses reinforcement learning to adapt the search towards the most promising areas of the search space as it progresses. We use a state-of-the-art deep reinforcement learning algorithm, proximate policy optimization, to more efficiently search the large and continuous state space. We find that this approach significantly improves the performance of AST compared to a prior approach based on Monte Carlo tree search. We perform experiments using AST to find sNMAC events under various encounter configurations, varying parameters pertaining to dynamics and coordination. Our experiments show AST to be very effective at finding sNMAC scenarios. We summarize our findings, presenting high-level categories of discovered sNMACs and specific examples of encounters in each category.

aircraft collision avoidance↗
NASA NTRSDate not supplied

Alternate Methods of Model Reduction to Avoid Dynamic Modal Truncation Error

Loads analysis is traditionally performed using dynamically reduced models, which provides the benefit to reduce run time. If the reduced model frequency cutoff is not chosen appropriately, the model will lack the dynamic content required to fully represent the response of the non-reduced model. Standard guidance for reduced model frequency content, provided in NASA-STD-5002, is to solve fixed base modes up to a minimum of 1.5x the model frequency content of interest and to employ static modal truncation methods such as residual vectors, the mode acceleration method, and the residual flexibility method to account for the truncated flexibility of the missing modes. Fixed base modes require the selection of a set of degrees of freedom to be constrained which, if not properly selected, may affect the accuracy of the reduced model by excluding some of the dynamic characteristic of the full model. In this case, the standard NASA guidance would be insufficient, but it may not be readily apparent that a portion of the reduced model response is missing. This error was encountered during an independent verification and validation (IV&V) effort, where it was observed that the resulting dynamic response was lower than the inline analysis. In this specific case, despite following the standard NASA model reduction guidelines in the selection of the frequency cutoff, the inline model still did not fully capture the necessary dynamic content. As part of the IV&V, an alternate reduction methodology was employed using an unconstrained mode acceleration method. The original model initially performed a constrained reduction to twice the frequency content of interest before doing a free-free run, employing the mode acceleration method to account for the truncated modes. In contrast in the IV&V, the unconstrained model reduced directly to the needed frequency content of the free-free run, avoiding any interactions between constraints and dynamic content. To verify the model, the original reduction methodology was used to generate a series of Hurty-Craig-Bampton reductions, each with a higher frequency cutoff than the previous. The results were shown to converge once the frequency cutoff increased past eight times the frequency content of interest. At the same frequency cutoff, the results of the Hurty-Craig-Bampton model converged with the results of the unconstrained mode acceleration model. This comparative study provided confidence that the results of the unconstrained modal acceleration reduced model were correct and that the Hurty-Craig-Bampton needed to increase its frequency cutoff to fully capture the dynamic response.

Erin Simmons↗
NASA NTRSDate not supplied

Alternate Methods of Model Reduction to Avoid Dynamic Modal Truncation Error

Loads analysis is traditionally performed using dynamically reduced models, which provides the benefit to reduce run time. If the reduced model frequency cutoff is not chosen appropriately, the model will lack the dynamic content required to fully represent the response of the non-reduced model. Standard guidance for reduced model frequency content, provided in NASA-STD-5002, is to solve fixed base modes up to a minimum of 1.5x the model frequency content of interest and to employ static modal truncation methods such as residual vectors, the mode acceleration method, and the residual flexibility method to account for the truncated flexibility of the missing modes. Fixed base modes require the selection of a set of degrees of freedom to be constrained which, if not properly selected, may affect the accuracy of the reduced model by excluding some of the dynamic characteristic of the full model. In this case, the standard NASA guidance would be insufficient, but it may not be readily apparent that a portion of the reduced model response is missing. This error was encountered during an independent verification and validation (IV&V) effort, where it was observed that the resulting dynamic response was lower than the inline analysis. In this specific case, despite following the standard NASA model reduction guidelines in the selection of the frequency cutoff, the inline model still did not fully capture the necessary dynamic content. As part of the IV&V, an alternate reduction methodology was employed using an unconstrained mode acceleration method. The original model initially performed a constrained reduction to twice the frequency content of interest before doing a free-free run, employing the mode acceleration method to account for the truncated modes. In contrast in the IV&V, the unconstrained model reduced directly to the needed frequency content of the free-free run, avoiding any interactions between constraints and dynamic content. To verify the model, the original reduction methodology was used to generate a series of Hurty-Craig-Bampton reductions, each with a higher frequency cutoff than the previous. The results were shown to converge once the frequency cutoff increased past eight times the frequency content of interest. At the same frequency cutoff, the results of the Hurty-Craig-Bampton model converged with the results of the unconstrained mode acceleration model. This comparative study provided confidence that the results of the unconstrained modal acceleration reduced model were correct and that the Hurty-Craig-Bampton needed to increase its frequency cutoff to fully capture the dynamic response.

Erin Simmons↗
NASA NTRS1999

Model Checking Verification and Validation at JPL and the NASA Fairmont IV and V Facility

We show how a technology transfer effort was carried out. The successful use of model checking on a pilot JPL flight project demonstrates the usefulness and the efficacy of the approach. The pilot project was used to model a complex spacecraft controller. Software design and implementation validation were carried out successfully. To suggest future applications we also show how the implementation validation step can be automated. The effort was followed by the formal introduction of the modeling technique as a part of the JPL Quality Assurance process.

Schneider, Frank↗
NASA NTRS2018

Dynamic Power Convertor Development for Radioisotope Power Systems at NASA Glenn Research Center

The Thermal Energy Conversion Branch at NASA Glenn Research Center is supporting the development of high-efficiency power convertors for use in radioisotope power systems (RPS). Significant progress was made toward such a system that utilized Stirling conversion during the 2001 to 2015 timeframe. Flight development of the Advanced Stirling Radioisotope Generator (ASRG) was canceled in 2013 by the U.S. Department of Energy (DOE) and NASA Headquarters primarily due to budget constraints, and the Advanced Stirling Convertor (ASC) technology contract was subsequently concluded in 2015. A new chapter of technology development has recently been initiated by the NASA RPS Program. This effort is considering all dynamic power convertor options, such as Stirling and Brayton cycles. Four convertor development contracts supporting this effort were awarded in 2017. The awarded contracts include two free-piston Stirling, one thermoacoustic Stirling, and one turbo-Brayton design. The technology development contracts each consist of up to three phases: design, fabrication, and test. As of May 2018, all contracts have completed the design phase, and each underwent a design review with an independent review board. Three of the contracts are planned to execute the phase 2 option for fabrication. Convertors manifesting from these development efforts will then undergo independent validation and verification (IV and V) at NASA facilities, which will consist of convertor performance and RPS viability demonstrations. Example tests include launch vibration simulation, performance mapping over the environmental temperature range, and static acceleration exposure. In parallel with this renewed development effort, Glenn is still demonstrating free-piston Stirling convertor technology using assets from previous projects. The Stirling Research Laboratory (SRL) is still operating several convertors from previous development projects, which have similarities and relevance to current contract designs. Four of these are flexure-bearing based and another six are gas bearing based. One of the flexure-bearing convertors has accumulated over 110,000 h of operation and holds the current record for maintenance-free heat-engine run time. Another flexure-bearing convertor was recently manually shut down after 105,620 h of operation, then disassembled and inspected. This inspection produced a wealth of information about the effects of this amount of run time on the technology's components. One of the engineering unit flexure-bearing convertors recently underwent a launch simulation vibration test, a static acceleration exposure up to 20 g, and was then placed on extended operation. Among the gas-bearing convertors, the longest running unit has accumulated over 70,000 h of operation. Four high-fidelity gas-bearing convertors from the ASRG project are still operating continuously for which the longest run time has reached 28,000 h.

Oriti, Salvatore M.↗
NASA NTRSDate not supplied

Software-Only Key Management and Cryptography (KMC) for Spacecraft Command Encryption

The Jet Propulsion Laboratory (JPL) Advanced Multi-Mission Operations System (AMMOS) in partnership with NASA’s Katherine Johnson Independent Verification and Validation (NASA IV&V) Jon McBride Software Testing and Research (JSTAR) laboratory have jointly developed a command encryption capability that provides a software-only encryption solution utilizing the CCSDS Space Data Link Security Protocol. This solution, named Key Management and Cryptography (KMC), incorporates JPL’s encryption service with the JSTAR open-source CryptoLib library, providing a solution for spacecraft command encryption, which is required per NASA standard NASA-STD-1006-A. This paper will focus on its design, usage, and operation for two SmallSat missions, along with its future expanded functionality of end-to-end spacecraft encryption.

command encryption↗
NASA NTRSDate not supplied

ARRISTOTLE Simulation Demo

NASA’s Katherine Johnson Independent Verification and Validation (NASA IV&V) Jon McBride Software Testing and Research (JSTAR) laboratory has developed the Advanced Risk Reduction Integrated Software Test and Operations Tri-program Lightweight Environment (ARRISTOTLE). ARRISTOTLE is a digital twin of NASA’s Artemis mission. ARRISTOLE executes the true SLS and Orion flight binaries integrated on a single laptop computer. This demo will showcase the execution of the full Artemis liftoff. Once airborne, an inflight anomaly will be injected, resulting in Orion detaching from the booster. The Launch Abort System (LAS) will safely bring Orion back to earth. The Unity game engine is used to visualize the flyout as flight software executes.

emulation↗
NASA NTRS2016

Integrating MBSE into Ongoing Projects: Requirements Validation and Test Planning for the ISS SAFER

The International Space Station (ISS) Simplified Aid for Extra Vehicular Activity (EVA) Rescue (SAFER) is the spacewalking astronaut's final safety measure against separating from the ISS and being unable to return safely. Since the late 1990s, the SAFER has been a standard element of the spacewalking astronaut's equipment. The ISS SAFER project was chartered to develop a new block of SAFER units using a highly similar design to the legacy SAFER (known as the USA SAFER). An on-orbit test module was also included in the project to enable periodic maintenance/propulsion system checkout on the ISS SAFER. On the ISS SAFER project, model-based systems engineering (MBSE) was not the initial systems engineering (SE) approach, given the volume of heritage systems engineering and integration (SE&I) products. The initial emphasis was ensuring traceability to ISS program standards as well as to legacy USA SAFER requirements. The requirements management capabilities of the Cradle systems engineering tool were to be utilized to that end. During development, however, MBSE approaches were applied selectively to address specific challenges in requirements validation and test and verification (T&V) planning, which provided measurable efficiencies to the project. From an MBSE perspective, ISS SAFER development presented a challenge and an opportunity. Addressing the challenge first, the project was tasked to use the original USA SAFER operational and design requirements baseline, with a number of additional ISS program requirements to address evolving certification expectations for systems operating on the ISS. Additionally, a need to redesign the ISS SAFER avionics architecture resulted in a set of changes to the design requirements baseline. Finally, the project added an entirely new functionality for on-orbit maintenance. After initial requirements integration, the system requirements count was approaching 1000, which represented a growth of 4x over the original USA SAFER system. This presented the challenge - How to confirm that this new set of requirements set would result in the creation of the desired capability.

Anderson, Herbert A.↗
NASA NTRS2012

SMAP Verification and Validation Project - Final Report

In 2007, the National Research Council (NRC) released the Decadal Survey of Earth science. In the future decade, the survey identified 15 new space missions of significant scientific and application value for the National Aeronautics and Space Administration (NASA) to undertake. One of these missions was the Soil Moisture Active Passive (SMAP) mission that NASA assigned to the Jet Propulsion Laboratory (JPL) in 2008. The goal of SMAP1 is to provide global, high resolution mapping of soil moisture and its freeze/thaw states. The SMAP project recently passed its Critical Design Review and is proceeding with its fabrication and testing phase.Verification and Validation (V&V) is widely recognized as a critical component in system engineering and is vital to the success of any space mission. V&V is a process that is used to check that a system meets its design requirements and specifications in order to fulfill its intended purpose. Verification often refers to the question "Have we built the system right?" whereas Validation asks "Have we built the right system?" Currently the SMAP V&V team is verifying design requirements through inspection, demonstration, analysis, or testing. An example of the SMAP V&V process is the verification of the antenna pointing accuracy with mathematical models since it is not possible to provide the appropriate micro-gravity environment for testing the antenna on Earth before launch.

Soil Moisture Active Passive (SMAP)↗