Search NASA⌕ Search

SEARCH · Search NASA

Results for “Verification and Validation (V&V)”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10
All sourcesNASA NTRSDOE OSTI
NASA NTRS1998

Credible Software and Simulation Uncertainty

The utility of software primarily depends on its reliability and performance; whereas, its significance depends solely on its credibility for intended use. The credibility of simulations confirms the credibility of software. The level of veracity and the level of validity of simulations determine the degree of credibility of simulations. The process of assessing this credibility in fields such as computational mechanics (CM) differs from that followed by the Defense Modeling and Simulation Office in operations research. Verification and validation (V&V) of CM simulations is not the same as V&V of CM software. Uncertainty is the measure of simulation credibility. Designers who use software are concerned with management of simulation uncertainty. Terminology and concepts are presented with a few examples from computational fluid dynamics.

Mehta, Unmeel B.↗
NASA NTRS2001

In-Flight Edge Response Measurements for High Spatial Resolution Remote Sensing Systems

In-flight measurement of spatial resolution were conducted as part of the ASA Scientific Data Purchase (SDP) Validation and Verification (V&V) process. Characterization included remote sensing systems with ground sample distance (GSD) of 1 meter or less, such as the panchromatic imager on-board the ICONOS satellite and the airborne ADAR System 5500 multispectral instrument. Final image products were used to evaluate the effect of both the image acquisition system (e.g., optics, electronics, motion, jitter, atmosphere) and image post-processing (e.g., resampling, modulation trasfer function (MTF) compensator). Spatial resolution was characterized by full width at half maximum (FWHM) of an edge response-derived line spread function. This was found to be a more robust measure of spatial resolution than the value of NTF at Nyquist frequency The edge responses were analysed using the tilted-edge technique that ovecomes the spatial sampling limitations of the digital imaging systems. As an enhancement to existing algorithms, the slope of the edge response and the orientation of the edge target were determined by a single computational process. Adjacent black and white square panels, either painted on a flat surface or deployed as traps, formed the ground-based edge targets used in the tests. Orientation of the deployable tarps was optimized beforehand, based on simulations of the imaging system. Numerous edge target images were analyzed for each of the tested sensors. The effect of such factors as acquisition geometry, temporal variability, MTF compensation, and GSD on spatial resolution were investigated.

Blonski, Slawomir↗
NASA NTRS2003

Experimental Evaluation of Verification and Validation Tools on Martian Rover Software

To achieve its science objectives in deep space exploration, NASA has a need for science platform vehicles to autonomously make control decisions in a time frame that excludes intervention from Earth-based controllers. Round-trip light-time is one significant factor motivating autonomy capability, another factor is the need to reduce ground support operations cost. An unsolved problem potentially impeding the adoption of autonomy capability is the verification and validation of such software systems, which exhibit far more behaviors (and hence distinct execution paths in the software) than is typical in current deepspace platforms. Hence the need for a study to benchmark advanced Verification and Validation (V&V) tools on representative autonomy software. The objective of the study was to access the maturity of different technologies, to provide data indicative of potential synergies between them, and to identify gaps in the technologies with respect to the challenge of autonomy V&V. The study consisted of two parts: first, a set of relatively independent case studies of different tools on the same autonomy code, second a carefully controlled experiment with human participants on a subset of these technologies. This paper describes the second part of the study. Overall, nearly four hundred hours of data on human use of three different advanced V&V tools were accumulated, with a control group that used conventional testing methods. The experiment simulated four independent V&V teams debugging three successive versions of an executive controller for a Martian Rover. Defects were carefully seeded into the three versions based on a profile of defects from CVS logs that occurred in the actual development of the executive controller. The rest of the document is structured a s follows. In section 2 and 3, we respectively describe the tools used in the study and the rover software that was analyzed. In section 4 the methodology for the experiment is described; this includes the code preparation, seeding of defects, participant training and experimental setup. Next we give a qualitative overview of how the experiment went from the point of view of each technology; model checking (section 5), static analysis (section 6), runtime analysis (section 7) and testing (section 8). The find section gives some preliminary quantitative results on how the tools compared.

Brat, Guillaume↗
NASA NTRS2003

Functional Flight Test Report for Positive Systems' ADAR System 5500 Sensor SN8 Linear

This report describes results of the functional flight test conducted with the Positive Systems' ADAR 5500 sensor system (serial number 8, linear configuration) near Winslow, Arizona on June 30 and July 1, 1999. The in-flight test is one component of the NASA Scientific Data Purchase (SDP) Validation and Verification (V&V) process. It allows to measure characteristics of the entire sensor system affected by both performance of the sensor during a flight and post-flight image processing. The following characteristics were analyzed: changes of dark digital numbers (DN's), radiometric linearity, signal-to-noise ratio (SNR), spatial resolution, and geolocation accuracy. The measured characteristics were compared with the image product specifications defined in the Positive Systems' SDP contract. Dependence of the dark DN's on several factors was analyzed, but no significant correlation was found. However, the observed changes in dark DN's were relatively small, which justifies usage of a constant value in the dark DN subtraction procedure during post-processing. Dependence of measured at-sensor, in-band radiance (in arbitrary units) on measured in-band ground reflectance is very well described by a linear function - The sensor fulfills the linearity requirement. Measured SNR values lower than the contract specifications, but accuracy of that test was possibly affected by non-uniformity of the employed gray-scale panels. The SNR values are generally sufficiently high for most applications. SNR can also be improved during standard flights by using longer exposure times. Full width at half maximum (FWHM) of an edge response derived line spread function was used as a measure of spatial resolution. FWHM was generally smaller than twice the ground sample distance (GSD), in agreement with the contract specifications. Accuracy of the geolocation information, which is provided for the particular images in a metadata file, was found to meet contract requirements as well.

Stanley, Tom↗
NASA NTRS2004

Monitoring the Performance of a Neuro-Adaptive Controller

Traditional control has proven to be ineffective to deal with catastrophic changes or slow degradation of complex, highly nonlinear systems like aircraft or spacecraft, robotics, or flexible manufacturing systems. Control systems which can adapt toward changes in the plant have been proposed as they offer many advantages (e.g., better performance, controllability of aircraft despite of a damaged wing). In the last few years, use of neural networks in adaptive controllers (neuro-adaptive control) has been studied actively. Neural networks of various architectures have been used successfully for online learning adaptive controllers. In such a typical control architecture, the neural network receives as an input the current deviation between desired and actual plant behavior and, by on-line training, tries to minimize this discrepancy (e.g.; by producing a control augmentation signal). Even though neuro-adaptive controllers offer many advantages, they have not been used in mission- or safety-critical applications, because performance and safety guarantees cannot b e provided at development time-a major prerequisite for safety certification (e.g., by the FAA or NASA). Verification and Validation (V&V) of an adaptive controller requires the development of new analysis techniques which can demonstrate that the control system behaves safely under all operating conditions. Because of the requirement to adapt toward unforeseen changes during operation, i.e., in real time, design-time V&V is not sufficient.

Schumann, Johann↗
NASA NTRS2004

Validation of Orthorectified Interferometric Radar Imagery and Digital Elevation Models

This work was performed under NASA's Verification and Validation (V&V) Program as an independent check of data supplied by EarthWatch, Incorporated, through the Earth Science Enterprise Scientific Data Purchase (SDP) Program. This document serves as the basis of reporting results associated with validation of orthorectified interferometric interferometric radar imagery and digital elevation models (DEM). This validation covers all datasets provided under the first campaign (Central America & Virginia Beach) plus three earlier missions (Indonesia, Red River: and Denver) for a total of 13 missions.

Smith Charles M.↗
NASA NTRS2005

Remote Sensing Product Verification and Validation at the NASA Stennis Space Center

Remote sensing data product verification and validation (V&V) is critical to successful science research and applications development. People who use remote sensing products to make policy, economic, or scientific decisions require confidence in and an understanding of the products' characteristics to make informed decisions about the products' use. NASA data products of coarse to moderate spatial resolution are validated by NASA science teams. NASA's Stennis Space Center (SSC) serves as the science validation team lead for validating commercial data products of moderate to high spatial resolution. At SSC, the Applications Research Toolbox simulates sensors and targets, and the Instrument Validation Laboratory validates critical sensors. The SSC V&V Site consists of radiometric tarps, a network of ground control points, a water surface temperature sensor, an atmospheric measurement system, painted concrete radial target and edge targets, and other instrumentation. NASA's Applied Sciences Directorate participates in the Joint Agency Commercial Imagery Evaluation (JACIE) team formed by NASA, the U.S. Geological Survey, and the National Geospatial-Intelligence Agency to characterize commercial systems and imagery.

Stanley, Thomas M.↗
NASA NTRS2005

Verification and Validation of NASA-Supported Enhancements to Decision Support Tools of PECAD

This section of the evaluation report summarizes the verification and validation (V&V) of recently implemented, NASA-supported enhancements to the decision support tools of the Production Estimates and Crop Assessment Division (PECAD). The implemented enhancements include operationally tailored Moderate Resolution Imaging Spectroradiometer (MODIS) products and products of the Global Reservoir and Lake Monitor (GRLM). The MODIS products are currently made available through two separate decision support tools: the MODIS Image Gallery and the U.S. Department of Agriculture (USDA) Foreign Agricultural Service (FAS) MODIS Normalized Difference Vegetation Index (NDVI) Database. Both the Global Reservoir and Lake Monitor and MODIS Image Gallery provide near-real-time products through PECAD's CropExplorer. This discussion addresses two areas: 1. Assessments of the standard NASA products on which these enhancements are based. 2. Characterizations of the performance of the new operational products.

Ross, Kenton W.↗
NASA NTRS2006

Verification and Validation of NASA-Supported Enhancements to the Near Real Time Harmful Algal Blooms Observing System (HABSOS)

This report discusses verification and validation (V&V) assessment of Moderate Resolution Imaging Spectroradiometer (MODIS) ocean data products contributed by the Naval Research Laboratory (NRL) and Applied Coherent Technologies (ACT) Corporation to National Oceanic Atmospheric Administration s (NOAA) Near Real Time (NRT) Harmful Algal Blooms Observing System (HABSOS). HABSOS is a maturing decision support tool (DST) used by NOAA and its partners involved with coastal and public health management.

Spruce, Joseph P.↗
NASA NTRS2006

NASA's M and S Accreditation Process Plan and Specification for Space Exploration

NASA's Exploration Systems Mission Directorate (ESMD) is implementing a management approach for modeling and simulation (M&S) that will provide decision-makers information on the model's fidelity, credibility, and quality. This information will allow the decision-maker to understand the risks involved in using a model's results in the decision-making process. This presentation will discuss NASA's overall approach to achieving formal accreditation of its models or simulations supporting space exploration. The development of a formal Accreditation Plan is a key component in the preliminary activities for modeling and simulation (M&S) assessment. This presentation will describe NASA's process for identifying risks associated with M&S use and the associated M&S assessments that will dictate the level of data certification and M&S verification and validation (V&V) activities required to support the decision-making process. The M&S Accreditation Plan and Report templates for ESMD will also be illustrated.

O'Neil, David↗
NASA NTRS2009

Systematic Benchmarking of Diagnostic Technologies for an Electrical Power System

Automated health management is a critical functionality for complex aerospace systems. A wide variety of diagnostic algorithms have been developed to address this technical challenge. Unfortunately, the lack of support to perform large-scale V&V (verification and validation) of diagnostic technologies continues to create barriers to effective development and deployment of such algorithms for aerospace vehicles. In this paper, we describe a formal framework developed for benchmarking of diagnostic technologies. The diagnosed system is the Advanced Diagnostics and Prognostics Testbed (ADAPT), a real-world electrical power system (EPS), developed and maintained at the NASA Ames Research Center. The benchmarking approach provides a systematic, empirical basis to the testing of diagnostic software and is used to provide performance assessment for different diagnostic algorithms.

Kurtoglu, Tolga↗
NASA NTRS2006

SIM-PlanetQuest: Progress Report

SIM-PlanetQuest is a NASA astrophysics mission that is implementing the National Research Counsel's recommended Astrometric Interferometry Mission (AIM) to develop the first, in-space, optical, long-baseline Michelson Stellar Interferometer for performing micro-arcsecond-level astrometry. This level of astrometric precision will enable characterization of planetary systems around nearby stars and enable a number of key investigations in astrophysics including calibration of the cosmological distance scale, stellar and galactic structure and evolution, and dark matter/energy distribution. This paper provides an update on the SIM-PlanetQuest Mission covering the results of the 2005 mission redesign and the recent completion of the last in a series of technology "gates." The SIM-PlanetQuest mission redesign was directed by NASA to recover eroded mass and power margins and to meet specific implementation cost targets. The resulting mission redesign met all redesign objectives with minimal impact to mission science performance. This paper provides the mission redesign objectives and describes the resulting mission and system design including changes in science capability. SIM-PlanetQuest also completed the last of eight major technology development gates that were established in 2001 by NASA, completing the enabling technology development. The technology development program, the last gate, and its significance to the project's flight verification and validation (V&V) approach are briefly described (covered in more detail in a separate paper at this conference). An update on project programmatic status and plans is also provided.

astrometry↗
NASA NTRS2010

Verification and Validation of Flight-Critical Systems

For the first time in many years, the NASA budget presented to congress calls for a focused effort on the verification and validation (V&V) of complex systems. This is mostly motivated by the results of the VVFCS (V&V of Flight-Critical Systems) study, which should materialize as a a concrete effort under the Aviation Safety program. This talk will present the results of the study, from requirements coming out of discussions with the FAA and the Joint Planning and Development Office (JPDO) to technical plan addressing the issue, and its proposed current and future V&V research agenda, which will be addressed by NASA Ames, Langley, and Dryden as well as external partners through NASA Research Announcements (NRA) calls. This agenda calls for pushing V&V earlier in the life cycle and take advantage of formal methods to increase safety and reduce cost of V&V. I will present the on-going research work (especially the four main technical areas: Safety Assurance, Distributed Systems, Authority and Autonomy, and Software-Intensive Systems), possible extensions, and how VVFCS plans on grounding the research in realistic examples, including an intended V&V test-bench based on an Integrated Modular Avionics (IMA) architecture and hosted by Dryden.

Brat, Guillaume↗
NASA NTRS2009

Software Health Management: A Short Review of Challenges and Existing Techniques

Modern spacecraft (as well as most other complex mechanisms like aircraft, automobiles, and chemical plants) rely more and more on software, to a point where software failures have caused severe accidents and loss of missions. Software failures during a manned mission can cause loss of life, so there are severe requirements to make the software as safe and reliable as possible. Typically, verification and validation (V&V) has the task of making sure that all software errors are found before the software is deployed and that it always conforms to the requirements. Experience, however, shows that this gold standard of error-free software cannot be reached in practice. Even if the software alone is free of glitches, its interoperation with the hardware (e.g., with sensors or actuators) can cause problems. Unexpected operational conditions or changes in the environment may ultimately cause a software system to fail. Is there a way to surmount this problem? In most modern aircraft and many automobiles, hardware such as central electrical, mechanical, and hydraulic components are monitored by IVHM (Integrated Vehicle Health Management) systems. These systems can recognize, isolate, and identify faults and failures, both those that already occurred as well as imminent ones. With the help of diagnostics and prognostics, appropriate mitigation strategies can be selected (replacement or repair, switch to redundant systems, etc.). In this short paper, we discuss some challenges and promising techniques for software health management (SWHM). In particular, we identify unique challenges for preventing software failure in systems which involve both software and hardware components. We then present our classifications of techniques related to SWHM. These classifications are performed based on dimensions of interest to both developers and users of the techniques, and hopefully provide a map for dealing with software faults and failures.

Pipatsrisawat, Knot↗
NASA NTRS2010

Validation and Verification of Future Integrated Safety-Critical Systems Operating under Off-Nominal Conditions

Loss of control remains one of the largest contributors to aircraft fatal accidents worldwide. Aircraft loss-of-control accidents are highly complex in that they can result from numerous causal and contributing factors acting alone or (more often) in combination. Hence, there is no single intervention strategy to prevent these accidents and reducing them will require a holistic integrated intervention capability. Future onboard integrated system technologies developed for preventing loss of vehicle control accidents must be able to assure safe operation under the associated off-nominal conditions. The transition of these technologies into the commercial fleet will require their extensive validation and verification (V and V) and ultimate certification. The V and V of complex integrated systems poses major nontrivial technical challenges particularly for safety-critical operation under highly off-nominal conditions associated with aircraft loss-of-control events. This paper summarizes the V and V problem and presents a proposed process that could be applied to complex integrated safety-critical systems developed for preventing aircraft loss-of-control accidents. A summary of recent research accomplishments in this effort is also provided.

Belcastro, Christine M.↗
NASA NTRS2010

TEAMS Model Analyzer

The TEAMS model analyzer is a supporting tool developed to work with models created with TEAMS (Testability, Engineering, and Maintenance System), which was developed by QSI. In an effort to reduce the time spent in the manual process that each TEAMS modeler must perform in the preparation of reporting for model reviews, a new tool has been developed as an aid to models developed in TEAMS. The software allows for the viewing, reporting, and checking of TEAMS models that are checked into the TEAMS model database. The software allows the user to selectively model in a hierarchical tree outline view that displays the components, failure modes, and ports. The reporting features allow the user to quickly gather statistics about the model, and generate an input/output report pertaining to all of the components. Rules can be automatically validated against the model, with a report generated containing resulting inconsistencies. In addition to reducing manual effort, this software also provides an automated process framework for the Verification and Validation (V&V) effort that will follow development of these models. The aid of such an automated tool would have a significant impact on the V&V process.

Tijidjian, Raffi P.↗
NASA NTRS2010

Model Transformation for a System of Systems Dependability Safety Case

Software plays an increasingly larger role in all aspects of NASA's science missions. This has been extended to the identification, management and control of faults which affect safety-critical functions and by default, the overall success of the mission. Traditionally, the analysis of fault identification, management and control are hardware based. Due to the increasing complexity of system, there has been a corresponding increase in the complexity in fault management software. The NASA Independent Validation & Verification (IV&V) program is creating processes and procedures to identify, and incorporate safety-critical software requirements along with corresponding software faults so that potential hazards may be mitigated. This Specific to Generic ... A Case for Reuse paper describes the phases of a dependability and safety study which identifies a new, process to create a foundation for reusable assets. These assets support the identification and management of specific software faults and, their transformation from specific to generic software faults. This approach also has applications to other systems outside of the NASA environment. This paper addresses how a mission specific dependability and safety case is being transformed to a generic dependability and safety case which can be reused for any type of space mission with an emphasis on software fault conditions.

Murphy, Judy↗