Engineering topics

Voas, Jeffrey M.

Publications and source records attributed to Voas, Jeffrey M..

9 records

NASA NTRS1993

An empirical comparison of a dynamic software testability metric to static cyclomatic complexity

This paper compares the dynamic testability prediction technique termed 'sensitivity analysis' to the static testability technique termed cyclomatic complexity. The application that we chose in this empirical study is a CASE generated version of a B-737 autoland system. For the B-737 system we analyzed, we isolated those functions that we predict are more prone to hide errors during system/reliability testing. We also analyzed the code with several other well-known static metrics. This paper compares and contrasts the results of sensitivity analysis to the results of the static metrics.

Voas, Jeffrey M.↗

NASA NTRS1993

Automating test case generation for coverages required by FAA standard DO-178B

Coverage testing techniques are required by the FAA for various levels of subsystem criticality at the unit testing level. Higher levels of criticality require coverage schemes that frequently require more and more test cases, particularly when the number of conditions in a decision grows. For example, if we have a decision with n conditions of the form: if (/ci/ or /c2/ or .... or /cn/) then there are 2(n) possible combinations of condition outcomes. Given the enormous number of inputs that may be required to satisfy different coverages, and given that there are no automated tools for determining these inputs (to our knowledge), we will show how one alternative testing technique, mutation testing, can be coerced into generating inputs that satisfy a code coverage scheme X, i.e., if we modify the rules for mutant generation during mutation testing, this technique will provide test cases that satisfy X.

Voas, Jeffrey M.↗

NASA NTRS1993

Software testability and its application to avionic software

Randomly generated black-box testing is an established yet controversial method of estimating software reliability. Unfortunately, as software applications have required higher reliabilities, practical difficulties with black-box testing have become increasingly problematic. These practical problems are particularly acute in life-critical avionics software, where requirements of 10 exp -7 failures per hour of system reliability can translate into a probability of failure (POF) of perhaps 10 exp -9 or less for each individual execution of the software. This paper describes the application of one type of testability analysis called 'sensitivity analysis' to B-737 avionics software; one application of sensitivity analysis is to quantify whether software testing is capable of detecting faults in a particular program and thus whether we can be confident that a tested program is not hiding faults. We so 80 by finding the testabilities of the individual statements of the program, and then use those statement testabilities to find the testabilities of the functions and modules. For the B-737 system we analyzed, we were able to isolate those functions that are more prone to hide errors during system/reliability testing.

Voas, Jeffrey M.↗

NASA NTRS1992

Estimating the probability of failure when testing reveals no failures

Formulas for estimating the probability of failure when testing reveals no errors are introduced. These formulas incorporate random testing results, information about the input distribution, and prior assumptions about the probability of failure of the software. The formulas are not restricted to equally likely input distributions, and the probability of failure estimate can be adjusted when assumptions about the input distribution change. The formulas are based on a discrete sample space statistical model of software and include Bayesian prior assumptions. Reusable software and software in life-critical applications are particularly appropriate candidates for this type of analysis.

Miller, Keith W.↗

NASA NTRS1991

Factors That Affect Software Testability

Software faults that infrequently affect software's output are dangerous. When a software fault causes frequent software failures, testing is likely to reveal the fault before the software is releases; when the fault remains undetected during testing, it can cause disaster after the software is installed. A technique for predicting whether a particular piece of software is likely to reveal faults within itself during testing is found in [Voas91b]. A piece of software that is likely to reveal faults within itself during testing is said to have high testability. A piece of software that is not likely to reveal faults within itself during testing is said to have low testability. It is preferable to design software with higher testabilities from the outset, i.e., create software with as high of a degree of testability as possible to avoid the problems of having undetected faults that are associated with low testability. Information loss is a phenomenon that occurs during program execution that increases the likelihood that a fault will remain undetected. In this paper, I identify two brad classes of information loss, define them, and suggest ways of predicting the potential for information loss to occur. We do this in order to decrease the likelihood that faults will remain undetected during testing.

Voas, Jeffrey M.↗

NASA NTRS1991

Improving the Software Development Process Using Testability Research

Software testability is the the tendency of code to reveal existing faults during random testing. This paper proposes to take software testability predictions into account throughout the development process. These predictions can be made from formal specifications, design documents, and the code itself. The insight provided by software testability is valuable during design, coding, testing, and quality assurance. We further believe that software testability analysis can play a crucial role in quantifying the likelihood that faults are not hiding after testing does not result in any failures for the current version.

Voas, Jeffrey M.↗

NASA NTRS1991

PISCES: A Tool for Predicting Software Testability

Before a program can fail, a software fault must be executed, that execution must alter the data state, and the incorrect data state must propagate to a state that results directly in an incorrect output. This paper describes a tool called PISCES (developed by Reliable Software Technologies Corporation) for predicting the probability that faults in a particular program location will accomplish all three of these steps causing program failure. PISCES is a tool that is used during software verification and validation to predict a program's testability.

Voas, Jeffrey M.↗

NASA NTRS1991

A Model for Assessing the Liability of Seemingly Correct Software

Current research on software reliability does not lend itself to quantitatively assessing the risk posed by a piece of life-critical software. Black-box software reliability models are too general and make too many assumptions to be applied confidently to assessing the risk of life-critical software. We present a model for assessing the risk caused by a piece of software; this model combines software testing results and Hamlet's probable correctness model. We show how this model can assess software risk for those who insure against a loss that can occur if life-critical software fails.

Voas, Jeffrey M.↗

NASA NTRS1990

PIE: A Dynamic Failure-Based Technique

This paper presents a dynamic technique for statistically estimating three program characteristics that affect a program's computational behavior: (1) the probability that a particular section of a program is executed, (2) the probability that the particular section affects the data state, and (3) the probability that a data state produced by that section has an effect on program output. These three characteristics can be used to predict whether faults are likely to be uncovered by software testing. Index Terms: Software testing, data state, fault, failure, testability. 1 Introduction

Voas, Jeffrey M.↗