Software testing and inspection

Software testing and inspection

Procedures for the detection of software faults. When software does not operate as it is intended to do, a software failure is said to occur. Software failures are caused by one or more sections of the software program being incorrect. Each of these incorrect sections is called a software fault. The fault could be as simple as a wrong value. A fault could also be complete omission of a decision in the program. Faults have many causes, including misunderstanding of requirements, overlooking special cases, using the wrong variable, misunderstanding of the algorithm, and even typing mistakes. Software that can cause serious problems if it fails is called safety-critical software. Many applications in aircraft, medicine, nuclear power plants, and transportation involve such software.

Software testing is the execution of the software with the purpose of detecting faults. Software inspection is a manual process of analyzing the source code to detect faults. Many of the same techniques are used in both procedures. Other techniques can also be used to minimize the possibility of faults in the software. These techniques include the use of formal specifications, formal proofs of correctness, and model checking. However, even with the use of these techniques, it is still important to execute software with test cases to detect possible faults.

Software testing involves selecting test cases, determining the correct output of the software, executing the software with each test case, and comparing the actual output with the expected output. More testing is better, but costs time and effort. The value of additional testing must be balanced against the additional cost of effort and the delay in delivering the software. Another consideration is the potential cost of failure of the software. Safety-critical software is usually tested much more thoroughly than any other software.

The number of potential test cases is huge. For example, in the case of a simple program that multiplies two integer numbers, if each integer is internally represented as a 32-bit number (a common size for the internal representation), then there are 232 possible values for each number. Thus, the total number of possible input combinations is 264, which is more than 1019. If a test case can be done each microsecond (10-6 second), then it will take hundreds of thousands of years to try all of the possible test cases. Trying all possible test cases is called exhaustive testing and is usually not a reasonable approach because of the size of the task.

One approach to software testing is to find test cases so that all statements in a program are executed. A more extensive criterion for test selection is “every branch coverage.” This means that each branch coming out of every decision is tested. Instead of just requiring the whole decision to be true or false, the “multiple condition coverage” criterion requires all combinations of truth values for each simple comparison in a decision to be covered. Another approach is called dataflow coverage. The basis for coverage is the execution paths between the statement where a variable is assigned a value (a def or definition) and a statement where that value is used. These paths must be free of other definitions of the variable of interest. See Dataflow systems

Functional testing compares the actual behavior of the software with the expected behavior. That expected behavior is usually described in a specification. More involved functional test case selection involves analyzing the conditions inherent in the task.

Another approach to test selection concentrates on the boundaries between the subdomains. This approach recognizes that many faults are related to the boundary conditions. Test cases are chosen to check whether the boundary is correct. Test cases on the boundary and test cases just off the boundary are chosen. See Software, Software engineering