Software Engineering for Self-Directed Learners »

Quality Assurance → Testing →

Test coverage

What

Can explain test coverage

Test coverage is a metric used to measure the extent to which testing exercises the code i.e., how much of the code is 'covered' by the tests.

Here are some examples of different coverage criteria:

  • Function/method coverage : based on functions executed e.g., testing executed 90 out of 100 functions.
  • Statement coverage : based on the number of lines of code executed e.g., testing executed 23k out of 25k LOC.
  • Decision/branch coverage : based on the decision points exercised e.g., an if statement evaluated to both true and false with separate test cases during testing is considered 'covered'.
  • Condition coverage : based on the boolean sub-expressions, each evaluated to both true and false with different test cases. Condition coverage is not the same as the decision coverage.

if(x > 2 && x < 44) is considered one decision point but two conditions.

For 100% branch or decision coverage, two test cases are required:

  • (x > 2 && x < 44) == true : [e.g. x == 4]
  • (x > 2 && x < 44) == false : [e.g. x == 100]

For 100% condition coverage, three test cases are required:

  • (x > 2) == true , (x < 44) == true : [e.g. x == 4] [see note 1]
  • (x < 44) == false : [e.g. x == 100]
  • (x > 2) == false : [e.g. x == 0]

Note 1: A case where both conditions are true is needed because most execution environments use a short circuiting behavior for compound boolean expressions e.g., given an expression c1 && c2, c2 will not be evaluated if c1 is false (as the final result is going to be false anyway).

  • Path coverage measures coverage in terms of possible paths through a given part of the code executed. 100% path coverage means all possible paths have been executed. A commonly used notation for path analysis is called the Control Flow Graph (CFG).
  • Entry/exit coverage measures coverage in terms of possible calls to and exits from the operations in the SUT.
    Entry points refer to all places from which the method is called from the rest of the code i.e., all places where the control is handed over to the method in concern.
    Exit points refer to points at which the control is returned to the caller e.g., return statements, throwing of exceptions.

Exercises



How

Can explain how test coverage works

Measuring coverage is often done using coverage analysis tools. Most IDEs have inbuilt support for measuring test coverage, or at least have plugins that can measure test coverage.

Coverage analysis can be useful in improving the quality of testing e.g., if a set of test cases does not achieve 100% branch coverage, more test cases can be added to cover missed branches.

Measuring code coverage in Intellij IDEA (watch from 4 minutes 50 seconds mark)