Expected time to detection of interaction faults

Software interaction test suites serve two complementary roles. They are employed to systematically verify that, for some strength t, no t-way interaction of a system's parameters causes a fault. They are also employed to locate a faulty configuration, when at least one interaction fault remains. Algorithms to find such test suites employing a number of tests close to the minimum have been extensively explored, in order to test all t-way interactions. However, when faults remain, the expected number of tests needed to reveal an interaction fault is also important. One might anticipate that the test suites of minimum size also have the lowest expected time to detection of an interaction fault; or, at the very least, that some test suite of minimum size does. However, in this paper it is shown that minimum test suite size and lowest expected time to fault detection are incompatible objectives. This underlies a challenging problem of how to generate test suites that have early coverage of t-way interactions, in order to reduce time to fault detection. A hybrid approach is developed that combines a simple greedy algorithm with heuristic search to construct one test at a time while attempting to maximize the number of t-way interactions covered by the earliest tests.