Constructing test suites for interaction testing

Myra B. Cohen, Peter B. Gibbons, Warwick B. Mugridge, Charles Colbourn

Research output: Chapter in Book/Report/Conference proceedingConference contribution

260 Citations (Scopus)

Abstract

Software system faults are often caused by unexpected interactions among components. Yet the size of a test suite required to test all possible combinations of interactions can be prohibitive in even a moderately sized project. Instead, we may use pairwise or t-way testing to provide a guarantee that all pairs or t-way combinations of components are tested together. This concept draws on methods used in statistical testing for manufacturing and has been extended to software system testing. A covering array, CA(N; t, k, v), is an N × k array on v symbols such that every N × t sub-array contains all ordered subsets from v symbols of size t at least once. The properties of these objects, however, do not necessarily satisfy real software testing needs. Instead we examine a less studied object, the mixed level covering array and propose a new object, the variable strength covering array, which provides a more robust environment for software interaction testing. Initial results are presented suggesting that heuristic search techniques are more effective than some of the known greedy methods for finding smaller sized test suites. We present a discussion of an integrated approach for finding covering arrays and discuss how application of these techniques can be used to construct variable strength arrays.

Original languageEnglish (US)
Title of host publicationProceedings - International Conference on Software Engineering
Pages38-48
Number of pages11
StatePublished - 2003
Event25th International Conference on Software Engineering - Portland, OR, United States
Duration: May 3 2003May 10 2003

Other

Other25th International Conference on Software Engineering
CountryUnited States
CityPortland, OR
Period5/3/035/10/03

Fingerprint

Testing
Software testing

ASJC Scopus subject areas

  • Software

Cite this

Cohen, M. B., Gibbons, P. B., Mugridge, W. B., & Colbourn, C. (2003). Constructing test suites for interaction testing. In Proceedings - International Conference on Software Engineering (pp. 38-48)

Constructing test suites for interaction testing. / Cohen, Myra B.; Gibbons, Peter B.; Mugridge, Warwick B.; Colbourn, Charles.

Proceedings - International Conference on Software Engineering. 2003. p. 38-48.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Cohen, MB, Gibbons, PB, Mugridge, WB & Colbourn, C 2003, Constructing test suites for interaction testing. in Proceedings - International Conference on Software Engineering. pp. 38-48, 25th International Conference on Software Engineering, Portland, OR, United States, 5/3/03.
Cohen MB, Gibbons PB, Mugridge WB, Colbourn C. Constructing test suites for interaction testing. In Proceedings - International Conference on Software Engineering. 2003. p. 38-48
Cohen, Myra B. ; Gibbons, Peter B. ; Mugridge, Warwick B. ; Colbourn, Charles. / Constructing test suites for interaction testing. Proceedings - International Conference on Software Engineering. 2003. pp. 38-48
@inproceedings{5d3cc145d8804af184f45e3c5db57f4b,
title = "Constructing test suites for interaction testing",
abstract = "Software system faults are often caused by unexpected interactions among components. Yet the size of a test suite required to test all possible combinations of interactions can be prohibitive in even a moderately sized project. Instead, we may use pairwise or t-way testing to provide a guarantee that all pairs or t-way combinations of components are tested together. This concept draws on methods used in statistical testing for manufacturing and has been extended to software system testing. A covering array, CA(N; t, k, v), is an N × k array on v symbols such that every N × t sub-array contains all ordered subsets from v symbols of size t at least once. The properties of these objects, however, do not necessarily satisfy real software testing needs. Instead we examine a less studied object, the mixed level covering array and propose a new object, the variable strength covering array, which provides a more robust environment for software interaction testing. Initial results are presented suggesting that heuristic search techniques are more effective than some of the known greedy methods for finding smaller sized test suites. We present a discussion of an integrated approach for finding covering arrays and discuss how application of these techniques can be used to construct variable strength arrays.",
author = "Cohen, {Myra B.} and Gibbons, {Peter B.} and Mugridge, {Warwick B.} and Charles Colbourn",
year = "2003",
language = "English (US)",
pages = "38--48",
booktitle = "Proceedings - International Conference on Software Engineering",

}

TY - GEN

T1 - Constructing test suites for interaction testing

AU - Cohen, Myra B.

AU - Gibbons, Peter B.

AU - Mugridge, Warwick B.

AU - Colbourn, Charles

PY - 2003

Y1 - 2003

N2 - Software system faults are often caused by unexpected interactions among components. Yet the size of a test suite required to test all possible combinations of interactions can be prohibitive in even a moderately sized project. Instead, we may use pairwise or t-way testing to provide a guarantee that all pairs or t-way combinations of components are tested together. This concept draws on methods used in statistical testing for manufacturing and has been extended to software system testing. A covering array, CA(N; t, k, v), is an N × k array on v symbols such that every N × t sub-array contains all ordered subsets from v symbols of size t at least once. The properties of these objects, however, do not necessarily satisfy real software testing needs. Instead we examine a less studied object, the mixed level covering array and propose a new object, the variable strength covering array, which provides a more robust environment for software interaction testing. Initial results are presented suggesting that heuristic search techniques are more effective than some of the known greedy methods for finding smaller sized test suites. We present a discussion of an integrated approach for finding covering arrays and discuss how application of these techniques can be used to construct variable strength arrays.

AB - Software system faults are often caused by unexpected interactions among components. Yet the size of a test suite required to test all possible combinations of interactions can be prohibitive in even a moderately sized project. Instead, we may use pairwise or t-way testing to provide a guarantee that all pairs or t-way combinations of components are tested together. This concept draws on methods used in statistical testing for manufacturing and has been extended to software system testing. A covering array, CA(N; t, k, v), is an N × k array on v symbols such that every N × t sub-array contains all ordered subsets from v symbols of size t at least once. The properties of these objects, however, do not necessarily satisfy real software testing needs. Instead we examine a less studied object, the mixed level covering array and propose a new object, the variable strength covering array, which provides a more robust environment for software interaction testing. Initial results are presented suggesting that heuristic search techniques are more effective than some of the known greedy methods for finding smaller sized test suites. We present a discussion of an integrated approach for finding covering arrays and discuss how application of these techniques can be used to construct variable strength arrays.

UR - http://www.scopus.com/inward/record.url?scp=0037587652&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0037587652&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:0037587652

SP - 38

EP - 48

BT - Proceedings - International Conference on Software Engineering

ER -