Balancing Frequencies and Fault Detection in the In-Parameter-Order Algorithm

Shi Wei Gao, Jiang Hua Lv, Bing Lei Du, Charles Colbourn, Shi Long Ma

Research output: Contribution to journalArticle

6 Citations (Scopus)

Abstract

The In-Parameter-Order (IPO) algorithm is a widely used strategy for the construction of software test suites for combinatorial testing (CT) whose goal is to reveal faults triggered by interactions among parameters. Variants of IPO have been shown to produce test suites within reasonable amounts of time that are often not much larger than the smallest test suites known. When an entire test suite is executed, all faults that arise from t-way interactions for some fixed t are surely found. However, when tests are executed one at a time, it is desirable to detect a fault as early as possible so that it can be repaired. The basic IPO strategies of horizontal and vertical growth address test suite size, but not the early detection of faults. In this paper, the growth strategies in IPO are modified to attempt to evenly distribute the values of each parameter across the tests. Together with a reordering strategy that we add, this modification to IPO improves the rate of fault detection dramatically (improved by 31% on average). Moreover, our modifications always reduce generation time (2 times faster on average) and in some cases also reduce test suite size.

Original languageEnglish (US)
Pages (from-to)957-968
Number of pages12
JournalJournal of Computer Science and Technology
Volume30
Issue number5
DOIs
StatePublished - Sep 22 2015

Fingerprint

Fault Detection
Fault detection
Balancing
Order Parameter
Fault
Testing
Reordering
Interaction
Horizontal
Vertical
Entire
Software
Strategy

Keywords

  • combinatorial testing
  • expected time to fault detection
  • IPO
  • software under test
  • test suite generation

ASJC Scopus subject areas

  • Hardware and Architecture
  • Software
  • Computational Theory and Mathematics
  • Theoretical Computer Science
  • Computer Science Applications

Cite this

Balancing Frequencies and Fault Detection in the In-Parameter-Order Algorithm. / Gao, Shi Wei; Lv, Jiang Hua; Du, Bing Lei; Colbourn, Charles; Ma, Shi Long.

In: Journal of Computer Science and Technology, Vol. 30, No. 5, 22.09.2015, p. 957-968.

Research output: Contribution to journalArticle

Gao, Shi Wei ; Lv, Jiang Hua ; Du, Bing Lei ; Colbourn, Charles ; Ma, Shi Long. / Balancing Frequencies and Fault Detection in the In-Parameter-Order Algorithm. In: Journal of Computer Science and Technology. 2015 ; Vol. 30, No. 5. pp. 957-968.
@article{94a600f01f564e4f8eca0f234da27d9b,
title = "Balancing Frequencies and Fault Detection in the In-Parameter-Order Algorithm",
abstract = "The In-Parameter-Order (IPO) algorithm is a widely used strategy for the construction of software test suites for combinatorial testing (CT) whose goal is to reveal faults triggered by interactions among parameters. Variants of IPO have been shown to produce test suites within reasonable amounts of time that are often not much larger than the smallest test suites known. When an entire test suite is executed, all faults that arise from t-way interactions for some fixed t are surely found. However, when tests are executed one at a time, it is desirable to detect a fault as early as possible so that it can be repaired. The basic IPO strategies of horizontal and vertical growth address test suite size, but not the early detection of faults. In this paper, the growth strategies in IPO are modified to attempt to evenly distribute the values of each parameter across the tests. Together with a reordering strategy that we add, this modification to IPO improves the rate of fault detection dramatically (improved by 31{\%} on average). Moreover, our modifications always reduce generation time (2 times faster on average) and in some cases also reduce test suite size.",
keywords = "combinatorial testing, expected time to fault detection, IPO, software under test, test suite generation",
author = "Gao, {Shi Wei} and Lv, {Jiang Hua} and Du, {Bing Lei} and Charles Colbourn and Ma, {Shi Long}",
year = "2015",
month = "9",
day = "22",
doi = "10.1007/s11390-015-1574-6",
language = "English (US)",
volume = "30",
pages = "957--968",
journal = "Journal of Computer Science and Technology",
issn = "1000-9000",
publisher = "Springer New York",
number = "5",

}

TY - JOUR

T1 - Balancing Frequencies and Fault Detection in the In-Parameter-Order Algorithm

AU - Gao, Shi Wei

AU - Lv, Jiang Hua

AU - Du, Bing Lei

AU - Colbourn, Charles

AU - Ma, Shi Long

PY - 2015/9/22

Y1 - 2015/9/22

N2 - The In-Parameter-Order (IPO) algorithm is a widely used strategy for the construction of software test suites for combinatorial testing (CT) whose goal is to reveal faults triggered by interactions among parameters. Variants of IPO have been shown to produce test suites within reasonable amounts of time that are often not much larger than the smallest test suites known. When an entire test suite is executed, all faults that arise from t-way interactions for some fixed t are surely found. However, when tests are executed one at a time, it is desirable to detect a fault as early as possible so that it can be repaired. The basic IPO strategies of horizontal and vertical growth address test suite size, but not the early detection of faults. In this paper, the growth strategies in IPO are modified to attempt to evenly distribute the values of each parameter across the tests. Together with a reordering strategy that we add, this modification to IPO improves the rate of fault detection dramatically (improved by 31% on average). Moreover, our modifications always reduce generation time (2 times faster on average) and in some cases also reduce test suite size.

AB - The In-Parameter-Order (IPO) algorithm is a widely used strategy for the construction of software test suites for combinatorial testing (CT) whose goal is to reveal faults triggered by interactions among parameters. Variants of IPO have been shown to produce test suites within reasonable amounts of time that are often not much larger than the smallest test suites known. When an entire test suite is executed, all faults that arise from t-way interactions for some fixed t are surely found. However, when tests are executed one at a time, it is desirable to detect a fault as early as possible so that it can be repaired. The basic IPO strategies of horizontal and vertical growth address test suite size, but not the early detection of faults. In this paper, the growth strategies in IPO are modified to attempt to evenly distribute the values of each parameter across the tests. Together with a reordering strategy that we add, this modification to IPO improves the rate of fault detection dramatically (improved by 31% on average). Moreover, our modifications always reduce generation time (2 times faster on average) and in some cases also reduce test suite size.

KW - combinatorial testing

KW - expected time to fault detection

KW - IPO

KW - software under test

KW - test suite generation

UR - http://www.scopus.com/inward/record.url?scp=84942017367&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84942017367&partnerID=8YFLogxK

U2 - 10.1007/s11390-015-1574-6

DO - 10.1007/s11390-015-1574-6

M3 - Article

VL - 30

SP - 957

EP - 968

JO - Journal of Computer Science and Technology

JF - Journal of Computer Science and Technology

SN - 1000-9000

IS - 5

ER -