5 Citations (Scopus)

Abstract

Complex engineered systems arise throughout computing, communications, and networking. Many factors, each having a finite number of levels, impact the behaviour of the system either singly or in interaction with one another. Testing or evaluating such a system involves formulating a set of tests, when executed, responses or outcomes from the tests are analyzed. A single round of testing is conducted. To witness the effect of an interaction, some test must cover it, this does not suffice in general to locate the interaction or to measure its effect. When there are few factors or many tests, experimental designs can measure (and hence locate) the interactions. When there are many factors and few tests, can we locate the interaction(s)? Can we efficiently detect them? Combinatorial arrays, locating and detecting arrays, are introduced to address such location and detection in the context of combinatorial testing. Locating and detecting arrays are contrasted with covering arrays and with experimental designs. An application to a 75 factor protocol stack for file transfer is given to demonstrate their practical use. Finally, their place in the literature of combinatorial testing is discussed and some directions are outlined.

Original languageEnglish (US)
Title of host publicationProceedings - 2016 IEEE International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2016
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages19-25
Number of pages7
ISBN (Electronic)9781509018260
DOIs
StatePublished - Aug 1 2016
Event9th IEEE International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2016 - Chicago, United States
Duration: Apr 10 2016Apr 15 2016

Other

Other9th IEEE International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2016
CountryUnited States
CityChicago
Period4/10/164/15/16

Fingerprint

Testing
Design of experiments
Large scale systems
Communication

Keywords

  • covering array
  • detecting array
  • experimental design
  • locating array

ASJC Scopus subject areas

  • Software
  • Safety, Risk, Reliability and Quality

Cite this

Colbourn, C., & Syrotiuk, V. (2016). Coverage, Location, Detection, and Measurement. In Proceedings - 2016 IEEE International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2016 (pp. 19-25). [7528941] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICSTW.2016.38

Coverage, Location, Detection, and Measurement. / Colbourn, Charles; Syrotiuk, Violet.

Proceedings - 2016 IEEE International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2016. Institute of Electrical and Electronics Engineers Inc., 2016. p. 19-25 7528941.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Colbourn, C & Syrotiuk, V 2016, Coverage, Location, Detection, and Measurement. in Proceedings - 2016 IEEE International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2016., 7528941, Institute of Electrical and Electronics Engineers Inc., pp. 19-25, 9th IEEE International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2016, Chicago, United States, 4/10/16. https://doi.org/10.1109/ICSTW.2016.38
Colbourn C, Syrotiuk V. Coverage, Location, Detection, and Measurement. In Proceedings - 2016 IEEE International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2016. Institute of Electrical and Electronics Engineers Inc. 2016. p. 19-25. 7528941 https://doi.org/10.1109/ICSTW.2016.38
Colbourn, Charles ; Syrotiuk, Violet. / Coverage, Location, Detection, and Measurement. Proceedings - 2016 IEEE International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2016. Institute of Electrical and Electronics Engineers Inc., 2016. pp. 19-25
@inproceedings{160f5af5e0fc461bb8f0af5d88739d00,
title = "Coverage, Location, Detection, and Measurement",
abstract = "Complex engineered systems arise throughout computing, communications, and networking. Many factors, each having a finite number of levels, impact the behaviour of the system either singly or in interaction with one another. Testing or evaluating such a system involves formulating a set of tests, when executed, responses or outcomes from the tests are analyzed. A single round of testing is conducted. To witness the effect of an interaction, some test must cover it, this does not suffice in general to locate the interaction or to measure its effect. When there are few factors or many tests, experimental designs can measure (and hence locate) the interactions. When there are many factors and few tests, can we locate the interaction(s)? Can we efficiently detect them? Combinatorial arrays, locating and detecting arrays, are introduced to address such location and detection in the context of combinatorial testing. Locating and detecting arrays are contrasted with covering arrays and with experimental designs. An application to a 75 factor protocol stack for file transfer is given to demonstrate their practical use. Finally, their place in the literature of combinatorial testing is discussed and some directions are outlined.",
keywords = "covering array, detecting array, experimental design, locating array",
author = "Charles Colbourn and Violet Syrotiuk",
year = "2016",
month = "8",
day = "1",
doi = "10.1109/ICSTW.2016.38",
language = "English (US)",
pages = "19--25",
booktitle = "Proceedings - 2016 IEEE International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2016",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
address = "United States",

}

TY - GEN

T1 - Coverage, Location, Detection, and Measurement

AU - Colbourn, Charles

AU - Syrotiuk, Violet

PY - 2016/8/1

Y1 - 2016/8/1

N2 - Complex engineered systems arise throughout computing, communications, and networking. Many factors, each having a finite number of levels, impact the behaviour of the system either singly or in interaction with one another. Testing or evaluating such a system involves formulating a set of tests, when executed, responses or outcomes from the tests are analyzed. A single round of testing is conducted. To witness the effect of an interaction, some test must cover it, this does not suffice in general to locate the interaction or to measure its effect. When there are few factors or many tests, experimental designs can measure (and hence locate) the interactions. When there are many factors and few tests, can we locate the interaction(s)? Can we efficiently detect them? Combinatorial arrays, locating and detecting arrays, are introduced to address such location and detection in the context of combinatorial testing. Locating and detecting arrays are contrasted with covering arrays and with experimental designs. An application to a 75 factor protocol stack for file transfer is given to demonstrate their practical use. Finally, their place in the literature of combinatorial testing is discussed and some directions are outlined.

AB - Complex engineered systems arise throughout computing, communications, and networking. Many factors, each having a finite number of levels, impact the behaviour of the system either singly or in interaction with one another. Testing or evaluating such a system involves formulating a set of tests, when executed, responses or outcomes from the tests are analyzed. A single round of testing is conducted. To witness the effect of an interaction, some test must cover it, this does not suffice in general to locate the interaction or to measure its effect. When there are few factors or many tests, experimental designs can measure (and hence locate) the interactions. When there are many factors and few tests, can we locate the interaction(s)? Can we efficiently detect them? Combinatorial arrays, locating and detecting arrays, are introduced to address such location and detection in the context of combinatorial testing. Locating and detecting arrays are contrasted with covering arrays and with experimental designs. An application to a 75 factor protocol stack for file transfer is given to demonstrate their practical use. Finally, their place in the literature of combinatorial testing is discussed and some directions are outlined.

KW - covering array

KW - detecting array

KW - experimental design

KW - locating array

UR - http://www.scopus.com/inward/record.url?scp=84992183966&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84992183966&partnerID=8YFLogxK

U2 - 10.1109/ICSTW.2016.38

DO - 10.1109/ICSTW.2016.38

M3 - Conference contribution

SP - 19

EP - 25

BT - Proceedings - 2016 IEEE International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2016

PB - Institute of Electrical and Electronics Engineers Inc.

ER -