Covering array constructors: An experimental analysis of their interaction coverage and fault detection

Rubing Huang, Haibo Chen, Yunan Zhou, Tsong Yueh Chen, Dave Towey, Man Fai Lau, Sebastian Ng, Robert Merkel, Jinfu Chen

Research output: Journal PublicationArticlepeer-review

1 Citation (Scopus)

Abstract

Combinatorial interaction testing (CIT) aims at constructing a covering array (CA) of all value combinations at a specific interaction strength, to detect faults that are caused by the interaction of parameters. CIT has been widely used in different applications, with many algorithms and tools having been proposed to support CA construction. To date, however, there appears to have been no studies comparing different CA constructors when only some of the CA test cases are executed. In this paper, we present an investigation of five popular CA constructors: ACTS, Jenny, PICT, CASA and TCA. We conducted empirical studies examining the five programs, focusing on interaction coverage and fault detection. The experimental results show that when there is no preference or special justification for using other CA constructors, then Jenny is recommended - because it achieves better interaction coverage and fault detection than the other four constructors in many cases. Our results also show that when using ACTS or CASA, their CAs must be prioritized before testing. The main reason for this is that these CAs can result in considerable interaction coverage or fault detection capabilities when executing a large number of test cases; however, they may also produce the lowest rates of fault detection and interaction coverage.

Original languageEnglish
Pages (from-to)762-788
Number of pages27
JournalComputer Journal
Volume64
Issue number5
DOIs
Publication statusPublished - 1 May 2021

Keywords

  • combinatorial interaction testing
  • constructor
  • covering array
  • empirical study
  • software testing

ASJC Scopus subject areas

  • General Computer Science

Fingerprint

Dive into the research topics of 'Covering array constructors: An experimental analysis of their interaction coverage and fault detection'. Together they form a unique fingerprint.

Cite this