Prioritizing random combinatorial test suites

Rubing Huang, Yunan Zhou, Tsong Yueh Chen, Dave Towey, Jinfu Chen, Weiwen Zong

Research output: Chapter in Book/Conference proceedingConference contributionpeer-review

Abstract

The behaviour of a system under test can be influenced by several factors, such as system configurations, user inputs, and so on. It has also been observed that many failures are caused by only a small number of factors.combinatorial testing aims at generating a small-sized test suite that can identify such failures. Random testing, on the other hand, selects test cases in a random manner, but has sometimes achieved similar failure-finding performance to combinatorial testing. Due to limited testing resources, test case prioritization is often employed, to attempt to identify failures as early as possible. Many prioritization studies, however, have only focused on test suites constructed by combinatorial testing, ignoring randomly constructed test suites. In this paper, we report on an investigation of test case prioritization for random combinatorial test suites. We conducted a series of empirical studies involving three real-world programs, each of which used twelve random combinatorial test suites, and adopted the well-known test case prioritization technique fixed-strength interaction coverage based prioritization (FICBP). Results from the studies indicate that, FICBP with high prioritization strength achieves better interaction coverage rate than with low prioritization strength. Furthermore, when the size of the random test suite is small, it would be better to assign a prioritization strength of 2; but otherwise, a strength of 3 would be more appropriate.

Original languageEnglish
Title of host publication32nd Annual ACM Symposium on Applied Computing, SAC 2017
PublisherAssociation for Computing Machinery
Pages1183-1189
Number of pages7
ISBN (Electronic)9781450344869
DOIs
Publication statusPublished - 3 Apr 2017
Event32nd Annual ACM Symposium on Applied Computing, SAC 2017 - Marrakesh, Morocco
Duration: 4 Apr 20176 Apr 2017

Publication series

NameProceedings of the ACM Symposium on Applied Computing
VolumePart F128005

Conference

Conference32nd Annual ACM Symposium on Applied Computing, SAC 2017
Country/TerritoryMorocco
CityMarrakesh
Period4/04/176/04/17

Keywords

  • Combinatorial testing
  • Random combinatorial test suite
  • Random testing
  • Software testing
  • Test case prioritization

ASJC Scopus subject areas

  • Software

Fingerprint

Dive into the research topics of 'Prioritizing random combinatorial test suites'. Together they form a unique fingerprint.

Cite this