Using metamorphic relation violation regions to support a simulation framework for the process of metamorphic testing

Zhihao Ying, Anthony Bellotti, Dave Towey, Tsong Yueh Chen, Zhi Quan Zhou

Research output: Chapter in Book/Conference proceedingConference contributionpeer-review

Abstract

Metamorphic testing (MT) has been growing in pop-ularity, but it can still be quite challenging and time-consuming to assess its performance. Typical approaches to performance assessment can require a series of steps, and depend on a variety of factors, often requiring serendipity. This can be a bottleneck for some aspects of MT research. Central to MT, metamorphic relations (MRs) represent necessary properties of the system under test (SUT). In traditional software testing, simulations are often employed to examine and compare the performance of dif-ferent testing strategies. However, these simulations are typically designed based on the assumed availability (and applicability) of a test oracle - a mechanism to decide the correctness of the SUT output or behaviour. A key reason for the popularity of MT is its proven record of effective software testing, without the need for a test oracle. This strength, however, also means that traditional ways of using simulations to analyse software testing approaches are not applicable for MT. This lack of cheap and fast ways to conduct simulation analyses of MT is a hurdle for many aspects of MT research, and may be an obstacle to its more widespread adoption. To address this, in this paper we introduce the concept of MR-violation regions (MRVRs), and show how they can be used for a certain category of MRs, Deterministic MRs (DMRs), to build simulation tools for MT. We analyse the differences between MRVRs and traditional, oracle-defined failure regions; and report on a preliminary case study exploring MRVRs in numerical-input-domain systems from previous MT studies. We anticipate that the proposed MT simulation framework may facilitate more research into MT, and may help lead to its more widespread adoption.
Original languageEnglish
Title of host publication2022 IEEE 46th Annual Computers, Software, and Applications Conference (COMPSAC)
PublisherIEEE
Pages1722-1727
Number of pages6
ISBN (Electronic)9781665488105
ISBN (Print)9781665488112
DOIs
Publication statusPublished - 10 Aug 2022
Event2022 46th Annual International Computer Software and Applications Conference -
Duration: 27 Jun 20221 Jul 2022
Conference number: 46
https://ieeecompsac.computer.org/2022/

Conference

Conference2022 46th Annual International Computer Software and Applications Conference
Abbreviated titleCOMPSAC
Period27/06/221/07/22
Internet address

Keywords

  • software quality assurance
  • oracle problem
  • metamorphic relations (MRs)
  • metamorphic relation violation region (MRVR)
  • metamorphic testing
  • Simulations

Fingerprint

Dive into the research topics of 'Using metamorphic relation violation regions to support a simulation framework for the process of metamorphic testing'. Together they form a unique fingerprint.

Cite this