Using Metamorphic Relation Violation Regions to Support a Simulation Framework for the Process of Metamorphic Testing

Zhihao Ying, Anthony Bellotti, Dave Towey, Tsong Yueh Chen, Zhi Quan Zhou

Research output: Chapter in Book/Conference proceedingConference contributionpeer-review

Abstract

Metamorphic testing (MT) has been growing in pop-ularity, but it can still be quite challenging and time-consuming to assess its performance. Typical approaches to performance assessment can require a series of steps, and depend on a variety of factors, often requiring serendipity. This can be a bottleneck for some aspects of MT research. Central to MT, metamorphic relations (MRs) represent necessary properties of the system under test (SUT). In traditional software testing, simulations are often employed to examine and compare the performance of dif-ferent testing strategies. However, these simulations are typically designed based on the assumed availability (and applicability) of a test oracle - a mechanism to decide the correctness of the SUT output or behaviour. A key reason for the popularity of MT is its proven record of effective software testing, without the need for a test oracle. This strength, however, also means that traditional ways of using simulations to analyse software testing approaches are not applicable for MT. This lack of cheap and fast ways to conduct simulation analyses of MT is a hurdle for many aspects of MT research, and may be an obstacle to its more widespread adoption. To address this, in this paper we introduce the concept of MR-violation regions (MRVRs), and show how they can be used for a certain category of MRs, Deterministic MRs (DMRs), to build simulation tools for MT. We analyse the differences between MRVRs and traditional, oracle-defined failure regions; and report on a preliminary case study exploring MRVRs in numerical-input-domain systems from previous MT studies. We anticipate that the proposed MT simulation framework may facilitate more research into MT, and may help lead to its more widespread adoption.

Original languageEnglish
Title of host publicationProceedings - 2022 IEEE 46th Annual Computers, Software, and Applications Conference, COMPSAC 2022
EditorsHong Va Leong, Sahra Sedigh Sarvestani, Yuuichi Teranishi, Alfredo Cuzzocrea, Hiroki Kashiwazaki, Dave Towey, Ji-Jiang Yang, Hossain Shahriar
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1722-1727
Number of pages6
ISBN (Electronic)9781665488105
DOIs
Publication statusPublished - 2022
Event46th IEEE Annual Computers, Software, and Applications Conference, COMPSAC 2022 - Virtual, Online, United States
Duration: 27 Jun 20221 Jul 2022

Publication series

NameProceedings - 2022 IEEE 46th Annual Computers, Software, and Applications Conference, COMPSAC 2022

Conference

Conference46th IEEE Annual Computers, Software, and Applications Conference, COMPSAC 2022
Country/TerritoryUnited States
CityVirtual, Online
Period27/06/221/07/22

Keywords

  • metamorphic relation violation region (MRVR)
  • metamorphic relations (MRs)
  • metamorphic testing
  • oracle problem
  • simulations
  • software quality assurance

ASJC Scopus subject areas

  • Computer Science Applications
  • Hardware and Architecture
  • Software
  • Media Technology
  • Education

Fingerprint

Dive into the research topics of 'Using Metamorphic Relation Violation Regions to Support a Simulation Framework for the Process of Metamorphic Testing'. Together they form a unique fingerprint.

Cite this