A metamorphic testing approach for supporting program repair without the need for a test oracle

Mingyue Jiang, Tsong Yueh Chen, Fei Ching Kuo, Dave Towey, Zuohua Ding

Research output: Journal PublicationArticlepeer-review

33 Citations (Scopus)


Test suite based automated program repair (APR) relies on a test oracle to determine the execution result of individual test cases. The applicability of APR techniques, therefore, is limited by the fact that test oracles may not exist. Metamorphic Testing (MT) is a testing approach that, rather than checking the correctness of individual test outputs, checks testing results through verification of relations among multiple test cases and their outputs: MT can therefore be applied without test oracles. This paper presents an integration of MT with APR that enables application of APR without the need for a test oracle. Two important issues for this integration which have been thoroughly investigated and addressed are: (1) feasibility — which is addressed by proposing a framework to support the integration, and then presenting MT-GenProg, a tool incorporating MT with the popular APR technique GenProg; and (2) effectiveness — which is confirmed through an empirical study of GenProg and MT-GenProg on 1,143 program versions from the IntroClass benchmark suite, demonstrating MT-GenProg's comparable performance to GenProg, in terms of repair effectiveness. We conclude that the proposed integration is both practically feasible and effective, and thus successfully extends APR techniques to a broader application domain.

Original languageEnglish
Pages (from-to)127-140
Number of pages14
JournalJournal of Systems and Software
Publication statusPublished - 1 Apr 2017


  • Metamorphic testing
  • Test oracle
  • Test suite based automated program repair

ASJC Scopus subject areas

  • Software
  • Information Systems
  • Hardware and Architecture


Dive into the research topics of 'A metamorphic testing approach for supporting program repair without the need for a test oracle'. Together they form a unique fingerprint.

Cite this