TY - JOUR
T1 - A metamorphic testing approach for supporting program repair without the need for a test oracle
AU - Jiang, Mingyue
AU - Chen, Tsong Yueh
AU - Kuo, Fei Ching
AU - Towey, Dave
AU - Ding, Zuohua
N1 - Publisher Copyright:
© 2016
Copyright:
Copyright 2020 Elsevier B.V., All rights reserved.
PY - 2017/4/1
Y1 - 2017/4/1
N2 - Test suite based automated program repair (APR) relies on a test oracle to determine the execution result of individual test cases. The applicability of APR techniques, therefore, is limited by the fact that test oracles may not exist. Metamorphic Testing (MT) is a testing approach that, rather than checking the correctness of individual test outputs, checks testing results through verification of relations among multiple test cases and their outputs: MT can therefore be applied without test oracles. This paper presents an integration of MT with APR that enables application of APR without the need for a test oracle. Two important issues for this integration which have been thoroughly investigated and addressed are: (1) feasibility — which is addressed by proposing a framework to support the integration, and then presenting MT-GenProg, a tool incorporating MT with the popular APR technique GenProg; and (2) effectiveness — which is confirmed through an empirical study of GenProg and MT-GenProg on 1,143 program versions from the IntroClass benchmark suite, demonstrating MT-GenProg's comparable performance to GenProg, in terms of repair effectiveness. We conclude that the proposed integration is both practically feasible and effective, and thus successfully extends APR techniques to a broader application domain.
AB - Test suite based automated program repair (APR) relies on a test oracle to determine the execution result of individual test cases. The applicability of APR techniques, therefore, is limited by the fact that test oracles may not exist. Metamorphic Testing (MT) is a testing approach that, rather than checking the correctness of individual test outputs, checks testing results through verification of relations among multiple test cases and their outputs: MT can therefore be applied without test oracles. This paper presents an integration of MT with APR that enables application of APR without the need for a test oracle. Two important issues for this integration which have been thoroughly investigated and addressed are: (1) feasibility — which is addressed by proposing a framework to support the integration, and then presenting MT-GenProg, a tool incorporating MT with the popular APR technique GenProg; and (2) effectiveness — which is confirmed through an empirical study of GenProg and MT-GenProg on 1,143 program versions from the IntroClass benchmark suite, demonstrating MT-GenProg's comparable performance to GenProg, in terms of repair effectiveness. We conclude that the proposed integration is both practically feasible and effective, and thus successfully extends APR techniques to a broader application domain.
KW - Metamorphic testing
KW - Test oracle
KW - Test suite based automated program repair
UR - http://www.scopus.com/inward/record.url?scp=84966376822&partnerID=8YFLogxK
U2 - 10.1016/j.jss.2016.04.002
DO - 10.1016/j.jss.2016.04.002
M3 - Article
AN - SCOPUS:84966376822
SN - 0164-1212
VL - 126
SP - 127
EP - 140
JO - Journal of Systems and Software
JF - Journal of Systems and Software
ER -