Metamorphic testing of an automated parking system: an experience report

Dave Towey, Zepei Luo, Ziqi Zheng, Peijian Zhou, Junbo Yang, Puttipatt Ingkasit, Changyang Lao, Matthew Pike, Yifan Zhang

Research output: Chapter in Book/Conference proceedingConference contributionpeer-review


Automated Driving Systems (ADSs) have gained popularity recently. However, the unstable and unsafe ADSs have caused many traffic accidents and received widespread attention. One way to alleviate such issues is to enhance the correctness and efficiency of testing ADSs. Due to the difficulty of checking ADSs' behavior such as parking the car, confirming the correctness of the actual behavior may be non-trivial or impossible. This kind of problem is called the test oracle problem. Unlike traditional software testing, Metamorphic Testing (MT) does not focus on the correctness of the actual strategy but examines whether or not the inputs and outputs of multiple executions of a Software Under Test (SUT) satisfy certain relations of the SUT, called Metamorphic Relations (MRs). The paper also implements Mutation Analysis (MA) on Baidu Apollo ADS to evaluate our MT. MA involves small modifications to a program's source code to see if test-cases can detect these changes. This work was part of a larger endeavour to create an Open Educational Resource (OER) to support learning about how to apply MT to ADSs. This paper reports on an experience of implementing MT to test the Automated Parking System (APS) of Apollo ADS and applying MA to evaluate the MT.

Original languageEnglish
Title of host publicationProceedings - 2023 IEEE 47th Annual Computers, Software, and Applications Conference, COMPSAC 2023
EditorsHossain Shahriar, Yuuichi Teranishi, Alfredo Cuzzocrea, Moushumi Sharmin, Dave Towey, AKM Jahangir Alam Majumder, Hiroki Kashiwazaki, Ji-Jiang Yang, Michiharu Takemoto, Nazmus Sakib, Ryohei Banno, Sheikh Iqbal Ahamed
PublisherIEEE Computer Society
Number of pages6
ISBN (Electronic)9798350326970
Publication statusPublished - 2023
Event47th IEEE Annual Computers, Software, and Applications Conference, COMPSAC 2023 - Hybrid, Torino, Italy
Duration: 26 Jun 202330 Jun 2023

Publication series

NameProceedings - International Computer Software and Applications Conference
ISSN (Print)0730-3157


Conference47th IEEE Annual Computers, Software, and Applications Conference, COMPSAC 2023
CityHybrid, Torino


  • Automated Parking System
  • Baidu Apollo
  • Metamorphic Relation
  • Metamorphic Relation Output Pattern
  • Metamorphic Testing
  • Mutation Analysis
  • Test Oracle Problem

ASJC Scopus subject areas

  • Software
  • Computer Science Applications


Dive into the research topics of 'Metamorphic testing of an automated parking system: an experience report'. Together they form a unique fingerprint.

Cite this