Scenario-Driven Metamorphic Testing for Autonomous Driving Simulators

Yifan Zhang, Dave Towey, Matthew Pike, Jia Cheng Han, Zhi Quan Zhou, Chenghao Yin, Qian Wang, Chen Xie

Research output: Journal PublicationArticlepeer-review

Abstract

The proliferation of driver-assistance features in vehicles has resulted in a growing interest among the public in fully autonomous driving systems (ADSs). However, the integration of software and hardware in these complex systems presents significant testing challenges, particularly with respect to ensuring passenger safety. To address these challenges, simulation has emerged as a crucial step in the testing of ADSs. This paper presents a solution to the challenges faced in testing ADSs, with a focus on the validation of ADS simulators. The proposed approach involves using simulations and metamorphic testing (MT) to generate multiple concrete metamorphic relations (MRs) for testing ADS simulators. In order to accomplish this goal, we introduce three metamorphic relation patterns (MRPs). Each MRP is accompanied by a metamorphic relation input pattern (MRIP) that aids in generating detailed MRs. These MRs are designed to identify potential issues within the ADS simulator. To simplify the testing process and facilitate MT for testers, a self-evolving scenario-testing framework is also presented. The framework allows testers to improve test cases and MRs iteratively until issues detected are confirmed. The benefits and limitations of the framework are demonstrated using an industry case study. Overall, this study offers a practical solution to the challenges in testing ADSs and provides useful insights into improving testing efficiency for researchers and practitioners in the field.

Original languageEnglish
JournalSoftware Testing Verification and Reliability
DOIs
Publication statusAccepted/In press - 2024

Keywords

  • automated testing
  • autonomous driving system (ADS)
  • metamorphic exploration (ME)
  • metamorphic relation input pattern (MRIP)
  • metamorphic relation pattern (MRP)
  • metamorphic testing (MT)
  • oracle problem
  • simulation

ASJC Scopus subject areas

  • Software
  • Safety, Risk, Reliability and Quality

Fingerprint

Dive into the research topics of 'Scenario-Driven Metamorphic Testing for Autonomous Driving Simulators'. Together they form a unique fingerprint.

Cite this