Metamorphic testing of Advanced Driver-Assistance System (ADAS) simulation platforms: Lane Keeping Assist System (LKAS) case studies

Muhammad Iqbal, Jia Cheng Han, Zhi Quan Zhou, Dave Towey, Tsong Yueh Chen

Research output: Journal PublicationArticlepeer-review

4 Citations (Scopus)

Abstract

Context: Simulation-based testing is essential when developing Advanced Driver-Assistance Systems (ADASs) and autonomous driving (AD) systems, producing fast, high-quality test results, at relatively low cost. Simulation testing relies on the quality of the ADAS simulation platform: If the simulation platform is faulty, then the simulation results may be incorrect, and hence useless. However, because of the lack of suitable test oracles — mechanisms to determine the correctness of the software output or behavior — it can be too difficult (or expensive) to verify or validate ADAS/AD simulation platforms, a situation known as the oracle problem. Objective: To alleviate the oracle problem and better understand ADAS simulation software. Methods: We develop geometric-transformation-based metamorphic testing approaches, and report on empirical studies conducted on the verification and validation (V&V) of three popular simulation platforms for ADAS development: Simulink, CarMaker and 51Sim-One Cloud. Our examination focused on the platforms’ Lane Keeping Assist Systems (LKASs). Results: When tested with ordinary (traditional) test cases, no issues were identified on any simulation platform. However, after applying geometric-transformation-based metamorphic testing, issues were revealed, some of which were later confirmed by the MATLAB and IPG Automotive teams. To the best of our knowledge, this paper is the first to report on real bugs and issues in ADAS simulation platforms. Conclusion: Our research shows the simplicity, effectiveness and applicability of the proposed approach for ADAS simulation testing. This paper also provides successful examples of incorporating metamorphic testing into the testing of ADAS standards and protocols, and shows how practitioners can design effective metamorphic relations (MRs) inspired by using the symmetry metamorphic relation pattern.

Original languageEnglish
Article number107104
JournalInformation and Software Technology
Volume155
DOIs
Publication statusPublished - Mar 2023

Keywords

  • Advanced Driver-Assistance System (ADAS)
  • Lane Keeping Assist System (LKAS)
  • Metamorphic exploration
  • Metamorphic testing
  • Simulation platform
  • Software testing

ASJC Scopus subject areas

  • Software
  • Information Systems
  • Computer Science Applications

Fingerprint

Dive into the research topics of 'Metamorphic testing of Advanced Driver-Assistance System (ADAS) simulation platforms: Lane Keeping Assist System (LKAS) case studies'. Together they form a unique fingerprint.

Cite this