TY - GEN
T1 - Metamorphic testing harness for the Baidu Apollo perception-camera module
AU - Zhang, Yifan
AU - Towey, Dave
AU - Pike, Matthew
AU - Han, Jia Cheng
AU - Zhou, George
AU - Yin, Chenghao
AU - Wang, Qian
AU - Xie, Chen
N1 - Funding Information:
ACKNOWLEDGMENTS This work is supported by the Natural Science Foundation of China (project no. 61872167). The authors also acknowledge the financial support from the Artificial Intelligence and Optimisation (AIOP) and Sensors, Sensor Networks and Instrumentation (SSNI) research groups, the Faculty of Science and Engineering (FoSE), the International Doctoral Innovation Centre, Ningbo Education Bureau, Ningbo Science and Technology Bureau, and the University of Nottingham.
Publisher Copyright:
© 2023 IEEE.
PY - 2023
Y1 - 2023
N2 - As the complexity of autonomous driving systems (ADSs) increases, the question of how to organize testing in an efficient manner has become a serious issue. This study investigates the potential for metamorphic testing (MT) to evaluate the perception-camera module of an open-source autonomous driving system (ADS), namely Baidu's Apollo ADS. The experiments revealed inconsistent obstacle identification results when increasing the brightness of a specific region of the driving scenarios, both in individual and sequential frames, demonstrating the ability of MT to address the oracle problem when testing the perception module of ADSs. Furthermore, this paper presents an MT harness to facilitate ADS testing, which would increase efficiency and help testers to better organize the testing procedure. We also present an industry case study to demonstrate its use in actual production phases.
AB - As the complexity of autonomous driving systems (ADSs) increases, the question of how to organize testing in an efficient manner has become a serious issue. This study investigates the potential for metamorphic testing (MT) to evaluate the perception-camera module of an open-source autonomous driving system (ADS), namely Baidu's Apollo ADS. The experiments revealed inconsistent obstacle identification results when increasing the brightness of a specific region of the driving scenarios, both in individual and sequential frames, demonstrating the ability of MT to address the oracle problem when testing the perception module of ADSs. Furthermore, this paper presents an MT harness to facilitate ADS testing, which would increase efficiency and help testers to better organize the testing procedure. We also present an industry case study to demonstrate its use in actual production phases.
KW - automated testing
KW - Autonomous driving system (ADS)
KW - metamorphic relation (MR)
KW - metamorphic testing (MT)
KW - oracle problem
UR - http://www.scopus.com/inward/record.url?scp=85169102520&partnerID=8YFLogxK
U2 - 10.1109/MET59151.2023.00009
DO - 10.1109/MET59151.2023.00009
M3 - Conference contribution
AN - SCOPUS:85169102520
T3 - Proceedings - 2023 IEEE/ACM 8th International Workshop on Metamorphic Testing, MET 2023
SP - 9
EP - 16
BT - Proceedings - 2023 IEEE/ACM 8th International Workshop on Metamorphic Testing, MET 2023
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 8th IEEE/ACM International Workshop on Metamorphic Testing, MET 2023
Y2 - 14 May 2023
ER -