Abstract
This paper presents a pilot study that examines the capacity of novice testers to generate Metamorphic Relations (MRs) for autonomous driving systems (ADSs), specifically fo-cusing on parking functions. By comparing MRs generated by human participants with those generated by artificial intelligence (AI), we seek to understand the variances in quality, particularly in terms of correctness, applicability, novelty, and utility. Our findings indicate that despite receiving only minimal training, human participants were capable of producing MRs with a wide range of effectiveness. Notably, humans exhibited a potential for creative thinking, contrasting with AI's ability to generate MRs that adhere closely to technical and applicability standards. The study underscores the need for improved educational strategies aimed at enhancing the quality and confidence of MRs produced by humans. Future research directions will explore the optimization of training approaches, particularly within a constrained timeframe to create a positive learning experience and maintain participant engagement, to fully harness the creative capabilities of human learners in the context of ADS testing.
Original language | English |
---|---|
Title of host publication | 2024 IEEE 48th Annual Computers, Software, and Applications Conference (COMPSAC) |
Publisher | IEEE |
Pages | 2393-2398 |
Number of pages | 6 |
ISBN (Electronic) | 9798350376968 |
ISBN (Print) | 9798350376975 |
DOIs | |
Publication status | Published - 2024 |
Keywords
- Metamorphic testing
- autonomous driving system
- metamorphic relation
- driving scenarios
- large language models
- artificial intelligence