Deep Learning Approach for Enhanced Object Recognition and Assembly Guidance with Augmented Reality

Boon Giin Lee, Xiaoying Wang, Renzhi Han, Linjing Sun, Matthew Pike, Wan-Young Chung

Research output: Chapter in Book/Conference proceedingBook Chapterpeer-review


In an effort to enhance the efficiency and precision of manual part assembly in industrial settings, the development of software for assembly guidance becomes imperative. Augmented reality (AR) technology offers a means to provide visual instructions for assembly tasks, rendering the guidance more comprehensible. Nevertheless, a significant challenge lies in the technology’s limited object detection capabilities, especially when distinguishing between similar assembled parts. This project proposes the utilization of deep learning neural networks to enhance the accuracy of object recognition within the AR guided assembly application. To achieve this objective, a dataset of assembly parts, known as the Visual Object Classes (VOC) dataset, was created. Data augmentation techniques were employed to expand this dataset, incorporating scale HSV (hue saturation value) transformations. Subsequently, deep learning models for the recognition of assembly parts were developed which were based on the Single Shot Multibox Detector (SSD) and the YOLOv7 detector. The models were trained and fine-tuned, targeting on the variations of the positions of detected parts. The effectiveness of this approach was evaluated using a case study involving an educational electronic blocks circuit science kit. The results demonstrated a high assembly part recognition accuracy of over 99% in mean average precision (MAP), along with favorable user testing outcomes. Consequently, the AR application was capable of offering high-quality guidance to users which holds promise for application in diverse scenarios and the resolution of real-world challenges.
Original languageEnglish
Title of host publicationIntelligent Human Computer Interaction
Subtitle of host publication15th International Conference, IHCI 2023, Daegu, South Korea, November 8–10, 2023, Revised Selected Papers, Part II
EditorsBong Jun Choi, Dhananjay Singh, Uma Shanker Tiwary, Wan-Young Chung
PublisherSpringer, Cham
ISBN (Electronic)9783031538308
ISBN (Print)9783031538292
Publication statusPublished - 29 Feb 2024

Publication series

Name Lecture Notes in Computer Science
Volumevolume 14532


  • Augmented Reality
  • Assembly Tasks
  • Object Detection
  • Object Recognition


Dive into the research topics of 'Deep Learning Approach for Enhanced Object Recognition and Assembly Guidance with Augmented Reality'. Together they form a unique fingerprint.

Cite this