SDViT: Stacking of Distilled Vision Transformers for Hand Gesture Recognition

Chun Keat Tan, Kian Ming Lim, Chin Poo Lee, Roy Kwang Yang Chang, Ali Alqahtani

Research output: Journal PublicationArticlepeer-review

4 Citations (Scopus)

Abstract

Hand gesture recognition (HGR) is a rapidly evolving field with the potential to revolutionize human–computer interactions by enabling machines to interpret and understand human gestures for intuitive communication and control. However, HGR faces challenges such as the high similarity of hand gestures, real-time performance, and model generalization. To address these challenges, this paper proposes the stacking of distilled vision transformers, referred to as SDViT, for hand gesture recognition. An initially pretrained vision transformer (ViT) featuring a self-attention mechanism is introduced to effectively capture intricate connections among image patches, thereby enhancing its capability to handle the challenge of high similarity between hand gestures. Subsequently, knowledge distillation is proposed to compress the ViT model and improve model generalization. Multiple distilled ViTs are then stacked to achieve higher predictive performance and reduce overfitting. The proposed SDViT model achieves a promising performance on three benchmark datasets for hand gesture recognition: the American Sign Language (ASL) dataset, the ASL with digits dataset, and the National University of Singapore (NUS) hand gesture dataset. The accuracies achieved on these datasets are 100.00%, 99.60%, and 100.00%, respectively.

Original languageEnglish
Article number12204
JournalApplied Sciences (Switzerland)
Volume13
Issue number22
DOIs
Publication statusPublished - Nov 2023
Externally publishedYes

Keywords

  • hand gesture recognition
  • knowledge distillation
  • sign language recognition
  • stacking
  • vision transformer

ASJC Scopus subject areas

  • General Materials Science
  • Instrumentation
  • General Engineering
  • Process Chemistry and Technology
  • Computer Science Applications
  • Fluid Flow and Transfer Processes

Fingerprint

Dive into the research topics of 'SDViT: Stacking of Distilled Vision Transformers for Hand Gesture Recognition'. Together they form a unique fingerprint.

Cite this