Interventionalist Hand Motion Recognition With Convolutional Neural Network in Robot-Assisted Coronary Interventions

Toluwanimi Oluwadara Akinyemi, Olatunji Mumini Omisore, Wenjing Du, Wenke Duan, Xing Yu Chen, Guanlin Yi, Lei Wang

Research output: Journal PublicationArticlepeer-review

3 Citations (Scopus)


The development of procedural competencies by interventional cardiology fellows for minimally invasive treatment of cardiac diseases depends on their effective training and evaluation. Learning tool manipulation for safe robotic PCIs requires expert supervision and use of high-fidelity systems, however, with limited proficiency for real-time hand motion recognition. Therefore, this study proposes a deep learning-based model for identifying operators' actions. The proposed model is based on the convolutional neural network (CNN) that consists of 1-D convolutional layers for automatic feature map extraction, downsampling, and fully connected layers (FCLs) for inference. The developed models were evaluated using a multimodal dataset collected from sensory glove, electromagnetic (EM), and surface electromyography (sEMG) sensors and real-time angiograms in the course of in vivo catheterization trials performed by nine subjects (two experts and seven novices) using a custom-built robotic catheter system (RCS). The results indicate that the model achieves a 92%-96% accuracy in identifying five actions across four clusters compared with a recognition performance of 83% when recognizing all six actions. Furthermore, we compared the proposed model performance with the existing studies, and the analyses show that our model has a 2%-3% higher accuracy for five-action recognition. Therefore, the proposed model could be employed for real-time hand motion recognition in robot-assisted percutaneous coronary intervention (R-PCI) trials.

Original languageEnglish
Pages (from-to)17725-17736
Number of pages12
JournalIEEE Sensors Journal
Issue number15
Publication statusPublished - 1 Aug 2023
Externally publishedYes


  • Convolutional neural network (CNN)
  • deep learning
  • hand motion classification
  • interventionalist
  • percutaneous coronary interventions (PCIs)
  • robotic catheterization
  • sensor-based data acquisition
  • signal processing

ASJC Scopus subject areas

  • Instrumentation
  • Electrical and Electronic Engineering


Dive into the research topics of 'Interventionalist Hand Motion Recognition With Convolutional Neural Network in Robot-Assisted Coronary Interventions'. Together they form a unique fingerprint.

Cite this