A Deep Multimodal Network for Classification and Identification of Interventionists' Hand Motions during Cyborg Intravascular Catheterization

Olatunji Mumini Omisore, Wenjing Du, Wenke Duan, Thanh Do, Rita Orji, Lei Wang

Research output: Chapter in Book/Conference proceedingConference contributionpeer-review

1 Citation (Scopus)

Abstract

Recent insights from human-robot intelligence and deep learning raise hope towards task-specific autonomy in robotic intravascular coronary interventions. However, lack of learning-based methods for characterizing the interventionists' kinesthetic data hinders the drive for shared control and robotic autonomy during cyborg catheterization. In this study, a deep multimodal network model is proposed for classification and recognition of interventionists' hand movements during cyborg intravascular catheterization. The model has two modules for extracting salient features in electromyography signal datasets, and classification of hand motions made during intravascular catheterization procedures. Network training and evaluation observed for in-vitro and in-vivo datasets obtained from trained novice subjects and expert with about 5 years of experience in percutaneous coronary interventions. Performance evaluation shows the learning model could classify interventionists' hand movements accurately in manual and robot-assisted navigations, respectively. This study is suggested to further stimulate the development of appropriate skill level assessments towards cyborg catheterization for cardiac interventions.

Original languageEnglish
Title of host publication2021 IEEE 17th International Conference on Automation Science and Engineering, CASE 2021
PublisherIEEE Computer Society
Pages1182-1187
Number of pages6
ISBN (Electronic)9781665418737
DOIs
Publication statusPublished - 23 Aug 2021
Externally publishedYes
Event17th IEEE International Conference on Automation Science and Engineering, CASE 2021 - Lyon, France
Duration: 23 Aug 202127 Aug 2021

Publication series

NameIEEE International Conference on Automation Science and Engineering
Volume2021-August
ISSN (Print)2161-8070
ISSN (Electronic)2161-8089

Conference

Conference17th IEEE International Conference on Automation Science and Engineering, CASE 2021
Country/TerritoryFrance
CityLyon
Period23/08/2127/08/21

Keywords

  • Hand Motion Classification
  • Machine Learning
  • Robotic Catheterization
  • sEMG-based Control
  • Skill Assessment

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'A Deep Multimodal Network for Classification and Identification of Interventionists' Hand Motions during Cyborg Intravascular Catheterization'. Together they form a unique fingerprint.

Cite this