Instrument-tissue Interaction Quintuple Detection in Surgery Videos

Wenjun Lin, Yan Hu, Luoying Hao, Dan Zhou, Mingming Yang, Huazhu Fu, Cheekong Chui, Jiang Liu

Research output: Chapter in Book/Conference proceedingConference contributionpeer-review

2 Citations (Scopus)


Instrument-tissue interaction detection in surgical videos is a fundamental problem for surgical scene understanding which is of great significance to computer-assisted surgery. However, few works focus on this fine-grained surgical activity representation. In this paper, we propose to represent instrument-tissue interaction as ⟨ instrument bounding box, tissue bounding box, instrument class, tissue class, action class ⟩ quintuples. We present a novel quintuple detection network (QDNet) for the instrument-tissue interaction quintuple detection task in cataract surgery videos. Specifically, a spatiotemporal attention layer (STAL) is proposed to aggregate spatial and temporal information of the regions of interest between adjacent frames. We also propose a graph-based quintuple prediction layer (GQPL) to reason the relationship between instruments and tissues. Our method achieves an mAP of 42.24% on a cataract surgery video dataset, significantly outperforming other methods.

Original languageEnglish
Title of host publicationMedical Image Computing and Computer Assisted Intervention – MICCAI 2022 - 25th International Conference, Proceedings
EditorsLinwei Wang, Qi Dou, P. Thomas Fletcher, Stefanie Speidel, Shuo Li
PublisherSpringer Science and Business Media Deutschland GmbH
Number of pages11
ISBN (Print)9783031164484
Publication statusPublished - 2022
Externally publishedYes
Event25th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2022 - Singapore, Singapore
Duration: 18 Sept 202222 Sept 2022

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume13437 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Conference25th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2022


  • Instrument-tissue interaction quintuple detection
  • Surgery video
  • Surgical scene understanding

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science (all)


Dive into the research topics of 'Instrument-tissue Interaction Quintuple Detection in Surgery Videos'. Together they form a unique fingerprint.

Cite this