Combating Ambiguity for Hash-Code Learning in Medical Instance Retrieval

Jiansheng Fang, Huazhu Fu, Dan Zeng, Xiao Yan, Yuguang Yan, Jiang Liu

Research output: Journal PublicationArticlepeer-review

4 Citations (Scopus)


When encountering a dubious diagnostic case, medical instance retrieval can help radiologists make evidence-based diagnoses by finding images containing instances similar to a query case from a large image database. The similarity between the query case and retrieved similar cases is determined by visual features extracted from pathologically abnormal regions. However, the manifestation of these regions often lacks specificity, i.e., different diseases can have the same manifestation, and different manifestations may occur at different stages of the same disease. To combat the manifestation ambiguity in medical instance retrieval, we propose a novel deep framework called Y-Net, encoding images into compact hash-codes generated from convolutional features by feature aggregation. Y-Net can learn highly discriminative convolutional features by unifying the pixel-wise segmentation loss and classification loss. The segmentation loss allows exploring subtle spatial differences for good spatial-discriminability while the classification loss utilizes class-aware semantic information for good semantic-separability. As a result, Y-Net can enhance the visual features in pathologically abnormal regions and suppress the disturbing of the background during model training, which could effectively embed discriminative features into the hash-codes in the retrieval stage. Extensive experiments on two medical image datasets demonstrate that Y-Net can alleviate the ambiguity of pathologically abnormal regions and its retrieval performance outperforms the state-of-the-art method by an average of 9.27% on the returned list of 10.

Original languageEnglish
Pages (from-to)3943-3954
Number of pages12
JournalIEEE Journal of Biomedical and Health Informatics
Issue number10
Publication statusPublished - Oct 2021
Externally publishedYes


  • Medical instance retrieval
  • content-based image retrieval
  • convolutional features
  • deep hashing methods

ASJC Scopus subject areas

  • Biotechnology
  • Computer Science Applications
  • Electrical and Electronic Engineering
  • Health Information Management


Dive into the research topics of 'Combating Ambiguity for Hash-Code Learning in Medical Instance Retrieval'. Together they form a unique fingerprint.

Cite this