Eye tracking based deep learning analysis for the early detection of diabetic retinopathy: A pilot study

Hongyang Jiang, Yilin Hou, Hanpei Miao, Haili Ye, Mengdi Gao, Xiaoling Li, Richu Jin, Jiang Liu

Research output: Journal PublicationArticlepeer-review


Deep neural networks (DNNs) have exhibited impressive performance in the diabetic retinopathy (DR) computer-aided diagnosis (CAD) systems. However, DNNs are quite hungry for enormous labeled images. The limited data volume may degrade both the accuracy and interpretability of DNNs seriously. To alleviate this situation, it is significant to excavate valuable prior information. We aim to explore how to utilize ophthalmologist's eye tracking information into an early DR detection system thus to improve the classification accuracy and interpretability. In this paper, ophthalmologists’ gaze maps are firstly collected from their eye movements through eye tracker during DR diagnosis. Then we investigate an eye tracking based early DR detection model based on ophthalmologists’ gaze maps. First, we analysis the effect of the gaze map integrated with the original fundus image based on two image fusion approaches. Second, the weighted gaze map is regarded as a supervised mask to guide the learning of the attention of a DNN model. Additionally, we propose a novel difficulty-aware and class-adaptive gaze map attention learning strategy to enhance the model interpretability. Comparative experiments prove that the weighted gaze map contains more medical knowledge for diagnostic decision. Meanwhile, the attention guidance method via class activate map (CAM) regularization demonstrates its superiority on improving both the accuracy and interpretability of early DR detection model. These investigations indicate that ophthalmologists’ gaze maps, as medical prior knowledge, can contribute to the design of early DR detection model.

Original languageEnglish
Article number104830
JournalBiomedical Signal Processing and Control
Publication statusPublished - Jul 2023
Externally publishedYes


  • Class activate map
  • Computer-aided diagnosis
  • Diabetic retinopathy
  • Eye tracking
  • Gaze map

ASJC Scopus subject areas

  • Signal Processing
  • Biomedical Engineering
  • Health Informatics


Dive into the research topics of 'Eye tracking based deep learning analysis for the early detection of diabetic retinopathy: A pilot study'. Together they form a unique fingerprint.

Cite this