Eye tracking based deep learning analysis for the early detection of diabetic retinopathy: A pilot study

Deep neural networks (DNNs) have exhibited impressive performance in the diabetic retinopathy (DR) computer-aided diagnosis (CAD) systems. However, DNNs are quite hungry for enormous labeled images. The limited data volume may degrade both the accuracy and interpretability of DNNs seriously. To alleviate this situation, it is significant to excavate valuable prior information. We aim to explore how to utilize ophthalmologist's eye tracking information into an early DR detection system thus to improve the classification accuracy and interpretability. In this paper, ophthalmologists’ gaze maps are firstly collected from their eye movements through eye tracker during DR diagnosis. Then we investigate an eye tracking based early DR detection model based on ophthalmologists’ gaze maps. First, we analysis the effect of the gaze map integrated with the original fundus image based on two image fusion approaches. Second, the weighted gaze map is regarded as a supervised mask to guide the learning of the attention of a DNN model. Additionally, we propose a novel difficulty-aware and class-adaptive gaze map attention learning strategy to enhance the model interpretability. Comparative experiments prove that the weighted gaze map contains more medical knowledge for diagnostic decision. Meanwhile, the attention guidance method via class activate map (CAM) regularization demonstrates its superiority on improving both the accuracy and interpretability of early DR detection model. These investigations indicate that ophthalmologists’ gaze maps, as medical prior knowledge, can contribute to the design of early DR detection model.