Ultrasound-based deep learning model as an assistant improves the diagnosis of ovarian tumors: a multicenter study

Background: Deep learning (DL) models based on ultrasound (US) images can enhance the ability of radiologists to diagnose ovarian tumors. Materials and methods: This retrospective study included 916 women with ovarian tumors in southeast China who underwent surgery with clear pathology and preoperative US examination. The data set was divided into a training (80%) and a validation (20%) set. The test set consisted of 81 women with ovarian tumors from southwest and northeast China. DL models based on three backbone architectures, ResNet-50 (residual CNN), VGG16 (plain CNN), and Vision Transformer (ViT), were trained to classify benign, borderline, and malignant ovarian tumors. The diagnostic efficiency of primary US doctors combined with the DL model was compared with the ADNEX model and a US expert. Additionally, we compared the diagnostic performance of primary US doctors before and after being assisted by the integrated framework combining visual DL models and large language models. Results: (1) The accuracy of the ResNet50-based DL model for benign, malignant, and borderline ovarian tumors was 91.8%, 84.61%, and 82.60% for the test sets, respectively. (2) After visual and linguistic DL assistance, the accuracy of primary US doctors all improved in the test set (doctor A: 76.62% to 90.90%, doctor B: 76.62% to 90.90%, doctor C: 79.22% to 94.54%, doctor D: 76.62% to 95.95%, doctor E: 76.60% to 95.95%, respectively). (3) The diagnostic consistency of primary US doctors for validation and test sets also increased (doctor A: 0.671 to 0.912, doctor B: 0.762 to 0.916, doctor C: 0.412 to 0.629, doctor D: 0.588 to 0.701, doctor E: 0.528 to 0.710, respectively). Conclusions: A DL system combining an image-based model (vision model) and a language model was developed to assist radiologists in classifying ovarian tumors in US images and enhance diagnostic efficacy. Critical relevance statement: The established model can assist primary US doctors in preoperative diagnosis and improve the early detection and timely treatment of ovarian tumors. Key Points: An ultrasound-based deep learning (DL) model was developed for ovarian tumors using multi-center patients. An image-based DL model was combined with a large language model to establish a diagnostic framework for ovarian tumor classification. Our DL model can improve the diagnosis of primary US doctors to the level of experts and might assist in surgical decision-making.