Ophthalmology in China ›› 2021, Vol. 30 ›› Issue (5): 355-359.doi: 10.13281/j.cnki.issn.1004-4469.2021.05.006

Previous Articles     Next Articles

Evaluation of the diagnostic accuracy in three kinds of artificial intelligence diagnosis model for detection of diabetic retinopathy on fundus photographs

Zhao Qi, Yang Wenli, Wei Wenbin, Zhang Yongpeng, Li Lei, Zhang Mengyu, Guo Jing   

  1. Beijing Tongren Eye Center, Beijing Key Laboratory of Intraocular Tumor Diagnosis and Treatment, Beijing Key Laboratory of Ophthalmology and Visual Sciences, Medical Artificial Intelligence Research and Verification Key Laboratory of the Ministry of Industry and Information Technology, Beijing Tongren Hospital, Capital Medical University, Beijing 100730, China

  • Received:2021-06-16 Online:2021-09-25 Published:2021-09-24
  • Contact: Wei Wenbin, Email: weiwenbintr@163.com
  • Supported by:
    The Capital Health Research and Development of Special (2020-1-2052); Science & Technology Project of Beijing Municipal Science & Technology Commission (Z201100005520045, Z181100001818003); the Beijing Municipal Administration of Hospitals’ Ascent Plan (DFL20150201)

Abstract: Objective To compare the consistency of artificial intelligence (AI) diagnosis models and ophthalmologist in grading diabetic retinopathy (DR). Design Diagnosis tests. Participants 236 cases (468 eyes) of diabetic patients in Beijing Tongren Hospital were selected. Methods Two-field non-mydriatic macula- or optic disc-centered fundus photographs were performed. Using ophthalmologist diagnosis as the golden standard, three kinds of AI diagnosis model(model 1, 2, 3) based on Convolutional Neural Network (CNN) were evaluated, including DR yes/no, referable DR and DR stagingtests. Main Outcome Measures Sensitivity (Se), specificity (Sp), area under curve (AUC). Results The Se, Sp and AUC of DR yes/no were 95.8%, 90.1% and 0.930 on model 1, 96.5%, 85.2% and 0.908 on model 2, and 96.2%，83.5% and 0.917 on model 3, respectively. The Se, Sp and AUC of referable DR were 93.9%, 90.1% and 0.933 on model 1, 97.7%，89.3% and 0.935 on model 2, and 95.4%, 89.8% and 0.926 on model 3, respectively. The Se, Sp and AUC of stages of DR were 72.9%~90.1%, 93.9%~97.8% and 0.854~0.930 on model 1, 68.8%~92.1%， 90.6%~98.2% and 0.831~0.914 on model 2, and 75.0%~83.5%, 89.2%~96.8% and 0.849~0.917 on model 3, respectively. Conclusion Three kinds of AI diagnosis model have good sensitivity and specificity for the determination of DR and referable DR, and it could be used as an auxiliary tool for DR screening of diabetic patients. (Ophthalmol CHN, 2021, 30: 355-359)


Key words: diabetic retinopathy/diagnosis, fundus photography, artificial intelligence, consistency