Application of Deep Learning Algorithm in the Grading Assessment of Corneal Fluorescein Staining
-
摘要:
目的 探讨深度学习算法在角膜荧光染色分级评估中的应用价值。 方法 采用横断面研究,共纳入koko体育app 华西医院隐形眼镜门诊就诊患者的角膜染色图片600张。其中500张用于构建算法,其余100张用于验证算法有效性并比较人工智能(artificial intelligence, AI)与视光初学者间评级准确性(accuracy, ACC)及诊断时长的差异。在完成初次评级1个月后,进行第二次评估,比较两次评估的一致性指数(kappa值)。3位经验丰富的视光师综合分级的结果作为本研究的“金标准”。 结果 通过全集、训练集、测试集交叉验证4种深度学习模型,ResNet34模型预测准确性最高。ResNet34深度学习模型在角膜染色分级上的准确性为93.0%,敏感性为89.5%,特异性为89.6%。AI与两位初学者比较,AI的准确性较高(ACCAI=87.0%、ACCS1=78.0%、ACCS2=52.0%,PACC=0.001),同时AI的平均诊断时长短于视光初学者(tAI=1.00 s,tS1=11.86 s,tS2=13.25 s,Pt=0.001)。在两次评级的一致性比较中,AI(kappaAI=0.658,PAI=0.001)的一致性程度高于视光初学者(kappaS1=0.575, PS1=0.001; kappaS2=0.609,PS2=0.001)。 结论 将深度学习算法应用于角膜染色分级评估具有一定的可行性及临床价值。在与初学者的比较中,AI表现出了较高的准确性和较好的一致性,能够指导视光师在临床工作中快速准确地评定角膜染色分级。 -
关键词:
- 人工智能 /
- koko体育app: 接触镜 /
- koko体育app: 角膜染色
Abstract:Objective To explore the application value of applying deep learning (DL) algorithm in the grading assessment of corneal fluorescein staining. Methods A cross-sectional study was carried out, covering 600 corneal fluorescein staining photos acquired in the Contact Lens Clinic, West China Hospital, Sichuan University between 2020 and 2022. Out of the 600 photos, 500 were used to construct the algorithm and the remaining 100 were used for the validation of the algorithm and a comparative analysis of the difference in grading accuracy (ACC) and the length of diagnostic time between artificial intelligence (AI) and optometry students. One month after finishing the first grading analysis, assessment by AI and optometry students was conducted for a second time and results from the two rounds of assessment were compared to examine the intrarater agreement (kappa value) of the two analyses. The grading analysis results of 3 experienced optometrists were used as the gold standard in the study. Results Findings of the cross validation with the complete dataset, the training dataset, and the test dataset showed that ResNet34 had the highest predictive accuracy among four DL models. ResNet34 DL model achieved an accuracy of 93.0%, sensitivity of 89.5%, and specificity of 89.6% in the grading of corneal staining. In the comparison of the grading accuracy of AI and two optometry students, AI showed better accuracy, with the respective grading accuracy being 87.0%, 78.0%, and 52.0% for AI, student 1, and student 2 (PACC=0.001). In addition, the average diagnostic time of AI was shorter than that of optometry students (tAI=1.00 s, tS1=11.86 s, tS2=13.25 s, Pt=0.001). In the comparative analysis of the intrarater agreement between the two assessments, AI (kappaAI=0.658, PAI=0.001) achieved better consistency than the two optometry students did (kappaS1=0.575, PS1=0.001; kappaS2=0.609, PS2=0.001). Conclusion Applying deep learning algorithms in the grading assessment of corneal fluorescein staining has considerable feasibility and clinical value. In the performance comparison between AI and optometry students, AI achieved higher accuracy and better consistency, which indicates that AI has potential application value for assisting optometrists to make clinical decisions with speed and accuracy. -
Key words:
- Artificial intelligence /
- koko体育app: Contact lens /
- Corneal staining
-
koko体育app
图 2 ResNet34模型损失和准确率变化图
Figure 2. ResNet34 model loss and accuracy graph for the t🧸rain💜ing and valid dataset
图 3 AI与初学者在角膜染色评估的等级分布情况(原图)
Figure 3. Distribution of grading results for corneal staining assessment by AI and♑ students 🍸(original photos)
AI: artificial intelligence; S1: student 1; S2: student 2.图 4 AI与初学者在角膜染色评估的等级分布情况(水平镜像图)
Figure 4. Distribution of grading results for corneal staining assessment results by AI and students (horizont𒅌ally-rotated mirror photos)
AI: artificial intelligence; S1: student 1; S2: student 2.图 5 AI未正确评级的图像示例
Figure 5. E🌄xamples of t𒐪he photos that AI did not grade correctly
A, Uneven staining; B, staining fading; C, excessive slit lamp luminance; D, tear film breakup.表 1 ResNet34深度学习模型预测分级的数据分布及准确性
Table 1. Data distribution and aꦗccuracy in 🍨ResNet34 deep learning model for predicted grading results
Dataset Number Zero deviation of grading
from estimated true
Efron scaleAccuracy/% Train dataset 870 815 93.7 Test dataset 100 87 87.0 Complete dataset 970 902 93.0 下载: 导出CSV
表 2 AI与初学者在分级准确性和诊断时长上的比较
Table 2. Comparison of graꦅding accuracy and dia𒐪gnostic time among AI and students
Variable AI Student 1 Student 2 Accuracy/% 87.0 78.0 52.0△ t/s 1.00 11.86△ 13.25△ △ P<0.017, vs. AI. 下载: 导出CSV
表 3 AI与初学者在分级准确性和诊断时长上的比较(水平镜像图)
Table 3. Comparison of grading accuracy and diagnostic time among AI and꧟ students (horizontally✤-rotated mirror photos)
Variable AI Student 1 Student 2 Accuracy/% 76.0 70.0 40.0△ t/s 1.00 9.91△ 10.99△ △ P<0.017, vs. AI. 下载: 导出CSV
表 4 AI与初学者在角膜染色评估一致性上的比较(kappa值)
Table 4. Comparison of intra-rater agreement between AI and students (kappa value)
Grader First grading vs. standard Second grading vs. standard First grading vs.
second gradingAI 0.726 0.517 0.658 Student 1 0.561 0.469 0.575 Student 2 0.230 0.110 0.609 下载: 导出CSV
-
[1] DOLGIN E. The myopia boom. Nature,2015,519(7543): 276–278. doi: [2] VERKICHARLA P K, OHNO-MATSUI K, SAW S M. Current and predicted demographics of high myopia and an update of its associated pathological changes. Ophthalmic Physiol Opt,2015,35(5): 465–475. doi: [3] HOLDEN B A, FRICKE T R, WILSON D A, et al. Global prevalence of myopia and high myopia and temporal trends from 2000 through 2050. Ophthalmology,2016,123(5): 1036–1042. doi: [4] BRON A J, ARGÜESO P, IRKEC M, et al. Clinical staining of the ocular surface: mechanisms and interpretations. Prog Retin Eye Res,2015,44: 36–61. doi: [5] NICHOLS J J, SINNOTT L T. Tear film, contact lens, and patient factors associated with corneal staining. Invest Ophthalmol Vis Sci,2011,52(2): 1127–1137. doi: [6] BEGLEY C, CAFFERY B, CHALMERS R, et al. Review and analysis of grading scales for ocular surface staining. Ocul Surf,2019,17(2): 208–220. doi: [7] RAHIMY E. Deep learning applications in ophthalmology. Curr Opin Ophthalmol,2018,29(3): 254–260. doi: [8] CHASE C, ELSAWY A, ELEIWA T, et al. Comparison of autonomous AS-OCT deep learning algorithm and clinical dry eye tests in diagnosis of dry eye disease. Clin Ophthalmol,2021,15: 4281–4289. doi: [9] LAVRIC A, VALENTIN P. KeratoDetect: Keratoconus detection algorithm using convolutional neural networks. Comput Intell Neurosci,2019,2019: 8162567. doi: [10] EFRON N. Grading scales for contact lens complications. Ophthalmic Physiol Opt,1998,18(2): 182–186. doi: [11] EFRON N, MORGAN P B, KATSARA S S. Validation of grading scales for contact lens complications. Ophthalmic Physiol Opt,2001,21(1): 17–29. doi: [12] GUO Y, LIU Y, OERLEMANS A, et al. Deep learning for visual understanding: a review. Neurocomputing,2016,187: 27–48. doi: [13] ANWAR S M, MAJID M, QAYYUM A, et al. Medical image analysis using convolutional neural networks: a review. J Med Syst,2018,42(11): 226. doi: [14] HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). USA: Las Vegas, NV, 2016: 770−778. doi: . [15] HUANG G, LIU Z, PLEISS G, et al. Convolutional networks with dense connectivity. IEEE Trans Pattern Anal Mach Intell,2022,44(12): 8704–8716. doi: [16] BUDA M, MAKI A, MAZUROWSKI M A. A systematic study of the class imbalance problem in convolutional neural networks. Neural Netw,2018,106: 249–259. doi: [17] HORN G V, PERONA P. The devil is in the tails: fine-grained classification in the wild. ArXiv, 2017. doi: . [18] LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection. IEEE Trans Pattern Anal Mach Intell,2020,42(2): 318–327. doi: [19] MACLURE M, WILLETT W C. Misinterpretation and misuse of the kappa statistic. Am J Epidemiol,1987,126(2): 161–169. doi: [20] LANDIS J R, KOCH G G. The measurement of observer agreement for categorical data. Biometrics,1977,33(1): 159–174. doi: [21] CHUN Y S, YOON W B, KIM K G, et al. Objective assessment of corneal staining using digital image analysis. Invest Ophthalmol Vis Sci,2014,55(12): 7896–7903. doi: [22] RODRIGUEZ J D, LANE K J, OUSLER G W, et al. Automated grading system for evaluation of superficial punctate keratitis associated with dry eye. Invest Opthalmol Vis Sci,2015,56(4): 2340–2347. doi: [23] AMPARO F, WANG H, YIN J, et al. Evaluating corneal fluorescein staining using a novel automated method. Invest Opthalmol Vis Sci,2017,58(6): BIO168. doi: [24] STEAD W W. Clinical implications and challenges of artificial intelligence and deep learning. JAMA,2018,320(11): 1107–1108. doi: [25] TAN D N H, TAN M, LIEW H, et al. A data-driven approach to evaluate factors affecting resident performance in cataract surgery. Int Ophthalmol,2023,43(9): 3269–3277. doi: [26] KOURUKMAS R, ROTH M, GEERLING G. Automated vs. human evaluation of corneal staining. Graefes Arch Clin Exp Ophthalmol,2022,260(8): 2605–2612. doi: -
开花修改 本文遵循知识共享署名—非商业性使用4.0国际许可协议(CC BY-NC 4.0),允许第三方对本刊发表的论文自由共享(♏即在任何媒介以任何形式复制、发行原文)、演绎(即修改、转换或以原文为基础进行创作),必须给出适当的署名,提供指向本文许可协议的链接,同时标明是否对原文作了修改;不得将本文用于商业目的。CC BY-NC 4.0许可协议详情请访问 //cr🐠eativecommons.org/licenses/by-nc/4.0