Abstract
Purpose
We developed an Infant Retinal Intelligent Diagnosis System (IRIDS), an automated system to aid early diagnosis and monitoring of infantile fundus diseases and health conditions to satisfy urgent needs of ophthalmologists.
Methods
We developed IRIDS by combining convolutional neural networks and transformer structures, using a dataset of 7697 retinal images (1089 infants) from four hospitals. It identifies nine fundus diseases and conditions, namely, retinopathy of prematurity (ROP) (mild ROP, moderate ROP, and severe ROP), retinoblastoma (RB), retinitis pigmentosa (RP), Coats disease, coloboma of the choroid, congenital retinal fold (CRF), and normal. IRIDS also includes depth attention modules, ResNet-18 (Res-18), and Multi-Axis Vision Transformer (MaxViT). Performance was compared to that of ophthalmologists using 450 retinal images. The IRIDS employed a five-fold cross-validation approach to generate the classification results.
Results
Several baseline models achieved the following metrics: accuracy, precision, recall, F1-score (F1), kappa, and area under the receiver operating characteristic curve (AUC) with best values of 94.62% (95% CI, 94.34%-94.90%), 94.07% (95% CI, 93.32%-94.82%), 90.56% (95% CI, 88.64%-92.48%), 92.34% (95% CI, 91.87%-92.81%), 91.15% (95% CI, 90.37%-91.93%), and 99.08% (95% CI, 99.07%-99.09%), respectively. In comparison, IRIDS showed promising results compared to ophthalmologists, demonstrating an average accuracy, precision, recall, F1, kappa, and AUC of 96.45% (95% CI, 96.37%-96.53%), 95.86% (95% CI, 94.56%-97.16%), 94.37% (95% CI, 93.95%-94.79%), 95.03% (95% CI, 94.45%-95.61%), 94.43% (95% CI, 93.96%-94.90%), and 99.51% (95% CI, 99.51%-99.51%), respectively, in multi-label classification on the test dataset, utilizing the Res-18 and MaxViT models. These results suggest that, particularly in terms of AUC, IRIDS achieved performance that warrants further investigation for the detection of retinal abnormalities.
Conclusions
IRIDS identifies nine infantile fundus diseases and conditions accurately. It may aid non-ophthalmologist personnel in underserved areas in infantile fundus disease screening. Thus, preventing severe complications. The IRIDS serves as an example of artificial intelligence integration into ophthalmology to achieve better outcomes in predictive, preventive, and personalized medicine (PPPM / 3PM) in the treatment of infantile fundus diseases.
Similar content being viewed by others
Data availability
These authors do not have permission to share data except the first author.
Code availability
The source code used in this study is available upon request from the first author only.
Abbreviations
- AI:
-
Artificial intelligence
- AUC:
-
Area under the receiver operating characteristic curve
- CNNs:
-
Convolutional neural networks
- CRF:
-
Congenital retinal fold
- DA:
-
Depth attention
- DL:
-
Deep Learning
- EPMA:
-
European Association for Predictive, Preventive and Personalized Medicine
- eps:
-
Epsilon
- F1:
-
F1 score
- IRIDS:
-
Infant Retinal Intelligent Diagnosis System
- Max ViT:
-
Multi-Axis Vision Transformer
- PPPM/3PM:
-
Predictive, preventive, and personalized medicine
- RB:
-
Retinoblastoma
- ROC:
-
Receiver operating characteristic
- ROP:
-
Retinopathy of prematurity
- RP:
-
Retinitis pigmentosa
- Res-18:
-
ResNet-18
- SGD:
-
Stochastic gradient descent
- T-SNE:
-
T-distributed stochastic neighbor embedding
References
Chiang MF, Quinn GE, Fielder AR, Ostmo SR, Paul Chan RV, Berrocal A, et al. International Classification of Retinopathy of Prematurity, Third Edition. Ophthalmology. 2021;128(10):e51–68. https://doi.org/10.1016/j.ophtha.2021.05.031.
Shields JA, Shields CL, Honavar SG, Demirci H. Clinical variations and complications of Coats disease in 150 cases: the 2000 Sanford Gifford Memorial Lecture. Am J Ophthalmol. 2001;131(5):561–71. https://doi.org/10.1016/s0002-9394(00)00883-7.
Spitznas M, Joussen F, Wessing A, Meyer-Schwickerath G. Coat’s disease. An epidemiologic and Fluorescein angiographic study. Albrecht Von Graefes Arch Klin Exp Ophthalmol. 1975;195(4):241–50. https://doi.org/10.1007/BF00414937.
Rao R, Honavar SG. Retinoblastoma. Indian J Pediatr. 2017;84(12):937–44. https://doi.org/10.1007/s12098-017-2395-0.
Pagon RA. Retinitis pigmentosa. Surv Ophthalmol. 1988;33(3):137–77. https://doi.org/10.1016/0039-6257(88)90085-9.
Giles K, Raoul C, Yannick B, Peter W. Uveal coloboma: about 3 cases at the University Teaching Hospital, Yaounde, Cameroon. Pan Afr Med J. 2016;24:201. https://doi.org/10.11604/pamj.2016.24.201.9770.
Nishina S, Suzuki Y, Yokoi T, Kobayashi Y, Noda E, Azuma N. Clinical features of congenital retinal folds. Am J Ophthalmol. 2012;153(1):81-7 e1. https://doi.org/10.1016/j.ajo.2011.06.002.
Liche F, Majji AB. Familial exudative vitreoretinopathy. Ophthalmology. 2012;119(5):1093. https://doi.org/10.1016/j.ophtha.2012.02.025.
Fielder A, Blencowe H, O’Connor A, Gilbert C. Impact of retinopathy of prematurity on ocular structures and visual functions. Arch Dis Child Fetal Neonatal Ed. 2015;100(2):F179–84. https://doi.org/10.1136/archdischild-2014-306207.
Golubnitschaja O, Costigliola V, Epma. General report & recommendations in predictive, preventive and personalised medicine 2012: white paper of the European Association for Predictive, Preventive and Personalised Medicine. EPMA J. 2012;3(1):14. https://doi.org/10.1186/1878-5085-3-14.
Good WV. Retinopathy of Prematurity Incidence in Children. Ophthalmology. 2020;127(4S):S82–3. https://doi.org/10.1016/j.ophtha.2019.11.026.
Dimaras H, Corson TW, Cobrinik D, White A, Zhao J, Munier FL, et al. Retinoblastoma. Nat Rev Dis Primers. 2015;1:15021. https://doi.org/10.1038/nrdp.2015.21.
Global Retinoblastoma Study G, Fabian ID, Abdallah E, Abdullahi SU, Abdulqader RA, Adamou Boubacar S, et al. Global Retinoblastoma Presentation and Analysis by National Income Level. JAMA Oncol. 2020;6(5):685–95. https://doi.org/10.1001/jamaoncol.2019.6716.
Chen HY, Lehmann OJ, Swaroop A. Genetics and therapy for pediatric eye diseases. EBioMedicine. 2021;67:103360. https://doi.org/10.1016/j.ebiom.2021.103360.
Coleman K, Coleman J, Franco-Penya H, Hamroush F, Murtagh P, Fitzpatrick P, et al. A New Smartphone-Based Optic Nerve Head Biometric for Verification and Change Detection. Transl Vis Sci Technol. 2021;10(8):1. https://doi.org/10.1167/tvst.10.8.1.
Golubnitschaja O, Potuznik P, Polivka J Jr, Pesta M, Kaverina O, Pieper CC, et al. Ischemic stroke of unclear aetiology: a case-by-case analysis and call for a multi-professional predictive, preventive and personalised approach. EPMA J. 2022;13(4):535–45. https://doi.org/10.1007/s13167-022-00307-z.
Topol EJ. High-performance medicine: the convergence of human and artificial intelligence. Nat Med. 2019;25(1):44–56. https://doi.org/10.1038/s41591-018-0300-7.
Baek SU, Lee WJ, Park KH, Choi HJ. Health screening program revealed risk factors associated with development and progression of papillomacular bundle defect. EPMA J. 2021;12(1):41–55. https://doi.org/10.1007/s13167-021-00235-4.
Li S, Li M, Wu J, Li Y, Han J, Cao W, et al. Development and validation of a routine blood parameters-based model for screening the occurrence of retinal detachment in high myopia in the context of PPPM. EPMA J. 2023;14(2):219–33. https://doi.org/10.1007/s13167-023-00319-3.
Golub TR, Slonim DK, Tamayo P, Huard C, Gaasenbeek M, Mesirov JP, et al. Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. Science. 1999;286(5439):531–7. https://doi.org/10.1126/science.286.5439.531.
Wang Y, Tetko IV, Hall MA, Frank E, Facius A, Mayer KF, et al. Gene selection from microarray data for cancer classification–a machine learning approach. Comput Biol Chem. 2005;29(1):37–46. https://doi.org/10.1016/j.compbiolchem.2004.11.001.
Yu KH, Levine DA, Zhang H, Chan DW, Zhang Z, Snyder M. Predicting Ovarian Cancer Patients’ Clinical Response to Platinum-Based Chemotherapy by Their Tumor Proteomic Signatures. J Proteome Res. 2016;15(8):2455–65. https://doi.org/10.1021/acs.jproteome.5b01129.
Yu KH, Fitzpatrick MR, Pappas L, Chan W, Kung J, Snyder M. Omics AnalySIs System for PRecision Oncology (OASISPRO): a web-based omics analysis tool for clinical phenotype prediction. Bioinformatics. 2018;34(2):319–20. https://doi.org/10.1093/bioinformatics/btx572.
Check Hayden E. The automated lab. Nature. 2014;516(7529):131–2. https://doi.org/10.1038/516131a.
Chew EY. Age-related Macular Degeneration: Nutrition, Genes and Deep Learning-The LXXVI Edward Jackson Memorial Lecture. Am J Ophthalmol. 2020;217:335–47. https://doi.org/10.1016/j.ajo.2020.05.042.
Shon K, Sung KR, Shin JW. Can Artificial Intelligence Predict Glaucomatous Visual Field Progression? A Spatial-Ordinal Convolutional Neural Network Model. Am J Ophthalmol. 2022;233:124–34. https://doi.org/10.1016/j.ajo.2021.06.025.
Ee CL, Samsudin A. Comparison of Smartphone-Based and Automated Refraction with Subjective Refraction for Screening of Refractive Errors. Ophthalmic Epidemiol. 2022;29(5):588–94. https://doi.org/10.1080/09286586.2021.1986550.
Dai L, Wu L, Li H, Cai C, Wu Q, Kong H, et al. A deep learning system for detecting diabetic retinopathy across the disease spectrum. Nat Commun. 2021;12(1):3242. https://doi.org/10.1038/s41467-021-23458-5.
Panwar N, Huang P, Lee J, Keane PA, Chuan TS, Richhariya A, et al. Fundus Photography in the 21st Century–A Review of Recent Technological Advances and Their Implications for Worldwide Healthcare. Telemed J E Health. 2016;22(3):198–208. https://doi.org/10.1089/tmj.2015.0068.
Zhao J, Lei B, Wu Z, Zhang Y, Li Y, Wang L, et al. A Deep Learning Framework for Identifying Zone I in RetCam Images. IEEE Access. 2019;7:103530–7. https://doi.org/10.1109/access.2019.2930120.
Zhang Y, Wang L, Wu Z, Zeng J, Chen Y, Tian R, et al. Development of an Automated Screening System for Retinopathy of Prematurity Using a Deep Neural Network for Wide-Angle Retinal Images. IEEE Access. 2019;7:10232–41. https://doi.org/10.1109/access.2018.2881042.
Rugang Zhang JZ. Hai Xie, Tianfu Wang, Automatic diagnosis for aggressive posterior retinopathy of prematurity via deep attentive convolutional neural network. Expert Syst Appl. 2022;187:115843.
Maji D, Sekh AA. Automatic grading of retinal blood vessel in deep retinal image diagnosis. J Med Syst. 2020;44(180). https://doi.org/10.1007/s10916-020-01635-1.
Xie HLH, Zeng X, He Y, Chen G. AMD-GAN: attention encoder and multi-branch structure based generative adversarial networks for fundus disease detection from scanning laser ophthalmoscopy images. Neural Netw. 2020;132:477–90.
Dong L, He W, Zhang R, Ge Z, Wang YX, Zhou J, et al. Artificial Intelligence for Screening of Multiple Retinal and Optic Nerve Diseases. JAMA Netw Open. 2022;5(5):e229960. https://doi.org/10.1001/jamanetworkopen.2022.9960.
Cen LP, Ji J, Lin JW, Ju ST, Lin HJ, Li TP, et al. Automatic detection of 39 fundus diseases and conditions in retinal photographs using deep neural networks. Nat Commun. 2021;12(1):4828. https://doi.org/10.1038/s41467-021-25138-w.
Patel TP, Kim TN, Yu G, Dedania VS, Lieu P, Qian CX, et al. Smartphone-Based, Rapid, Wide-Field Fundus Photography for Diagnosis of Pediatric Retinal Diseases. Transl Vis Sci Technol. 2019;8(3):29. https://doi.org/10.1167/tvst.8.3.29.
Golubnitschaja O, Baban B, Boniolo G, Wang W, Bubnov R, Kapalla M, et al. Medicine in the early twenty-first century: paradigm and anticipation - EPMA position paper 2016. EPMA J. 2016;7(1):23. https://doi.org/10.1186/s13167-016-0072-4.
World Medical A. World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects. JAMA. 2013;310(20):2191–4. https://doi.org/10.1001/jama.2013.281053.
Reynolds JD and Olitsky SE, pediatric retina. 1st ed. Heidelberg: Springer-Verlag Berlin. 2010. https://doi.org/10.1007/978-3-642-12041-11.
He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. in Proceedings of the IEEE conference on computer vision and pattern recognition. 2016;770–778. https://doi.org/10.48550/arXiv.1512.03385.
Tu Z, Talebi H, Zhang H, Yang F, Peyman Milanfar, Alan Bovik, et al. Maxvit: Multi-axis vision transformer. arXiv preprint arXiv. 2022: p. 01697. https://doi.org/10.48550/arXiv.2204.01697.
Chen C, Rameswar P, Fan Q. Regionvit: Regional-to-local attention for vision transformers. arXiv preprint arXiv. 2022. https://doi.org/10.48550/arXiv.2106.02689.
Lee SH, Lee S, Song BC. Vision transformer for small-size datasets. arXiv preprint arXiv. 2021. https://doi.org/10.48550/arXiv.2112.13492.
Yu S, Ma K, Bi Q, Bian C. Mil-vt: Multiple instance learning enhanced vision transformer for fundus image classification[C]//Medical Image Computing and Computer Assisted Intervention-MICCAI 2021: 24th International Conference, Strasbourg, France, September 27-October 1, 2021, Proceedings, Part VIII 24. Springer International Publishing. 2021: p. 45-54. https://doi.org/10.1007/978-3-030-87237-3_5.
Sun R, Li Y, Zhang T, Mao Z, Wu F. Lesion-aware transformers for diabetic retinopathy grading[C]. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021: p. 10938–10947. https://doi.org/10.1109/CVPR46437.2021.01079.
Playout C, Duval R, Boucher MC, Cheriet F. Focused Attention in Transformers for interpretable classification of retinal images. Med Image Anal. 2022;82:102608. https://doi.org/10.1016/j.media.2022.102608.
Wong TY, Bressler NM. Artificial Intelligence With Deep Learning Technology Looks Into Diabetic Retinopathy Screening. JAMA. 2016;316(22):2366–7. https://doi.org/10.1001/jama.2016.17563.
Ng WY, Zhang S, Wang Z, Ong CJT, Gunasekeran DV, Lim GYS, Ting DSW. Updates in deep learning research in ophthalmology. Clin Sci. 2021;135(20):2357–76.
Huang YPBH, Kang EYC, Chen KJ, Hwang YS, Lai CC, Wu WC. Automated detection of early-stage ROP using a deep convolutional neural network. Br J Ophthalmol. 2021;105(8):1099–103.
Durai C, Jebaseeli TJ, Alelyani S, Mubharakali A. Early Prediction and Diagnosis of Retinoblastoma Using Deep Learning Techniques. arXiv preprint arXiv. 2021. https://doi.org/10.48550/arXiv.2103.07622.
Stevenson CH, Hong SC, Ogbuehi KC. Development of an artificial intelligence system to classify pathology and clinical features on retinal fundus images. Clin Exp Ophthalmol. 2019;47(4):484–9. https://doi.org/10.1111/ceo.13433.
Son J, Shin JY, Kim HD, Jung KH, Park KH, Park SJ. Development and Validation of Deep Learning Models for Screening Multiple Abnormal Findings in Retinal Fundus Images. Ophthalmology. 2020;127(1):85–94. https://doi.org/10.1016/j.ophtha.2019.05.029.
Lin D, Xiong J, Liu C, Zhao L, Li Z, Yu S, et al. Application of Comprehensive Artificial intelligence Retinal Expert (CARE) system: a national real-world evidence study. Lancet Digit Health. 2021;3(8):e486–95. https://doi.org/10.1016/S2589-7500(21)00086-8.
Cen L-P, Ji J, Lin J-W, Ju S-T, Lin H-J, Li T-P, et al. Automatic detection of 39 fundus diseases and conditions in retinal photographs using deep neural networks. Nat Commun. 2021;12(1). https://doi.org/10.1038/s41467-021-25138-w.
Ju L, Yu Z, Wang L, Zhao X, Wang X, Bonnington P, et al. Hierarchical Knowledge Guided Learning for Real-world Retinal Disease Recognition. IEEE Trans Med Imaging. 2023;PP. https://doi.org/10.1109/TMI.2023.3302473.
Gu C, Wang Y, Jiang Y, Xu F, Wang S, Liu R, et al. Application of artificial intelligence system for screening multiple fundus diseases in Chinese primary healthcare settings: a real-world, multicentre and cross-sectional study of 4795 cases. Br J Ophthalmol. 2023. https://doi.org/10.1136/bjo-2022-322940.
Zhu H, Cheng C, Yin H, Li X, Zuo P, Ding J, et al. Automatic multilabel electrocardiogram diagnosis of heart rhythm or conduction abnormalities with deep learning: a cohort study. Lancet Digit Health. 2020;2(7):e348–57. https://doi.org/10.1016/S2589-7500(20)30107-2.
Golubnitschaja O, Topolcan O, Kucera R, Costigliola V, Epma. 10th Anniversary of the European Association for Predictive, Preventive and Personalised (3P) Medicine - EPMA World Congress Supplement 2020. EPMA J. 2020;11(Suppl 1):1–133. https://doi.org/10.1007/s13167-020-00206-1.
Funding
This study was supported by National Natural Science Foundation of China (No. 82271103, 82301269, 82301226, 62376164, 62106153, U22A2024), Sanming Project of Medicine in Shenzhen (No. SZSM202311018), Guangdong Basic and Applied Basic Research Foundation (No. 2022A1515012326, 22201910240002529), Shenzhen Medical Research Fund (No. C2301005), Shenzhen Key Medical Discipline Construction Fund (No. SZXK038), Shenzhen Fund for Guangdong Provincial High Level Clinical Key Specialties (No. SZGSP014), Shenzhen Science and Technology R&D Fund Project (No. JCYJ20220530153607015), China Ophthalmology New Technology Incubation Project.
Author information
Authors and Affiliations
Contributions
GZ has full access to all data in the study and is responsible for the integrity of the data and the accuracy of the data analysis. YL, HX, and XZ contributed equally and are considered co-first authors. Concept and design: YL, HX, and XZ. Acquisition, analysis, or interpretation of data: YL, HX, XZ, and SZ. Drafting of the manuscript: YL and HX. Critical revision of the manuscript for important intellectual content: GZ, JT, XZ, and DPN. Statistical analysis: YL and JT. Obtained funding: GZ. Administrative, technical, or material support: ZY, ZW, RT, YC, MC, YD, TC, YH, and BL. Supervision: GZ.
Corresponding authors
Ethics declarations
Conflicts interests
The authors have no relevant financial or non-financial interests to disclose.
Ethics approval
This diagnostic study was approved by the Ethics Committee of Shenzhen Eye Hospital. All institutions abided by the tenets of the Declaration of Helsinki.
Consent to participate
Written informed consent was obtained from the parents of all enrolled infants.
Consent for publication
This article has been approved for publication by the authors.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Liu, Y., Xie, H., Zhao, X. et al. Automated detection of nine infantile fundus diseases and conditions in retinal images using a deep learning system. EPMA Journal 15, 39–51 (2024). https://doi.org/10.1007/s13167-024-00350-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13167-024-00350-y