Automated Detection of Nine Infantile Fundus Diseases and Conditions in Retinal Images Using a Deep Learning System

doi:10.21203/rs.3.rs-3303322/v1

Download PDF

Research Article

Automated Detection of Nine Infantile Fundus Diseases and Conditions in Retinal Images Using a Deep Learning System

https://doi.org/10.21203/rs.3.rs-3303322/v1

This work is licensed under a CC BY 4.0 License

Journal Publication

published 15 Feb, 2024

Read the published version in EPMA Journal →

You are reading this latest preprint version

Purpose

We developed an Infant Retinal Intelligent Diagnosis System (IRIDS), an automated system to aid early diagnosis and monitoring of infantile fundus diseases and conditions due to a shortage of ophthalmologists. Our aim is to provide personalized monitoring and early intervention to prevent complications associated with infantile fundus diseases, aligning with predictive, preventive, and personalized medicine (PPPM).

Methods

We developed IRIDS by combining convolutional neural networks and transformer structures, using a dataset of 7697 retinal images from four hospitals. It identifies nine fundus diseases and includes depth attention modules, Res-18, and MaxViT. Performance was compared to that of ophthalmologists using 450 retinal images. The IRIDS employed a five-fold cross-validation approach to generate the classification results.

Results

Several baseline models achieved the following metrics: accuracy, precision, recall, F1-score (F1), kappa, and area under the receiver operating characteristic curve (AUC) with values of 90.25%, 87.69%, 83.38%, 85.48%, 83.51, and 98.04%, respectively. In comparison, IRIDS outperformed the ophthalmologists with an average accuracy, precision, recall, F1, kappa, and AUC of 96.45%, 95.86%, 94.37%, 95.03%, 94.43, and 99.51%, respectively, in multi-label classification on the test dataset, utilizing the Res-18 and MaxViT models. Compared with ophthalmologists, the IRIDS achieved a higher AUC for the detection of retinal abnormalities.

Conclusions

IRIDS identifies nine infantile fundus diseases and conditions accurately. It can aid non-ophthalmologist personnel in underserved areas in infantile fundus disease screening. Thus, preventing severe complications. The IRIDS serves as an example of artificial intelligence integration into ophthalmology to achieve better outcomes in PPPM services.

Deep Learning

infant

fundus disease

retinal image

predictive

preventive and personalized medicine

Early diagnosis and treatment of infantile fundus diseases and conditions are key factors in improving the quality of life for infants and their families. Common infantile fundus diseases include retinopathy of prematurity (ROP) [1], Coats disease [2, 3], retinoblastoma (RB) [4], retinitis pigmentosa (RP) [5], choroid coloboma [6], congenital retinal fold (CRF) [7], and familial exudative vitreoretinopathy [8]. Delayed diagnosis and management of these conditions can lead to reduced treatment effectiveness and the potential for severe, sometimes irreversible structural and functional complications, such as refractive errors, night blindness, strabismus, and neovascular glaucoma [2, 9]. Identifying high-risk individuals is a vital step in preventing these serious complications, as emphasized in the European Association for PPPM white paper [10].

ROP, a typical example of an infantile fundus abnormality, is a leading cause of visual impairment and blindness in early life, requiring timely diagnosis and intervention [11]. Approximately 8000 new cases of infantile RB worldwide require enucleation surgery every year [12–14]. Due to the relative rarity of some fundus diseases in infants, even when encountered, making an accurate diagnosis in certain children’s hospitals and maternal and child health hospitals may be challenging. This can have detrimental consequences not only for individuals and their families but also for society at large. However, numerous underserved areas worldwide lack access to experienced ophthalmologists [15], placing infants with fundus diseases at immediate risk. It is critical to develop a unique strategy that adheres to the PPPM principles for early detection of infants with fundus diseases [16]. Therefore, the latest technological innovations in digital computing can be applied to develop an effective automated diagnostic tool for the timely diagnosis and management of infantile fundus diseases.

Deep Learning (DL) is a mature yet rapidly developing technology, especially in the realm of computer-aided diagnosis of human diseases [17–19]. This frontier continues to expand into other areas of medicine, such as clinical practice, translational medical research, and basic biomedical research [20–24]. DL has demonstrated outstanding performances in automating the screening and diagnosis of various fundus diseases, such as diabetic retinopathy, age-related macular degeneration, refraction errors, glaucoma, and retinal tumors. Some of these applications are already progressing towards clinical implementation [25–29]. Regarding infantile fundus diseases, our team successfully used DL to identify ROP, zone I of ROP, and A-ROP, effectively applying augmentation algorithms for retinal grading [30–34]. Most relevant studies have focused on detecting a single fundus disease using retinal images. Recently, DL-assisted systems have been developed to detect multiple fundus diseases [35, 36], including one that utilized a smartphone-based wide-field retinal imaging system for classifying multiple diseases in children [37]. It's worth noting that the above-mentioned studies have primarily focused on single fundus diseases or examined retinal images of adults and older children.

Working hypothesis and purpose of the study

No DL studies involving the detection of multiple infantile fundus diseases have been conducted thus far. In clinical practice, especially in remote areas lacking specialized ophthalmologists, such a multi-disease detection system for infantile fundus diseases would not only be necessary but also potentially highly beneficial. Many infantile fundus diseases are difficult for parents to detect, and they are often identified when a child seeks treatment for conditions such as amblyopia at a later age. Moreover, the diagnosis of infantile fundus diseases is challenging, leading to an increased incidence of severe complications. We hypothesized that utilizing DL models for automatic detection of infantile fundus diseases through retinal images can improve the diagnostic rate in infants, prevent the occurrence of severe complications, and ultimately enhance their quality of life. We believe that this approach can be effectively applied in clinical settings, providing more accurate and reliable tools for screening and managing infantile fundus diseases.

To achieve this, we developed an automated disease detection system for infantile fundus diseases called the Infantile Retinal Intelligence Diagnosis System (IRIDS). It can classify nine common infantile fundus diseases and conditions using 7679 retinal images collected from hospitals throughout China. We anticipate significant benefits in improving individual outcomes for preventable infantile fundus diseases, accompanied by a positive cost-effectiveness impact on advanced medical services for the population. This includes the use of innovative DL screening technologies that utilize predictive disease modeling and treatment algorithms tailored to each patient's personalized profile [38].

2.1 Datasets and labeling

Retinal images from four diverse data sources were collected to develop and validate the IRIDS. The primary dataset for training, validation, and testing was collected from the Department of Ophthalmology, Shenzhen Eye Hospital, Shenzhen, Guangdong, China, between April 2004 and January 2021. A smaller portion of the data came from three other maternal and child health hospitals in Guangdong Province. This diagnostic study was approved by the Ethics Committee of Shenzhen Eye Hospital. All institutions abided by the tenets of the Declaration of Helsinki [39], and written informed consent was obtained from the parents of all enrolled infants.

The data acquisition and processing methods are shown in Fig. 1A and Supplementary Fig. 2. All images collected from the various data sources underwent quality screening by two junior ophthalmologists. To maintain a balance among different categories, not all the images were included in the database. For example, if there were too many ROP images, only a representative portion was included to ensure data diversity while still ensuring the total number of images was adequate. Subsequently, the included images were discarded if they were unclear, too dark, or too bright or if the image quality was poor. In total, 2933 images were excluded before the labeling procedure. The included images were independently labeled by two trained junior ophthalmologists to ensure the accuracy of classification by the IRIDS algorithms. Images with conflicting labels underwent review by a senior ophthalmologist, and in cases of uncertainty, the final label was determined by an experienced retina specialist. Finally, images with agreed-upon labels were retained for training, validation, and test datasets for the automated system. Unclassifiable images, either due to uncertain features or being rare diseases not falling into any of the nine categories, were excluded from the analysis. This led to the exclusion of an additional 1874 images, resulting in a final inclusion of 7697 images.

2.2 Classification

All retinal images were classified into nine primary classes/subclasses of diseases and conditions (Supplementary Table 1 and Supplementary Fig. 1) according to fundus signs described in Pediatric Retina [40]. Common fundus diseases with distinct retinal characteristics recognizable on retinal images were classified as independent major classes. To account for real-world scenarios, we included normal retinal images. These major classes included ROP, RP, RB, Coats’ disease, choroid coloboma, and normal and congenital retinal folds.

Some diseases share similar characteristics and are not easily distinguishable solely through fundus imaging. For instance, the symptoms of infants with stage 1 or 2 ROP are milder than those in other stages. To address this, we clustered them together and classified them as “mild” ROP. Likewise, infants with stage 3 ROP are more likely to require treatment with laser photocoagulation or intravitreal injection of anti-vascular endothelial growth factor than infants with mild ROP; thus, we classified this stage as “moderate” ROP. Infants with stage 4 and 5 ROP typically require vitrectomy and/or other surgical treatments, so we clustered them together and classified as “severe” ROP.

Following this clustering, the nine types of diseases and conditions were divided into seven main classes and three subclasses. Each class was assigned a unique class identity number (ID) (Supplementary Table 1).

2.3 Architecture of the IRIDS

The workflow and architecture of the IRIDS are shown in Fig. 1B and 1C. Convolutional neural networks (CNNs) mainly focus on local features, which leads to an inadequate representation of the global features, whereas the transformer network pays more attention to the extraction of global features. As a result, the classification performance was limited when only one structure was used to extract features from the analyzed images. Hence, we applied a hybrid model that combines CNNs and transformer structures, as shown in Fig. 1B. ResBlocks were derived from the ResNet-18 model [41], which is represented as a residual block. A residual block is defined as shown in Fig. 1C (a). The transformer blocks used were the same as those of MaxViT [42], whose main transformer structure is shown in Fig. 1C (b). After obtaining high-level features from the Res-blocks and Transformer blocks, we used the depth-attention fusion module to fuse the local and global features from the paths of the ResNet and MaxViT modules, as illustrated in Fig. 1C (c). Thus, the entire network can express features related to each specific disease and focus on extracting key features for each condition.

The Pytorch library was used to train and test the datasets in all the models. An NVIDIA TITAN XP GPU was used to accelerate the training and testing. The Adam with the betas = (0.9, 0.997), eps = 10^− 8 was chosen as the optimizer for the student network. The discriminators used stochastic gradient descent (SGD) as the optimizer with a momentum of 0.9 and a weight decay of 5×10^− 5. The initial learning rates of the student and discriminator networks were set to 0.001 and then multiplied by 0.1 every 20 epochs. The maximum epoch was set to 100, and the size of the mini-batch was set to 16. All images were resized to 224×224 pixels and used as inputs for all networks.

2.4 Comparison of IRIDS performance with that of ophthalmologists

To compare the performance between the IRIDS and ophthalmologists with clinical experience with fundus diseases, two retina specialists with 10 to 15 years of clinical experience and two senior ophthalmologists with 5 to 10 years of experience were invited to independently classify the same images in the reader study. For this comparison, we used an additional dataset of 450 retinal images obtained from Shenzhen Eye Hospital in 2022. This dataset included both subclass and main class categories, with 50 images in each category.

2.5 Statistical analysis

We evaluated the performance of the IRIDS model using several evaluation metrics, including accuracy, precision, recall, kappa, area under the receiver operating characteristic curve (AUC), and F1-score (F1). Specifically, we employed a 5-fold cross-validation technique to obtain the classification results for all categories, and these metrics were calculated and analyzed for each fold to evaluate the overall performance of the model.

3.1 Data characteristics and prediction system architecture

The entire workflow, simplified architecture of the IRIDS, and primary dataset processing flow are shown in Fig. 1A, 1B, and Supplementary Fig. 2. A total of 7697 images (1089 cases) were collected for model training (80%) and testing (20%). The image data distributions and results for various classes of each dataset are shown in Fig. 1A, Supplementary Fig. 2, and Supplementary Table 2.

3.2 Performance in data test

First, we conducted several comparative experiments using baseline models. The classification results are listed in Table 1. The Res-18 model achieved the best classification performance among the CNN models (Table 1). Among the Transformer models, the MaxViT model achieved the highest accuracy and precision for all classification metrics. Therefore, the Res-18 model was selected to extract the local contextual feature information, and considering its low complexity and high classification performance, the MaxViT model was utilized to extract the global features. In addition, we plotted receiver operating characteristic (ROC) curves for these baseline methods to compare the performance of the models. From this, we found that the Res-18 and MaxVit models achieved the best performance (Fig. 2A).

Table 1

The classification results of different baseline methods (%). The values outside and inside the brackets represent the mean value and variance, respectively.
Method	Accuracy	Precision	Recall	F1-score	Kappa	AUC
Res-18 [41]	93.19 (0.28)	90.62 (2.19)	87.08 (1.47)	88.90 (0.79)	86.93 (0.83)	99.12 (0.001)
Res-34 [42]	93.01 (0.21)	91.60 (0.66)	86.19 (1.69)	88.76 (0.64)	87.20 (0.31)	98.98 (0.001)
Res-50 [43]	92.69 (0.24)	90.73 (0.35)	87.68 (0.34)	89.21 (0.23)	87.01 (0.23)	98.27 (0.005)
ViT-small [44]	92.37 (0.02)	90.19 (0.05)	89.97 (0.09)	90.02 (0.03)	88.56 (0.06)	98.82 (0.001)
RegionViT [43]	90.25 (0.03)	87.69 (0.07)	83.38 (0.14)	85.48 (0.07)	83.51 (0.07)	98.04 (0.02)
MaxViT [42]	94.62 (0.10)	94.07 (0.27)	90.56 (0.69)	92.34 (0.17)	91.15 (0.28)	99.08 (0.005)

3.3 Performance in the advanced test

Through experiments comparing baseline methods, the Res-18 and MaxViT models were chosen as the main structures to build the IRIDS. To fuse the extracted features from the Res-blocks and Transformer blocks, we employed a depth attention (DA) fusion module to efficiently fuse high-level features. Hence, the IRIDS consists of the Res-18, MaxViT, and DA modules. We conducted ablation experiments on these modules to verify their effectiveness; Res-18 + MaxViT + DA represents the proposed method. The experimental results are presented in Table 2. Our proposed method achieved the best classification performance, with accuracy, precision, recall, F1-score, Kappa, and AUC of 96.45%, 95.86%, 94.37%, 95.03%, 94.43%, and 99.51%, respectively.

Table 2

The ablation experiments of the proposed model (%). The values outside and inside of the brackets represent the mean value and variance, respectively.
Method	Accuracy	Precision	Recall	F1-score	Kappa	AUC
Res-18	93.19 (0.28)	90.62 (2.19)	87.08 (1.47)	88.90 (0.79)	86.93 (0.83)	99.12 (0.001)
MaxViT	94.62 (0.10)	94.07 (0.27)	90.56 (0.69)	92.34 (0.17)	91.15 (0.28)	99.08 (0.005)
Res-18 + MaxViT	95.64 (0.02)	96.58 (0.09)	90.77 (0.30)	93.56 (0.08)	92.50 (0.16)	99.39 (0.001)
Res-18 + MaxViT + DA	96.45 (0.03)	95.86 (0.47)	94.37 (0.15)	95.03 (0.21)	94.43 (0.17)	99.51 (0.001)

We observed that the IRIDS achieved the best ROC performance and the highest AUC for each disease in terms of all evaluation metrics (Fig. 2B and 3A). Specifically, for RP and CRF, the proposed method's correct prediction probability exceeded that of other comparison methods by at least five percentage points. All images in the normal category were correctly detected using our method.

3.4 T-distributed stochastic neighbor embedding (T-SNE) visualization

We conducted t-SNE visualization to prove the effectiveness of the extracted features of our proposed method by applying the features of the last feature extraction layer to all the comparison methods, as shown in Fig. 3B. We observed that all predicted categories were better separated by our method than by the other methods (Fig. 3B).

3.5 Comparison of IRIDS with ophthalmologists

The results of the comparison between the IRIDS and ophthalmologists are shown in Fig. 2B. For the diagnosis of six main classes of fundus disease, the AUC of the IRIDS ranged from 0.976 to 0.999, which was higher than or equal to that of retina specialists for three of the six diseases (Coats, Coloboma, and RB). The IRIDS achieved an AUC comparable to that of retina specialists for other fundus diseases (RP, ROP, and CRF). The IRIDS showed a superior AUC compared to senior ophthalmologists for the six main classes.

In addition, we performed a human-machine comparison of the ROP staging model (Fig. 2C and Supplementary Table 3). The results of the IRIDS were better than those of the ophthalmologists for severe ROP, but for the other categories of ROP, the difference was insignificant.

In this study, we developed the IRIDS model, which automatically identified nine infantile fundus diseases and conditions. By comparing multiple algorithms, we established that our model had the highest performance, with an AUC of 99.51%. Furthermore, a five-fold cross-validation confirmed the excellent performance of our model. Compared to retina specialists, the IRIDS exhibits superior diagnostic accuracy.

We first conducted several comparative experiments using baseline models, and their results are presented in Table 1. Finally, we selected the Res-18 and MaxViT models that combined DA modules as the main structures to build the IRIDS. The IRIDS was used to analyze data collected from four hospitals. Despite potential variations in image quality, imaging techniques, and machine types across these hospitals, the model consistently performed excellently, indicating its robust adaptability.

Many CNN-based DL methods focus on local feature extraction, while overlooking valuable information from the pan-retinal viewing field. To address this issue, a transformer model was adopted for vision tasks [45–47], which can be employed to extract long-distance dependent features from input images. Therefore, the integration of CNNs and transformer structures within a single model enables the extraction of both local and global features, ultimately improving the model's classification performance. The inherent opacity of CNNs in image analysis poses a potential obstacle that hinders clinicians from adopting DL techniques [48]. However, our model can effectively extract key local and global features relevant to specific diseases, as demonstrated by the results of the t-SNE analysis. This characteristic allows for an improved classification performance and provides promising potential for its practical clinical applications.

In this study, we investigated the automated detection of infantile fundus diseases using the IRIDS. We have previously reported the use of ultrawide-field retinal images to diagnose ROP ^[31]. Studies have also been conducted on the staging of ROP and the diagnosis of RB and RP in infants [49–51]. However, these studies have mainly focused on detecting a single infantile fundus disease using retinal images. Subsequently, an increasing number of studies have shifted focus towards automating the diagnosis of multiple fundus diseases. Some studies have aimed to distinguish among several fundus diseases using a single algorithm [35]. Stevenson et al. [52] developed a multiclass DL model for detecting fundus diseases using a publicly available retinal image database. This study demonstrated the feasibility of developing artificial intelligence (AI) classifiers using standard laptops without prior programming or AI knowledge. Son et al. [53] used a larger dataset to build a DL model for detecting 12 retinal abnormalities. The external testing set in their study achieved an AUC of up to 98%, indicating exceptional performance. Moreover, the model's interpretable and reliable classification outputs suggest its potential for clinical use as an automated screening system for retinal images. Because these systems detect abnormalities rather than fundus diseases, these models cannot be used directly for screening. Lin et al. [54] developed a DL-based system that prospectively evaluated 18,163 color fundus photographs and detected 14 fundus diseases, achieving an AUC of 0.98. Li et al. [35] developed and prospectively validated a retinal image-based DL algorithm to simultaneously identify ten fundus diseases in clinical practice. This is in line with general screening methods and is practical. Cen et al. [55] used a large database of 600,000 retinal images to establish a Digital Light Processing system with a high F1 score, sensitivity, specificity, and AUC for the detection of 39 fundus diseases and conditions. Ju et al. [56] introduced a novel method that enabled deep neural networks to learn from a comprehensive fundus database with over one million images, facilitating the recognition of various fundus diseases. Gu et al. [57] evaluated the performance of an AI system in detecting multiple fundus diseases in real-world scenarios, specifically primary healthcare settings, and achieved high accuracy. However, these studies only focused on adult fundus diseases and did not include infantile fundus diseases. The IRIDS utilizes non-invasive retinal images, which are easily obtainable, to facilitate automatic screening for multiple infantile fundus diseases and conditions, rendering it more clinically applicable and adaptable than existing approaches.

5.1 Conclusions

By utilizing cutting-edge DL capabilities, the IRIDS accurately detects nine infantile fundus diseases and conditions. Thanks to its exceptional diagnostic precision and reliability, the IRIDS can improve visual outcomes and quality of life for children and families worldwide, especially in regions with a shortage of retina specialists. It can effectively identify and prevent the occurrence of severe complications by promptly detecting retinal lesions. The deployment of IRIDS holds great promise in enhancing early detection and intervention, ultimately leading to improved prognosis and better healthcare access for children in need.

5.2 Predictive approach

By using the depth-attention fusion module to fuse the local and global features from the paths of the ResNet and MaxViT modules, the IRIDS not only achieved automated screening for multiple infantile fundus conditions and diseases, but also demonstrated high diagnostic accuracy comparable to or better than that of ophthalmologists, including retina specialists. As shown in Supplementary Table 1, the IRIDS can automatically detect the most common fundus diseases and conditions in infants, such as Coats’ disease, RB, RP, choroid coloboma, and CRF, with AUC values close to 1.0, indicating excellent performance. The high accuracy of our model, which may be due to the strict dataset selection process [53], was further confirmed by the high AUC results achieved by the IRIDS, despite the high imbalance rate, as reported in a study using a multilabel setting [58]. Within the context of PPPM [38], it would be advantageous to strive to incorporate this new modality into current clinical workflows because it has the potential to provide more personalized and quantifiable methods for the diagnosis of multiple infantile diseases. This further confirms the robustness and reliability of our model, making it a promising tool for the early detection and treatment of infantile fundus disease.

5.3 Targeted prevention

In the context of prevention [38], due to its outstanding performance in human-machine comparison, the IRDS system enables us to identify and promptly treat some infantile fundus diseases that are prone to be overlooked or misdiagnosed, even in the absence of retina specialists. This helps prevent further deterioration of the patient’s condition and halt the progression of the disease. The IRIDS is helpful in reducing the burden on ophthalmologists and improving screening efficiency, especially in less-developed regions.

5.4 Personalization of medical services

We included nine different infantile fundus diseases and conditions from four hospitals. To address the limitations of using a single structure to extract features from image analysis, we employed a hybrid model that combines CNNs and transformer structures. This hybrid model allows simultaneous attention to local and global features. As a result, the AUC (Area Under Curve) for the classification of the nine disease categories reached an impressive 99.51%. Consequently, the entire network can capture disease-specific features and focus on extracting key features for each condition. Therefore, IRIDS can diagnose individual infants based on a single retinal image. Owing to its simplicity and ease of training, the IRIDS can also be applied in environments where there are no professional ophthalmologists, potentially helping to eliminate the problems of underdiagnosis, misdiagnosis, and overexamination.

5.5 Limitations and outlook in the framework of 3PM

This study has a few limitations. First, similar to many other fundus diseases, some infantile fundus diseases lack universal diagnostic criteria or consensus. Mitigating this issue can involve referencing multiple authoritative sources or relevant studies. Second, although we collected data on up to nine common infantile fundus diseases and conditions, we did not include all infantile fundus diseases or conditions. Diseases such as Familial Exudative Vitreoretinopathy and retinal detachment were omitted due to insufficient image data. Adding an “others” class can be an effective way to address this limitation. Third, our research dataset was relatively small and limited to a single geographic region. Furthermore, this dataset includes only Chinese individuals, limiting its universality. Therefore, images from databases in other regions of the world should be included to increase diversity and expand the applicability of the database.

This DL system is the first of its kind to automatically recognize infantile fundus diseases in a non-invasive manner. It also aligns with a crucial aspect of the PPPM approach emphasized at the EPMA World Congress 2019, which focused on customizing and continuously monitoring patients’ clinical parameters to improve treatment outcomes [59]. To facilitate the future application of DL in this field, it is strongly recommended to consider the following points:

1. Ensuring comprehensive detection of infantile fundus diseases or conditions across a wide range of categories, while also enabling intelligent clinical diagnosis applicable to each specific category.
2. Enabling comprehensive prediction of disease progression and facilitating treatment decision assessments related to clinical settings, enabling early application in clinical monitoring. Through the PPPM approach, it aims to provide better clinical outcomes for patients by offering a complete prediction from diagnosis to treatment decision.

Funding/Support: This study was supported by National Natural Science Foundation of China (No. 82271103), Guangdong Basic and Applied Basic Research Foundation (No. 2022A1515012326), Shenzhen Key Medical Discipline Construction Fund (No. SZXK038), Shenzhen Fund for Guangdong Provincial High Level Clinical Key Specialties（No. SZGSP014）, Shenzhen Science and Technology Program (Grant No. JSGG20201102174200001), National Natural Science Foundation of China (Nos. 62106153, U22A2024, 82271103 and 62001302), and supported by Guangdong Basic and Applied Basic Research Foundation (No. 2020A1515110605).

Conflicts interests The authors have no relevant financial or non-financial interests to disclose.

Ethics approval This diagnostic study was approved by the Ethics Committee of Shenzhen Eye Hospital. All institutions abided by the tenets of the Declaration of Helsinki.

Consent to participate Written informed consent was obtained from the parents of all enrolled infants.

Consent for publication This article has been approved for publication by the authors.

Availability of data and material These authors do not have permission to share data except the firsts author.

Code availability The source code used in this study is available upon request from the first author only.

Author Contributions: GZ has full access to all data in the study and is responsible for the integrity of the data and the accuracy of the data analysis. YL, HX and XZ contributed equally and are considered co–first authors. Concept and design: YL, HX and XZ. Acquisition, analysis, or interpretation of data: YL, HX, XZ and SZ. Drafting of the manuscript: YL and HX. Critical revision of the manuscript for important intellectual content: GZ, JT, XZ and DPN. Statistical analysis: YL and JT. Obtained funding: GZ. Administrative, technical, or material support: ZY, ZW, RT, YC, MC, YD, TC, YH and BL. Supervision: GZ.

Chiang MF, Quinn GE, Fielder AR, et al. International Classification of Retinopathy of Prematurity, Third Edition. Ophthalmology. 2021;128(10): e51-e68. doi:10.1016/j.ophtha.2021.05.031
Shields JA, Shields CL, Honavar SG, Demirci H. Clinical variations and complications of Coats disease in 150 cases: the 2000 Sanford Gifford Memorial Lecture. Am J Ophthalmol. 2001;131(5):561-571. doi:10.1016/s0002-9394(00)00883-7.
Spitznas M, Joussen F, Wessing A, Meyer-Schwickerath G. Coat's disease. An epidemiologic and Fluorescein angiographic study. Albrecht Von Graefes Arch Klin Exp Ophthalmol. 1975;195(4):241-250. doi:10.1007/BF00414937.
Rao R, Honavar SG. Retinoblastoma. Indian J Pediatr. 2017;84(12):937-944. doi:10.1007/s12098-017-2395-0.
Pagon RA. Retinitis pigmentosa. Surv Ophthalmol. 1988;33(3):137-177. doi:10.1016/0039-6257(88)90085-9.
Giles K, Raoul C, Yannick B, Peter W. Uveal coloboma: about 3 cases at the University Teaching Hospital, Yaounde, Cameroon. Pan Afr Med J. 2016;24:201. Published 2016 Jul 8. doi:10.11604/pamj.2016.24.201.9770.
Nishina S, Suzuki Y, Yokoi T, Kobayashi Y, Noda E, Azuma N. Clinical features of congenital retinal folds. Am J Ophthalmol. 2012;153(1):81-7. e1. doi:10.1016/j.ajo.2011.06.002.
Liche F, Majji AB. Familial exudative vitreoretinopathy. Ophthalmology. 2012;119(5):1093. doi:10.1016/j.ophtha.2012.02.025.
Fielder A, Blencowe H, O'Connor A, Gilbert C. Impact of retinopathy of prematurity on ocular structures and visual functions. Arch Dis Child Fetal Neonatal Ed. 2015;100(2):F179-F184. doi:10.1136/archdischild-2014-306207.
Golubnitschaja O, Costigliola V; EPMA. General report & recommendations in predictive, preventive and personalised medicine 2012: white paper of the European Association for Predictive, Preventive and Personalised Medicine. EPMA J. 2012;3(1):14. Published 2012 Nov 1. doi:10.1186/1878-5085-3-14.
Good WV. Retinopathy of Prematurity Incidence in Children. Ophthalmology. 2020;127(4S):S82-S83. doi:10.1016/j.ophtha.2019.11.026.
Dimaras H, Corson TW, Cobrinik D, et al. Retinoblastoma. Nat Rev Dis Primers. 2015;1:15021. Published 2015 Aug 27. doi:10.1038/nrdp.2015.21.
Global Retinoblastoma Study Group, Fabian ID, Abdallah E, et al. Global Retinoblastoma Presentation and Analysis by National Income Level [published correction appears in JAMA Oncol. 2020 Nov 1;6(11):1815]. JAMA Oncol. 2020;6(5):685-695. doi:10.1001/jamaoncol.2019.6716.
Chen HY, Lehmann OJ, Swaroop A. Genetics and therapy for pediatric eye diseases. EBioMedicine. 2021;67:103360. doi:10.1016/j.ebiom.2021.103360.
Coleman K, Coleman J, Franco-Penya H, et al. A New Smartphone-Based Optic Nerve Head Biometric for Verification and Change Detection. Transl Vis Sci Technol. 2021;10(8):1. doi:10.1167/tvst.10.8.1.
Golubnitschaja O, Potuznik P, Polivka J Jr, et al. Ischemic stroke of unclear aetiology: a case-by-case analysis and call for a multi-professional predictive, preventive and personalised approach. EPMA J. 2022;13(4):535-545. Published 2022 Nov 17. doi:10.1007/s13167-022-00307-z.
Topol EJ. High-performance medicine: the convergence of human and artificial intelligence. Nat Med. 2019;25(1):44-56. doi:10.1038/s41591-018-0300-7.
Baek SU, Lee WJ, Park KH, Choi HJ. Health screening program revealed risk factors associated with development and progression of papillomacular bundle defect. EPMA J. 2021;12(1):41-55. Published 2021 Mar 4. doi:10.1007/s13167-021-00235-4.
Li S, Li M, Wu J, et al. Development and validation of a routine blood parameters-based model for screening the occurrence of retinal detachment in high myopia in the context of PPPM. EPMA J. 2023;14(2):219-233. Published 2023 Mar 15. doi:10.1007/s13167-023-00319-3.
Golub TR, Slonim DK, Tamayo P, et al. Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. Science. 1999;286(5439):531-537. doi:10.1126/science.286.5439.531.
Wang Y, Tetko IV, Hall MA, et al. Gene selection from microarray data for cancer classification--a machine learning approach. Comput Biol Chem. 2005;29(1):37-46. doi:10.1016/j.compbiolchem.2004.11.001.
Yu KH, Levine DA, Zhang H, Chan DW, Zhang Z, Snyder M. Predicting Ovarian Cancer Patients' Clinical Response to Platinum-Based Chemotherapy by Their Tumor Proteomic Signatures. J Proteome Res. 2016;15(8):2455-2465. doi:10.1021/acs.jproteome.5b01129.
Yu KH, Fitzpatrick MR, Pappas L, Chan W, Kung J, Snyder M. Omics AnalySIs System for PRecision Oncology (OASISPRO): a web-based omics analysis tool for clinical phenotype prediction. Bioinformatics. 2018;34(2):319-320. doi:10.1093/bioinformatics/btx572.
Check Hayden E. The automated lab. Nature. 2014;516(7529):131-132. doi:10.1038/516131a.
Dai L, Wu L, Li H, et al. A deep learning system for detecting diabetic retinopathy across the disease spectrum. Nat Commun. 2021;12(1):3242. Published 2021 May 28. doi:10.1038/s41467-021-23458-5.
Shon K, Sung KR, Shin JW. Can Artificial Intelligence Predict Glaucomatous Visual Field Progression? A Spatial-Ordinal Convolutional Neural Network Model. Am J Ophthalmol. 2022;233:124-134. doi:10.1016/j.ajo.2021.06.025.
Ee CL, Samsudin A. COMPARISON OF SMARTPHONE-BASED AND AUTOMATED REFRACTION WITH SUBJECTIVE REFRACTION FOR SCREENING OF REFRACTIVE ERRORS. Ophthalmic Epidemiol. 2022;29(5):588-594. doi:10.1080/09286586.2021.1986550.
Chew EY. Age-related Macular Degeneration: Nutrition, Genes and Deep Learning-The LXXVI Edward Jackson Memorial Lecture [published correction appears in Am J Ophthalmol. 2021 Nov;231:212]. Am J Ophthalmol. 2020;217:335-347. doi:10.1016/j.ajo.2020.05.042.
Panwar N, Huang P, Lee J, et al. Fundus Photography in the 21st Century--A Review of Recent Technological Advances and Their Implications for Worldwide Healthcare. Telemed J E Health. 2016;22(3):198-208. doi:10.1089/tmj.2015.0068.
J. Zhao et al., "A Deep Learning Framework for Identifying Zone I in RetCam Images," in IEEE Access, vol. 7, pp. 103530-103537, 2019, doi: 10.1109/ACCESS.2019.2930120.
Y. Zhang, L Wang, Z Wu, et al., Development of an Automated Screening System for Retinopathy of Prematurity Using a Deep Neural Network for Wide-Angle Retinal Images, IEEE Access, 2019. 7, 10232-10241. doi: 10.1109/access.2018.2881042.
Rugang Zhang, Jinfeng Zhao, Hai Xie, et al., Automatic diagnosis for aggressive posterior retinopathy of prematurity via deep attentive convolutional neural network. Expert Systems with Applications, 2022. 187. 115843. doi: 10.1016/j.eswa.2021.115843.
Maji D, Sekh AA. Automatic Grading of Retinal Blood Vessel in Deep Retinal Image Diagnosis. J Med Syst. 2020;44(10):180. Published 2020 Sep 1. doi:10.1007/s10916-020-01635-1.
Xie H, Lei H, Zeng X, et al. AMD-GAN: Attention encoder and multi-branch structure based generative adversarial networks for fundus disease detection from scanning laser ophthalmoscopy images. Neural Netw. 2020;132:477-490. doi:10.1016/j.neunet.2020.09.005.
Dong L, He W, Zhang R, et al. Artificial Intelligence for Screening of Multiple Retinal and Optic Nerve Diseases. JAMA Netw Open. 2022;5(5):e229960. Published 2022 May 2. doi:10.1001/jamanetworkopen.2022.9960.
Cen LP, Ji J, Lin JW, et al. Automatic detection of 39 fundus diseases and conditions in retinal photographs using deep neural networks. Nat Commun. 2021;12(1):4828. Published 2021 Aug 10. doi:10.1038/s41467-021-25138-w.
Patel TP, Kim TN, Yu G, et al. Smartphone-Based, Rapid, Wide-Field Fundus Photography for Diagnosis of Pediatric Retinal Diseases. Transl Vis Sci Technol. 2019;8(3):29. Published 2019 May 30. doi:10.1167/tvst.8.3.29.
Golubnitschaja O, Baban B, Boniolo G, et al. Medicine in the early twenty-first century: paradigm and anticipation - EPMA position paper 2016. EPMA J. 2016;7(1):23. Published 2016 Oct 25. doi:10.1186/s13167-016-0072-4.
YS W, pediatric retina [M]. Xi'an: Fourth Military Medical University Press, 2013.
World Medical Association. World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects. JAMA. 2013;310(20):2191-2194. doi:10.1001/jama.2013.281053.
K He, X Zhang, S Ren, et al., Deep residual learning for image recognition. in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016: p. 770-778.
Z Tu, H Talebi, H Zhang, et al., Maxvit: Multi-axis vision transformer. arXiv preprint arXiv, 2022: p. 2204.01697. doi: arxiv-2204.01697.
Chen CF, R Panda and Q Fan, Regionvit: Regional-to-local attention for vision transformers. arXiv preprint arXiv:2106.02689, 2021. doi: arxiv-2106.02689.
L S Lee, S H Song and BC Song, Vision transformer for small-size datasets. arXiv preprint arXiv:2112.13492, 2021. doi: arxiv-2112.13492
Yu S, Ma K, Bi Q, et al., Mil-vt: Multiple instance learning enhanced vision transformer for fundus image classification[C]. International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, Cham, 2021: p. 2021: 45-54.
Playout C, Duval R, Boucher MC, Cheriet F. Focused Attention in Transformers for interpretable classification of retinal images. Med Image Anal. 2022;82:102608. doi:10.1016/j.media.2022.102608.
Sun R, Li Y, Zhang T, et al., Lesion-aware transformers for diabetic retinopathy grading. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition., 2021: p. 10938-10947.
Wong TY, Bressler NM. Artificial Intelligence With Deep Learning Technology Looks Into Diabetic Retinopathy Screening. JAMA. 2016;316(22):2366-2367. doi:10.1001/jama.2016.17563.
Ng WY, Zhang S, Wang Z, et al. Updates in deep learning research in ophthalmology. Clin Sci (Lond). 2021;135(20):2357-2376. doi:10.1042/CS20210207.
Huang YP, Basanta H, Kang EY, et al. Automated detection of early-stage ROP using a deep convolutional neural network. Br J Ophthalmol. 2021;105(8):1099-1103. doi:10.1136/bjophthalmol-2020-316526.
J T Durai C, Alelyani S and Mubharakali A. Early Prediction and Diagnosis of Retinoblastoma Using Deep Learning Techniques. arXiv, 2021. 1(2103). doi: arxiv-2103.07622
Stevenson CH, Hong SC, Ogbuehi KC. Development of an artificial intelligence system to classify pathology and clinical features on retinal fundus images. Clin Exp Ophthalmol. 2019;47(4):484-489. doi:10.1111/ceo.13433.
Son J, Shin JY, Kim HD, Jung KH, Park KH, Park SJ. Development and Validation of Deep Learning Models for Screening Multiple Abnormal Findings in Retinal Fundus Images. Ophthalmology. 2020;127(1):85-94. doi:10.1016/j.ophtha.2019.05.029.
Lin D, Xiong J, Liu C, et al. Application of Comprehensive Artificial intelligence Retinal Expert (CARE) system: a national real-world evidence study. Lancet Digit Health. 2021;3(8):e486-e495. doi:10.1016/S2589-7500(21)00086-8.
Cen LP, Ji J, Lin JW, et al. Automatic detection of 39 fundus diseases and conditions in retinal photographs using deep neural networks. Nat Commun. 2021;12(1):4828. Published 2021 Aug 10. doi:10.1038/s41467-021-25138-w.
Ju L, Yu Z, Wang L, et al. Hierarchical Knowledge Guided Learning for Real-world Retinal Disease Recognition [published online ahead of print, 2023 Aug 7]. IEEE Trans Med Imaging. 2023;PP:10.1109/TMI.2023.3302473. doi:10.1109/TMI.2023.3302473.
Gu C, Wang Y, Jiang Y, et al. Application of artificial intelligence system for screening multiple fundus diseases in Chinese primary healthcare settings: a real-world, multicentre and cross-sectional study of 4795 cases [published online ahead of print, 2023 Mar 6]. Br J Ophthalmol. 2023;bjo-2022-322940. doi:10.1136/bjo-2022-322940.
Zhu H, Cheng C, Yin H, et al. Automatic multilabel electrocardiogram diagnosis of heart rhythm or conduction abnormalities with deep learning: a cohort study [published correction appears in Lancet Digit Health. 2020 Sep;2(9):e455]. Lancet Digit Health. 2020;2(7):e348-e357. doi:10.1016/S2589-7500(20)30107-2.
Golubnitschaja O, Topolcan O, Kucera R, Costigliola V; EPMA. 10th Anniversary of the European Association for Predictive, Preventive and Personalised (3P) Medicine - EPMA World Congress Supplement 2020. EPMA J. 2020;11(Suppl 1):1-133. Published 2020 Aug 19. doi:10.1007/s13167-020-00206-1.

Download PDF

Journal Publication

published 15 Feb, 2024

Read the published version in EPMA Journal →

Reviewers agreed at journal
02 Oct, 2023
Reviewers invited by journal
01 Oct, 2023
Editor assigned by journal
21 Sep, 2023
First submitted to journal
19 Sep, 2023

You are reading this latest preprint version

Automated Detection of Nine Infantile Fundus Diseases and Conditions in Retinal Images Using a Deep Learning System

Status:

Journal Publication

Version 1

Abstract

Figures

1. Introduction

2. Methods

2.1 Datasets and labeling

2.2 Classification

2.3 Architecture of the IRIDS

2.4 Comparison of IRIDS performance with that of ophthalmologists

2.5 Statistical analysis

3. Results

3.1 Data characteristics and prediction system architecture

3.2 Performance in data test

3.3 Performance in the advanced test

3.4 T-distributed stochastic neighbor embedding (T-SNE) visualization

3.5 Comparison of IRIDS with ophthalmologists

4. Discussion

5. Conclusions, expert recommendations, and outlook in the framework of 3PM

5.1 Conclusions

5.2 Predictive approach

5.3 Targeted prevention

5.4 Personalization of medical services

5.5 Limitations and outlook in the framework of 3PM

Declarations

References

Supplementary Files

Status:

Journal Publication

Version 1