This study investigated the diagnostic performances of CNN, ML, and ensemble models using OCT images obtained during eye checkups. The CNN models focused on the structural changes in the retina in abnormal eyes, with accuracy from 95–96% (Figs. 1 and 2). Our finding is consistent with earlier studies that reported a classification accuracy of a single CNN model of about 70–95%; this model may also miss some diseases using OCT images.10,11,13,24,25 RP was false-negatively predicted, suggesting insufficient training with abnormalities in the retinal pigment epithelium and photoreceptor layer, including interdigitation and ellipsoid zones in the external limiting membrane. In cases with no apparent edema in the inner retinal layer, our CNN models tended to predict an abnormality based on the peripheral temporal and nasal retinal shapes (Fig. 3). Russakoff et al.26 described CNN models trained with OCT images of age-related macular degeneration that focused on the temporal and nasal retinas and differentiated between progressors and non-progressors. These findings suggest that the CNN can detect subtle differences in the morphology of the peripheral temporal and nasal retinal regions to differentiate between abnormal and normal eyes or between progressors and non-progressors.
The ML model used the central macular (segment 3) and nasal (segments 4 and 5) areas as bases for determining eye abnormalities, as a significant difference was found between abnormal and normal retinal areas in each segment (Fig. 5). Of the five diseases that were misdiagnosed by the ML model, the retinal and choroidal areas were underestimated because the morphological transformation did not fill in the inner retinal layer with white in ERM (Figs. 7A and 7A′) and macular edema (Figs. 7B, 7B′, 7C, and 7C′). Furthermore, the OCT images of RP had small retinal and choroidal areas (Figs. 7D, 7D′, 7E, and 7E′). Most diseases were correctly classified by the ML model, although cases with an underestimated macular area were misclassified. Therefore, the features of the retinal and choroidal areas extracted by dividing the OCT images into five segments were useful.
The ensemble model had a significant better diagnostic performance than the single CNN models (Fig. 6). The findings show that the OCT images misclassified by each CNN model varied. Conversely, the ensemble model improved the diagnostic performance for retinal disease. However, the risk of misjudging normal features as abnormal remains.
Nonetheless, the ensemble with the ML model significantly improved that risk. The ML model, which evaluated for disease using the nasal retinal area, thereby improving on the CNN models, which misrecognized the nasal peripheral retinal structures (Figs. 3 and 5). Thus, the ensemble model combining the CNN models trained with OCT images and the ML model trained in the retinal area can improve disease prediction during an eye or health checkup in which only OCT images are acquired. Furthermore, the ensemble with the ML model may be useful to clinicians, given its diagnostic accuracy of 0.999 at 0.025 image/s.
The purpose of screening during an eye checkup is to detect abnormalities. The ensemble model used in our study successfully performed screening using already labeled data. However, we did not evaluate whether our ensemble model could be adapted to anonymous screening data. Moreover, we excluded OCT images in which the ophthalmologists had difficulty determining the disease by reading the images alone. The OCT image quality is higher at the clinic than during the eye checkup because the eye checkup often prioritizes speed and minimizes image processing (e.g., the number of images is decreased to enable averaging). The shift in the dataset is one of the problems encountered in applying artificial intelligence from the laboratory level to actual practice.27 The performance of artificial intelligence depends on the training dataset. Thus, data distribution (e.g., image quality or unknown disease) differs from the training dataset and, consequently, decreases prediction performance. To prevent a dataset shift in this study, we used single shot OCT images captured during eye checkups. However, because we did not divide into segmention of each retinal layer in the OCT images, the ML model included information about the choroid, and we expected data shifts to occur. Therefore, the accuracy of the ensemble model at actual eye checkups will need to be confirmed in a future investigation.