Recently, CNN-based image classification networks have been used to diagnose gastrointestinal malignancies from endoscopic images, demonstrating the rapid prediction capability of computer-vision methods prior to receiving pathological results. Notably, such technology-enabled capabilities provide great assistance to endoscopists of all skill levels. Therefore, we developed and validated our novel EoE screening tool using sca4U-Net trained with a small dataset of endoscopic images, assessed after augmentation with the EREFS scoring system.
Development and evolution of endoscopy imaging, particularly, imaged-enhanced endoscopy shows great potential in diagnosing gastrointestinal lesions. However, endoscopists often have difficulty detecting and diagnosing EoE due to its rarity. For these reasons, diagnosis may be delayed or misdiagnosed. Therefore, early diagnosis is crucial to follow-up care and treatment. Our method improves overall EoE diagnosis with deep machine learning.
Our proposed sca4U-Net reliably outperformed the classical CNNs[10–12] in identifying EoE positive vs. EoE negative (accuracy: 86.9%; AUROC: 0.93). Additionally, we determined that the deepest skip connection of the U-Net [24] offers the best feature extraction. The classical CNNs sometimes show good (but mixed) performance, but the Grad-CAM [23] showed that they are inappropriate from the viewpoint of clinical usage. Table 2 shows the accuracy results of sca4U-Net in detecting the EoE features using the EREFS scoring system. We also compared the diagnostic performances of human endoscopists and sca4U-Net, finding that the sensitivity and NPV of sca4U-Net was significantly greater than all three groups of endoscopists, and the specificity and PPV of sca4U-Net was between the FAC and FEL2 groups. Moreover, our model sca4U-Net predicted probability between groups of low and high EREFS scores showed significant difference, which can imply (but not strictly) that our model can detect EoE regardless of its severity (see Supplementary Section A).
In the study, we divided the endoscopists into FAC, FEL2, and FEL1 groups based on their experience levels. FEL1 comprised medical fellows who lacked EoE endoscopic diagnosis experience, group FEL2 was consisted of fellows with some experience (fellows ≥ 1 year ), whereas those in FAC had abundant experience. The sensitivity and NPV of the sca4U-Net were significantly greater than those of the other groups, suggesting that sca4U-Net can be used by all endoscopist groups as a screening tool to aid in EoE diagnosis. However, it might not be a fair comparison, since endoscopists usually diagnose patients in video, not still image. The strength of this study is that, it is the first validating a novel EoE screening tool using a U-Net CNN in the real world with high accuracy (86.9%).
Our study has several limitations. First, it is a single-center study with a retrospective design based on observational data, which needs to be re-evaluated in a multi-center study. Secondly, the collected dataset was relatively small which increases the risk of overfitting. To overcome this problem, we applied both hard and soft augmentations when considering the rarity of EoE. However, the models cannot be trained further since their learning curves showed the symptoms of overfitting after epoch of mid 40’s. In addition, multi-center or multinational studies are required to further expand the validity of this tool. Third, this study was performed based on still images. For more clinically meaningful applications, this work needs to be applied on video data with a 3D CNN [25, 26]. However, it is the first study which conducted multi-tasking experiments with multiple CNN models with robust accuracy.
In conclusion, our method showed robust performance compared to expert endoscopists, which could assist endoscopists of all experience levels in the early detection of the infrequent but clinically significant EoE. Multi-center prospective studies with mature datasets are needed for clinical application.