Measurement of Retinal Nerve Fiber Layer Thickness with a Deep Learning Algorithm in Ischemic Optic Neuropathy and Optic Neuritis

Abstract

This work aims at determining the ability of a deep learning (DL) algorithm to measure retinal nerve fiber layer (RNFL) thickness from optical coherence tomography (OCT) scans in anterior ischemic optic neuropathy (NAION) and demyelinating optic neuritis (ON). The training/validation dataset included 750 RNFL OCT B-scans. Performance of our algorithm was evaluated on 194 OCT B-scans from 70 healthy eyes, 82 scans from 28 NAION eyes, and 84 scans of 29 ON eyes. Results were compared to manual segmentation as a ground-truth and to RNFL calculations from the built-in instrument software. The Dice coefficient for the test images was 0.87. The mean average RNFL thickness using our U-Net was not different from the manually segmented best estimate and OCT machine data in control and ON eyes. In NAION eyes, while the mean average RNFL thickness using our U-Net algorithm was not different from the manual segmented value, the OCT machine data were different from the manual segmented values. In NAION eyes, the MAE of the average RNFL thickness was 1.18 ± 0.69 µm and 6.65 ± 5.37 µm in the U-Net algorithm segmentation and the conventional OCT machine data, respectively (P = 0.0001).

Introduction

Optical coherence tomography (OCT) has been applied to measure peripapillary RNFL thickness at a micrometer scale in several optic neuropathies such as non-arteritic anterior ischemic optic neuropathy (NAION) and demyelinating optic neuritis (ON). [1–3] In the early stages of NAION and papillitis, OCT of the RNFL may show thickening that decreases in the subacute phase. An accurate measurement of the thickness is important as RNFL thickness may be useful for detection of disease progression or improvement.[2, 3] Commercial OCT machines use automated retinal layer segmentation algorithms to detect the difference in signal intensity between adjacent retinal layers to calculate RNFL thickness. However, misidentification of the anterior and posterior boundaries of the RNFL and incomplete segmentation are artifacts that are consistently described.[4,5 ] Of note, the baseline errors persist into the subsequent scans, and errors are propagated longitudinally.[6] Therefore, manual refinement of OCT retinal layer segmentation when assessing RNFL thickness by an operator is recommended, but this process is laborious and extremely time consuming. In addition, segmentation errors can lead to clinical misdiagnosis of neuro-ophthalmic diseases if they go unrecognized.[7]

Deep learning (DL) is a type of artificial intelligence that uses multilayer neural networks, and its algorithms outperform ophthalmologists in disease detection.[8] Furthermore, DL algorithms have been trained to detect errors in automated RNFL segmentation of OCT scans in glaucoma, identifying the probability of a segmentation artifact as well as highlighting the location of these errors using a heat-map with an accuracy of 92.4%.[9]

Since accurate RNFL thickness data is necessary for diagnosis and follow up of ischemic optic neuropathy and optic neuritis [10], we hypothesized that a DL approach with accurate segmentation of the retinal layers would allow estimation of the thickness of the RNFL which is comparable to manual segmentation. In addition, we hypothesized this approach would outperform conventional OCT machine RNFL data segmentation algorithms.

Results

The training/validation dataset included 750 RNFL OCT B-scans from 250 eyes (60 eyes with NAION, 50 eyes with ON, and 140 control eyes). These study eyes were split into training (80% of the sample) and validation (20%) datasets.

Performance of our algorithm was evaluated in 370 scans acquired in 132 eyes from the test set. There was no overlap between the training and testing sets.

After excluding 10 scans due to centration and quality issues, the final test set consisted of 194 OCT B-scans from 70 healthy eyes, 82 scans from 28 NAION eyes, and 84 scans of 29 ON eyes. The mean of the manually corrected average RNFL thickness (ground-truth) was significantly lower in NAION (69.7± 19.3 μm) and ON (76.0 ±16.2 μm) eyes compared to control (100.2 ±10.8 μm) eyes (P < 0.001, Kruskal-Wallis). There was no difference in average RNFL thickness between ON and NAION eyes (P=0.96). NAION scans had lower OCT quality scores than ON scans (23.7 ± 4.5 vs 25.7±4, P=0.03, Kruskal-Wallis) and control scans (26.8 ± 4, P< 0.001). In all datasets, the mean age at OCT scan was 62.8 ± 9.4 years in the NAION group, 30.8 ± 10.1 years in the ON group, and 42.3 ± 18.4 in the controls. 

  We conducted a two-step process to obtain our results: the first step was our U-net evaluation, and the second step was considering the RNFL thickness measurement using the U-net algorithm in the three different groups (control, ON, NAION) for the test images [11,12].

Our U-net model yielded high performance in the test and validation images. The sensitivity and specificity of our proposed model on the validation data sets were 0.91 and 0.90, respectively. The same measures on the test sets were 0.88 and 0.86, respectively. The Dice coefficient between our proposed segmentation and manual segmentation by an expert for validation data set was 0.90, and for the test images was 0.87.

We also compared the estimate of RNFL thickness measurements in seven sectors (average, nasal, temporal, superior-temporal, superior-nasal, inferior-temporal, inferior-nasal) by three different methods (our U-Net algorithm, conventional OCT machine data, and the manual segmented best estimate determined by the ophthalmologist) in the three different groups (Table 1):

Table 1

Comparison of estimate of RNFL thickness measurements (µm) in seven sectors by three different methods in the control, non-arteritic anterior ischemic optic neuropathy (NAION) and demyelinating optic neuritis (ON). 

1. RNFL thickness in the normal control group: 

There was no significant difference in average RNFL thickness amongst the three methods of measurements (P=0.69, ANOVA). The mean average RNFL thickness using our U-Net algorithm segmentation was not different from the manual segmented best estimate (ground truth) (101.1 ± 10.8 µm, vs 100.2 ± 10.8 µm; P>0.99). Similarly, there was no significant difference between the OCT machine average RNFL thickness (100.2±11.2) and the manually segmented value (P>0.99). Both RNFL thickness from U-Net algorithm segmentation and the conventional OCT machine data were strongly correlated with RNFL thickness obtained from manual segmentation (r² =0.99 and 0.98) with no significant difference between two correlations (P=0.33) (Figure 3). Mean absolute error (MAE) of the average RNFL thickness was 1.04 ± 0.74 μm and 0.18 ± 1.23 µm in the U-Net algorithm segmentation and the conventional OCT machine data, respectively. There was no significant difference between the two MAE numbers (P= 0.93).  

2.  RNFL thickness in the NAION group: 

The Kruskal-Wallis test showed a significant difference in average RNFL thickness amongst the three methods of measurement (P=0.02). While the mean average RNFL thickness using our U-Net algorithm was not different from the manually segmented value (ground truth) (70.5 ± 19.4 µm, vs 69.7 ± 19.3 µm, respectively; P>0.99), the OCT machine RNFL thickness (64.4 ± 21.3 µm ) was lower than the manual segmented value (P=0.04). Furthermore, a significant difference was also found between U-Net calculated RNFL thickness and OCT machine thickness (P=0.009). The correlation between the manually segmented RNFL thickness and the U-Net average RNFL thickness (r=0.99) was stronger than the correlation between manually segmented RNFL thickness and the OCT machine RNFL thickness (r=0.95) (P=0.02) (Figure 3, 4). The MAE of the average RNFL thickness was 1.18 ± 0.69 μm and 6.65 ± 5.37 μm in the U-Net algorithm segmentation and the conventional OCT machine data, respectively. There was a significant difference between the two MAE numbers (P=0.0001). Specifically, the MAE for nasal, nasal superior, and temporal superior RNFL thicknesses with U-Net segmentation were 1.48 ± 1.26 μm, 1.64 ± 2.19 μm, and 1.69 ± 1.38 μm, respectively. The MAE for the corresponding thickness sectors with the conventional OCT machine were 6.16 ± 10.02 μm, 5.08 ± 10.47 μm, and 6.26 ± 13.16 μm, respectively.  

3. RNFL thickness in the ON group

The average RNFL thickness was not significantly different amongst the three methods of measurement (P=0.66 Kruskal-Wallis). The mean average RNFL thickness was 76.1±16.2 µm with manual segmentation, and 77.1±16 µm versus 75.9±16.6 µm using U-net algorithm segmentation and the OCT machine, respectively. Both average RNFL thicknesses from U-net algorithm segmentation and the conventional OCT machine data were strongly correlated with RNFL thickness obtained from manual segmentation without a significant difference between­­ them (r =0.99 and 0.99, respectively, P=033) (Figure 3). The MAE of the average RNFL thickness was 0.2 ± 1 μm and 1.2 ± 0.71 μm with OCT machine and U-Net segmentation without a significant difference between them (P=0.93).

This study investigated quantification of peripapillary RNFL thickness on OCT with deep learning (U-Net) in NAION eyes, ON eyes and controls. We compared the manually segmented (ground-truth) estimate of RNFL thickness with our U-Net algorithm and conventional OCT machine data. First, we showed high performance of our U-Net model in the test and validation images. The Dice coefficient between our model and manual segmentation for the validation data set was 0.90, and for the test images was 0.87. Second, both the RNFL thickness from U-Net algorithm segmentation and the conventional OCT machine data were similar to the RNFL thickness obtained from manual segmentation in control and ON eyes. However, in NAION eyes, the mean average RNFL thickness using the OCT machine was different from the manual segmentation (Figure 5). In these eyes our U-Net algorithm was not different from the manually segmented value (ground truth).

Errors in segmentation of the RNFL and estimates of its thickness are not uncommon and could lead to disease misdiagnosis. Several studies have demonstrated high rates of errors on peripapillary RNFL segmentation in glaucomatous eyes.[4-6,13,14]  Mansberger et al.[5] have found that automated OCT machine data resulted in a 1.6 μm thinner RNFL thickness than the ground-truth measurements determined by manual refinement. Manual refinement changed 8.5% of scans to a different global glaucoma classification wherein 23.7% of borderline classifications become normal. A few studies have shown RNFL segmentation problems in neuro-ophthalmology and its impact on disease follow up.[7] In this study we showed that the mean global RNFL obtained from OCT machine automated segmentation has MAE of 6.65 ± 5.37 μm in NAION eyes compared to ground-truth manual segmentation. However, this error was significantly lower in ON eyes and controls. The reasons for segmentation error in NAION eyes are multifactorial. Another study found three common sources of RNFL imaging artifacts: posterior vitreous detachments, high myopia, and epiretinal membranes, with the third being the most common culprit.[13] Our NAION eyes, which were older than ON and control eyes, likely had a higher frequency of vitreous detachment and epiretinal membrane. Other studies also indicated that the difference between the automated and ground-truth thickness increased with older age, thinner RNFL thickness, and lower scan quality.[5] Miki et al.[14] also showed a 20.7 % segmentation failure in glaucoma eyes, which is significantly correlated with low signal strength index and large disc area.[14] Of note, our NAION scans had lower scan quality than ON and control scans. It seems that sub-optimal scan quality reduces the accuracy of automated segmentation. A reduction in signal strength from a media opacity such as dry eye, corneal opacities, and cataract or vitreous opacities can result in artifacts in layer segmentation and interpretation.⁷

Several studies have used deep learning algorithms for retinal segmentation in normal and age-related macular degeneration eyes.[15-17] In the optic nerve head scans, Devalla et al.[18] developed a DL algorithm which achieved good accuracy when compared to manual segmentation. Jammal et al.[9] also developed a DL algorithm that detects errors in RNFL segmentation in glaucoma and normal eyes. In this study, the test sample consisted of scans with at least one RNFL segmentation error and scans without error as defined by a human grader, and the algorithm was trained to output the probability of a segmentation error in test data. For a probability cut point of 0.5, the DL algorithm was 95.0% sensitive and correctly identified 1,172 of the 1,234 scans that had any segmentation error(s) in the test sample. The same group in another study predicted the RNFL thickness from raw unsegmented scans using DL.[19] In images without segmentation errors, they found a high correlation of segmentation-free DL RNFL predictions with conventional OCT RNFL thickness calculations. In low-quality images with segmentation errors, segmentation-free DL predictions had higher correlation with the best available estimate compared to those from the conventional OCT machine. The MAE was 4.98 ± 5.85 μm for DL RNFL estimates and 8.59 ± 11.26 μm for OCT machine estimates.[19] However, the ground truth in their study was considered the best available estimate from a good quality scan of the conventional OCT, rather than manual segmented data as a ground truth as in our work. We found MAE of the average RNFL thickness in NAION eyes with lower scan quality was 1.18 ± 0.69 μm and 6.65 ± 5.37 μm in the U-Net algorithm segmentation and the OCT machine data, respectively. Interestingly, in ON and controls, both the average RNFL thicknesses from U-net algorithm segmentation and the conventional OCT machine data were strongly correlated with RNFL thickness obtained from manual segmentation without a significant difference between­­ them. 

Our study had several limitations. First, our data set was smaller than in glaucoma studies, which is expected in light of the relative rarity of other optic neuropathies compared to glaucoma. In addition, we do not know if the segmentation performance would improve when trained upon a larger data set. Second, our U-Net was trained with the images from the Spectralis OCT machine and therefore, we could not extrapolate our algorithm to scans of other OCT devices. Finally, our supervised DL algorithm was trained to be only as good as the manual segmentation according to an ophthalmologist which was subject to bias. 

 Overall, using U-Net, we were able to segment the RNFL layer in three groups of eyes. When trained and tested on compensated images, there was good correlation with manual segmentation in control eyes, ON eyes, and NAION eyes. In contrast, conventional OCT machine segmentations were prone to errors in NAION eyes, resulting in inaccurate RNFL thickness measurements. In addition, in lower quality scans, our U-Net segmentation performance was similar to ground truth, and therefore this algorithm may provide robust RNFL thickness estimates both in good quality images as well as in those that are prone to segmentation errors such as may occur in NAION eyes. Such an algorithm could be helpful in clinical practice for assessing RNFL thickness in NAION eyes as well as ON eyes.

Material And Methods

Subjects

This was a cross-sectional comparative study in eyes with post-acute ON and NAION that were examined at Farabi Eye Hospital between May 2015 and April 2020. The study was approved by the Ethics Committee of Tehran University of Medical Sciences, and all investigations adhered to the tenets of the Declaration of Helsinki. In addition, informed consent was obtained from all subjects.

All subjects underwent a standard ophthalmic evaluation, including best-corrected visual acuity (BCVA), slit-lamp biomicroscopy, and SD-OCT imaging (Spectralis, HEYEX software 6.0 Heidelberg Engineering, Heidelberg, Jena, Germany) for peripapillary RNFL thickness measurement.

Diagnosis of NAION was defined by: 1) a history of sudden visual loss >3 months prior to enrollment, with documented acute optic disc edema, 2) complete resolution of disc edema at the time of study, and 3) optic disc-related visual field defects.[10] Diagnosis of ON was defined by: 1) an attack of painful, subacute vision loss between 3 and 12 months prior to enrollment in patients age 18 to 50 years, and 2) a gadolinium-enhanced MRI demonstrating optic nerve enhancement with or without periventricular plaques typical of multiple sclerosis.[10] In both groups, patients with other ocular pathology such as glaucoma and giant cell arteritis or a history of other autoimmune diseases were excluded.

Spectral-domain optical coherence tomography: Images of the peripapillary RNFL were acquired using the Spectralis SD-OCT, and 2-4 images for each study eye were used. Segmentation lines were delineated by the conventional OCT software in the Spectralis SD-OCT, including the anterior and posterior RNFL boundaries in the circle scan corresponded to the internal limiting membrane and the inner plexiform layer, respectively. The peripapillary RNFL measurements thus delineated by the OCT instrument were recorded. All OCT machine segmentations were inspected by an ophthalmologist (E.H) for possible segmentation errors, manual corrections of segmentation were made to the B-scans, and subsequently RNFL thickness data from corrected segmentations were recorded as “ground-truth” data.

Image processing and deep learning algorithm:

We developed an algorithm for estimating RNFL thickness based on deep learning. At the first step, a preprocessing method was used to reduce speckle noise. After that, a segmentation algorithm based on a U-Net algorithm was implemented to delineate the contours of the RNFL. In a post-processing step, to eliminate any speckled patterns in the segmentation results, a morphological method was applied. Finally, RNFL thickness was determined using a pipeline of image processing methods and the deep learning model.

Pre-processing

Speckle noise was reduced as the primary objective of the preprocessing step (Figure 1). To minimize these image artifacts, morphological filters (using OpenCV version 4.5.1, kernel dimension: square; size: 3×3) were used.

Retinal Nerve Fiber Layer Segmentation using U-Net

In this study, the U-Net is adopted for segmentation of the RNFL. U-Net is an accurate architecture developed by Ronneberger et al.[11] (a detailed architecture is shown in Figure 2). To train the network and make predictions on test images, images were resized to 256×256 pixels by zero padding. The input image was passed to the network, and at the last convolution layer a binary mask was produced by the network that included the RNFL region. There are skip connections between encoders and decoders in the U-Net architecture. Max-pooling is used to decrease the input size by a factor of two and to capture contextual information at different resolution levels. The resolution is restored to the original level in the decoder module by up-sampling using Up-Convolution, allowing for precise localization. Moreover, the input images are passed through a series of convolutional layers with the ReLU activation function. The designed U-net utilized increasing numbers of convolutional filters (16, 32, 64, 128, 256) in the encoder and the symmetric structure in the decoder. Binary cross entropy was used as a loss function for our model. The deep learning model was implemented with Python programming language (Python 3.5 Software Foundation, https://www.python.org/). After training and validating the U-Net model using RNFL OCT images and their ground truth, we predicted the output of the unlabeled RNFL images in the test set.

Post-processing and average RNFL Thickness estimation

After implementing the segmentation method, the resulting binary masks may contain gaps and speckles. The gaps were filled by implementing post-processing methods based on a morphological algorithm. After the post-processing step, the average thickness of RNFL was measured by using a Euclidean distance transform method.[12]

Performance metrics

We used specificity, sensitivity, and the Dice coefficient to evaluate the segmentation accuracy of our U-Net on the test sets. Specificity was used to assess the true negative rate of the proposed method and sensitivity was used to assess the true positive rate of the proposed method compared to the corresponding manually segmented images. The Dice coefficient, which measures the overlap between the manual and U-Net segmentation, is between 0 and 1, where 0 represents no overlap and 1 represents a complete overlap.

Finally, after calculating the RNFL thickness, mean absolute error (MAE) was reported to compare both the DL algorithmic RNFL results and the OCT machine RNFL data with the ground-truth, defined as the best available estimation of RNFL average thickness that the ophthalmologist determined manually.

Statistics

The normality assumption was verified using the Shapiro-Wilk test. Normal data were analyzed by one-way analysis of variance (ANOVA), and a Kruskal–Wallis test was used when we did not assume a normal distribution of the data. A Bonferroni correction was used to account for multiple comparisons. The correlations between data were analyzed with the Pearson test, and the differences between the correlations was evaluated through the package “cocor” in R (http://www.R-project.org/; provided in the public domain by the R Foundation for Statistical Computing, Vienna, Austria). Differences were considered significant if the P value was less than 0.05.

Declarations

Data Availability Statement:  The datasets used and/or analysed during the current study available from the corresponding author on reasonable request.

Funding/Support: None

Financial disclosures: No financial disclosures

Competing interests: The author(s) declare no competing interests.

Author contributions:

G.R., M.H, M.A.F., P.S, R.R and R.K conceived the methodology and experiments, M.A.F. G.R. and E.H. conducted the experiments, M.A.F and G.R analyzed the results, M.A.F., R.R, P.S. wrote the original draft, M.A.F, R.K, and P.S supervised the work. All authors reviewed, edited, and approved the manuscript.

References

1. Fard, M.A. et al. Optic Nerve Head Morphology in Nonarteritic Anterior Ischemic Optic Neuropathy Compared to Open-Angle Glaucoma. Invest Ophthalmol Vis Sci. 57, 4632-4640 (2016).
2. Fard, M.A., Ghahvehchian, H., Subramanian, P.S. Optical coherence tomography in ischemic optic neuropathy. Ann Eye Sci. 5, 6 (2020).
3. Yadegari, S., Gholizade, A., Ghahvehchian, H., Aghsaei Fard, M. Effect of phenytoin on retinal ganglion cells in acute isolated optic neuritis. Neurol Sci. 41, 2477-2483 (2020).
4. Liu, Y. et al. Patient characteristics associated with artifacts in Spectralis optical coherence tomography imaging of the retinal nerve fiber layer in glaucoma. Am J Ophthalmol. 159, 565-76 (2015).
5. Mansberger, S.L., Menda, S.A., Fortune, B.A., Gardiner, S.K., Demirel, S. Automated Segmentation Errors When Using Optical Coherence Tomography to Measure Retinal Nerve Fiber Layer Thickness in Glaucoma. Am J Ophthalmol. 174, 1-8 (2017).
6. Nagarkatti-Gude, N., Gardiner, S.K., Fortune, B., Demirel, S., Mansberger, S.L. Optical Coherence Tomography Segmentation Errors of the Retinal Nerve Fiber Layer Persist Over Time. J Glaucoma. 28, 368-374 (2019).
7. Chen, J.J., Kardon, R.H. Avoiding Clinical Misinterpretation and Artifacts of Optical Coherence Tomography Analysis of the Optic Nerve, Retinal Nerve Fiber Layer, and Ganglion Cell Layer. J Neuroophthalmol. 36, 417-438 (2016).
8. Mayro, E.L., Wang, M., Elze, T., Pasquale, L.R. The impact of artificial intelligence in the diagnosis and management of glaucoma. Eye (Lond). 34, 1-11 (2020).
9. Jammal, A.A. et al. Detecting Retinal Nerve Fibre Layer Segmentation Errors on Spectral Domain-Optical Coherence Tomography with a Deep Learning Algorithm. Sci Rep. 9, 9836 (2019).
10. Fard, M.A. et al. Optical Coherence Tomography Angiography of a Pale Optic Disc in Demyelinating Optic Neuritis and Ischemic Optic Neuropathy. J Neuroophthalmol. 39, 339-344 (2016).
11. Ronneberger, O., Fischer, P., Brox, T. U-Net: Convolutional Networks for Biomedical Image Segmentation. Medical Image Computing and Computer-Assisted Intervention (MICCAI), Springer, LNCS, 9351, 234—241 (2015).
12. Strutz, T. The Distance Transform and its Computation. TECH/2021/06, arXiv:2106.03503v1, https://arxiv.org/abs/2106.03503 (2021).
13. Asrani, S., Essaid, L., Alder, B.D., Santiago-Turla, C. Artifacts in spectral-domain optical coherence tomography measurements in glaucoma. JAMA Ophthalmol. 132, 396-402 (2014).
14. Miki, A. et al. Prevalence and Associated Factors of Segmentation Errors in the Peripapillary Retinal Nerve Fiber Layer and Macular Ganglion Cell Complex in Spectral-domain Optical Coherence Tomography Images. J Glaucoma. 26, 995-1000 (2017).
15. Kugelman, J., Alonso-Caneiro, D., Read, S.A., Vincent, S.J., Collins, M.J. Automatic segmentation of OCT retinal boundaries using recurrent neural networks and graph search. Biomed Opt Express. 9, 5759-5777 (2018).
16. Fang, L., Cunefare, D., Wang, C., Guymer, R.H., Li, S., Farsiu, S. Automatic segmentation of nine retinal layer boundaries in OCT images of non-exudative AMD patients using deep learning and graph search. Biomed Opt Express. 8, :2732-2744 (2017).
17. Roy, A.G. et al. ReLayNet: retinal layer and fluid segmentation of macular optical coherence tomography using fully convolutional networks. Biomed Opt Express. 8, 3627-3642 (2017).
18. Devalla, S.K. et al. DRUNET: a dilated-residual U-Net deep learning network to segment optic nerve head tissues in optical coherence tomography images. Biomed Opt Express. 9, 3244-3265 (2018).
19. Mariottoni, E.B. et al. Quantification of Retinal Nerve Fibre Layer Thickness on Optical Coherence Tomography with a Deep Learning Segmentation-Free Approach. Sci Rep. 10, 402. (2020).