Canal-Net: Automatic and Robust 3D Segmentation of Mandibular Canals in CBCT Images Using a Continuity-Aware Contextual Network

doi:10.21203/rs.3.rs-1537019/v1

Download PDF

Article

Canal-Net: Automatic and Robust 3D Segmentation of Mandibular Canals in CBCT Images Using a Continuity-Aware Contextual Network

https://doi.org/10.21203/rs.3.rs-1537019/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

The purpose of this study was to propose a continuity-aware contextual network (Canal-Net) for the automatic and robust 3D segmentation of the mandibular canal (MC) with high consistent accuracy throughout the entire MC volume in cone-beam CT (CBCT) images. The Canal-Net was designed based on a 3D U-Net with bidirectional convolutional LSTM (ConvLSTM) under a multi-task learning framework. Specifically, the Canal-Net learned the 3D anatomical context information of the MC by incorporating spatio-temporal features from ConvLSTM, and also the structural continuity of the overall MC volume under a multi-task learning framework using multi-planar projection losses (MPL) complementally. The Canal-Net achieved higher segmentation accuracies compared to popular deep learning networks in 2D performance metrics of the Dice similarity coefficient score (DSC), Jaccard index (JI), precision (PR), and recall (RC), 3D performance metrics of volume of error (VOE), relative volume distance (RVD), and mean curve distance (MCD). The Canal-Net demonstrated a higher consistent accuracy of segmentation compared to other networks throughout the entire MC volume in the areas of low visibility by the unclear and ambiguous cortical bone layer. Furthermore, the Canal-Net outperformed other state-of-the-art networks in MC segmentation. Therefore, the continuity-aware contextual network (Canal-Net) demonstrated substantially enhanced performance for the automatic and robust 3D segmentation of the MC throughout the entire MC volume in CBCT images.

3D Segmentation of the Mandibular Canal

CBCT image

Deep Learning Network

Multi-planar Projection Losses

ConvLSTM

The mandibular canal (MC) is an important mandibular structure that supplies sensation to the lower teeth, chin, and lower lip¹. Any injury to the MC can lead to temporary or permanent damage resulting in sensory disturbance sequelae such as paresthesia, hypoesthesia, and dysesthesia, which affects speech, mastication, and quality of life^2–5. Therefore, knowing the exact localization of the MC is essential in planning appropriate oral-maxillofacial surgeries such as implant placement and third molar extractions^6,7. In preoperative assessments and surgical planning in dental clinics, panoramic radiographs are used as a standard dental imaging tool^8,9, but presents limitations in that it is challenging to determine the actual 3D rendering of the entire canal structure as the panoramic radiograph only shows the canal in a single view¹⁰. Therefore, additional investigations using CT may be recommended to verify the exact position of the canal in a 3D view⁸. Due to the advantages of CBCT such as a lower radiation dose, inexpensive image acquisition cost, and high spatial resolution, CBCT has been widely used in dental clinics for 3D diagnosis and treatment planning in the field of oral and maxillofacial surgery^11–13. However, the manual segmentation of the MC that is generally performed using 3D cross-sectional slices in CBCT images is time-consuming and labor-intensive^10,14. In addition, the ambiguous cortical bone layer surrounding the canal and the unclear medulla pattern also makes it difficult to distinguish the entire MC because of the lower contrast of CBCT images¹⁵. Therefore, automatic segmentation of the MC is required to alleviate the workload of dental clinicians by overcoming the limitations of CBCT images.

Among studies for automatic MC segmentation in CBCT images, atlas-based segmentation (ARS) and statistical shape model (SSM) methods have been proposed as two conventional representatives of MC segmentation methods^16–18. The SSM method utilized the prior knowledge of shape models to perform MC segmentation^17,18. This prior knowledge was required to reconstruct a 3D model of CBCT images, which highly affects the segmentation result^17,18. On the other hand, the ARS method only requires the atlas image for MC segmentation, which is independent of prior knowledge¹⁶. However, both the SSM and ARS methods exhibit a limitation in dealing with new forms of data beyond the predefined standard since they depend on prior knowledge or other preprocessing techniques^16–18. Recently, deep learning methods have been widely used for the detection^19–21, classification^22–24, segmentation^25,26, and enhancement^27,28 of medical and dental images. Several convolutional neural networks (CNN) such as 3D U-Net, a type of deep learning method, were used for MC segmentation in CBCT images exhibiting a high accuracy of segmentation^10,14. However, these CNNs failed to segment the MC with high consistent accuracy throughout its entire range because of the occasional unclear and ambiguous cortical bone layer caused by the overall lower contrast of CBCT images^10,14. CNNs for the segmentation of the entire MC exhibited lower accuracy around the mandibular and mental foramens compared to other parts of the canal^10,14 since discrimination of the canal from its surroundings became increasingly less clear towards the mental foramen region, and visibility of the MC clearly decreased on cross-sectional images of more distal regions of the MC¹⁵. Precise MC segmentation with high consistent accuracy throughout the entire MC is essential for avoiding nerve injury in oral and maxillofacial surgeries such as mandibular osteotomy and implant surgery²⁹.

In this study, we proposed a continuity-aware contextual network (Canal-Net) for the automatic and robust 3D segmentation of the MC with high consistent accuracy throughout the entire MC volume in CBCT images and compared our network with other networks in terms of volumetric accuracy over the entire canal. Our main contributions were as follows: 1) We designed a continuity-aware contextual network (Canal-Net) that was robust to large anatomical variations of the MC and lower contrast of CBCT images in 3D segmentations of the entire MC. 2) We applied bidirectional convolutional LSTM (ConvLSTM) in order to learn 3D anatomical contextual information of the MC by incorporating spatio-temporal features. 3) We used a multi-task learning framework with multi-planar projection losses (MPL) in three anatomical planes in order to evaluate the global structural continuity of the MC.

Data Acquisition and Preparation

We included 50 patients (27 women and 23 men; mean age 25.56 ± 6.73 years) who underwent dental implant surgeries or third molar extractions at the Seoul National University Dental Hospital (2019–2020). The patient data were obtained at 80 kVp and 8 mA using CBCT (CS9300®, Carestream Health, New York, USA). The CBCT images had dimensions of 841×841×289 pixels, voxel sizes of 0.2×0.2×0.2 mm³, and 16-bit depth. This study was performed with approval from the institutional review board of the Seoul National University Dental Hospital (ERI18001). The ethics committee approved the waiver for the informed consent because this was a retrospective study. The study was performed in accordance with the Declaration of Helsinki.

The mandibular canals including the surrounding cortical bone was manually annotated by an oral and maxillofacial radiologist using a software (3D Slicer, MIT, Massachusetts, US)³⁰. We used the cropped images consisting of 200 slices of 128×128 pixels that were centered at the left and right mandibular regions in order to reduce the memory requirement. Zero-padding was performed to maintain the input volume of the same length for all patients showing the mandibular canals of different lengths. For deep learning, we prepared 60 volume images from 30 patients for the training dataset, 20 from ten patients for the validation dataset, and 20 from ten patients for the test dataset where the right mandible images were horizontally flipped to match the left.

Continuity-Aware Contextual Network (Canal-Net)

We designed a continuity-aware contextual network (Canal-Net) which had 3D encoder-decoder architecture under a multi-task learning framework consisting of time-distributed convolution blocks, multi-scale inputs³¹, skip connections, and bidirectional convolutional LSTM (ConvLSTM) with side-output layers^31,32 (Fig. 1). The bidirectional ConvLSTM was used to capture anatomical context information in concatenated feature maps extracted from the corresponding encoding path and the previous decoding up-sampling layer. A multi-task learning approach was adopted to simultaneously output the entire MC volume and its 2D multi-planar projections in three anatomical planes, which helped the network learn the overall MC volume and structural continuity (Fig. 1). The network under multi-task learning was optimized in an end-to-end manner, where the MC segmentation output was generated directly from the input volumes of the CBCT images.

At the encoder, the time-distributed convolution blocks processed sequential information from 3D volumetric inputs as series of features for 2D slices³³ (Fig. 1). It was a typical convolution passed to a time-distributed wrapper that could be applied to every temporal frame of the input independently³³. Convolutional blocks were comprised of two repeated modules of two 3×3×3 convolutions, batch normalization, ReLU, and 2×2×2 max-pooling at the encoder path. The number of feature maps gradually decreased from 128 to 64, 32, and 16. To mitigate spatio-temporal information loss caused by max-pooling operations, the multi-scale inputs down-sampled from the original input volume by 2×2×2 average pooling were concatenated at each level of the encoder (Fig. 1).

At the decoder, the features from time-distributed convolutions at the encoder³³ were concatenated with the corresponding up-sampling layer and fed to bidirectional ConvLSTM blocks (Fig. 1). Long short-term memory (LSTM), one of the recurrent neural networks (RNN)³⁴, was an efficient network for handling spatio-temporal data and was widely used in contextual processing such as natural language processing³⁵ and video segmentation³⁶. The internal matrix multiplication of the original LSTM was replaced by the convolution operation to maintain the input dimension in ConvLSTM³⁷. The ConvLSTM blocks were composed of two repeated modules of two 3×3×3 bidirectional ConvLSTMs, batch normalization, ReLU, and 2×2×2 up-sampling at the decoder path. The number of feature maps gradually increased from 16 to 32, 64, and 128 (Fig. 1). The ConvLSTM captured 3D local anatomical contextual information more effectively by learning the spatio-temporal features of the 3D volumetric data³⁷.

At the output layer, the averaged side-outputs generated from a local output map from every level of the decoder were merged and fed to the bidirectional ConvLSTM, which mitigated the gradient vanishing problem for encouraging the back-propagation of gradient flow (Fig. 1). The 3D volume loss and multi-planar projection losses (MPL) from the 2D projections simultaneously encouraged the network to learn the global structural continuity information of the canal under the multi-task learning framework. The MPL were calculated from the 2D projection maps of the output in three anatomical planes (Fig. 2). The Dice similarity coefficient score (DSC) was used for the two loss functions³⁸. The loss function (\(L= \alpha {DL}_{vol}+\beta {({DL}_{ap}+ DL}_{cp}+{DL}_{sp})\) of the Canal-Net consisted of 3D volume loss (\({DL}_{vol}\)) for the entire canal volume, and the MPL as sum of the 2D projection losses in axial- (\({DL}_{ap}\)), coronal- (\({DL}_{cp}\)), and sagittal- (\({DL}_{sp}\)) planes, where α and β were constant weights for the 3D volume loss and the summation of the 2D projection map losses, respectively (Fig. 1). The weights of α and β were optimized for the best performance through an ablation study. The weights of 0.7 and 0.3 for the 3D volume loss and MPL, respectively, exhibited the best performance compared to other weight options (Table 1).

Table 1

Comparison of Dice similarity coefficient scores (DSC) for α, weight of 3D volume loss, and β, weight of the MPL in axial, coronal, and sagittal planes to determine the optimal loss weights for α and β for the Canal-Net.
Loss weight	3D volume	2D axial	2D coronal	2D sagittal
α = 0.1, β = 0.9	0.84 ± 0.14	0.92 ± 0.07	0.91 ± 0.17	0.91 ± 0.14
α = 0.2, β = 0.8	0.85 ± 0.12	0.91 ± 0.10	0.90 ± 0.12	0.89 ± 0.13
α = 0.3, β = 0.7	0.84 ± 0.16	0.90 ± 0.14	0.87 ± 0.17	0.88 ± 0.17
α = 0.4, β = 0.6	0.83 ± 0.16	0.90 ± 0.15	0.83 ± 0.19	0.88 ± 0.16
α = 0.5, β = 0.5	0.86 ± 0.13	0.91 ± 0.10	0.88 ± 0.16	0.91 ± 0.11
α = 0.6, β = 0.4	0.84 ± 0.14	0.89 ± 0.13	0.86 ± 0.17	0.89 ± 0.13
α = 0.7, β = 0.3	0.87 ± 0.03	0.93 ± 0.07	0.91 ± 0.14	0.94 ± 0.08
α = 0.8, β = 0.2	0.87 ± 0.09	0.92 ± 0.09	0.91 ± 0.14	0.92 ± 0.09
α = 0.9, β = 0.1	0.86 ± 0.13	0.91 ± 0.10	0.87 ± 0.16	0.92 ± 0.10

The proposed networks were trained using an Adam optimizer, and the learning rate of 0.00025 was reduced on plateau by a factor of 0.5 every 25 epochs in 300 epochs with the batch size of 1. They were implemented with Python3 based on Keras with a Tensorflow backend using a single NVIDIA Titan RTX GPU 24G.

Performance Evaluation of Canal-Net for MC Segmentation

We compared the performance of the MC segmentation by Canal-Net with those by other networks of 2D U-Net³⁹, SegNet⁴⁰, 3D U-Net⁴¹, 3D U-Net with MPL (MPL 3D U-Net), and 3D U-Net with ConvLSTM (ConvLSTM 3D U-Net). To evaluate the performances quantitatively, we compared the 2D segmentation performance metrics of the Dice similarity coefficient score (\(DSC=\frac{2TP}{2TP+FN+FP}\)), Jaccard index (\(JI=\frac{TP}{TP+FN+FP}\)), precision (\(PR=\frac{TP}{TP+FP}\)), recall (\(RC=\frac{TP}{TP+FN}\)) among networks, where TP, FP, and FN denoted true positives, false positives, and false negatives, and also 3D volumetric performance metrics of volume of error (\(VOE=1- \frac{{V}_{gt}\cap {V}_{pred}}{{V}_{gt}\cup {V}_{pred}}\)) and relative volume difference (\(RVD=\frac{|{V}_{gt}-{V}_{pred}|}{{V}_{gt}}\)), where \({V}_{gt}\) and \({V}_{pred}\) represented the number of voxels for the ground truth and for the predicted volume, respectively. We also evaluated the mean curve distance (\(MCD=\frac{\sum _{tϵC\left({V}_{gt}\right)}dist\left(t,C\left({V}_{pred}\right)\right)}{\left|C\left({V}_{gt}\right)\right|}\))), where\(dist\left(x, Y\right)={min}_{yϵY}\left\{{|x-y|}^{2}\right\}\), and \(\text{t}\) denotes coordinates of a ground truth voxel¹⁴, and \(C(·)\) is an operation which extracted the center curve line through skeletonization for the set of voxels¹⁴. The higher values of DSC, JI, PR, and RC, and the lower values of VOE, RVD, and MCD indicated better segmentation performance. We used paired two-tailed t-tests (SPSS v26, SPSS Inc., Chicago, IL, USA) to compare performances between Canal-Net and others. The statistical significance level was set at 0.05.

The performances of Canal-Net, convLSTM 3D U-Net, MPL 3D U-Net, 3D U-Net, SegNet, and 2D U-Net were evaluated for a total of 20 mandibular canals not used for training. Among them, convLSTM 3D U-Net, MPL 3D U-Net, and 3D U-Net were evaluated to demonstrate the effectiveness of the corresponding components in Canal-Net, while the other networks were used for performance comparisons between 2D and 3D CNN-based approaches. In addition, the Canal-Net was evaluated for the impacts of the weights of α and β on 3D volume loss and MPL, respectively. The Canal-Net with loss weights of α = 0.7 and β = 0.3 achieved the best segmentation performance of 0.87, 0.93, 0.91, and 0.94 DSC for 3D volume, axial, coronal, and sagittal planes, respectively (Table 1).

Table 2 shows the quantitative results of the segmentation performance by the networks. The performances of Canal-Net, ConvLSTM 3D U-Net, MPL 3D U-Net, 3D U-Net, SegNet, and 2D U-Net were compared using 20 total mandibular canals. The Canal-Net achieved the highest values of 0.87 DSC, 0.78 JI, 0.88 PR, and 0.88 RC in 2D performance metrics, and also the lowest values of 0.14 RVD, 0.16 VOE, and 0.81 MCD in 3D performance metrics (Table 2). The Canal-Net outperformed all the other networks in DSC, JI, PR, RC, RVD, and VOE, and significantly so in MCD (p < 0.05) (Table 2). The MPL 3D U-Net and ConvLSTM 3D U-Net also outperformed other networks of 3D U-Net, SegNet, and 2D U-Net in DSC, JI, PR, RC, RVD, VOE, and MCD (Table 2). The performance of the networks is also plotted in boxplots in Fig. 2. The Canal-Net achieves the higher performances than the other networks with a smaller dispersion of data, shorter length of whiskers, and rare existence of outliers (Fig. 2).

Table 2

Mean (SD) Dice similarity coefficient score (DSC), Jaccard index (JI), precision (PR), recall (RC), volume of error (VOE), relative volume difference (RVD), and mean curve distance (MCD) by Canal-Net (ours), ConvLSTM 3D U-Net (ours), MPL 3D U-Net (ours), 3D U-Net, SegNet, and 2D U-Net (*: significant difference between Canal-Net and 3D U-Net (p < 0.05), †: between Canal-Net and SegNet (p < 0.05), ‡: between Canal-Net and 2D U-Net (p < 0.05).
	DSC	JI	PR	RC	RVD	VOE	MCD (mm)
Canal-Net	0.87 ± 0.03	0.78 ± 0.04	0.88 ± 0.07	0.88 ± 0.06	0.14 ± 0.06	0.16 ± 0.03	0.81 ± 0.15
ConvLSTM 3D U-Net	0.82 ± 0.07	0.73 ± 0.08	0.87 ± 0.08	0.82 ± 0.12	0.19 ± 0.07	0.21 ± 0.06	1.15 ± 0.42
MPL 3D U-Net	0.82 ± 0.05	0.71 ± 0.06	0.87 ± 0.08	0.80 ± 0.10	0.17 ± 0.06	0.22 ± 0.04	1.10 ± 0.26
3D U-Net	0.81 ± 0.06	0.70 ± 0.07	0.86 ± 0.08	0.80 ± 0.11	0.18 ± 0.07	0.23 ± 0.04	1.19 ± 0.33 *
SegNet	0.79 ± 0.04	0.71 ± 0.04	0.82 ± 0.05	0.79 ± 0.08	0.22 ± 0.07	0.26 ± 0.04	1.61 ± 0.60 †
2D U-Net	0.77 ± 0.06	0.70 ± 0.06	0.80 ± 0.05	0.78 ± 0.10	0.25 ± 0.10	0.29 ± 0.07	2.05 ± 0.75 ‡

In Fig. 3, the Canal-Net exhibited more accurate predictions with more true positives (yellow) and less false positives (red) and false negatives (green) compared to other networks in the MC region with unclear and ambiguous cortical bone layers and CBCT images of low contrast (Fig. 3). In the 3D segmentation results, the Canal-Net exhibited better prediction results with less false positives and false negatives compared to other networks in the mental foramen area with large anatomical variations (Fig. 4). Furthermore, compared with other networks, the Canal-Net predicted more accurately the entire MC volume and exhibited improved structural continuity and boundary details of the MC from the mental foramen to the mandibular foramen (Fig. 4).

The DSC and MCD for the entire test dataset were plotted from the mental foramen to the mandibular foramen, and the 3D networks generally exhibited less variations of the performances compared to the 2D networks (Figs. 5 and 6). The Canal-Net demonstrated the most consistent performances with the smallest fluctuations of true segmentation compared to the other networks throughout the entire MC volume (Figs. 5 and 6). As a result, the Canal-Net represented the best 3D segmentation accuracies of RVD, VOE, and MCD throughout the entire MC volume among the networks (Table 2). Therefore, the Canal-Net represented more robust 3D segmentation performance of the entire MC volume compared to the other networks.

The accurate identification of the MC structure in the mandible is an essential prerequisite for the preoperative planning of third molar extractions and implant surgeries to avoid any surgical complications⁷. However, the exact recognition of the entire canal structure is considered to be a challenging and delicate task for several reasons¹⁵. CBCT, the most commonly used 3D dental imaging tool, has lower contrast than CT, which negatively affects the ability to distinguish MCs^10,42. As a result, the low visibility of MCs, such as in ambiguous or unclear cortical bone regions, affects the structural continuity of MC segmentation in CBCT images^10,14. Furthermore, the visibility of the MC itself is low due to variable cortications and bone densities of the canal wall, the diverse travel courses of the canal, and the spread of vessels and nerve branches^15,43−47. In this study, we proposed a continuity-aware contextual network (Canal-Net) which learned 3D local anatomical contextual information of the MC by incorporating spatio-temporal features and the global continuity of MCs by using multi-planar projection losses (MPL) in order to segment the MC with high consistent accuracy throughout its entire range. We employed time-distributed convolution layers for handling time-distributed sequential features with multi-scale inputs at the encoder path³³, and bidirectional ConvLSTM layers for extracting spatio-temporal features at the decoder path³⁷. The Canal-Net was able to learn the local anatomical variations of the MC by incorporating the spatio-temporal features effectively, and the global structural continuity information of the MC under the multi-task learning framework, complementally. The Canal-Net used optimized weights for 3D volume loss and multi-planar projection losses in multi-task learning. Therefore, the Canal-Net improved the performance of automatic segmentation of the MC by combining anatomical context information and global structural continuity information, resulting in higher consistent accuracy throughout the entire MC volume in the CBCT images.

We compared the Canal-Net with other popular segmentation networks such as 2D U-Net, SegNet, and 3D U-Net, and also with our MPL 3D U-Net and ConvLSTM 3D U-Net for MC segmentation. In performances of MC segmentation in CBCT images, 2D U-Net and SegNet exhibited lower accuracies compared to the 3D networks, generally. False negatives and positives were observed at a higher rate around the mental foramen area with ambiguous or unclear cortical bone layers. Since the 2D networks were not able to learn the 3D contextual features of the MC volume in CBCT images, the 2D networks exhibited coarser 3D segmentation volumes with more fluctuations of 3D performance accuracy from the mental to the mandibular foramen regions. In terms of learning 3D spatial contextual information between image slices of the 3D anatomical structures, 3D U-Net was generally expected to generate more accurate segmentation results compared to 2D networks⁴¹. In the present study, the 3D U-Net predicted the more accurate segmentation of the MC with fewer false positives and negatives compared to the 2D U-Net and SegNet. However, the 3D U-Net had still limitations in segmenting the MC regions with unclear cortical bone layers accurately by only learning 3D spatial information between image slices, and exhibited inaccurate segmentation results with disconnections around the mental foramen area.

Both MPL 3D U-Net and ConvLSTM 3D U-Net demonstrated better performances than 3D U-Net. The MPL 3D U-Net showed an improved travel course of the MC compared to 3D U-Net because its spatial information was complemented with the global structural continuity information by learning through multi-planar projections. Although the structural continuity of the MC volume was improved by multi-task learning, the MPL 3D U-Net exhibited difficulties in producing segmentation boundaries in detail around the mental foramen area. On the other hand, the ConvLSTM learned anatomical context information through spatio-temporal features, and the MC volume showed smooth boundaries with more consistent accuracies even in unclear cortical bone layer regions in the CBCT images. Therefore, the Canal-Net demonstrated the most accurate segmentation of the entire MC volume compared to the other networks by simultaneously learning global structural continuity through MPL, and anatomical context information through ConvLSTM. Compared with previous studies using 3D U-Net^10,14, our Canal-Net achieved 0.87 of DSC and 0.78 of the mean intersection of union (IoU) while two previous studies reported 0.58 of DSC^{10, 14} and 0.58 of mean IoU^10,14. The Canal-Net outperformed the 3D networks used in previous studies in MC segmentation.

In the Canal-Net, the MPL provided global structural continuity from three anatomical projection maps with ConvLSTM anatomical context information by spatio-temporal features, complementally. In the MC areas of low visibility with ambiguous or unclear cortical bone layers in CBCT images, the Canal-Net exhibited the best outcomes with continuous and consistent MC volumes from the mental to mandibular foramens. The Canal-Net especially surpassed other networks by showing continuous MC volumes around the mental foramen area where the visibility of the MC tended to diminish¹⁵, and in areas affected by metallic objects such as implant fixtures or dental crowns in CBCT images. As a result, the Canal-Net demonstrated the most robust MC segmentations with high consistent accuracy throughout the entire MC volume in CBCT images.

The primary reason for improved segmentation performance by Canal-Net was that its network architecture was constructed to complementally learn the 3D anatomical context information of the MC by the spatio-temporal features from the bidirectional ConvLSTM layers and the global structural continuity information by MPL. In the Canal-Net, the complementary context information was successfully learned in the proposed framework, leading to maintaining continuous and consistent MC volumes from the mental to the mandibular foramen areas. The proposed learning process has several advantages. First, it could increase the discriminative capability of intermediate feature representations with multiple regularizations on disentangling subtly correlated tasks⁴⁸, potentially improving the robustness of the segmentation performance. Second, in the application of MC segmentation, the multi-task learning framework could also provide complementary context information that would serve well to segment the MC maintaining overall continuous and consistent volumes. This could improve the performance accuracy of MC segmentations substantially, especially in MC regions with ambiguous or unclear cortical bone layers in lower contrast CBCT images.

However, our study had several limitations. First, as there was the problem of reducing the memory requirements for dealing with large amounts of data when using deep 3D networks running on the GPU, it was necessary to optimize the way the memory was used in order to maximize GPU utilization. Therefore, we used the cropped images with smaller dimensions than the original, and preprocessing of the images required additional time and labor. Second, our study had a potential limitation of generalization ability due to using a relatively small amount of data from a single organization. Overfitting of training a deep learning model, which resulted in the model learning statistical regularity specific to the training dataset, could negatively impact the model’s ability to generalize to a new dataset⁴⁹. The proposed network needs to be trained and evaluated for datasets from multiple organizations. Third, the results presented in this study were based on datasets from 50 patients. The proposed method needs to be evaluated for datasets from more patients with various dental restorations and implants. In future studies, we improve the generalization ability and clinical efficacy of the Canal-Net by using large patient datasets from multiple organizations or devices.

In this study, we proposed a continuity-aware contextual network (Canal-Net) that was robust to large anatomical variations of the MC and lower contrast of CBCT images in 3D segmentations of the entire MC. The Canal-Net was designed based on a 3D U-Net with the ConvLSTM under the multi-task learning framework using MPL in order to complementally learn anatomical contexts and global structural continuity information. As a result, the Canal-Net outperformed other networks such as 2D U-Net, SegNet, 3D U-Net, MPL 3D U-Net, and ConvLSTM 3D U-Net in 2D and 3D performances. Therefore, the continuity-aware contextual network (Canal-Net) demonstrated substantially enhanced performances for automatic and robust 3D segmentation of the MC throughout the entire MC volume in CBCT images.

Acknowledgements

This work was supported by the Korea Medical Device Development Fund Grant funded by the Korean Government (The Ministry of Science and ICT, The Ministry of Trade, Industry, and Energy, The Ministry of Health & Welfare, and The Ministry of Food and Drug Safety) (1711138289, RS-2020-KD00014, 1711137883, KMDF_PR_20200901_0011)

Conflicts of Interest

None of the authors have conflicts of interest to declare.

Author Contributions

Bo-Soung Jeoun: Contributed to the conception and design, data acquisition, analysis and interpretation, and drafted and critically revised the manuscript

Su Yang: Contributed to the conception and design, data analysis and interpretation, and drafted and critically revised the manuscript

Sang-Jeong Lee: Contributed to the data analysis and interpretation, and drafted the manuscript

Tae-Il Kim: Contributed to conception and design, data interpretation, and drafted the manuscript

Jun-Min Kim: Contributed to conception and design, data interpretation, and drafted the manuscript

Jo-Eun Kim: Contributed to the conception and design, data interpretation, and drafted the manuscript

Kyung-Hoe Huh: Contributed to the conception and design, data interpretation, and drafted the manuscript

Sam-Sun Lee: Contributed to the conception and design, data interpretation, and drafted the manuscript

Min-Suk Heo: Contributed to the conception and design, data interpretation, and drafted the manuscript

Won-Jin Yi: Contributed to the conception and design, data acquisition, analysis and interpretation, and drafted and critically revised the manuscript

“All authors gave their final approval and agreed to be accountable for all aspects of the work.”

Data availability

The datasets generated and/or analysed during the current study are not publicly available due to the restriction by the Institutional Review Board (IRB) of Seoul National University Dental Hospital in order to protect patients’ privacy but are available from the corresponding author on reasonable request.

Ghatak, R. N., Helwany, M. & Ginglen, J. G. Anatomy, Head and Neck, Mandibular Nerve. in StatPearls (2020).
Shavit, I. & Juodzbalys, G. Inferior alveolar nerve injuries following implant placement - importance of early diagnosis and treatment: a systematic review. J Oral Maxillofac Res 5, e2, doi:10.5037/jomr.2014.5402 (2014).
Sarikov, R. & Juodzbalys, G. Inferior alveolar nerve injury after mandibular third molar extraction: a literature review. J Oral Maxillofac Res 5, e1, doi:10.5037/jomr.2014.5401 (2014).
Phillips, C. & Essick, G. Inferior alveolar nerve injury following orthognathic surgery: a review of assessment issues. J Oral Rehabil 38, 547–554, doi:10.1111/j.1365-2842.2010.02176.x (2011).
Loescher, A. R., Smith, K. G. & Robinson, P. P. Nerve damage and third molar removal. Dent Update 30, 375–380, 382, doi:10.12968/denu.2003.30.7.375 (2003).
Ai, C. J., Jabar, N. A., Lan, T. H. & Ramli, R. Mandibular Canal Enlargement: Clinical and Radiological Characteristics. J Clin Imaging Sci 7, 28, doi:10.4103/jcis.JCIS_28_17 (2017).
Jung, Y. H. & Cho, B. H. Radiographic evaluation of the course and visibility of the mandibular canal. Imaging Sci Dent 44, 273–278, doi:10.5624/isd.2014.44.4.273 (2014).
Ghaeminia, H. et al. Position of the impacted third molar in relation to the mandibular canal. Diagnostic accuracy of cone beam computed tomography compared with panoramic radiography. Int J Oral Maxillofac Surg 38, 964–971, doi:10.1016/j.ijom.2009.06.007 (2009).
Vinayahalingam, S., Xi, T., Berge, S., Maal, T. & de Jong, G. Automated detection of third molars and mandibular nerve by deep learning. Sci Rep 9, 9007, doi:10.1038/s41598-019-45487-3 (2019).
Kwak, G. H. et al. Automatic mandibular canal detection using a deep convolutional neural network. Sci Rep 10, 5711, doi:10.1038/s41598-020-62586-8 (2020).
Ludlow, J. B., Davies-Ludlow, L., Brooks, S. & Howerton, W. Dosimetry of 3 CBCT devices for oral and maxillofacial radiology: CB Mercuray, NewTom 3G and i-CAT. Dentomaxillofacial Radiology 35, 219–226 (2006).
Arai, Y., Tammisalo, E., Iwai, K., Hashimoto, K. & Shinoda, K. Development of a compact computed tomographic apparatus for dental use. Dentomaxillofac Radiol 28, 245–248, doi:10.1038/sj/dmfr/4600448 (1999).
Pauwels, R. et al. Variability of dental cone beam CT grey values for density estimations. Br J Radiol 86, 20120135, doi:10.1259/bjr.20120135 (2013).
Jaskari, J. et al. Deep Learning Method for Mandibular Canal Segmentation in Dental Cone Beam Computed Tomography Volumes. Sci Rep 10, 5842, doi:10.1038/s41598-020-62321-3 (2020).
Oliveira-Santos, C. et al. Visibility of the mandibular canal on CBCT cross-sectional images. J Appl Oral Sci 19, 240–243, doi:10.1590/s1678-77572011000300011 (2011).
Kroon, D.-J. Segmentation of the mandibular canal in cone-beam CT data. Citeseer (2011).
Abdolali, F. et al. Automatic segmentation of mandibular canal in cone beam CT images using conditional statistical shape model and fast marching. Int J Comput Assist Radiol Surg 12, 581–593, doi:10.1007/s11548-016-1484-2 (2017).
Kainmueller, D., Lamecker, H., Seim, H., Zinser, M. & Zachow, S. Automatic extraction of mandibular nerve and bone from cone-beam CT data. in International Conference on Medical Image Computing and Computer-Assisted Intervention. 76–83 Springer (2009).
Ahn, J. M. et al. A deep learning model for the detection of both advanced and early glaucoma using fundus photography. PloS one 13, e0207982 (2018).
Phan, S., Satoh, S. i., Yoda, Y., Kashiwagi, K. & Oshika, T. Evaluation of deep convolutional neural networks for glaucoma detection. Japanese journal of ophthalmology 63, 276–283 (2019).
Chang, H.-J. et al. Deep learning hybrid method to automatically diagnose periodontal bone loss and stage periodontitis. Scientific reports 10, 1–8 (2020).
Shen, W. et al. Multi-crop convolutional neural networks for lung nodule malignancy suspiciousness classification. Pattern Recognition 61, 663–673 (2017).
Kumar, A., Kim, J., Lyndon, D., Fulham, M. & Feng, D. An ensemble of fine-tuned convolutional neural networks for medical image classification. IEEE journal of biomedical and health informatics 21, 31–40 (2016).
Yu, Y. et al. Deep transfer learning for modality classification of medical images. Information 8, 91 (2017).
Cheng, J. Z. et al. Computer-Aided Diagnosis with Deep Learning Architecture: Applications to Breast Lesions in US Images and Pulmonary Nodules in CT Scans. Sci Rep 6, 24454, doi:10.1038/srep24454 (2016).
Christ, P. F. et al. Automatic liver and tumor segmentation of CT and MRI volumes using cascaded fully convolutional neural networks. arXiv preprint arXiv:1702.05970 (2017).
Yong, T.-H. et al. QCBCT-NET for direct measurement of bone mineral density from quantitative cone-beam CT: A human skull phantom study. Scientific Reports 11, 1–13 (2021).
Heo, M.-S. et al. Artificial intelligence in oral and maxillofacial radiology: what is currently possible? Dentomaxillofacial Radiology 50, 20200375 (2021).
Greenstein, G. & Tarnow, D. The mental foramen and nerve: clinical and anatomical factors related to dental implant placement: a literature review. J Periodontol 77, 1933–1943, doi:10.1902/jop.2006.060197 (2006).
Fedorov, A. et al. 3D Slicer as an image computing platform for the Quantitative Imaging Network. Magn Reson Imaging 30, 1323–1341, doi:10.1016/j.mri.2012.05.001 (2012).
Fu, H. et al. Joint Optic Disc and Cup Segmentation Based on Multi-Label Deep Network and Polar Transformation. IEEE Trans Med Imaging 37, 1597–1605, doi:10.1109/TMI.2018.2791488 (2018).
Yin, P., Yuan, R., Cheng, Y. & Wu, Q. Deep Guidance Network for Biomedical Image Segmentation. IEEE Access 8, 116106–116116 (2020).
Novikov, A. A., Major, D., Wimmer, M., Lenis, D. & Buhler, K. Deep Sequential Segmentation of Organs in Volumetric Medical Scans. IEEE Trans Med Imaging 38, 1207–1215, doi:10.1109/TMI.2018.2881678 (2019).
Yu, Y., Si, X., Hu, C. & Zhang, J. A review of recurrent neural networks: LSTM cells and network architectures. Neural computation 31, 1235–1270 (2019).
Sutskever, I., Vinyals, O. & Le, Q. V. Sequence to sequence learning with neural networks. arXiv preprint arXiv:1409.3215 (2014).
Ventura, C. et al. Rvos: End-to-end recurrent network for video object segmentation. in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 5277–5286 IEEE (2019).
Shi, X. et al. Convolutional LSTM network: A machine learning approach for precipitation nowcasting. arXiv preprint arXiv:1506.04214 (2015).
Sudre, C. H., Li, W., Vercauteren, T., Ourselin, S. & Cardoso, M. J. Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations. in Deep learning in medical image analysis and multimodal learning for clinical decision support. 240–248 Springer (2017).
Ronneberger, O., Fischer, P. & Brox, T. U-net: Convolutional networks for biomedical image segmentation. in International Conference on Medical image computing and computer-assisted intervention. 234–241 Springer (2015).
Badrinarayanan, V., Kendall, A. & Cipolla, R. SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation. IEEE Trans Pattern Anal Mach Intell 39, 2481–2495, doi:10.1109/TPAMI.2016.2644615 (2017).
Çiçek, Ö., Abdulkadir, A., Lienkamp, S. S., Brox, T. & Ronneberger, O. 3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation. in International Conference on Medical image computing and computer-assisted intervention. 424–432 Springer (2016).
Moris, B., Claesen, L., Sun, Y. & Politis, C. Automated tracking of the mandibular canal in cbct images using matching and multiple hypotheses methods. in 2012 Fourth International Conference on Communications and Electronics. 327–332 IEEE (2012).
Denio, D., Torabinejad, M. & Bakland, L. K. Anatomical relationship of the mandibular canal to its surrounding structures in mature mandibles. J Endod 18, 161–165, doi:10.1016/S0099-2399(06)81411-1 (1992).
Gowgiel, J. M. The position and course of the mandibular canal. J Oral Implantol 18, 383–385 (1992).
Monsour, P. A. & Dudhia, R. Implant radiography and radiology. Aust Dent J 53 Suppl 1, S11–25, doi:10.1111/j.1834-7819.2008.00037.x (2008).
Wadu, S. G., Penhall, B. & Townsend, G. C. Morphological variability of the human inferior alveolar nerve. Clin Anat 10, 82–87, doi:10.1002/(SICI)1098-2353(1997)10:2<82::AID-CA2>3.0.CO;2-V (1997).
Carter, R. B. & Keen, E. N. The intramandibular course of the inferior alveolar nerve. J Anat 108, 433–440 (1971).
Ruder, S. An overview of multi-task learning in deep neural networks. arXiv preprint arXiv:1706.05098 (2017).
Kwon, O. et al. Automatic diagnosis for cysts and tumors of both jaws on panoramic radiographs using a deep convolution neural network. Dentomaxillofacial Radiology 49, 20200185 (2020).

No competing interests reported.

Download PDF

Editorial decision: Major revision
13 May, 2022
Reviews received at journal
25 Apr, 2022
Reviewers agreed at journal
15 Apr, 2022
Reviewers agreed at journal
14 Apr, 2022
Reviews received at journal
14 Apr, 2022
Reviewers agreed at journal
14 Apr, 2022
Reviewers invited by journal
14 Apr, 2022
Editor assigned by journal
14 Apr, 2022
Editor invited by journal
14 Apr, 2022
Submission checks completed at journal
14 Apr, 2022
First submitted to journal
08 Apr, 2022

You are reading this latest preprint version

Canal-Net: Automatic and Robust 3D Segmentation of Mandibular Canals in CBCT Images Using a Continuity-Aware Contextual Network

Status:

Version 1

Abstract

Figures

Introduction

Materials And Methods

Data Acquisition and Preparation

Continuity-Aware Contextual Network (Canal-Net)

Performance Evaluation of Canal-Net for MC Segmentation

Results

Discussion

CONCLUSIONS

Declarations

References

Additional Declarations

Status:

Version 1