The deep learning procedures which has started with suggestion of the LeNET in 198829 has been started to be used with inclusion of popular algorithms such as ImageNET, AlexNET,25 ResNET,26 VGG,30 GoogleNET,24 Inception24 into the literature. Researchers continue to propose new models in order to improve their accuracy. Hardware with high processing power is required in order to train deep learning models. However, since each researcher has limited access to the hardware, the weights of the trained networks with standard data sets are shared as open source. The values of the previous weights may be used practically by applying these weights to new data sets with transfer learning methods.31
Osteoporosis is a silent disease and individuals are not aware of this condition until they experience bone fractures. Therefore, dentists will be able to detect patients at an early stage and direct them for early treatment with an effective CAD system for detection of osteoporosis on panoramic radiographs.32 Accordingly, deep learning models of AlexNET, GoogleNET, ResNET-50, SqueezeNET and ShuffleNET were trained through the transfer learning method in this study in order to predict osteoporosis by using MCI over panoramic radiographs of female patients over 50 years old. The results were evaluated by performance criteria including accuracy, sensitivity, specificity, F1-score, AUC, and training duration.
There are several studies in which the presence of osteoporosis was investigated on panoramic radiographs with some classical image processing methods, 16, 18, 33–35 machine learning algorithms and deep learning algorithms in the literature32, 36. Jae-Seo Lee et al.32 estimated the presence and absence of osteoporosis through the single-column deep convolutional neural network (SC-DCNN), multi-column deep convolutional neural network (MC-DCNN) methods by cutting the right and left mandibular regions of 1268 panoramic radiographs labeled with MCI. A trial conducted with 200 test data, the MC-DCNN method provided an accuracy rate of 98.5% and an AUC value of 0.9987. It is obvious in the study where the cross-validation method was not used that a different result would be obtained in each trial. Different from the study stating that healthy and osteoporotic individuals were differentiated only,32 estimation of osteopenic findings scored as C2 was performed in this research. The most striking result was in the C1-C3 dataset where the radiological change related to osteoporosis is most evident among the model performances given in Table 5 where the highest prediction performances are shown. Classification of this dataset was performed through the AlexNET model with a total training duration of 18 minutes and 38 seconds with an accuracy rate of 98.56%, and an AUC value of 0.9987. The feature extraction map of the model is seen when the Grad-CAM images given in Figure 6 are reviewed for differentiation of C1-C3 where the cortical bone change was most evident. As the score progresses from C1 to C3, the cortical change differentiates at length to include the entire area of interest, the areas shown in red are the regions on which the model bases on for distinction between C1 and C3. The total training duration of 43 minutes and 32 seconds with GoogleNET in differentiation of C1-C2 scores has realized with an accuracy rate of 88.94% and a value of 0.9560 AUC. The C2 score shows radiological features close to the C1 score; this may be explained by the lower AUC values in differentiation of C1 and C2. Since the data is unevenly distributed (imbalanced because the number of labels in the datasets is not evenly distributed), the classification estimation performance of the model may be evaluated over the AUC. The three-class C1-C2-C3 dataset has realized with AlexNET with 35 minutes and 43 seconds of total training duration with an accuracy rate of 81.14% and AUC value of 0.9363. The review of GradCAM images given in Figure 7 revealed that the strong features in C1 and C3 scores were acquired from the mandibular cortex at length, and C2 score was acquired from certain regions where the mandibular cortex show porosity in a patchy pattern. In the C1-(C2+C3) dataset where the C2-C3 dataset was labeled as osteoporotic-osteopenic changes, the C1 dataset was labeled as no osteoporosis, presence and absence of osteoporosis and osteopenia were estimated only; an accuracy rate of 92.79% and an AUC value of 0.9787 were acquired with a total training duration of 58 minutes and 35 seconds. Values above 0.9 were obtained in the review of AUC results of four datasets. The heat maps from which the model weights are obtained in the Grad-CAM images and the regions where the oral radiologist interprets the textural differences between the disease scores are consistent (Figures 5-8). These results show that the models may be used as a CAD system in order to assist the oral radiologist to estimate osteoporosis and mapping the localization sites of tissue change through MCI.
Ki-Sun Lee et al.36 classified panoramic images of 680 patients as osteoporosis and no-osteoporosis according to BMD values; 20% of the data was used for random test set, the remaining data was used for training and validation, data were given to deep learning inputs with 5-fold cross validation. The system performances of deep learning models of CNN, VGG-16, VGG-16_TF, VGG-16_TF_FT trained with simple three convolution layers are discussed. The highest values were obtained by VGG-16_TF_FT model with an accuracy rate of 84.00% and AUC of 0.858. Grad-CAM images were created through the model weights acquired with VGG-16_TF_FT. According to these images, the weak lower border of the mandibular cortical bone and the region presenting with less intense, spongy bone tissue in the periphery were evaluated as the regions where strong features were obtained in accurate estimation of osteoporosis. In the normal labeled image, the region presenting with the strong lower border of the mandibular cortical bone and the surrounding dense tissue were determined as the region where strong features were obtained for estimation.
ROI cutting of panoramic radiographs was performed with ImageJ program in this study. The subsampling was then performed according to the row and column sizes consistent for the input of deep learning models. A homogeneous dataset was thereby created in which osteoporosis may be examined in the most adequate anatomical region in each panoramic radiograph. Subsampling panoramic radiographs first followed by cutting through a designated region would create a heterogeneous dataset for the area of interest. Therefore, deep learning models will result in biased training by searching for features in different mandible points and image spaces. Retrieving the entire mandible instead of just taking the ROI would cause search for more features in unnecessary regions by the model and increase the training duration. Although BMD indexing is defined as the gold standard by the World Health Organization, the MCI index suggested by Klemetti22 has been used in many studies. A random test dataset of 20% was determined in the study; the remaining data were given to the model inputs with 5-fold cross validation as training and validation. Undifferentiation of the random test data from the data and not providing to system inputs by 5-folc cross validation both causes failure to interpret the reaction of models against data which were not seen before, and average output of the model obtained through cross validation. Models would produce different results in each run without cross validation.
The limitation of the present study was a lack of a gold standard for diagnosis of osteoporosis in each of the cases. However, it is known that the mandibular cortical shape is significantly compatible with the skeletal BMD obtained with DXA.32 The images were acquired manually in order to include the mandibular inferior margin in the center of the mandibular body as a preoperative preparation for classification of osteoporosis. The construction of a network for scanning of osteoporosis from dental panoramic radiographs through automated detection of the ROI from untrimmed dental panoramic radiographs is required.