Accurate Automatic Detection Method to Assist Physicians in Diagnosing Early Nondisplaced Fractures of Femoral Neck


 Background - Nondisplaced femoral neck fractures are sometimes misdiagnosed by radiographs. We developed an automatic detection method using deep learning networks to pinpoint femoral neck fractures on radiographs to assist the physicians in making accurate diagnosis in the first place. Results - A total of approximately 3,840 images of non-displaced Garden type I and II femoral neck fracture cases collected from the Radiology Information System (RIS) from the Picture Archiving and Communication System (PACS) database between 2018 and 2020 from the China Medical University Hospital (CMUH). Two senior orthopedic surgeons from the China Medical University Hospital participated in independently labeling the femoral neck margin and fracture line on these images as the training dataset for the deep learning network. Our proposed accurate automatic detection method, called direction-aware fracture detection network (DAFDNet), consists of two steps, namely region of interest (ROI) segmentation and fracture detection. The first step removes the noise region and pinpoints the femoral neck region. The fracture detection step uses direction-aware deep learning algorithm to mark the exact femoral neck fracture location in the region detected in the first step.Conclusions - Our proposed DAFDNet demonstrated over 94.8% accuracy in differentiating non-displaced Garden type I and type II femoral neck fracture cases. Our DAFDNet method outperforms the diagnostic accuracy of general practitioners and orthopedic surgeons in accurately locating Garden type I and type II fractures locations. This study can determine the feasibility of applying artificial intelligence in a clinical setting and how the use of deep learning networks assist physicians in improving the correct diagnosis compared to current traditional orthopedic manual assessments.


Background
Femoral neck fractures are one of the most common osteoporotic fractures in the elderly and cause substantial morbidity and mortality [1][2][3]. According to the radiograph-based Garden classi cation system for assessing fracture severity, femoral neck fractures can be classi ed into four types, namely nondisplaced Garden I and II and displaced Garden III and IV [4]. Nondisplaced Garden I and II indicate incomplete nondisplaced fractures and complete nondisplaced fractures, respectively. Displaced Garden III and IV are incomplete displaced fractures and complete displaced fractures.
The features of displaced femoral neck fractures are clinically and imaging distinct, whereas those of nondisplaced femoral neck fractures are challenging and receive less attention [5][6][7][8]. Radiographic imaging of nondisplaced femoral neck fractures can be compromised by osteoporosis, obesity, patient position-related reasons, or the use of portable radiographic equipment, and poor image quality, which creates additional di culties for clinicians [6,9].
Recent advances in arti cial intelligence using deep learning techniques, such as deep convolutional neural networks (DCNN), have shown remarkable results for a range of medical tasks as well as for human experts [10][11][12][13][14]. A growing number of studies support that deep learning networks can be trained to identify fractures in orthopedic radiographs with satisfactory accuracy [14][15][16]. Although deep learning has been applied to fracture detection for radiological diagnosis, nondisplaced femoral neck fractures are often overlooked for misdiagnosis, which may result in patients with nondisplaced fractures deteriorating into displaced fractures. Therefore, we propose a new direction-aware fracture detection network, termed as DAFDNet, for automatic detection of femoral neck fractures on anterior-posterior pelvic radiographs. It is well known that Gabor lter is a differentiable band-pass lter with adjustable scales and orientations, and therefore it has been integrated into DCNN [17][18][19]. Garden type I and Garden type II of femoral neck fractures present different orientations and frequencies in frequency space depending on the patient's imaging location and conditions. By integrating a Gabor lter in the DCNN, the lter is able to x optimal parameters and help the DCNN learn robust feature presentations. We present this study to validate the accuracy of DCNN in detecting nondisplaced femoral neck fractures, and it shows substantial improvements in performance.

Dataset and metrics
The original radiographic images for the experiments, including left or right femoral neck, were acquired from patients and approved by the local Institutional Review Board. We extracted approximately 3,840 radiological images of anterior-posterior views of pelvis from China medical university hospital (CMUH, Taichung, Taiwan) between 2018 and 2020 for nondisplaced ipsilateral femoral neck fracture (Garden type I and II) noted in relevant radiologist reports taken from the Picture Archiving and Communication System database identi ed through the Radiology Information System. Two senior orthopedic surgeons were involved in the annotation of the images, independently annotating the femoral neck part and the fracture line. In our algorithm, the femoral neck part was used to train the DCNN for ROI segmentation and the fracture line was used to train the DCNN for fracture detection. All labeled images were made under the guidance of professional orthopedist, and new images of 1024×1024 pixels size were extracted accordingly to reduce the computing time. We used intersection over union (IOU) value between the femoral neck fracture region and the labeled region as assessing metrics, de ned as IOU=(A∩B)/(A∪B) where ∩ and ∪ denote the intersection and union of two sets, A is the intersection of the predicted region and label region and B is the predicted region.
Implementation details Fig. 1 shows the femoral neck fracture detection strategy used in this paper, which consists of two phases, namely femoral neck localization and fracture detection. In the rst stage, the original image is fed into a segmentation network with matching and alternative methods to accurately localize the femoral neck. In the second stage, a surgeon-made label-trained network is used to localize the exact location of the fracture after the output femoral neck image in the rst stage.
We augment the data by rotating and rescaling the images and labels with various degrees and scales. During training process, 12 images were randomly chosen as input in each training batch. The model was trained by the Adam optimizer with a learning rate initialized to 1e -5 and set to 4e -5 in steps of 1e -5 . Two typical DCNNs, i.e., namely U shape Connected convolutional network (U-Net) [21] and Densely Connected convolutional network (DenseNet) [24] were used as the comparing algorithms and their codes were downloaded from GitHub shared by the original authors. The corresponding parameters of these three methods are optimized until the network converges. The detection results show that our proposed method is closest to the actual size and area of the label region, while the other methods obtain results with an area several times larger than the labeled region. Therefore, our proposed method gains the largest IOU value among the three methods, which indicates that the fracture detected by our proposed method is the closest to the ground truth. In Table 1, we divided the IOU values into three categories. There are 73.1% of DAFDNet results above 0.5 (or 50%), while none of the other two methods. 21.7% of the DAFDNet results are between 0.2 and 0.5 (or [20%,50%)), while more than 90% of DenseNet and U-Net comparison methods have results close to 0.1 (or [0%,20%)). The table also calculates the average IOU values, which are 0.648, 0.084 and 0.062 (or 64.8%, 8.4% and 6.2%) for DAFDNet, DenseNet and U-Net, respectively.

Discussion
In this study, we propose a new method for detecting femoral neck fractures. The results show that our method is effective in detecting the precise fracture location and outperforms other comparative methods. Our proposed method is implemented in the localization and detection phases, i.e., localization of the femoral neck and detection of fracture detection. The bene t of the localization phase is that by localizing the ROI from the original image, the input data size of DCNN can be greatly reduced. On the one hand, the computation time is saved to a great extent, and on the other hand, the disturbs, such as regions with similar gray distribution in the pelvis image, can be excluded to improve the accuracy of detection. In the fracture detection stage, because the orientation of fracture is random, DAFDNet introduces an orientation-aware algorithm to detect fracture directionality. The use of band-pass lter Gabor enables the network to detect image gray changes by adjusting its frequency or orientation. In addition, attention mechanism and ghost convolution are also involved to improve the performance of DAFDNet.

Conclusion
Although deep learning has made great advances in medical image processing, few publications have shown clinical utility for detecting femoral neck fractures, especially non-displaced Garden I and Garden II fracture detection. The success of our study in detecting the precise location of nondisplaced fractures provides the rst evidence that DCNN can help physicians improve the diagnostic accuracy of nondisplaced Garden I and Garden II fractures. As shown by the predicted rectangular and IOU values of the fractures, our proposed method obtained better results than physician diagnosis. However, fractures were not detected in more than 5.2% of the images tested, due to the poor contrast of the images.
Therefore, better radiographic image quality would greatly improve our approach.

DCNN for femoral neck fractures
Recently, DCNN-based methods have shown great potential e ciency in many areas of medical diagnosis and have encouraged further applied research [20]. The use of DCNN can reduce the need for expensive Computed Tomography (CT) and Magnetic Resonance Imaging (MRI) scans, and its automotive and accurate detection results can reduce the burden on clinicians for urgent identi cation of fractures [12]. However, the feasibility and e ciency of DCNN in detecting femoral neck fractures remains challenging and has not been fully investigated, needless to say the occult representations of the Garden I and II [14]. To the best of our knowledge, typical DCNN-based medical image segmentation methods [16, [21][22][23][24], such as U-Net [21] and DenseNet [24], can be applied to detect femoral neck fractures. These types of fractures may disappear after a series of convolutional operations with the depth of the layers due to the tiny variations in the grayscale distribution in these regions in the radiographic images.

Gabor lter
Two dimensional Gabor lter is a directional band-pass wavelet lter that is the multiplication of a Gaussian function and a cosine function, de ned as follows [17]: and orientation, respectively. Substantially, the Gabor transform is a windowed short-time Fourier transform that is able to extract features locally or of certain frequency components. The direction and frequency selection properties make the Gabor lter sensitive to certain types of boundaries.
In radiography, the orientations of nondisplaced femoral neck fracture is related to the patient's position, where the frequency components lie in certain ranges. Therefore, in our study, a multiple-direction Gabor lter with adjusted frequency bands was engaged as a input layer to the DCNN to detect the tiny changes in the grayscale distribution in Garden type I and type II.

Attention mechanism
The attention mechanism was invented to tell DCNN where or what features to focus on, which was demonstrated to signi cantly improve the effectiveness of model performance [25,26]. Squeeze-and-Excitation attention network (SENet) exploits the squeeze function and the excitation function, i.e., the global average pooling operation and the Sigmoid function, respectively, to encode inter-channel information [27]. This simple and innovative model provides a signi cant performance improvement for DCNN, but ignores the location information that is important for capturing features [25]. Therefore, several extension studies such as Bottle-Attention-Module (BAM) [28], Convolutional Block-Attention-Module (CBAM) [29], and Spatial and Channel-wise Attention (SCA) [30] have been proposed to further extract spatial and channel information and improve the network effectiveness. Self-attention DCNN models Attention in Attention network (A 2 Net) divides the attention branches into attention and nonattention branches to maximize the use of high-contributing information and minimize the suppression of redundant information [31]. Although A 2 Net exhibits excellent performance, the large amount of computation requires signi cant hardware facility costs. In summary, this study uses the SCA strategy to obtain spatial and channel-wise information.
Direction-aware segmentation network In this section, we introduce the implementation details of the proposed DAFDNet model, including the attention mechanism Ghost convolution and the details of the model architecture.

Squeeze-and-Excitation Ghost convolution
GhostNet was rst proposed in reference [32] to reduce the computation consumption by replacing the ordinary convolution with a simple linear transformation. A Ghost module divides the result of convolution into two parts. The rst part involves ordinary convolution, while the other part uses a series of linear transformations to generate more feature maps, as shown in Fig.5a. By using this strategy, the lightweight Ghost module produces more feature maps with inexpensive operations and performs better than other lightweight DCNNs, which also accelerates the learning process. However, the linear transformation does not focus on cross-channel relationships, which have proven to be robust in object detection. Therefore, each Squeeze-and-Excitation (SE) block consisting of a global average pooling and two fully connected layers is embedded in the Ghost module instead of linear transformation, as show in Fig. 5b. The weights calculated from the SE blocks are then multiplied with the input convolution results by channel and concatenated with the original convolution results to generate the nal feature maps.

Model architecture
As shown in Fig.6, DAFDNet uses the popular encoder-decoder framework to rst encode the input image by focusing on the attention Ghost convolutional module and Gabor-lter convolution. The input image is manipulated with the Ghost module, and then two down-sampling operations are performed to gain global feature maps with different resolutions, i.e., GhoM 1 , GhoM 2 , and GhoM 3 , respectively. The output feature map GhoM i in the Ghost module is represented as follows In the process of bypassing the Gabor convolution, we use an 8-direction Gabor with diverse scales to extract the boundary of the femoral neck fracture, and then perform two down-sampling operations to generate two additional batches of shrinking feature maps. Then, we obtain the feature maps Gab i from the Gabor convolution, which is represented as follows where GF i (θ,s) is the Gabor-lter with directions θ and scales s. Afterwards, in our study, Gab 1 , Gab 2 and Gab 3 are concatenated with GhoM 1 , GhoM 2 and GhoM 3 , respectively, to get aggregated feature maps GG 1 , GG 2 and GG 3 expressed by Then, the extracted features are re ned with a 2×2 average pooling layer, a 3×3 convolution layer and a Batch Normalization layer to obtain the PF i feature map. We concatenate the corresponding GhoM i and PF i with the same resolution to obtain A i , where the results are processed with the attention module, respectively. The feature maps with lower resolution, such as A 3 , A 2 as shown in Fig.6, are resized with a 2×2 up-sampling layer before being concatenated to the feature maps with higher resolution. In general, the process can be expressed as follows Hence, the output of our network is result of the sequential operations of expressions (2-5) with a 2×2 upsampling layer to recover the feature size as the input image and followed by a 1×1 convolution layer to reduce the dimension of channels.
The loss function of DAFDNet is mean square error that can be expressed as Figure 1 Work ow of the fracture detection strategy. Two phases are included, femoral neck localization and fracture detection.    Original Ghost convolution and SE Ghost module. (a) Ghost convolution, which use a simple linear transformation to generate more features; (b) SE Ghost module, which incorporates the SE attention mechanism into the Ghost module to discriminate the weight of each channel.