All procedures performed in this study involved human participants in accordance with the ethical standards of the institutional review board and the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. Approval from the Institutional Review Board of our hospital was obtained, with the project approval number of “2021 Medical Review 088”. The HIPAA requirements were followed. Informed consent was not required.
Subjects
In this study, we selected patients who underwent standing X-ray examinations of the lower limbs from the period of March 2021 to June 2021. The exclusion criteria were as follows: (1) patients who could not meet the measurement requirements after external fixation because of blurring landmarks, (2) patients who had repeated examinations, and (3) patients who had poor image quality. A total of 1000 patients were enrolled in this study (Fig. 1).
X-ray examinations and ground truth labelling
A double-planar X-ray scanner (Discovery XR656, GE Healthcare, Milwaukee, WI, USA) was used to perform full-length X-ray examinations of the lower limbs in the standing position. The patients stood naturally; both hands held both sides of the shelf, kept their feet shoulder-width apart, the median sagittal plane of the body was perpendicular to the detector and the horizontal plane, the coronal plane of the body was perpendicular to the horizontal plane, the patella was facing forward, and the lower limbs were completely extended. Tube voltage, tube current, and target-film distance were set to 75 KV, 25 mAs, and 180 cm, respectively.
The measurement parameters included: 1) the mechanical axis of the femur, the line between the centre of the femoral head and the lowest point of the intercondylar fossa of the femur, where the centre of the femoral head is determined by mose concentric circles, 2) the mechanical axis of the tibia, i.e., the line between the midpoint of the intercondylar ridge of the tibia, and the midpoint of the talus, 3) mLDFA: lateral angle of the tangent line of the distal femoral articular surface intersects the mechanical axis of the femur, 4) MPTA: medial angle of intersection of articular surface tangent of tibiae plateau with mechanical axis of tibia, 5) LDTA: lateral angle of intersection of tangent of distal articular surface of tibia and mechanical axis of tibia, 6) JLCA: angle of intersection of tangent lines of the distal femur and tibial plateau, and 7) mechanical axis of the lower limb, i.e., the distance between the centre of the femoral head and midpoint of the talus. The ground truth of the training, validation, and test sets was measured by two experienced radiologists (YQL, engaged in imaging diagnosis of musculoskeletal system for 10 years, and ZFL, engaged in imaging diagnosis of musculoskeletal system for 7 years). We considered the average values from the two above radiologists as the ground truth to decrease the individual differences between markers [13]. Finally, another senior radiologist (XHM, engaged in imaging diagnosis of musculoskeletal system for 12 years) reviewed all the generated ground truth and revised some inconsistent cases.
All measurements were performed independently on a local measuring tool based on Python 3.6. The radiologist first opened the software and imported full-length X-ray images of both lower limbs. Next, the centre of the femoral head, the lowest points of the lateral and medial condyles of the femur, the lowest point of the intercondylar fossa of the femur, the lateral and medial point of the tibial plateau, the midpoint of the intercondylar spine of the tibia, and the lowest points of the lateral and medial articular surface of the distal tibia were marked. The study also enrolled some patients who had total knee arthroplasty and marked the medial and lateral points, and the middle points between the medial and lateral points of the joint prosthesis in the distal femur and proximal tibia. Following marking, the software automatically calculated and displayed the mLDFA, MPTA, LDTA, JLCA, and the mechanical axis of the lower limbs. Finally, the measured data were saved and exported to Excel.
Data splitting, pre-processing, and augmentation
According to examined dates, we divided 70% of the enrolled X-rays as a training set (n = 700), 10% as the validation set (n = 100), and the other 20% of the X-rays were treated as the test set (n = 200).
To eliminate the scanning differences between subjects, we applied a series of pre-processing to normalize all the enrolled X-rays. First, we resampled the pixel spacing to 1×1 mm. Min-max normalization was next used to scale pixel values. We also employed rotation as the augmentation strategy to increase the variance of the training set. In the training phase, the input X-rays were rotated at a random angle in the range of 5° to 5°.
Deep learning methods of landmark location
In this study, we employed the VB-Net architecture as the basic model to build a coarse-to-fine system. The architecture of VB-Net is shown in Fig. 2. In the novel VB-Net, a bottle-neck structure replaces the conventional convolutional layers in the convolutional U-Net and thus contributes to a significant decrease in the model size. In this study, we considered a 20×20 region around each ground truth as the network input. In addition, we output the centre of the largest connected component in probability maps. In this manner, we realize the landmark detection using a segmentation network.
As shown in Fig. 2(a), a coarse VB-Net first located the greater trochanter, fossa intercondyle of the femur, and the lateral malleolus on both the left and right sides of the X-ray, respectively. Next, three pairs of image patches were extracted around the above-mentioned regions. The cropped field of view (FOV) was 180 mm2 for the greater trochanter regions, 128 mm2 for the fossa intercondyle regions, and 104 mm2 for the lateral malleolus regions. In the second stage, three fine VB-Net were constructed to precisely locate the greater trochanter, head centre of the femur in greater trochanter-centric patches, lateral femoral condyle, fossa intercondyle, medial femoral condyle, lateral tibial condyle, eminentia intercondyle, medial tibial condyle in fossa intercondyle-centric patches, lateral malleolus, and medial malleolus in lateral malleolus-centric patches. The output FOV of both the coarse and fine networks was 5 mm2.
In this study, we use the training set to build a two-stage network. We set the loss function as the focal loss and the constant learning rate as 0.0001 based on the validation set. We use the test set to evaluate the performance of the proposed AI-aided lower limb measuring system. The deep learning algorithm was developed on PyTorch with an NVIDIA GeForce GTX TITAN X graphic card.
Lower Limbs alignment automatic measurement
In this study, we aimed to obtain mLDFA, MPTA, LDTA, JLCA, and the mechanical axis of the lower limbs for preoperative measurements. The related parameters of lower limb alignment can be calculated automatically using the 10-pairs core regions. The calculations are detailed in Fig. 3.
Statistical Analysis
Statistical analyses were performed using SPSS 26.0 software (version 26.0; SPSS Inc., Chicago, IL, USA). For key landmark location estimation, we determined the percentage of points of the correct key (PCK) [14] with a threshold of 3 mm. Furthermore, some metrics were employed to estimate the angle prediction performance of the DCNN system and human experts. Intraclass correlation coefficients (ICCs) with 95% confidence intervals and Pearson correlation were used to analyse the correlation between the measurements of the DCNN system and the ground truth. For the variability analysis, the mean absolute error (MAE) and root mean squared error (RMSE) were calculated. The measurement time of the DCNN system and the ground truth were compared using an independent sample t-test. Statistical significance was set at P < 0.05. Additionally, to visually demonstrate the distribution of the metrics, Bland-Altman plots were also drawn.