In this study, we demonstrated that radiographic textural features of distal metacarpal bones could indicate early signs of RA. Because of the complexity of the high-dimensional textural features in radiographs, simple mathematical operations such as FSA cannot be used to describe them. In contrast, deep learning methods can provide an overall insight into the complex textural bone properties and yield risk scores based on them, thereby enabling the classification early RA and stratifying patients into different risk groups. Thus, deep learning methods can be used for automatic reporting of RA risk based on plain radiographs; this risk information could then be incorporated into standard clinical risk analysis for early RA prediction.
We compared two deep learning models, namely the Deep-TEN and ResNet-50 models, for RA classification. Based on our results, the performance of both models is similar in terms of binary classification into RA and non-RA radiographs. However, the primary difference between both models is that the Deep-TEN model only takes into account the textural information from radiographs for RA prediction, while the ResNet-50 model considers both their textural and structural features. For example, bone erosions resulting in a change in bone contour are not considered by the Deep-TEN model because it is a structural feature change. Therefore, the ResNet-50 model performs slightly better at identifying patients at high risk of RA. In contrast, the Deep-TEN model is better at separating the patients into three risk groups for RA based on changes in the texture, thereby forming a more homogenous risk continuum. Hence, the selection of a deep learning model for RA prediction in clinical settings would depend on clinical needs.
The 1987 ACR classification criteria for RA(28) define erosion or unequivocal bony decalcification (periarticular osteoporosis) in hand and wrist posteroanterior radiographs as one of the radiographic features relevant to RA diagnosis. Periarticular osteoporosis, which is a bone textural feature, is an osseous morphologic indication that is observed before the occurrence of bone erosions and joint space narrowing.(29, 30) Early periarticular osteoporosis, which is characterized by the loss of trabecular size and reduction in the number of metaphyseal regions, is difficult to detect and quantify via traditional hand radiography; therefore, X-ray radiogrammetry,(31) CT,(32) and MRI(33) have been applied to detect periarticular osteoporosis in previous studies. However, the application of these approaches in clinical settings is hampered by their high costs. In the 2010 EULAR-ACR classification criteria for RA,(21) information on RA diagnoses based on clinical features such as joint involvement or symptom duration as well as using laboratory tests for anti-citrullinated peptide antibodies, rheumatoid factor, C-reactive protein, and erythrocyte sedimentation rate were included. Radiographic bone texture changes were not emphasized as in the previous 1987 ACR criteria(28) because early indications of bone erosion and periarticular osteoporosis were difficult to assess objectively from plain radiographs, and this could have led to delayed RA diagnosis. Traditionally, conventional radiography was considered to be less sensitive to early indications of RA. Nevertheless, in recent times, with the assistance of machine learning techniques, as we have observed in our study, conventional radiography could perhaps be useful for early RA classification.
In many clinical situations, the automatic evaluation of radiographs using deep learning will be of great medical value, because such a system could potentially support RA diagnosis as a screening tool for RA in both general clinics and specialized hospitals. Furthermore, our proposed CNN model can estimate the bone texture score and predict RA from radiographs within one second per image, which is considerably faster than analyses by human clinicians. Thus, our proposed model could save time and be used as a diagnostic tool in countries where the number of available rheumatologists or radiologists is low. Furthermore, it can be used by family physicians to refer their patients to RA specialists based on the diagnostic predictions by the model. Moreover, because this is a computerized model, intraobserver and interobserver variabilities can be avoided if it is applied in clinical trials related to RA research.
Compared with our current work, previous attempts to use CNNs for the interpretation of hand radiography images of RA patients did not consider the distinctive textural or structural changes that occur in the joints of RA patients. In particular, Kemal et al. used 180 hand radiographs to train their CNN model for RA diagnosis and achieved an accuracy, sensitivity, and specificity of 73.33%, 0.68, and 0.78, respectively.(34) Toru et al. proposed a model that achieved accuracies of 49.3–65.4% for joint space narrowing and 70.6–74.1% for bone erosion detection on 30 hand radiographs; their model was trained using 186 radiographs.(35) Because these two studies used downsampled images of the entire radiographs, subtle textural changes were not considered. The CNN model has been applied not only to radiography images but also to other image modalities. For example, Jakob et al. used CNN to assess synovitis activity from ultrasound images and achieved an accuracy of 86.4%, sensitivity 0.864, and specificity of 0.864.(36) Lun et al. developed a CNN-based segmentation method for the wrist using T2-weighted fat-suppressed MRI images for early RA detection.(37)
Despite the advantages of our proposed CNN-based approach for the detection of early RA indications, our study has the following limitations. First, we only analyzed the texture of the second, third, and fourth distal metacarpal bones for RA classification of radiographs. Thus, further investigation is required to confirm whether the inclusion of radiographic images of other parts of the hand as input to the proposed CNN model would increase its RA risk classification performance. Second, the training data used in the current study are from patients with early RA (for most patients, RA was diagnosed less than a year prior to the study). Thus, later temporal changes in the bone texture or structure due to RA as the disease progresses were not considered in our work. Third, the complexity of the proposed deep learning model with millions of parameters prevents a straightforward interpretation of the results by human doctors and clinicians.