This preprint is under consideration at BMC Musculoskeletal Disorders. A preprint is a preliminary version of a manuscript that has not completed peer review at a journal. Research Square does not conduct peer review prior to posting preprints. The posting of a preprint on this server should not be interpreted as an endorsement of its validity or suitability for dissemination as established information or for guiding clinical practice.
Learn more about In Review
Research article
Artificial Intelligence Improves the Accuracy of Residents in the Diagnosis of Hip Fractures –A Multicenter Study
YOICHI SATO
JCHO Tokyo Yamate Medical Center: Tokyo Yamate Medical Center
Corresponding Author
ORCiD: https://orcid.org/0000-0001-6144-3735
License:
This work is licensed under a CC BY 4.0 License. Read Full License
Background
Less experienced clinicians sometimes make misdiagnosis of hip fractures. We developed computer-aided diagnosis (CAD) system for hip fractures on plain X-rays using a deep learning model trained on a large dataset. In this study, we examined whether the accuracy of the diagnosis of hip fracture of the residents could be improved by using this system.
Methods
A deep convolutional neural network approach was used for machine learning. Pytorch 1.3 and Fast.ai 1.0 were applied as frameworks, and an EfficientNet-B4 model (a pre-trained ImageNet model) was used. We handled the 5295 X-rays from the patients with femoral neck fracture or femoral trochanteric fracture from 2009 to 2019. We excluded cases in which the bilateral hips were not included within an image range, and cases of femoral shaft fracture and periprosthetic fracture. Finally, we included 5242 AP pelvic X-rays from 4,851 cases. These images were divided into 5242 images that included the fracture site and 5242 images that did not. Thus, a total of 10484 images were used for machine learning. The accuracy, sensitivity, specificity, F-value, and area under the curve (AUC) were assessed. Gradient-weighted class activation mapping (Grad-CAM) was used to conceptualize the basis for the diagnosis of the fracture by the deep learning algorithm. Secondly, we conducted a controlled experiment with clinicians. Thirty-one residents and four orthopedic surgery fellows were tested using 300 hip fracture images that were randomly extracted from the dataset. We evaluated the diagnostic accuracy with and without the use of the CAD system for each of the 300 images.
Results
The accuracy, sensitivity, specificity, F-value, and AUC were 96.1%, 95.2%, 96.9%, 0.961, and 0.99, respectively, with the correct diagnostic basis generated by Grad-CAM. In the controlled experiment, the diagnostic accuracy of the residents significantly improved when they used the CAD system.
Conclusions
We developed a newly CAD system with a deep learning algorithm from a relatively large dataset from multiple institutions. Our system achieved high diagnostic performance. Our system improved the diagnostic accuracy of residents for hip fractures.
Level of Evidence
Level Ⅲ, Foundational evidence, before-after study. Clinical relevance: high
Keywords
misdiagnosis , hip fractures, CAD, plain X-rays
Figure 1
Figure 2
Figure 3
Figure 4
This is a list of supplementary files associated with this preprint. Click to download.
Badges
Prescreen
This manuscript passed our Prescreen assessment
Complete author information
Appropriate declaration statements
Potential risks to human health
Prescreen
Peer Review Timeline
CURRENT STATUS: Under Review
Version 1
Posted 25 Sep, 2020
No community comments so far
Editorial decision: Minor revision
On 21 Nov, 2020
Review #2 received
Received 12 Nov, 2020
Review #1 received
Received 22 Oct, 2020
Reviewer #2 agreed
On 21 Oct, 2020
Reviewers invited
Invitations sent on 01 Oct, 2020
Reviewer #1 agreed
On 01 Oct, 2020
Editor assigned
On 14 Sep, 2020
Submission checks complete
On 13 Sep, 2020
Editor invited
On 13 Sep, 2020
First submitted
On 11 Sep, 2020
Metrics
Comments: 0
PDF Downloads: ...
HTML Views: ...
Subject Areas
Orthopedics
Learn more about our company.
This preprint is under consideration at BMC Musculoskeletal Disorders. A preprint is a preliminary version of a manuscript that has not completed peer review at a journal. Research Square does not conduct peer review prior to posting preprints. The posting of a preprint on this server should not be interpreted as an endorsement of its validity or suitability for dissemination as established information or for guiding clinical practice.
Learn more about In Review
Research article
Artificial Intelligence Improves the Accuracy of Residents in the Diagnosis of Hip Fractures –A Multicenter Study
YOICHI SATO
JCHO Tokyo Yamate Medical Center: Tokyo Yamate Medical Center
Corresponding Author
ORCiD: https://orcid.org/0000-0001-6144-3735
Badges
Prescreen
This manuscript passed our Prescreen assessment
Complete author information
Appropriate declaration statements
Potential risks to human health
Prescreen
Peer Review Timeline
CURRENT STATUS: Under Review
Version 1
Posted 25 Sep, 2020
No community comments so far
Editorial decision: Minor revision
On 21 Nov, 2020
Review #2 received
Received 12 Nov, 2020
Review #1 received
Received 22 Oct, 2020
Reviewer #2 agreed
On 21 Oct, 2020
Reviewers invited
Invitations sent on 01 Oct, 2020
Reviewer #1 agreed
On 01 Oct, 2020
Editor assigned
On 14 Sep, 2020
Submission checks complete
On 13 Sep, 2020
Editor invited
On 13 Sep, 2020
First submitted
On 11 Sep, 2020
Metrics
Comments: 0
PDF Downloads: ...
HTML Views: ...
Subject Areas
Orthopedics
License:
This work is licensed under a CC BY 4.0 License. Read Full License
Background
Less experienced clinicians sometimes make misdiagnosis of hip fractures. We developed computer-aided diagnosis (CAD) system for hip fractures on plain X-rays using a deep learning model trained on a large dataset. In this study, we examined whether the accuracy of the diagnosis of hip fracture of the residents could be improved by using this system.
Methods
A deep convolutional neural network approach was used for machine learning. Pytorch 1.3 and Fast.ai 1.0 were applied as frameworks, and an EfficientNet-B4 model (a pre-trained ImageNet model) was used. We handled the 5295 X-rays from the patients with femoral neck fracture or femoral trochanteric fracture from 2009 to 2019. We excluded cases in which the bilateral hips were not included within an image range, and cases of femoral shaft fracture and periprosthetic fracture. Finally, we included 5242 AP pelvic X-rays from 4,851 cases. These images were divided into 5242 images that included the fracture site and 5242 images that did not. Thus, a total of 10484 images were used for machine learning. The accuracy, sensitivity, specificity, F-value, and area under the curve (AUC) were assessed. Gradient-weighted class activation mapping (Grad-CAM) was used to conceptualize the basis for the diagnosis of the fracture by the deep learning algorithm. Secondly, we conducted a controlled experiment with clinicians. Thirty-one residents and four orthopedic surgery fellows were tested using 300 hip fracture images that were randomly extracted from the dataset. We evaluated the diagnostic accuracy with and without the use of the CAD system for each of the 300 images.
Results
The accuracy, sensitivity, specificity, F-value, and AUC were 96.1%, 95.2%, 96.9%, 0.961, and 0.99, respectively, with the correct diagnostic basis generated by Grad-CAM. In the controlled experiment, the diagnostic accuracy of the residents significantly improved when they used the CAD system.
Conclusions
We developed a newly CAD system with a deep learning algorithm from a relatively large dataset from multiple institutions. Our system achieved high diagnostic performance. Our system improved the diagnostic accuracy of residents for hip fractures.
Level of Evidence
Level Ⅲ, Foundational evidence, before-after study. Clinical relevance: high
Figure 1
Figure 2
Figure 3
Figure 4