Protocol and Registration
The protocol will be registered on the PROSPERO international prospective register of systematic reviews. The systematic review will follow the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement and guidelines [12, 13].
For the initial literature search, the PICOS (Population, Intervention, Comparison, Outcome, Study Design) framework will be utilized for the eligibility criteria . The following characteristics for clinical studies are:
Studies involving human models from adult participants (18 years or older) will be considered for review, without any geographical exclusion criteria. Studies meeting eligibility criteria will apply to orthopedic interventions, acute and chronic orthopedic musculoskeletal injuries. Articles will be excluded from eligibility if they are animal studies or do not relate to orthopedic intervention.
The studies considered will present ML models employing deep learning as an intervention with the aim of providing diagnosis or clinical prognosis of an orthopedic surgery intervention. The intervention may be used by itself or with other methods. Studies will be excluded if no clinical data were used. Considering that there is no single best ML model, various models will be used.
Gold standard processes and practices to obtain a clinical diagnosis and predict post-operative outcomes shall be compared with and without the use of ML algorithms.
The primary outcome is the evaluation of ML models and how accurately they can provide a clinical diagnosis, and how accurately they can predict post-operative outcomes and complications of orthopedic surgery interventions.
Any case reports and other primary studies assessing the prediction rate of post-operative outcomes or the ability to identify a diagnosis in orthopedic surgery will be included. Systematic reviews or literature reviews will only be examined to identify further studies for inclusion, and results of meta-analysis will not be included in the analysis. Regarding publication year, all studies published to date will be included.
A systematic search will be conducted in PubMed, ScienceDirect, and Google Scholar databases of English, Italian, French, Spanish and Portuguese language articles published before July 2020. Secondary searching of reference lists of key articles and reviews will be undertaken in order to identify any additional studies potentially missed in electronic search.
The PRISMA checklist and flow diagram will be used as the eligibility and inclusion criteria during the search and selection process. A web-based reference software system (RefWorks) will be used for data management.
The study selection process will entail an initial review by two different reviewers of all titles and abstracts. These findings will be uploaded to the web-based reference software system. This will be followed by a second review, again by two independent reviewers, of the full text articles. This will ensure that the remaining articles meet the inclusion criteria. Discrepancies at any level will be resolved through discussion with a third reviewer.
Two independent reviewers will perform data extraction from articles which will meet the inclusion criteria. The data extracted and synthesized will include study characteristics, application of the ML model, outcome measurement, outcome assessment and complications or adverse events reported. Forms will be customized during the data extraction and collection process. Primary authors will be contacted via email if any information requires clarification.
Relevant items related to study characteristics such as authors, study design and year of publication will be extracted and included. The characteristics of the ML software and its specific use in the prediction of a diagnosis or treatment outcome will also be included. Outcomes measures relating to predicting a diagnosis will include negative predictive value, positive predictive value, sensitivity and specificity. Measures extracted and included for the prediction of post-operative outcomes will include function, general outcomes, complications, success and survival.
Risk of bias
Multiple resources will be used to assess the risk of biases and shall be reported as low risk, moderate risk, or high risk of biases. Ten domains will be addressed related to selection bias, performance bias, detection bias, attrition bias, reporting bias, etc. The Risk Of Bias In Non-randomized Studies of Interventions (ROBINS-I) tool assesses observational and quasi-randomized studies . Seven domains will be used in assessing risk, including confounding, participant selection bias, classification bias, deviation bias, bias from missing data, outcome measurement bias, and bias in selection of reported results. Studies will be categorized into no information or a low, moderate, serious, or critical risk of bias. For randomized control trials, the Risk of Bias 2 (RoB 2) tool will be used to establish risk of bias . Five domains of biases will be analyzed, including those from the randomization process, those arising from deviations from intended interventions, those from missing outcome data, those found in measuring the outcome, and those found in selection of the reported result. All included studies will be independently scored by two reviewers and a discussion will facilitate consensus of the biases risk levels.
Data synthesis and meta-analysis
Guidelines published by Hooijmans et al. will be used to for the data synthesis and meta-analysis. A random-effects meta-analysis followed by subgroup analysis will be performed if appropriate, given the anticipated heterogeneity amongst studies. Sources of potential heterogeneity include the ML software used, what it was used for (diagnosis or treatment), and the treatment population and indication. Results of meta-analysis will be summarized appropriately with emphasis on design and outcome measures.