Study selection
By adopting the described searching strategy, a total of 375 references were identified. After eliminating 161 duplicates, the titles and abstracts of the rest 214 references were screened, among which, 143 were excluded due to irrelevance. The full-texts of the remaining 71 references were screened. Finally, 22 references were included [11-13, 27-45]. The reference inclusion and exclusion process is shown in figure 1 (Flow diagram of study retrieval and selection).
Characteristics of included SRs/MAs
The characteristics of the enrolled SRs/MAs are presented in Table 1. Among the 22 eligible SRs/MAs, 17 were MAs [12, 13, 27-30, 32, 35, 37-45]. The publication year ranged from 2015 to 2021. In terms of the country of the first author, 50% (n=11) of the publications were from China [13, 27-29, 35, 37, 38, 41-44], 18.18% (n=4) from Korea [12, 32, 33, 40], 9.09% (n=2) from US [31, 36], while 4.55% (n=1) from Netherlands [11], Japan [30], Germany [34], India [39] and Singapore [45], respectively. Regarding the study type, 54.55% (n=12) of SRs/MAs only contained RCTs [12, 13, 29, 35-42, 44], while the remaining SRs/MAs involved both RCTs and non-RCTs [11, 27, 28, 30-34, 43, 45]. More details about the characteristics of the enrolled SRs/MAs are displayed in Additional file 2 (Supplementary Table 1. Details of characteristics in included studies).
All the included studies searched the PubMed database, 81.82% searched the Cochrane Library, 77.27% searched Embase, and 27.27% searched Web of Science. A few SRs/MAs searched the Trial Registry Platforms. For example, only 27.27% (n=6) of SRs/MAs searched the Clinical Trials.gov [11, 27-29, 37, 43]. Meanwhile, four SRs/MAs searched the Chinese databases [27, 29, 35, 44], involving CBM, CNKI and Wanfang Database. (Additional file 3 Supplementary Table 2. Retrieval source data analysis, Supplementary Table 3. The overall of retrieval source data analysis)
Assessment of methodological quality of the included SRs/MAs
Figure 2 (Methodology adherence of each item for included systematic reviews based on AMSTRA 2) summarizes the methodological adherence of each item in AMSTAR 2 for the included studies. The detailed methodological quality assessment results are shown in Additional file 4 (Supplementary Table 4. Methodological quality assessment of systematic reviews by AMSTAR 2).
Item 1. Research questions and inclusion criteria
36.36% (n=8) of the included SRs/MAs presented the PICO description (population, intervention, comparison, and outcome) to clarify the research questions and inclusion criteria. In particular, two SRs/MAs also described the follow-up timeframe [27, 35], while the remaining six only contained PICO description [11, 13, 37-39, 44] (all rated as “Yes”). However, there were still 63.64% (n=14) of the SRs/MAs rated as “No”, among which, 13 failed to specify the intervention of control group [12, 28-30, 32-34, 36, 40-43, 45] and one did not specify the intervention of control group or selected outcomes [31].
Item 2*. Protocol registration (Critical item)
No SRs/MAs were rated as “Yes”. Specifically, 18.18% (n=4) of the enrolled SRs/MAs were rated as “Partial Yes”, including three reporting a protocol and PROSPERO registration number. However, one SR/MA had inconsistent search strategy with protocol [11], while two lacked complete inclusion and exclusion criteria in the protocol, and did not describe the inconsistency between the implementation process and the protocol [12, 40]. Another SR/MA reported that a protocol was developed in advance and followed the PRISMA reporting guidelines, but the protocol was not registered or published [43]. Meanwhile, 81.82% (n=18) of the SRs/MAs were rated as “No”. Among these 18 SRs/MAs, three referred to a protocol [30, 41, 42], of which, two completed the PROSPERO registration [41, 42]. Still, one failed to provide a PROSPERO registration number [42], one was completely inconsistent with the research content [41], and the remaining one did not describe the protocol content [30]. The remaining 15 SRs/MAs did not refer to a protocol. It was impossible to determine the review methods in advance according to the content of the studies.
Item 3. Study designs for inclusion
There were 27.27% (n=6) of the enrolled SRs/MAs reporting the reasons why specific study designs were included [12, 30, 36, 37, 41, 44] (rated as “Yes”). Of them, one included prospective cohort study with long-term follow-up and emphasized that RCTs alone did not provide sufficient evidence [30], one included RCTs to offer reliable evidence due to the controversial conclusion of previous studies [12], one was carried out because a previously unpublished SRs/MAs involved RCTs alone [36], two included the newly published or high-quality RCTs to update clinical evidence [37, 41], and one considered that the overall quality of previous studies was poor and carried out MAs including RCTs to evaluate the clinical efficacy [44]. The remaining 72.73% (n=16) of the SRs/MAs did not explain the reasons why certain study designs were included (rated as “No”).
Item 4*. Literature search (Critical item)
None of the enrolled SRs/MAs used a comprehensive literature search strategy that satisfied all the components required. In addition, 68.18% (n=15) of these SRs/MAs searched two or more databases, provided keywords and search strategy, and justified language or retrieval time restrictions, but supplementary retrieval contents (like published reviews, clinical trial registration platform, field experts, gray literature) were not comprehensively searched. Among these SRs/MAs, one did not consult field experts [11], while for the rest 14 SRs/MAs, one did not retrieve gray literature [43], three did not search the reference list of the included studies or gray literature [27, 29, 37], three did not search the clinical trial registration platform or gray literature [13, 33, 41], seven did not retrieve the reference list of the included studies, gray literature, or clinical trial registration platform [30, 32, 34, 38, 40, 42, 45], which met the minimal requirement (rated as “Partial Yes”). However, 31.82% (n=7) of the SRs/MAs failed to meet the minimal requirement (rated as “No”), among which, five did not provide complete search strategies [28, 35, 36, 39, 44], one did not explain the reason for restricting the English language [12], and one did not offer comprehensive search strategy or explain the reason for restricting the English language [31].
Item 5. Duplicate study selection, literature screen
68.18% (n=15) of the enrolled SRs/MAs stated that the literature selection process was conducted by two reviewers independently [11-13, 27-30, 32-34, 36-40] (rated as “Yes”). Of them, only one used kappa value [40]. The remaining 31.82% (n=7) of the SRs/MAs did not describe the literature selection process, and it was difficult to judge whether it was independently completed by two reviewers [31, 35, 41-45] (rated as “No”).
Item 6. Duplicate data extraction
77.27% (n=17) of the enrolled SRs/MAs stated that the data extraction process was conducted by two reviewers independently [11-13, 28-30, 32-34, 36-41, 43, 44] (rated as “Yes”). Notably, none SRs/MAs provided kappa value to demonstrate that the process involved two reviewers. The remaining 22.73% (n=5) of the SRs/MAs did not describe the data extraction process, and it was difficult to judge whether it was independently completed by two reviewers [27, 31, 35, 42, 45] (rated as “No”).
Item 7*. Study exclusion and justification (Critical item)
Only 13.64% (n=3) of the SRs/MAs provided a list of excluded studies and justified the reasons for exclusions [27, 28, 44] (rated as “Yes”). The remaining 86.36% (n=19) of the SRs/MAs only described the summary reasons of exclusions and did not provide a list of specific exclusion studies [11-13, 29-43, 45] (rated as “No”).
Item 8. Description of included studies
There were no clear indicators to distinguish “Partial Yes” and “Yes”. Therefore, we rated all the studies that described PICOs as “Partial Yes” for conservative evaluation. 81.82% (n=18) of the SRs/MAs described the elements of PICOS (rated as “Partial Yes”). The remaining 18.18% (n=4) of the SRs/MAs were rated as “No”. In particular, three did not describe the control measures [32, 33, 45], and one did not describe the control measures and outcomes [34].
Item 9*. Risk of bias assessment (Critical item)
45.45% (n=10) of the enrolled SRs/MAs appropriately used the RoB assessment to assess all of the essential biases (rated as “Yes”). In particular, nine only contained RCTs and used the Cochrane RoB tool to assess all the essential biases [12, 13, 35, 37, 39-42, 44], whereas one included both RCTs and non-RCTs (RCTs: the Cochrane RoB tool, non-RCTs: ROBINS-I) [45]. 9.09% (n=2) of the enrolled SRs/MAs assessed part of the essential biases (RCTs: concealed allocation and blinding; non-RCTs: confounding and selection bias), which met the minimal requirement [30, 43] (rated as “Partial Yes”). Among them, one used the Downs and Black scale but did not evaluate the selective reporting bias [30], whereas the other adopted the modified Newcastle–Ottawa scale for non-RCTs and did not evaluate the selective reporting bias [43]. The remaining 45.45% (n=10) of the enrolled SRs/MAs were rated as “No”, including two adopting the Jadad scale for RCTs [29, 38], five not reporting the significant bias results [11, 27, 28, 32, 33], and three not reporting the tool used [31, 34, 36].
Item 10. Report on the sources of funding for included studies
There were 95.45% (n=21) of the SRs/MAs that failed to report the funding information of included studies (rated as “No”). Only 4.55% (n=1) of the SRs/MAs reported it by chart [30] (rated as “Yes”).
Item 11*. Methods for statistical combination (Critical item)
36.36% (n=8) of the SRs/MAs included RCTs alone, used appropriate effect sizes and statistical analysis models to pool the data, investigated the potential sources of heterogeneity and adjusted for the existing heterogeneity [13, 35, 38-42, 44] (rated as “Yes”). In addition, 40.91% (n=9) of the SRs/MAs were rated as “No”. In particular, six of these SRs/MAs included both RCTs and non-RCTs, but incorrectly combined two different types of studies [27, 28, 30, 32, 43, 45]. Three SRs/MAs only included RCTs, but did not correct the existing heterogeneity [12, 29, 37]. The remaining 22.73% (n=5) of the SRs/MAs were qualitative studies without data combination [11, 31, 33, 34, 36] (rated as “No MAs conducted”).
Item 12. Assessment of the potential impact of risk of bias on the results
13.64% (n=3) of the SRs/MAs included both RCTs and/or non-RCTs with different RoB and investigated the potential impact of RoB on MAs or other evidence integration [13, 30, 43] (rated as “Yes”). Of them, one conducted meta-regression [30], one carried out subgroup analysis [13], and one performed sensitivity analysis to evaluate the high-quality studies [43]. However, there were 63.64% (n=14) of the SRs/MAs not investigating the impact of existing RoB on the total effect [12, 27-29, 32, 35, 37-42, 44, 45] (rated as “No”), whereas the remaining 22.73% (n=5) of the enrolled SRs/MAs were qualitative studies without data combination [11, 31, 33, 34, 36] (rated as “No MAs conducted”).
Item 13*. Result interpretation with risk of bias (Critical item)
68.18% (n=15) of the SRs/MAs included both RCTs and/or non-RCTs with different RoB and investigated the impact of RoB on results [11, 13, 27-32, 34, 35, 39, 42-45] (rated as “Yes”). Among the 15 SRs/MAs, two discussed the RoB resulting from methodological quality [13, 27]; two mentioned the RoB of different study designs [28, 32]; one reported the RoB of outcome measurement, blinding and randomization [29]; one addressed the RoB of outcome measurement, blinding, randomization, allocation concealment and selective reporting [11]; one discussed the RoB of methodological quality and study design [30]; two addressed the risk of selection bias [31, 34]; one discussed the RoB caused by randomization and small sample size [35]; one reported the RoB of outcome measurement and blinding [39]; one discussed the RoB of implementation [42]; one addressed the RoB of outcome measurement and implementation [43]; one discussed the RoB caused by small sample size [44]; and one addressed the RoB of confounding factors [45]. The remaining 31.82% (n=7) of the SRs/MAs did not investigate the impact of RoB on the total effect [12, 33, 36-38, 40, 41] (rated as “No”).
Item 14. Exploration and explanation of heterogeneity
There were 81.82% (n=18) of the SRs/MAs investigating the sources of heterogeneity and discussing the potential impact on the conclusions (rated as “Yes”). The rest 18.18% (n=4) of the SRs/MAs were rated as “No” [12, 28, 29, 38], among which, three investigated the source of heterogeneity but did not thoroughly investigate the existing heterogeneity [12, 28, 38], and one did not report the source of heterogeneity [29].
Item 15*. Investigation and discussion of publication bias (Critical item)
13.64% (n=3) of the SRs/MAs employed funnel charts or statistics (Egger’s test, Begg’s test) to investigate the publication bias and discuss the possible publication bias and its potential impact on the results [29, 43, 45] (rated as “Yes”). However, 63.64% (n=14) of the SRs/MAs were rated as “No”, among which, six did not describe the publication bias [27, 32, 35, 38, 42, 44], and eight investigated the publication bias by funnel chart or statistics (Egger’s test, Begg’s test), but did not discuss the effects on the results [12, 13, 28, 30, 37, 39-41]. The remaining 22.73% (n=5) of the SRs/MAs were qualitative studies without data combination [11, 31, 33, 34, 36] (rated as “No MAs conducted”).
Item 16. Report of sources of conflict of interest
There were 45.45% (n=10) of the SRs/MAs rated as “Yes”, including five stating that they did not receive any funding related to the research and described the potential conflicts of interest [13, 34, 39, 43, 44]. The other five provided information on the sources of funding and explained the role of funds in research (like participation in research design, data collection and analysis, article writing) as well as the potential conflicts of interest [11, 12, 29, 33, 40]. The remaining 54.55% (n=12) of the SRs/MAs were rated as “No”. In particular, three of them reported the source of funding but did not describe the role of funds in research or the potential conflicts of interest [27, 28, 37]; two described the funding and potential conflicts of interest, but did not explain the role of funds in research [30, 41]; one stated that it did not receive any research-related funding, but did not describe the potential conflicts of interest [42]; and six did not describe the source of funding [31, 32, 35, 36, 38, 45].
Rating of each issue and overall confidence
Methodological adherence of each item is presented in figure 2 (Methodology adherence of each item for included systematic reviews based on AMSTRA 2). Overall confidence is displayed in figure 3 (Methodology adherence of overall confidence for included systematic reviews based on AMSTRA 2). All the SRs/MAs were rated as “Critically low”, among which, eight items rated as “No” accounted for more than 50%, including Items 1, 2, 3, 7, 10, 12, 15, 16. Of them, Items 2, 7, 15 were the critical items.
Risk of bias for the included SRs/MAs
The details of ROBIS in assessing the RoB of each SRs/MAs are presented in Table 2. The figure 4 (Risk of bias for included SRs/MAs based on ROBIS) showed the bias judgments in Phase 2 and Phase 3 by ROBIS in the form of percentages, and all SRs/MAs were rated as “High”.
Phase 2.1 Study eligibility criteria
Only 4.55% (n=1) of the SRs/MAs were rated as “Low” [11]. However, 63.64% (n=14) of the SRs/MAs were rated as “High”. In particular, 13 SRs/MAs did not describe the intervention measures of the control group [12, 28, 29, 30, 32-34, 36, 40-43, 45], and one did not present the intervention measures of the control group, nor did it report the selected outcomes [31]. The remaining 31.82% (n=7) of the SRs/MAs were rated as “Unclear” because they did not describe whether the protocol was registered in advance [13, 27, 35, 37-39, 44]. It was difficult to judge whether the predetermined inclusion criteria were followed based on the study contents and whether the limitations were appropriate based on the research characteristics.
Phase 2.2 Identification and selection of studies
9.09% (n=2) of the SRs/MAs were rated as “Low” [11, 33]. 86.36% (n=19) of the SRs/MAs did not carry out a comprehensive database search, and PubMed, Embase, Cochrane Library, and Web of Science databases were not all searched [12, 13, 27, 29-32, 34-45] (rated as “High”). In addition, eight did not use methods other than database-based literature retrieval [30-32, 34, 35, 40, 42, 45], seven did not meet the inclusion criteria [12, 29, 36, 37, 39, 43, 44], and one had both of the above [38]. The remaining 4.55% (n=1) of the SRs/MAs were rated as “Unclear” because the search strategy was not described, nor was the related information described [28].
For the 22 SRs/MAs, the 42 original studies involved were considered as RCTs [46-87]. After screening the full-texts, 16 did not meet the inclusion criteria. Of them, eight were not RCTs [49-51, 70, 71, 75, 76, 86], including two cohort studies [49, 70], one protocol [75], one case-control study [51], one with insufficient data [50], one conference abstract [71], one commentary [76], and one with abstract only [86]. Another three RCTs did not focus on patients with KOA [59, 60, 68], with two enrolling patients with cartilage defects of the knee joint [59, 60], and one including patients after anterior cruciate ligament reconstruction [68]. The intervention measures of two RCTs were bone marrow aspirate concentrate (BMAC) [67, 83], and those of three RCTs were Stromal vascular fraction (SVF) [79, 85, 87].
Phase 2.3 Data collection and study appraisal
There were 50.00% (n=11) of the SRs/MAs rated as “High”, among which, one did not describe the basic characteristics of the included studies in detail [45], seven did not use appropriate tools to evaluate the RoB [11, 27-29, 31, 36, 38], and three had both of the above two issues [32-34]. The remaining 50.00% (n=11) of the SRs/MAs were rated as “Unclear”, since it was difficult to judge whether all the relevant research results were extracted for data synthesis, and five of them did not describe the process of RoB assessment. It was also not easy to judge whether the two reviewers completed the RoB assessment process independently [35, 37, 42-44].
Phase 2.4 Synthesis and findings
There were 81.82% (n=18) of the SRs/MAs rated as “High”, of which, four with different types of studies were combined and analyzed [30, 34, 36, 43], eight did not deal with RoB in data synthesis [29, 35, 38-42, 44], and six had both of the above two issues [12, 27, 28, 32, 37, 45]. In addition, three SRs/MAs did not deal with the heterogeneity between studies [12, 29, 37]. The remaining 18.18% (n=4) of the SRs/MAs were rated as “Unclear” [11, 13, 31, 33].