Protocol design and registration
The design, conduct, and reporting of this systematic review will follow the Cochrane Handbook of Systematic Reviews of Interventions [22]. The protocol has been registered with PROSPERO, an international database of prospectively registered systematic reviews to prevent duplication (Trial registration number: CRD42020223133). Considering this is a methodological review, we will adopt the Studies, Data, Methods, Outcomes (SDMO) framework to address critical questions in this review [23]. The study design (S) focuses on RCTs due to its level of evidence [24]. The data (D) are the results from original published studies in biomedical journals (PubMed, Embase, Cochrane Central Register of Controlled Trials (CENTRAL), and Web of Science). The method (M) compares the quality of reporting subgroup analyses between pre- and post-publication of the updated the Consolidated Standards of Reporting Trials (CONSORT) guidelines [25-27]. Lastly, the outcome (O) is to determine if there is an improvement in the quality subgroup analysis reporting in stroke clinical trials after the publication of the CONSORT statement.
Eligibility criteria and search strategy
We will include studies with subgroup analyses in stroke randomized controlled trials by searching the electronic biomedical databases up to November 30th, 2020. Studies in this systematic review will include (1) randomized controlled trials, (2) stroke trials, (3) trials with subgroup analysis, effect modification, or interaction term, and (4) trials with published protocols or trial registration. Hence, we will exclude (1) subgroup analysis defined by intervention characteristics, (2) studies in not in human, (3) systematic review, literature review, or meta-analysis, (4) grey literature or conference abstracts, and (5) non-English language publications. The flowchart of the inclusion and exclusion is in Figure 1. The search strategy will include three main concepts, i.e., randomized controlled trial, stroke, and subgroup analysis/effect modifications. Below in Table 1 is the detailed Medical Subject Headings (MeSH) for Medline electronic database and Figure 1A in the appendix for the Medline’s search result.
Two reviewers (AA and SB) will independently execute the search strategy in each of the preselected electronic databases and compare the number of articles obtained from each database. The reviewers will upload these identified studies into Covidence, a web-based application for undertaking systematic reviews [28]. Covidence will automatically remove duplicated studies and provide a standard template where the reviewers can independently review them.
Selection of studies
AA and SB will separately screen titles and abstracts of identified articles using both inclusion and exclusion criteria. Likewise, AA and SB will individually perform a full-text review of all eligible articles and extract relevant information. Unresolved disagreement on any article between the two reviewers will be marked “Further review required.” They will forward such articles to LT, MDH, and TTS, who serves as an adjudicator making the final decision. The results will summarize in the PRISMA flow diagram [29]. Also, we will compute and report the interrater reliability score.
Data Items and Outcomes
The primary outcome is the credibility of subgroup analyses results of published stroke randomized controlled trials. The outcome will be obtained using ICEMAN to evaluate each subgroup analyses. The main measure of effect is the proportion of subgroup analysis with credible claim. AA and SB will extract relevant characteristics reported for each study’ subgroup analyses after a thorough full-text review. Data items to be extracted are the first author name, country of the first author, year of publication, journal of publication, study design, trial intervention, prespecified analysis in the original protocol, and the number of subgroups analyzed. Other information includes sample size, type of endpoint for subgroup analysis (primary or secondary), type of endpoints (binary, continuous, and ordinal), analysis type (linear regression, generalized linear model, or proportional regression). Also, AA and SB will extract the statistical significance of effect modification (subgroup analysis), and the magnitude of overall treatment effect size reported (small, median, large). More detailed relevant characteristics to be extracted from each study are in Appendix Table 1A.
Risk of bias and assessment of subgroup analyses’ credibility.
We will use two different validated assessment tools to appraise the study design’s quality and the subgroup analyses’ trustworthiness of each study. A poorly conducted trial invalidates such trial’s results; therefore, we will assess each study design through the Cochrane Risk-of-Bias tool for randomized trials version 2 (RoB 2) [30]. This instrument will evaluate pertinent study qualities such as participants’ selection, randomization, concealment, and allocation. Next, we will use the ICEMAN checklists to evaluate each study subgroup analyses’ credibility. The ICEMAN has five-core questions for RCTs. Each core question has four response options that ranged from “probably reduced credibility” to “probably increased credibility. Reviewers (AA and SB) will independently use these checklists to grade each study’s quality and credibility of reported subgroup analyses for these studies. LT, MDH, and TTS will adjudicate any disagreement between the two reviewers. Based on RoB 2, we will classify each study ranging from low to high quality. AA and SB will use the ICEMAN checklists to assess each subgroup analysis credibility through the four-rating score, i.e., from very low credibility to high credibility.
Data analysis (include analysis based on ICEMAN)
Descriptive analysis of all the extracted data from all the included studies will be conducted. We will analyze the extracted data using both ordinal, binary and cluster analysis. These results will be stratified by the reported studies’ credibility, type of intervention, whether the subgroup analysis result is significant, population group (acute versus non-acute stroke), and publication journal. Mixed-effect regression will be used to evaluate the impact of study characteristics (publication journal, sample size, type of analysis, type of endpoint [primary or secondary], disease population, intervention, and risk of bias) on the credibility of subgroup analysis reported in the studies. Trend analysis of the reporting will be performed to appraise any improvement in the quality of reporting over the last few years. Lastly, we will analyze the impact of publication journals on the reporting quality trials’ results [31, 32].