In the current study, we described the current methodological reporting quality in three major sports medicine journals. It is noted that 29.9% of all RCTs did not present adequate baseline data, 65.1% did not report adequate generation of the allocation sequence,73.9% did not report adequate allocation concealment, 44.8% did not report adequate blinding way, 73.1% did not perform adequate analysis and 78.9% did not report adequate handling of dropouts. These findings suggest that more efforts should be taken to improve the methodological reporting quality of RCTs in major sports medicine journals. And this is in accord with the conclusion by Brophy et al[10].
Although the history of clinical trials dates back to 600 B.C. [15], the clinical study of pulmonary tuberculosis by Austin Bradford Hill is recognized as the beginning of modern randomized controlled trials which ushered in a new era[16]. Currently the Cochrane Library has already collected over 150, 000 RCTs which have become the solid basis of the evidenced based medicine (EBM). Therefore, the quality of evidence is extremely important[17].
The quality of RCTs is largely dependent on the study design and methodology conducted. Although the quality of design and methodology is not necessarily correlated with methodological reporting quality[18], under most circumstances, we can only depend on the information contained in the written paper. There is a possibility that a trial with a biased design, if well reported, could be considered of high quality, whereas a well-designed trial with poor reporting quality could be neglected. Nevertheless, some studies proposed that the reporting quality of RCTs could be used as a means to evaluate the methodological quality[19]. Therefore, it’s clearly a poorly reported RCT could be very misleading. Thus, to improve the reporting quality has gradually become a common consensus worldwide. To improve the standards of written reports of RCTs, in 1996, the “CONSORT” (Consolidated Standards of Reporting Trials) was published[20] and revised in 2001[21] and 2010[13].The CONSORT statement of 2010 comprises a 25-item checklist and a flow diagram. The checklist describes how the trial is designed, analyzed and interpreted. Based on the 2010 version, we carried out this study. We selectively chose “sample size calculation, allocation concealment, blinding, type of analysis, handling of dropouts” to evaluate the reporting methodological quality, for these are the most important factors influencing the RCTs quality.
Studies have demonstrated that endorsement of the CONSORT statement is associated with improved RCTs reporting methodological quality[22–24]. In recent years, the number of studies reporting methodological quality of RCTs based on the CONSORT statement has been increasing dramatically[25].
As we searched and reviewed, trials in many fields[5, 8, 14, 26, 27] showed inconsistence and non-adherence to the CONSORT statement, including anesthesiology[26], spine surgery[8], gastroenterology[5], surgery[14] and four high-impact medical journals[27]. Therefore, it is safe to conclude that the CONSORT statement has not been widely recognized although it has been introduced for almost twenty years. The CONSORT statement was significantly revised in 2010. According to different strata, it is found generally that methodological reporting quality has not been significantly improved between 2008–2010 and 201–2017. However in this study only allocation sequence generation and handling of dropout is improved after 2010 and the difference is statistically significant (p < 0.01), which indicated that the improvement of the reporting quality of RCTs in sports medicine still needs more attention.
The methodological reporting quality of RCTs in sports medicine has not been previously published. We hypothesized that the CONSORT statement was not well endorsed in this area. Randomization is the key of RCTs procedure which assigns all participants to each study group by equal chance. An appropriate sequence generation method directly determines the randomization. In this study, we employed similar criteria previously reported[8]. A study considered with adequate sequence generation should clearly and properly address the sequence generation methods, such as random number table or a computer random number software. Sequence generation based on systematic methods such as the date of admission or with only simple statement like “we randomly assigned” was regarded as inadequate. In this study, we found that less than half (34.9%) reported adequate generation of allocation sequence.
Adequate sequence generation alone is not sufficient to prevent bias. If the sequence is open to investigators, health care providers and participants who will be aware of the upcoming assignment, selection bias is likely to generate[28]. Thus, allocation concealment is considered to be an important part of randomization[29] and authors who report randomized trials should provide enough details on how allocation concealment was implemented so that the reader is able to determine the likelihood of success. In this study, we found that only 26.1% trials reported adequate allocation concealment and envelope was the major means.
Blinding is an important means to reduce bias. However, in the treatment of sports-related diseases, it is difficult or impossible to compare surgical or rehabilitation interventions with a double-blinded method. Nevertheless, the 2011 Cochrane Handbook defines adequate blinding. First, the outcome and the outcome measurement are not likely to be influenced by lack of blinding. Second, either the participants or some key study personnel are not blinded, but the outcome assessment is blinded. Although it is not easy to conduct a double-blind surgical trial, it is basically possible to find a blinded outcome assessor. In this study, 262 (55.2%) trials reported adequate blinding including 138 with single blinding and 124 with double blinding.
An intention-to-treat (ITT) analysis of the results is based on the initial treatment assignment and not on the treatment eventually received. In contrast, a per-protocol analysis only focuses on patients who complete the entire clinical trial following the protocol. It could overestimate the practical value of an intervention. In this study, only 122 (25.7%) trials reported employing ITT analysis and 27 (5.7%) reported per-protocol analysis, which was far away from the standard of the CONSORT statement.
Missing data caused by patients dropping out of the study before completion is a major problem in the analysis of clinical trials. It can result in reduced sample size, biased treatment comparison and influence the overall statistical power of the study[30]. A review of 71 randomized controlled trials in four top medical journals showed dropout rates of 20% or more in 18% of trials[31]. Therefore, it is important to report an adequate handling of dropouts since it could notify readers’ unsatisfactory reasons for dropouts including adverse events, lack of efficacy, lost to follow-up, death and so on. In this study, only 100 (21.1%) reported handling of dropouts.
Except for the above items which could affect the process to draw a right conclusion, funding source is another item. The result of the item “funding source” indicated that trials with industrial funding showed better quality in adequate allocation concealment (p < 0.001), adequate type of analysis (p = 0.011) and adequate handling of dropout(p = 0.002). It suggested that we should focus on which kind institutions funded a trial, rather than whether there was a funding institution when we evaluate quality of a RCT.
The current study had several limitations. First, the methodological reporting quality could differ from the actual methodological quality. As we focused on the methodological methods of the trials, information not find in the published manuscript was deemed as deletion, which resulted that there is a possibility that a well-designed and well-conducted trial is considered with high risk of bias if the methodological methods were not reported adequately. Second, we only extracted major item instead of all items from the CONSORT 2010 statements. Third, although the three sports medicine journals with the highest impact factors were included, many sports medicine related RCTs are published in other journals. To some extent, this study reflects the methodological reporting quality of RCTs in sports medicine studies.