This study sought to investigate the quality of Iranian RCTs included in CRs based on the evaluations of Cochrane reviewers. To the best of the authors’ knowledge, this study pioneered the quality evaluation of RCTs at a national scale using the Cochrane RoB tool. The quality of Iranian RCTs was found to be low in terms of the majority of RoB domains with a high or unclear RoB, which were mostly associated with the study design and included random sequence generation, allocation concealment and blinding.
A few studies mainly assessed RCTs of special subjects or fields using the CONSORT checklist and yielded consistent findings suggesting the poor methodological quality and reporting of the Iranian RCTs [22–25].
The present study found the RoB of 44.9% of the Iranian RCTs to be low in terms of random sequence generation, while the status of the remaining RCTs was impossible to be evaluated in terms of randomization given the inadequate data. Similarly, randomization was invalidated in 35.5%-98.7% of Iranian RCTs owing to their failure to report their randomization method, as was the case for the RCTs conducted in other countries [22, 26–28]; for instance, a low RoB was reported in terms of randomization for 44% of RCTs performed in Saudi Arabia [15]. A low RoB was also reported for below 50% of 1286 RCTs included in CRs.[29] Given that this problem is not specific to Iran, it is recommended that efforts be made at a global scale to enhance the quality of RCTs in this domain.
Blinding was assessed as two subdomains, i.e. blinding of participants and personnel (performance bias) and blinding of outcome assessors and analysts (detection bias). In terms of performance bias, only 32.3% (n = 262) of the included studies were assessed as low-risk, while 299 studies used placebos as the method of blinding their participants and personnel. This difference between the number of low-risk studies and that of studies using placebos shows the failure of Iranian researchers to comprehend the mechanism of implementing placebos to hide interventions from the participants or indicates their inability to properly explain the mechanism in a way that Cochrane reviewers are persuaded with the explanation. One-third of the studies were also evaluated as low-risk in terms of detection bias, which can be explained by the unwillingness of Iranians to participate in completely-blinded trials and failure to report the study details, which made it difficult for the reviewers to assess bias. Research generally suggests inadequately-performed blinding in the RCTs conducted in Iran and other countries [15, 29, 30].
A low RoB was assigned to attrition bias in 56.3% of cases, suggesting a low number of patients failing to follow up in the Iranian RCTs. In line with studies assessing Iranian RCTs in this domain, the present study found Iranian researchers to be relatively successful in providing data on the patients withdraw the study. Studies conducted in countries other than Iran have also reported the RoB of many RCTs as low in this domain [15, 29]. A low RoB was assigned to selective outcome reporting in 436 (41.3%) RCTs and an unclear RoB to 453 (42.9%). This high frequency of unclear RoB can be explained by the failure of the majority of Iranian authors to register their studies in databases such as the IRCT, which resulted in failing to publish the study protocol before the final results were published. The reviewers had therefore difficulty evaluating this domain based on the outcomes reported in the article. Given the generally-difficult assessment of selective outcome reporting and other bias domains [31] these results should be cautiously interpreted.
The limitations of the Iranian RCTs included a small sample, which was below 60 in their majority. The distribution of the sites of conducting the RCTs was also heterogeneous and many of the studies were unicenter and conducted in Tehran, the capital of Iran, which can be explained by the significantly higher number of universities and top medical universities located in Tehran compared to in other cities and provinces. It is recommended that more collaborative, multi-center and high-quality studies be conducted by making proper research policies.
According to Chalmers and Glasziou, an estimated significant portion (85%) of medical research is wasted in many dimensions and phases, namely relevance of the research question to the patients and physicians, appropriateness of the study design, accessibility of full text and unbiased and usable reporting. The present findings suggest the limitations of Iranian RCTs include all these four dimensions. Some of these issues, e.g. low-quality reporting, could be simply avoided. The introduction of the CONSORT statement in 1996 [4] has increased the percentage of studies with a low RoB in randomization, blinding of outcome assessors, incomplete data reporting and other sources of bias. However, the high frequency of unclear RoB in all the domains can be an alarming sign of low-quality reporting in Iranian RCTs.
The strengths of the present study include the evaluation of the RCTs by third-party reviewers with no prejudices or bias. The RoB was, however, differently reported in different CRs for the same RCT. This discrepancy was also raised in previous assessments of the RoB based on the Cochrane tool [32].