GBM had been the focus of substantial clinical and basic science research activities aimed at discovering a treatment modality that can significantly improve survival [13]. The recently reported analysis of the 100 most cited GBM publications contained 27 clinical studies, 19 of which were trials that were included in this study (articles 1-18, 20) (Table S1) [1]. The review of the 44 neurosurgical RCTs in high impact journals that was published lately did not contain any GBM trials [11]. None of these publications assessed citation patterns.
The mean citation numbers for the 100 most cited GBM trials was 825, which was higher than the mean citation of 198 for the 100 most cited meningioma articles [19]. However, it is slightly lower than the reported median citation number of 935 for the 100 most cited general GBM articles [16]. The finding is not surprising as the mentioned review covered a bigger pool of GBM studies that included 52 basic science articles [16]. The latter are recognised to be associated with high citation numbers [2, 10, 16]. Variation in citation rates according to study topic or subject is well recognized in the literature relating to neurosurgery [1], spine [4], plastic surgery [7], and urology [9]. It is generally accepted that disciplines differ in their citation practices and that certain topics or subject area may be cited more than others [2].
It is also agreed that the numbers of citations is influenced by the size of literature in the field [2].
In this analysis, a significant association with study population was observed in both primary and secondary analysis implying that study population was a firm predictor of citation rates in GBM clinical trials. Similar findings relating to study population were reported by others [2, 4, 6, 9, 20]. A significant correlation with trial phase was also found in both primary and secondary analysis indicating that being a phase III trial was a solid predictor of citation rates. Citation rates, however, were not affected by randomization which is surprising as the correlation between RCT-type studies and bigger citation numbers is well reported in the literature [1, 8–10]. This finding could be unique to the GBM topic area or could be related to the relatively limited number of articles selected in this review. Citation rates were not affected by the trials’ number of arms, status of GBM and duration of study. The lack of impact of certain features of study designs on citation numbers was also noted by others [2, 20].
In this study, the primary analysis did not reveal a correlation between treatment protocols and citation rates. However, the secondary analysis demonstrated a significantly higher citation rates in trials in which the treatment protocol included radiotherapy and TZM. This probably reflects the current widely accepted standard treatment for newly diagnosed GBM which includes surgery followed by concurrent radiotherapy with TZM and further adjuvant TZM [21, 22]. No significant association was observed between citation rates and treatment protocols that included chemotherapy in general. This probably relates to the wide-ranging chemotherapeutic agents used in the studies and their mixed efficacy. Also, the chemotherapy in general group included older studies that were conducted before the use of standard TZM in the first line setting. The lack of association between citation numbers and treatment protocols including BVZ, nitrosourea, surgery and local treatment and immunotherapy may be influenced by the limited number of trials that focused on the treatment modality. However, it could reflect their undetermined role in the future direction in the management of GBM [21, 22]. Citation rates were also not affected by the duration from the time of publication (age of the study). This is not unusual as the study covered a long period (43 years). It is recognized that the number of citations increase in the first year after publication to reach a peak and then they are less cited as time passes [2]. The latter could be because the paper’s information becomes outdated with time [2].
In this article, significant association with journal’s IF was observed in both primary and secondary analysis denoting that journal’s IF was a strong predictor of citation rates in GBM clinical trials. Similar findings relating to the journal’s IF were reported by others [1, 2, 15]. Furthermore, the secondary analysis demonstrated a significantly higher citation rates in trials that were published in general medical journals. This was expected as the group of general medical journals in this study had a much higher mean IF than the oncological and neurosurgical groups (40.5 vs. 20.1 and 4 respectively). The association between certain journals and higher citations rates was documented in the literature relating to spine [4], plastic surgery [7] and transplantology [23].
In this review, no significant link was found between citation rates of GBM clinical trials and the numbers of authors, centres, countries and references. A similar finding was reported by others [2, 20]. However, in the literature, several publications identified the number of authors as a significant predictor of citation counts [1, 7, 15]. Significant relationships were also reported between the international and national collaboration of authors, number of organizations and number of countries producing the paper and the frequency of citations [1, 2]. A positive link with the number of references was stated by some authors [20]. Furthermore, some authors suggested that a proportion of variance in the number of citation an article receives can be explained by seemingly superficial factors that have nothing to do with the content of the article such as the title, the number of authors, the number of references, the number of sentences in the abstract, the presence of a colon in the title and the number of pages [24, 25].
The countries where the GBM clinical trials originated were not examined in this study.
It has been reported that country of origin can be a positive predictor of citation counts in research relating to spine [4], radiology [5] and urology [9]. Furthermore, a recent publication [3] investigated the influence of 66 factors on citations using samples of articles from 18 leading Chinese library and information science journals. They found 46 factors were significantly associated with citations. They also observed the most significant factors to be the number of downloads, the number of citations in the first 5 years, the author being an independent researcher and the percentage of monographs in the references. Several other potential predictors were not addressed in this study that were examined by other studies. These include increasing visibility through open access [5], selection for press release [26], funding [14, 20], disclosure of conflict of interest [7], statistically significant results [20] and the trial being referenced in ClinicalTrials.gov [27].
There are several limitations to the study. The study was reliant on the precision of the online search engines PubMed and Google Scholar. The selection of the 100 trials was based on their total cites at a certain point which was likely to change relatively quickly. This could have influenced the inclusion or exclusion of few of the lower impact trials. The wide duration from publication may have affected citations of the older trials. There may have been potential errors in the subgrouping of the treatment protocols. Also, variation in authors affiliation may have affected the number of centres. Collaborators were not counted in the number of authors, centres and countries. In addition, the impact of self-citation on the citation numbers was not examined.