Predictors of Citation Rates in High Impact Glioblastoma Trials

Clinical trials are at the top of research study designs and tend to attract high citation numbers. Glioblastoma multiforme (GBM) is a multidisciplinary disease that continues to be the subject of peak research interest. The literature relating to predictors of citation rates in clinical trials in general remains limited. We aim to identify the factors that inuence citation numbers in high impact GBM trials. Methods The 100 most cited published GBM trials were identied and reviewed. The primary analysis was correlating articles citation numbers with various trial and publication-related predictors using Pearson correlation coecient. The secondary analysis was comparing the mean citation numbers for the different subgroups using mean difference test. phase included


Introduction
Citation-based metrics are used for calculating journal impact factor (IF) and for evaluating academic productivity of researchers. The number of citations an article receives, also referred to as the citation rate, is arguably the most important measure of a study's impact and clinical weight [1]. An analysis of the various article, journal and author-related factors that may affect citation rates was reported in two publications [2,3]. The matter was also examined by other studies that focused on the identifying predictors of citations in published research relating to several specialties that included spine [4], neurosurgery [1], radiology [5], psychology [6], plastic Surgery [7], cardiovascular [8], urology [9] and orthopedic surgery [10].
Randomized controlled trials (RCTs) are recognized as the pinnacle of clinical study designs and evidence-based medicine [11]. They are frequently published in high impact journals and receive considerable visibility [11]. They are also likely to in uence opinions of clinicians, patients and policymakers [11]. The association between study designs (RCTs and meta-analyses in particular) and high citation numbers is well documented in the literature [2,[8][9][10]. However, clinical trials are not always randomized, and they vary in their characteristics, completion and publication rates [12]. Furthermore, studies that analysed citation patterns of clinical trials remain limited in the literature [11,[13][14][15].
Glioblastoma multiforme (GBM) is a malignant primary central nervous tumour that represent an enigma to clinicians because of its aggressive and heterogeneous nature [16]. It is primarily a topic of oncology but includes the disciplines of neurosurgery, neurology, radiotherapy, basic science, and general medicine [16]. A recent bibliometric evaluation of high impact GBM research did not address citation rates [16]. The objective of this study is to to determine the identify the different trial and publication-related predictors of citations in high impact GBM clinical trials.

Methods
PubMed and Google Scholar databases were searched in March 2021 for all the GBM-related trials that were available in the literature. The inclusion criteria were highly cited clinical trials at any phase that were published anytime. We also searched the web sites of the following journals: New England Journal of Medicine, Lancet, Journal of American Medical Association (JAMA), Journal of Clinical Oncology, Neuro-oncology, and Journal of Neurosurgery. The main keywords for the search were "Glioblastoma," "GBM," "Glioblastoma Multiforme," "Grade IV Glioma", "Trials" and "Randomized Controlled Trials".
Articles were assessed for suitability using the abstract and the full text were reviewed in case of ambiguity. Using article citation numbers provided by Google Scholar, the 100 most cited GBM trials were identi ed.
To minimize bias, the two authors conducted independent searches and prepared separate lists of the most cited studies. The two lists were compared, and any discrepancies were resolved by consensus. In view of the regular changes in the citation numbers, the search ndings on a single day (1st April 2021) were documented and used for analysis. In addition, journal impact factors (IFs) were taken from the various journals' web sites and were for 2019 as these were the latest available at the time of the study.
The selected trials were analysed, and information relating to the characteristics of the trials and publications was collected. The data was used to generate descriptive statistics relating to the 100 high impact GBM clinical trials.
The primary analysis was correlating the total citations number for the various studies with several trial and publication-related predictors. These were: study population, randomization, number of arms, phase, GBM status, treatment modality used in any of the trial arms [chemotherapy (including temozolomide, nitrosourea, bevacizumab, others), radiotherapy (including electrotherapy and proton/neutron irradiation), surgery, local treatment (chemotherapy, immunotherapy, and hyperthermia) and immunotherapy], trial duration in months, duration from publication in years, publishing journal's IF and eld (oncological, general medical, neurosurgical), and the number of authors, centres, countries, and references listed on the publication. The correlation analysis was done by calculating the Pearson correlation coe cient (R) using Social Sciences Statistics [17] and signi cance was determined when p<0.05.

Results
The median (range) and mean (± SD) total citation numbers for the 100 most cited GBM trials were 340 (135-16284) and 825 (± 1828) respectively. An analysis of the trials is shown in Table S1 (Supplementary material). The median (range) and ndings relating to the various trial and publication parameters are summarised in Table 1 Of the selected 100 trials, the trial treatment protocols included chemotherapy using one or more agents in 73% (TZM: 31%, Nitrosourea: 17%, BVZ: 12%, erlotinib and ge tinib: 7%, irenotican: 5%, and others: 30%), radiotherapy in 38% (including tumour treatment elds: 2%, photodynamic therapy: 2%, accelerated proton/photon irradiation: 1% and neutron capture therapy: 1%), surgery and local treatment in 13% and immunotherapy in 9%.  between the two subgroups relating to randomization, number of trial arms, GBM status, treatment protocols that included chemotherapy in general, nitrosourea, bevacizumab (BVZ), surgery, local treatment, and immunotherapy, as well as trial duration, period from publication, and number of authors, centres, countries, and references.

Discussion
GBM had been the focus of substantial clinical and basic science research activities aimed at discovering a treatment modality that can signi cantly improve survival [13]. The recently reported analysis of the 100 most cited GBM publications contained 27 clinical studies, 19 of which were trials that were included in this study (articles 1-18, 20) (Table S1) [1]. The review of the 44 neurosurgical RCTs in high impact journals that was published lately did not contain any GBM trials [11]. None of these publications assessed citation patterns.
The mean citation numbers for the 100 most cited GBM trials was 825, which was higher than the mean citation of 198 for the 100 most cited meningioma articles [19]. However, it is slightly lower than the reported median citation number of 935 for the 100 most cited general GBM articles [16]. The nding is not surprising as the mentioned review covered a bigger pool of GBM studies that included 52 basic science articles [16]. The latter are recognised to be associated with high citation numbers [2,10,16].
Variation in citation rates according to study topic or subject is well recognized in the literature relating to neurosurgery [1], spine [4], plastic surgery [7], and urology [9]. It is generally accepted that disciplines differ in their citation practices and that certain topics or subject area may be cited more than others [2].
It is also agreed that the numbers of citations is in uenced by the size of literature in the eld [2].
In this analysis, a signi cant association with study population was observed in both primary and secondary analysis implying that study population was a rm predictor of citation rates in GBM clinical trials. Similar ndings relating to study population were reported by others [2,4,6,9,20]. A signi cant correlation with trial phase was also found in both primary and secondary analysis indicating that being a phase III trial was a solid predictor of citation rates. Citation rates, however, were not affected by randomization which is surprising as the correlation between RCT-type studies and bigger citation numbers is well reported in the literature [1,[8][9][10]. This nding could be unique to the GBM topic area or could be related to the relatively limited number of articles selected in this review. Citation rates were not affected by the trials' number of arms, status of GBM and duration of study. The lack of impact of certain features of study designs on citation numbers was also noted by others [2,20].
In this study, the primary analysis did not reveal a correlation between treatment protocols and citation rates. However, the secondary analysis demonstrated a signi cantly higher citation rates in trials in which the treatment protocol included radiotherapy and TZM. This probably re ects the current widely accepted standard treatment for newly diagnosed GBM which includes surgery followed by concurrent radiotherapy with TZM and further adjuvant TZM [21,22]. No signi cant association was observed between citation rates and treatment protocols that included chemotherapy in general. This probably relates to the wide-ranging chemotherapeutic agents used in the studies and their mixed e cacy. Also, the chemotherapy in general group included older studies that were conducted before the use of standard TZM in the rst line setting. The lack of association between citation numbers and treatment protocols including BVZ, nitrosourea, surgery and local treatment and immunotherapy may be in uenced by the limited number of trials that focused on the treatment modality. However, it could re ect their undetermined role in the future direction in the management of GBM [21,22]. Citation rates were also not affected by the duration from the time of publication (age of the study). This is not unusual as the study covered a long period (43 years). It is recognized that the number of citations increase in the rst year after publication to reach a peak and then they are less cited as time passes [2]. The latter could be because the paper's information becomes outdated with time [2].
In this article, signi cant association with journal's IF was observed in both primary and secondary analysis denoting that journal's IF was a strong predictor of citation rates in GBM clinical trials. Similar ndings relating to the journal's IF were reported by others [1,2,15]. Furthermore, the secondary analysis demonstrated a signi cantly higher citation rates in trials that were published in general medical journals. This was expected as the group of general medical journals in this study had a much higher mean IF than the oncological and neurosurgical groups (40.5 vs. 20.1 and 4 respectively). The association between certain journals and higher citations rates was documented in the literature relating to spine [4], plastic surgery [7] and transplantology [23].
In this review, no signi cant link was found between citation rates of GBM clinical trials and the numbers of authors, centres, countries and references. A similar nding was reported by others [2,20]. However, in the literature, several publications identi ed the number of authors as a signi cant predictor of citation counts [1,7,15]. Signi cant relationships were also reported between the international and national collaboration of authors, number of organizations and number of countries producing the paper and the frequency of citations [1,2]. A positive link with the number of references was stated by some authors [20]. Furthermore, some authors suggested that a proportion of variance in the number of citation an article receives can be explained by seemingly super cial factors that have nothing to do with the content of the article such as the title, the number of authors, the number of references, the number of sentences in the abstract, the presence of a colon in the title and the number of pages [24,25].
The countries where the GBM clinical trials originated were not examined in this study.
It has been reported that country of origin can be a positive predictor of citation counts in research relating to spine [4], radiology [5] and urology [9]. Furthermore, a recent publication [3] investigated the in uence of 66 factors on citations using samples of articles from 18 leading Chinese library and information science journals. They found 46 factors were signi cantly associated with citations. They also observed the most signi cant factors to be the number of downloads, the number of citations in the rst 5 years, the author being an independent researcher and the percentage of monographs in the references. Several other potential predictors were not addressed in this study that were examined by other studies. These include increasing visibility through open access [5], selection for press release [26], funding [14,20], disclosure of con ict of interest [7], statistically signi cant results [20] and the trial being referenced in ClinicalTrials.gov [27].
There are several limitations to the study. The study was reliant on the precision of the online search engines PubMed and Google Scholar. The selection of the 100 trials was based on their total cites at a certain point which was likely to change relatively quickly. This could have in uenced the inclusion or exclusion of few of the lower impact trials. The wide duration from publication may have affected citations of the older trials. There may have been potential errors in the subgrouping of the treatment protocols. Also, variation in authors a liation may have affected the number of centres. Collaborators were not counted in the number of authors, centres and countries. In addition, the impact of self-citation on the citation numbers was not examined.

Conclusions
Clinical trials are the pinnacle of research study designs and tend to attract high citation numbers. GBM is a multidisciplinary disease that continues to be a subject of peak research interest. Increasing awareness of the factors that could affect citations may be useful to researchers. The literature relating to predictors of citation rates in clinical trials remains limited. The most consistent predictors of citation rates in GBM clinical trials were study population, trial phase, and journal IF. Treatment protocol was a positive predictor when it included the currently widely accepted treatment modalities (RT and TZM).