Use of systematic reviews to justify the conduct of urology clinical trials

Background Given the increased amount of research being funded in the eld of urology, reducing the amount of research waste is vital. Systematic reviews are an essential tool in aiding in reducing waste in research; they are a comprehensive summary of the current data on a clinical question. The aim of this study is to evaluate the use of systematic reviews as justication in conducting randomized clinical trials (RCT) in high impact urology journals. Methods On December 13, 2019, one of us (BJ) conducted a PubMed search for randomized controlled trials published in the top four urology journals according to their Google Scholar h5-index. Using a masked data extraction process each RCT was searched for systematic reviews. Then each review was evaluated for if it was justication for conducting the trial based on the context the systematic review was used. Results Of the 566 articles retrieved 281 were included. Overall 60.5% (170/281) trials cited a systematic review. We found only 47.6% (134/281) studies cited a systematic review as “verbatim” justication for conducting the trial. Regression analysis yielded a nding of statistical signicance in showing a correlation of studies over medical devices were more likely to cite a systematic review than other study topics ( adjusted odds ratio 2.01, 95% CI, 1.08 - 3.73) A total of 409 different systematic review citations were recorded in the 281 trials. Conclusion trials cited systematic review trial. If trials their studies believe would greatly reduce the amount of waste within clinical


Abstract
Background Given the increased amount of research being funded in the eld of urology, reducing the amount of research waste is vital. Systematic reviews are an essential tool in aiding in reducing waste in research; they are a comprehensive summary of the current data on a clinical question. The aim of this study is to evaluate the use of systematic reviews as justi cation in conducting randomized clinical trials (RCT) in high impact urology journals.
Methods On December 13, 2019, one of us (BJ) conducted a PubMed search for randomized controlled trials published in the top four urology journals according to their Google Scholar h5-index. Using a masked data extraction process each RCT was searched for systematic reviews. Then each review was evaluated for if it was justi cation for conducting the trial based on the context the systematic review was used.
Results Of the 566 articles retrieved 281 were included. Overall 60.5% (170/281) trials cited a systematic review. We found only 47.6% (134/281) studies cited a systematic review as "verbatim" justi cation for conducting the trial. Regression analysis yielded a nding of statistical signi cance in showing a correlation of studies over medical devices were more likely to cite a systematic review than other study topics ( adjusted odds ratio 2.01, 95% CI, 1.08 -3.73) A total of 409 different systematic review citations were recorded in the 281 trials.
Conclusion Less than half of clinical trials cited a systematic review as justi cation for conducting the trial. If clinical trials were required to support their studies with systematic reviews we believe this would greatly reduce the amount of research waste within clinical research.
Background Systematic reviews are important study designs for clinical practice. These reviews collate all available evidence for clinical interventions, and, if results from individual studies can be pooled by meta-analysis, a summary effect estimate may be estimated. Further, systematic reviews are useful for understanding whether particular studies contain design features that are prone to bias or whether poor reporting exists among them (1,2). This information is critical to the vertical advancement of the eld of urology research.
Finally, systematic reviews -because they summarize all available evidence on a particular topic -are useful for elucidating areas of research that have yet to be undertaken (3). A number of Cochrane systematic reviews have concluded that, upon having completed comprehensive searches, no eligible studies were found(4). Thus, systematic reviews are quite valuable for determining whether a new clinical trial is needed. In this study, we performed a review of urology clinical trials to determine whether the trial authors reported having rst consulted available systematic reviews to establish the need for conducting their trial. We also evaluated the ways in which systematic reviews were incorporated into clinical trial reports. Finally, given the strong presence of the Cochrane Collaboration's Urology Review Group(5), we wanted to determine whether urology clinical trials referenced Cochrane reviews when providing their justi cation of need. Both SS and AW were blinded and as they separately screened and extracted data. After nishing each data screening and extraction phase, authors, SS and AW, hosted a resolution meeting for any con icts. If any disagreements were not able to be resolved, BJ and MV were to intervene and provide guidance.

Methods
After receiving the search string results, authors, SS and AW, screened to assure the studies were RCTs using the screening tool Rayyan (https://rayyan.qcri.org/). The included RCTs were then extracted using a pilot-tested Google Form. We recorded the number of systematic reviews or meta-analyses cited in a RCT's introduction, methods, and/or discussion, and whether the RCT used these citations to justify their trial. Justi cation of trial was recorded as "Yes -Verbatim". "Yes -Inferred", or "Not Used as Justi cation".
Rationale for each classi cation was recorded for the resolution meeting. We also extracted the following study characteristics for each RCT: the publication year, journal of publication, study funding source, the RCT's intervention, the trial type, and whether or not RCT's primary outcome was statistically signi cant.
All data extraction was recorded and analyzed through Google Sheets and it's mathematical functions. We performed a logistic regression between trial characteristics and if the study justi ed their trial using Stata 15.1 (StataCorp, LLC, College Station, TX). Both "Yes -Verbatim" and "Yes -Inferred" were considered justi ed. This study is not subject to institutional review board standards because according to IRB's regulatory de nition, our study is not considered human subject research.

Results
Our search retrieved 566 records; of which, 275 were excluded for not being clinical trials and an additional 10 were excluded because they were inaccessible, yielding 281 clinical trials for analysis (Fig. 1). The most common journals in our sample were Journal of Urology (82/ 281; 29.8%) and European Urology (72/ 281; 25.6%). Drug therapies were the most common intervention type (120/281; 42.7%). Additional trial characteristics can be found in Table 1. Our results found that 47.6% (134/ 281) of the clinical trials directly stated the use of a systematic review as the means to justify conducting the trial ( Table 2). Eighty trials provided indirect reference to the use of a systematic review for justifying the need for initiating the trial (Table 2). Some trials were given the verbatim or inferred justi cation in the introduction and then the opposite in the discussion and were therefore counted as one for each. Fortyone systematic reviews were cited but not in a way to suggest that they were used as the basis for conducting the trial (Table 2). Clinical trials involving procedures were more likely to cite systematic reviews ( adjusted odds ratio 2.01, 95% CI, 1.08-3.73) no other characteristics were statistically signi cant (Table 2.) A total of 111 trials had no systematic review cited anywhere in their manuscript (Table 2). Of the 170 trials that cited a systematic review, we found 409 citations that were systematic reviews and meta-analyses (Table 3).

Discussion
Our results suggest that urology clinical trials infrequently report the use of systematic reviews as a means to justify their trial. Researchers have noted similar results in other areas of medicine. For example, in orthopedic trauma(6) only 33% of clinical trials reported the use of systematic reviews as means to justify conducting the trial. A similar study in ophthalmology found an even lower percentage, (7) with only 22.4% of their included 152 trials reporting the use of systematic reviews to establish justi cation. Enklinging, et al. (8) found that, among anesthesiology trials, only 56% of trials referenced systematic reviews to justify need. Taken together, the existing evidence suggests that more work is needed to understand the full utility of systematic reviews. While trialists, researchers, and healthcare providers are aware of their usefulness for advancing patient care, additional promotion is needed to also understand that systematic reviews should be consulted prior to conducting a new trial. Thus, we believe greater education in this area is needed. Some solutions include: (1) having funding agencies require statements from trialists attesting that they had consulted the literature for available systematic reviews when writing the grant application; (2) incorporating guidance into the Consolidated Standards of Reporting Trials (CONSORT) Statement(9) -the reporting guideline for clinical trials which has been widely adopted within the trial community -and within its Explanation & Elaboration document; and (3) having journals adopt procedures for trialists to include in their written reports whether systematic reviews were consulted for trial justi cation.
Why is this matter important? Primarily, we suggest that research waste is a signi cant problem in the United States and across the world. The Lancet's special edition on research waste shed signi cant light on the problem(10). The National Institute for Health Research (NIHR), which is the largest national clinical research funder in Europe, has implemented their own process for reducing waste in funding. The NIHR are now requiring that funding be provided only to research that is related to a review of the existing evidence. (11). Furthermore, if there are no such reviews that exist for the presented research, the applicants must carry out an appropriate systematic review themselves. Finally, the applicants must search the clinical trial registry to create a list of other studies currently being conducted that may overlap with their project. We commend the NIHR in their thoroughness and diligence to reduce wasteful research practices and hope other clinical research funders follow their example. This study has both strengths and limitations. Regarding its strengths, we used a duplicate and blinded data extraction, which is recommended by the Cochrane Handbook for Systematic Reviews of Interventions(12). In addition, we searched PubMed with a comprehensive string to increase the generalizability of our study sample within the urology literature. Regarding its limitations, we conducted a cross-sectional investigation, restricted to particular journals and to a particular time period. Our results should be interpreted descriptively rather than inferentially. Furthermore, we acknowledge that systematic reviews are not available for every topic; therefore, clinical trialists may have consulted existing systematic reviews and, having found none, omitted reporting this information in the trial publication. Thus, our results may underreport the number of trials that reported the use of systematic reviews.

Conclusion
Less than half of clinical trials cited a systematic review as justi cation for conducting the trial. If clinical trials were required to support their studies with systematic reviews we believe this would greatly reduce the amount of research waste within clinical research. We recommend future studies to assess the justi cation of clinical trials with systematic reviews in other specialties. from: https://training.cochrane.org/handbook/current Figures Figure 1