Impact of Investigator Initiated Trials and Industry Sponsored Trials on Medical Practice (Impact): Results of a Cohort Study

Decisions in healthcare are ideally made on basis of the results of clinical trials, published in study registries, as primary journal articles or summarized in secondary research articles. It is still unclear, whether and to what extent public and commercial expenses invested in clinical trials pays out in a way that their ndings have an impact on publication output and medical practice. The aim to examine and secondary research by determining the that We set-up a cohort of randomized controlled trials (n=691). We created and compared four sub-cohorts of investigator initiated trials (IITs) and industry sponsor trials (ISTs) with and without German contribution. For each trial, we searched for corresponding publications and citing systematic reviews and clinical guidelines. Additionally, we multivariable


Abstract
Background Decisions in healthcare are ideally made on basis of the results of clinical trials, published in study registries, as primary journal articles or summarized in secondary research articles. It is still unclear, whether and to what extent public and commercial expenses invested in clinical trials pays out in a way that their ndings have an impact on publication output and medical practice.

Methods
The aim of this project was to examine the lifecycle of clinical trials from their registration to their publication and citation in secondary research articles by determining the proportion of trials that were published and were included in systematic review and clinical guidelines.
We set-up a cohort of randomized controlled trials (n=691). We created and compared four sub-cohorts of investigator initiated trials (IITs) and industry sponsor trials (ISTs) with and without German contribution. For each trial, we searched for corresponding publications and citing systematic reviews and clinical guidelines.
Additionally, we investigated what study characteristics are associated with publication and impact by using multivariable logistic regressions.

Results
Of the 691 trials, 576 (83%) were published as method article or result article in a medical journal and/or the trial results were made available in study registries; results were available for 555 (80%) of the trials. More than half (52%) of the trials were cited by a systematic review and about a quarter (26%) reached impact in a clinical guideline. Drug trials and larger trials are associated with a higher probability to be published and to have an impact than non-drug trials and smaller trials. Results of IITs were more often published as journal article, results of ISTs more often in study registries. International ISTs gain less often impact by inclusion in systematic reviews or guidelines than publicly sponsored trials.

Conclusion
A considerable proportion of clinical trials investigated was published and had an impact on clinical practice, whereas the proportions depend on speci c study characteristic. Study registries are an important alternative or complement to journal articles for publishing study results. There is still a need to improve the transfer of knowledge generated in clinical research into practice.

Background
Clinical studies are conducted by academic, private, governmental or commercial institutions to investigate safety and effectiveness of preventive, therapeutic or diagnostic procedures in health care. Publication of the ndings from clinical studies is an essential requirement to inform clinicians, patients and other decision makers about the e cacy and safety of treatments tested. Without publishing the study results, the ndings are lost and cannot gain impact on medical practice or further research activities. Additionally, the publication of the detailed study methods is a prerequisite for the assessment of the validity of the study results, for replication of the study and veri cation of the study results, and nally, for a reliable implementation of interventions into clinical practice.
It has long been known that only part of the clinical studies conducted ultimately reaches the stage of full publication in peerreviewed journals [1]. For example, more than half of the study results presented as abstract at scienti c meetings fail to be published as full-text article [2]. Thus, important study information cannot be considered for health care decisions and further research planning, which in turn could expose patients and future study participants to unnecessary risks [3]. Systematic reviews and meta-analyses can come to an erroneous overall effect estimate and conclusion when unpublished data cannot be considered [4]. If experiences and results obtained from trials are not disseminated, they are not only lost for health care, but also for further research. Moreover, personnel resources and scarce research funds are badly invested or even wasted. An important step for increasing transparency in research and the visibility of unpublished studies was the implementation of study registries and the call for prospective study registration by several research organizations [5][6][7]. In Germany, funding organizations such as the German Research Foundation (Deutsche Forschungsgemeinschaft, DFG) and the Federal Ministry of Education and Research (Bundesministerium für Bildung und Forschung, BMBF) require the registration of the trial in a public registry and publication of the trial protocol following grant approval [8,9]. Prospective study registration is a major step forward, but it is equally important to make the results of a trial publicly available, which is possible through the study registries. But even several years after these urgent calls for a prospective study registration, there are still trials that are not included in a study registry [10], i.e. unpublished studies and their results are di cult to be identi ed.
In recent years several authorities and research organizations became aware of the problems arising from withholding study results. The World Health Organization, the World Medical Association and the All Trials initiative (http://www.alltrials.net/), have alerted that it is unethical to conduct human research without subsequently publishing the results and took various steps to prevent non-, incomplete or biased reporting of research results [11].
As an order of magnitude, in 2018 the German Research Foundation (DFG) alone spent 22 Million euros for the conduction of 47 trials within their clinical trials program [12]. A major step forward would be to identify the risk and risk factors for nonpublication or for having no impact on medical practice.
The prospective and longitudinal approach of the present project, following-up the lifecycle of a de ned set of trials from their registration up to their use in medical practice, allowed us to compare trials that were successful with those that were not, regarding the transfer of their results into practice and also to identify trial characteristics potentially associated with impact. An increased awareness of the risk factors could improve the feasibility and e ciency of future trials and the validity of trial results. Trials at high risk of having no impact could be adjusted beforehand to ensure a successful trial progress. Systematic reviews on retrospective projects have shown that several study factors are associated with publication of results, e.g. direction of study ndings, study size and duration [13,14]. Suspected to have an in uence on the lifecycle of a trial, i.e. going beyond publication, our impact measures are the study characteristics number of participants (sample size), phase of clinical research (study phase), number of prede ned primary outcome(s), medical eld, type of funding or sponsorship, and place of conduct (country of study sites).
To our knowledge, it is still unclear under what conditions expenses invested to support clinical trials pay off in a way that the ndings have an impact on healthcare decisions. Therefore, in this project, we aimed to investigate the publication output and research impact on medical practice of investigator initiated trials (IITs) initiated in Germany and funded by the independent research funding organizations DFG, BMBF and others. We aimed to nd out whether there is a difference in research impact between academic-driven IITs and trials initiated by commercial organizations (industry sponsored trials, ISTs) in Germany and whether there is a difference in publication and impact between trials initiated or conducted in Germany or internationally.

Objectives
The aim of this project was to examine the transfer process of clinical trial results into practice by determining the trial publication characteristics and by determining the use of trial ndings in secondary research articles: 1. Proportion of studies that is (not) published 2. Type of study information published (methods and/or results) 3. How study information is published (as journal article, register entry) 4. Proportion of studies included in secondary research articles (reviews) 5. Proportion of studies included in clinical guidelines 6. Association of pre-de ned study characteristics with publication or use by secondary research articles Methods Rationale and design of this cohort study is described in detail in a previous publication [15].

Study cohort
In brief, we set up a cohort of trials consisting of 691 therapeutic, multicenter, randomized controlled trials that started in 2005 or later and were completed by the end of 2016. To nd out whether sponsorship/funding or place of conduct in uence a trial's impact, we created and compared sub-cohorts of investigator initiated trials (IITs) and industry sponsor trials (ISTs) with and without German contribution (Table 1). For the IIT-sub-cohort "Public Germany" we included trials funded by the DFG and BMBF (Public Germany gov), which we retrieved from the funder's databases "German Project Information System" (GEPRIS) of the DFG and the website of the BMBF [16,17]. These IITs served as a reference sub-cohort relating to the study characteristics for the creation of the comparison sub-cohorts. We complemented this sub-cohort by an equal number of IITs funded by other German non-commercial organizations (Public Germany other), which we randomly drew from the trials registries ClinicalTrials.gov and German Clinical Trials Register (DRKS) ( Table 1). We balanced the three sub-cohorts Public International, Commercial Germany and Commercial International on the basis of the proportion of the speci c study phase for both drug trials and non-drug trials according to the reference sub-cohort. For further details please refer to the Methods paper [15]. To minimize possibly biasing study characteristics, we aimed to generate comparable sub-cohorts by balancing for effects of study phase and for the location of participating study sites (proportion of German study sites) according to the reference subcohort Public Germany. The largest trial of the reference sub-cohort included 4005 participants so that we only considered trials up to this sample size.
We independently double-extracted the pre-de ned study characteristics sample size, study phase, number of pre-de ned primary outcomes, and medical elds [18] from the study registries, as we were interested in whether they were associated with research impact [15].

Identi cation of corresponding publications
For each trial, we identi ed related publications and classi ed them according to the trial information published: method article, solely describing the study methods in detail, or result article, also reporting the study results. This classi cation allowed us to determine what kind of study information was used in secondary research articles and clinical guidelines.
Search strategy: sources where journal articles were identi ed We searched for publications in different biomedical databases and other sources using an incremental search strategy (Additional le 1). As search terms, we combined various study information such as the registry identi cation number, study title, acronym, PICO-aspects, and/or name of applicant or principal investigator. Searches for primary study reports were conducted between 6 February 2018 and 30 August 2018. We downloaded the references of all identi ed published articles into an Endnote database. We considered full articles reporting a trial's methods and/or results. We also downloaded all the study protocols we came across during our literature search.

Trial information in study registries
In addition to publication as a journal article, we determined whether or not study information was available in the study registries DRKS and ClinicalTrials.gov. Beside study registration, publishing study results in trial registries is required by several organizations since several years [19,20]. In order to meet those requirements and to implement the amended public law concerning result reporting of the U.S. Government [21], ClinicalTrial.gov launched its results database in 2008. Each country has its own requirements and national study registries have their individual options and possibilities. In some study registries (e.g. ClinicalTrials.gov), results can be entered directly into the trial record as a separate register tab, hereafter referred to as "results in registries". In other study registries (e.g. DRKS Register), study related documents can be attached to the trial record. Apart from information directly integrating in the study registry, references of a published journal article deriving from the study can be added manually to the register record or are automatically searched and attached by the study registries themselves [22]. In the DRKS, it is possible to link publication references and attached documents to the register record but there is no possibility to directly report the study results. Beside their registration in ClincialTrials.gov and DRKS, 189 (27%) of the trials were additionally registered in EudraCT and 35 (5%) in the ISRCTN registry. We considered study results of these trials as "results in registries" when they were available in these other/secondary study registries [15].

De nition
Hereinafter, we use the following de nitions: for publications in journal articles we use the expression "published articles". We distinguish between articles solely concerning a trial's methods, called "method articles", and articles also reporting study results ("result articles"). Beside publication as journal article, results can be published in study registries; in this case, we use the expression "results in registries". For published trial results, i.e. as result article or as results in registries, we use the general term "published results".

Identi cation of secondary research articles citing primary published article
To assess the research impact of the included trials, we investigated whether or not published articles were cited by secondary research articles, i.e. systematic reviews/meta-analyses and clinical guidelines.

Systematic reviews
For each published article, we downloaded all references listed under the functions "Cited by" in PubMed and "Times Cited" in Web of Science. To identify the systematic reviews and meta-analyses among the citing articles, we matched their Digital Object Identi er (DOI) with the record-DOIs included in the database Epistemonikos, which can be considered as the "largest source of systematic reviews relevant for health-decision making" [23]. Epistemonikos includes references of four categories: broad syntheses, systematic reviews, structured summaries and primary studies. In our project, we focused on references classi ed as systematic reviews or broad syntheses. Both categories are hereinafter referred to as "systematic reviews" (SRs).
If a DOI of a citing article was found in Epistemonikos, the publication type was veri ed and the citing article labelled as systematic review. We then manually assessed the context in which published articles were cited by the systematic reviews/meta-analysis: General information or methods of the trial were reported, e.g. in the introduction or discussion section, Study results were included in the systematic review/meta-analyses or Study results were excluded.

Clinical Guidelines
The ultimate step for a successful implementation of a trial's results in medical practice is their inclusion in clinical practice guidelines. To identify these, we manually searched in the clinical guidelines databases TRIP [24], and NICE evidence search [25] and AWMF (Association of the Scienti c Medical Societies) [26]. We searched for clinical guidelines citing the trial publications. As search terms we used (parts of) the title and name of the rst author of the published articles and the corresponding systematic review/meta-analysis; to identify guidelines citing results published in registries, we searched with the register identi cation number. The search period for guidelines citing the published articles was between December 2018 and March 2019, for guidelines including systematic reviews between April and August 2019, and for the registry identi er in February 2020. For each identi ed clinical guideline, we retrieved the full text and veri ed the citations.

Data collection
We extracted the following information about the publications into an Access database: 1) whether or not study results were reported in study registries, 2) bibliographic information of included publications and content (method article or result article), 3) bibliographic information of citing systematic reviews/meta-analyses, and 4) bibliographic information of citing guidelines.

Semi-automatic tool
Within this project one author (KN) developed a semi-automatic tool (called DoiScout) that facilitates large-scale literature searches and citation analyses in order to carry out extensive literature searches based on internet search engines more timee ciently.
DoiScout automatically identi es primary published articles that reference a particular study registry ID (e.g. NCT02179424). Bibliographic information about the identi ed articles is extracted and presented in a list that is formatted in a way that allows passing on the information to other software programs for further processing.
A second feature refers to citation analysis. Search engines behind platforms such as PubMed (www.pubmed.gov) and Web of Science (www.webofknowledge.com) can be used to identify other articles, e.g. primary research articles, systematic reviews and clinical guidelines that cite a given article. DoiScout extracts the bibliographic information of the citing articles and provides it to the user in a workable format. In addition, DoiScout can be used to identify articles citing the citing articles of the original source. This can be done for any pre-speci ed citation depth, thus providing a comprehensive overview of the extent of a project's academic impact.
The program of the DOIScout and a manual describing the features in more detail are available via the GitHub platform [27].

Data analysis
We used queries in MS Access 2010TM and tabulation in Microsoft Excel 2010 to obtain standard descriptive statistics. Multivariable logistic regression was used to determine the association of study characteristics with the probability of a trial to be published, cited by systematic reviews and included in guidelines. Based on the reference sub-cohort Public Germany, it was carried out for the other sub-cohorts, for study phase, number of participants, and number of primary outcomes. For time to publication, multivariable Cox regression was used to account for study characteristics. For distinguishing between rst publication in a journal or in a registry, a competing risk model, Aalen-Johansen estimates of the cumulative incidence functions, are presented [28].

Publication
For published trials, method articles, result articles and results in registers can be present or not. There are two possibilities to publish the results of a trial, as a result journal article or as results in registries, while methods are always published as a journal article. We rst describe the proportion of publication types for the total cohort and then for the different sub-cohorts. If not mentioned otherwise, all percentages of trials given for the entire cohort are calculated on the basis of the included 691 trials. Percentages given for the sub-cohorts are based on the number of trials in each sub-cohort. Minor differences in summed percentages derive from rounding to full integer.

Proportion of published trials
For our whole cohort, 576 (83%) of the 691 trials included were published as a method article or a result article in a medical journal and/or the trial results were made available in study registries; results were available for 555 (80%) of the trials ( Figure  1). For 107 (19%) trials, results were solely published in a registry.  (20) 16 (27) 28 (23) 26 (13) 24 (14) 37 (19)

Sub-Cohorts
For the sub-cohorts, the proportion of trials published varied between 77% and 87% ( Figure 2). Compared to the sub-cohort Public Germany (77%), the probability of a trial to be published is higher for the sub-cohorts Public International (87%), Commercial Germany (86%) and Commercial International (82%) ( Table 2). The publication of results ranged between 63% for Public Germany (58% for Public Germany gov and 67% Public Germany other) and 86% for Commercial Germany.
Obvious differences exist between the sub-cohorts regarding type of publication. IITs were more often published as journal article than ISTs (Table 2). Especially method articles are more present for IITs (Public Germany 38%, Public International 21%) than for the IST-sub-cohorts (Commercial Germany: 6%; Commercial International: 2%). Compared to the German sub-cohorts, results were more often published as journal article for Public International trials and less often for Commercial International trials. Looking at the number of publications per trials, multiple publications were more common in IITs (Germany: 33%, International: 32%) compared to ISTs (Germany: 17%, International 14%).

Trial information available in study registries
Study registries are basic sources containing information on the design and methodology of a trial. The assigned unique trial identi er facilitates the matching of published articles to the trial. Study registries use this identi er to automatically link corresponding published articles to the registry record. Furthermore, information can be added manually. Therefore, references of publications in journals, e. g. method and result articles, are often directly available in the registries and can easily be found. This additional information is extremely helpful to get a broader picture of the trial.

Cohort
For 293 (42%) of the 691 included trials, at least one reference to a corresponding journal article was reported in the study registry and/or a link to the original publication source or a database was provided. This means that 62% (293 of 472) of all published journal articles could be found in study registries (Table 2).
Information on results was available for 449 (65%) trials. For 305 (44%) trials, results were directly included in a study registry and for 276 (40%), a reference to a result article was reported. For 132 (19%) trials, both sources were present and for 144 (21%) solely a reference of a result article.

Sub-cohorts
The proportion of trials with a reference or link to the journal article was with 58% and 52% higher in the Public sub-cohorts than in the Commercial sub-cohorts with 33% and 32% ( Table 2). Results in registries ranged between 4% and 67%. The proportion of IST with results in registries was higher than for IITs.
For Public Germany, only 4% of the trials had results in registries. This small percentage can be explained by the fact that most of those trials derived from the DRKS register (summarized data, see Table 2 for more details), where results cannot directly be entered.
For the three other sub-cohorts, between 29% and 39% of the trials have results published in both registries and as journal articles and for 20% and 32% of the commercial sub-cohorts, results were solely available in registries.

Study characteristics associated with publication of results
The multivariable analysis con rmed our ndings regarding publication probability for the sub-cohorts. It also showed that additional study characteristics are associated with the probability to be published: drug trials are published more often than non-drug trials, larger trials more often than smaller trials and trials with more than one primary outcome more often than trials with one primary outcome (Additional le 4).
Each trial was allocated to one of 23 pre-de ned medical elds (Additional le 5). In our cohort, the median number of trials per medical eld was 25 and ranged between 2 (anaesthesiology) and 104 (surgery), the proportion of trials published ranged between 87% and 25%. Statistically signi cant differences were only found for medical elds with a su cient number of trials (≥ 39): higher publication rates were found for neurology (87%) and psychiatry/psychotherapy (84%), lower for surgery (64%) and ophthalmology (25%). Due to the limited number of trials per medical eld, an analysis for signi cant differences was not appropriate for the sub-cohorts. Further details on publication and impact are presented in the chapter "Overall impact".

Time to publication
The median time to any publication as a journal article or in a study registry, including method papers, was 4.07 years (95% CI: 3.79-4.33). If only counting result papers, the median time was longer (4.67 years, 95% CI: 4.36-5.03). The median time for any type of article (including method papers) to be published in a journal was 5.19 years (95% CI: 4.83-5.82); if only result articles were counted, the median was 6.09 years (95% CI: 5.66-6.62).
We analyzed the time to rst publication of study results in a journal or in a registry also in the framework of a competing risk model. This is visualized as Aalen-Johansen estimators in a stacked probability plot (Figure 3). The result shows that for the majority of studies (about 52%) the rst publication was found in a journal, while about 28% of studies were rst published in a study registry. Compared to Public Germany trials, results were published earlier for trials of the other sub-cohorts (Table 3, Figure 4). Furthermore, drug trials were published earlier than non-drug trials and larger trials earlier than smaller. In our cohort, we did not nd an association of time to publication with the number of primary outcomes (1 or more than 1). Multivariable analysis with estimated covariate effects for sub-cohorts, study phase, study size, number of primary outcomes. Hazard ratio with 95% con dence intervals. The intercept stands for the combination of IIT Public Germany, drug trial, n ≤ 150 and one primary outcome.
Impact: Proportion of trials included in systematic reviews and guidelines Systematic reviews The inclusion of a trial's results in a systematic review is one measure for the impact of a trial. It is based on the citation of the journal articles by systematic reviews. We identi ed 3429 references in 2631 systematic reviews, citing one or more of the 599 (63%) of the 947 identi ed journal articles.

Cohort
The 599 articles cited by systematic reviews corresponded to 360 of 691 (52%) trials (Table 4, Additional le 6). Of those, 27% were cited by only one systematic review, 73% by more than one. It is notable that 15% of the published articles were cited by 10 or more systematic reviews (Additional le 7). The median number of citing systematic review(s) per trial was 4 (range 1 to 99; mean=4.1).
Similar proportions were found for the subgroup of result articles (529 of 843; 63%) and the corresponding trials (335; 48%). Of the 104 method articles, 70 (67%) method articles corresponding to 70 (10%) trials were cited by a systematic review.
We not only examined whether retrieved publications were cited in systematic reviews but also how they were used (excluded, included or used otherwise). As publications included in secondary research articles are more likely to in uence clinical practice than excluded publications, this analysis is important for the assessment of the impact of trials. Of the citations in systematic reviews, 69% (2374 from 3429) were included and correspond to 45% (309 of 691) trials (Table 4), 6% (190 of 3429) were excluded and 25% (865 of 3429) were used otherwise. Nevertheless, 69 of the 87 trials with excluded publications in reviews had included publications in other reviews. For the remaining 18 trials, only exclusions were found. Frequently stated reasons for the exclusion of publications were failure to meet the eligibility criteria and not reporting the data of interest.

Sub-cohorts
For the public sub-cohorts and for Commercial Germany, citation by systematic reviews ranged between 52% and 63% and was higher than in Commercial International with 38% ( Figure 5 and Table 4). This difference might be explained by the lower proportion of trials published as journal articles in ISTs (compare Figure 2). Furthermore, a relevant proportion of articles cited by systematic reviews were method articles, which were rare in ISTs but mainly present in IITs. Study characteristics associated with citation by systematic review The multivariable analysis con rmed the signi cantly lower representation of Commercial International trials in systematic reviews compared to the other sub-cohorts. Study phase and number of primary outcomes are not associated with the inclusion probability, whereas larger trials are signi cantly more often included in reviews than smaller trials (Additional le 8).

Cohort
We found 574 citations of 178 trials (26%) in guidelines ( Figure 6). As a guideline can include information from several of our trials, these corresponded to 427 unique guidelines. On average, each of our trials was cited 3.2 times (574/178) in guidelines. This "guideline inclusion factor" ranged between 2.9 and 3.7 for the sub-cohorts.
One trial can be included in one guideline via several pathways, via a published article or via a systematic review. The following analysis shows via what publication type trials were included in guidelines: 69% (122 of 178) of the trials were included in 285 of 427 (67%) different guidelines via the citation of 382 systematic reviews. 58% (104 of 178) of the trials were included directly in 226 of 427 (53%) different guidelines via the citation of 262 result articles. In total, 93% (166 of 178) of the inclusions in guidelines come from result articles via directly or indirect pathway. 6% (10 of 178) of the trials were included in 12 of 427 (3%) different guidelines via citation of 12 method articles. 4% (7 of 178) of the trials were included in 6 (2%) different guidelines via citation of 7 registry information ( Table 5).

Sub-cohorts
In Figure 7 / Table 5 it is shown that for the sub-cohorts the inclusion of trials in guidelines ranged between 17% and 31%. For the subgroup Public Germany gov, even 45% (27 of 60) of the trials were cited in guidelines. Compared to Public Germany trials, the proportion of trials included in a guideline is about similar for Public International and Commercial Germany trials, whereas commercial International trials are less often included in guidelines.

Study characteristics associated with inclusion in a guideline
Similar to inclusion in reviews, the multivariable analysis con rmed a signi cantly lower representation of Commercial International trials in guidelines compared to the other sub-cohorts and demonstrated that study phase and number of primary outcomes are not associated with the inclusion in guidelines. Larger trials are about twice often included in guidelines than smaller trials (Additional le 9).
Overall Impact Lifecycle of trials Figure 8 shows the fate of the trials included in our cohort from registration to publication and to their impact on clinical practice. During their lifecycle from registration to impact in clinical practice, the number of relevant trials decreases with each step. 17% of the trials have no published results. Of the 576 (83%) published trials, 15% (107 of 691) have their results only published in registries and therefore might have a limited awareness in the scienti c community and a limited impact.
Trials published as journal article(s) (472; 68%) have a good chance to be cited in reviews or guidelines. Nevertheless, in our cohort, a relevant percentage did not nd an inclusion in clinical practice: only 309 (45%) of the trials were included in systematic reviews and 178 (26%) in guidelines.

Cohort
Of all trials, 274 (40%) generated no impact: 115 (17%) of the trials were not published and of the published trials, 160 (23%) were not cited by either a systematic review or a guideline (Figure 9).
Used by secondary research articles were 417 (60%) trials: 361 (52%) were cited by a systematic review, 178 (26%) by a guideline. Of those, 123 (18%) were cited by both, a systematic review and a guideline. This means that more than half (52%) of the trials were cited in a systematic review and that about a quarter (26%) reached impact in a clinical guideline.

Sub-cohort
Commercially funded trials, especially Commercial International trials, less often gain impact by inclusion in systematic reviews (52%; 39% for SRs, and 29%; 31% for guidelines) than publicly sponsored trials or guidelines (59%; 63% for SRs, and 29% and 17% for guidelines). The distribution of the three "impact-proportions" concerning inclusion in reviews and/or guidelines showed only minor differences between the sub-cohorts ( Figure 10).

Medical Fields
For our cohort, we found clear differences regarding publication and impact for the main medical elds (number of trials ≥ 39). The high proportion of guidelines in psychiatry and psychotherapy, cardiovascular disease and neurology is related to a high proportion of publications and systematic reviews for these elds ( Figure 11). When publication is low, this results in less reviews and guidelines (ophthalmology, surgery).

Discussion
In the current project we assessed the research impact on clinical practice of publicly sponsored trials and commercially sponsored trials conducted in Germany compared to those conducted internationally. By using a prospective strategy that followed the lifecycle of trials from their registration up to their inclusion in systematic reviews and clinical guidelines, we have collected and analyzed data not only for those trials that were 'successful', but also for trials that were not published and did not gain impact on clinical practice.

Interpretation of nding
We compared IITs and IST because they often focus on different clinical questions and pursue different aims and objectives. IITs play a crucial role in academic clinical research whereas ISTs usually focus on commercial interests, mainly of pharmaceutical companies, whose primary aim is to develop and approve drugs or other medical treatments [29]. In IITs, an academic investigator is responsible for the conduct of the clinical trial, which includes planning, registration and publishing the results of the study [30]. IITs are often conducted to expand product knowledge, including safety, and to identify new ways of using existing treatments, which might lead to the improvement of patient health [31]. IITs complement ISTs regarding the medical eld, such as physio-and psychotherapy, behavioral changes and complementary medicine.
Compared to literature, in our project a high proportion of trials (83%) were published. Systematic reviews and retrospective research projects investigating the publication proportion of RCTs resulted in considerably lower proportions of 60% to 71%. [1,32,33] Only 37% of RCTs presented as conference abstract were published in full as journal article [2]. The relatively high proportion of published trials and trial results in our cohort can be attributed to the fact that we also considered a trial as published, when its results were reported in a study registry.
Even though there are several advantages for posting results in registries, e.g. results can be presented fast and concisely, they are directly attached to the registry record, providing information about the trial methods as well as references and links to further trial information, the publication proportion is still relatively small. In a cross sectional study across academic medical centers in the United States, the publication proportion of completed trials that were registered in ClinicalTrials.gov was analyzed. Across the medical centers, 10.8% to 40.3% of the trials were published within 24 months of study completion, and for 1.6% to 40.7%, results were reported on ClinicalTrials.gov [34].
In our cohort, on average, for 45% (range of sub-cohorts: 36%-68%) of the trials we found results in study registries. This nding is in line with the results of a recent study, investigating the compliance with the Food and Drug Administration Amendments Act of 2007 (FDAAA) concerning reporting of trial results. The researchers found, that only about 40% of all applicable trials due to report results under the FDAAA reported their results in ClinicalTrials.gov within the one year deadline after study completion [35].
The possibility to add study results to a study in registrie, is certainly an important step to improve transparency in clinical research, but limited, incomplete or expired trial information in registries often make it di cult to get a complete picture of the trial and to appraise and interpret the results. Several initiatives such as AllTrials and TranspariMed work on the improvement of a trial's reporting by requiring clinical trials to be registered and to report their full methods and summary results [19,20].
In our cohort, of the trials with published results, 19% (107 of 556) were solely available in study registries. This has serious implications for the search process to identify relevant studies, i.e. which sources need to be searched, especially for systematic reviews and clinical guidelines. A search strategy should not only focus on journal articles, but should be accomplished by an additional search in study registries, which has already become mandatory for conducting Cochrane intervention reviews [36,37]. To improve the ndability of trial results, the registries themselves should improve their searchability. They should be constructed in a standardized format so that they are easily and reliably searchable, e.g. similar to biomedical databases by title, author, keywords and abstracts. Looking at our sub-cohorts, we found a signi cant difference between IITs and ISTs. For publicly sponsored trials, only 2%-4% were solely published in registries, whereas this was the case for 20% to 32% of the commercially sponsored trials. The reasons for this difference are unclear and future analyses would be worth to compare the characteristics and results of those trials published solely in registries with those published as journal articles, e.g. regarding publication bias. One explanation could be that for publicly funded trials publication of results in the form of a journal article is often demanded by the funding organization and is part of the funding conditions. Advantages of publishing results as a journal article ideally are a quality-assured peer-review, trial methods and results are considered and discussed in the context of the existing evidence and can be commented by other researches e. g. via response letters. For a great proportion of trials in our cohort, results were published as journal article (81%), and for more than half of the trials results were included in a registry.
Disclosure of detailed trial methods of a trial is essential with respect to the critical appraisal and interpretation of the results, and is the basis to enable other researchers to reproduce the trial and verify its results, which is a basic requirement for later implementation in medical practice. While in an original journal research article both methods and results of a trial are described, it is becoming more common to publish articles only describing the detailed methods of a trial and not the results. In our cohort, this was the case for 14% of the trials, for 3% only a method article could be identi ed. Moreover, it is important to point out that most of the method articles derive from publicly funded trials (87%), of which most of the German IITs were from Public Germany gov (67%). In scienti c research it is not unusual to publish results of one study in more than one article. One reason for this could be that in academia the reward system is often built on quantity of research output [38]. Scienti c success, such as reputation, career advancement, successfulness of applications for research funding, is directly associated with the publication output of a researcher. In our project, multiple publication was the case for 188 (40%) trials. They were more common for IITs (Germany: 33%, International: 32%) than for ISTs (Germany: 17%, International 14%). The trial with the highest number of identi ed publications (n=21) was a phase 4 study, conducted in the eld of cardiovascular disease, funded by the German Research Foundation. For this trial, one method article, two result articles and 18 sub-studies and secondary analysis were published. This trial and also the other high-frequently published trials (25 with more than 5 published articles) were conducted in academia. For this publication frequency, measured as the number of published articles per trial, we found a remarkable phenomenon: about one third of all published articles corresponded to only 8% of the trials. Even though this aforementioned reward system and its consequences have been in the focus of criticism for several years, structures have still not changed [39].
In contrast, in industry the central ( nancial) interest lies in the results, i.e. e cacy and safety of the tested treatment, whereas the study protocol and methods used are often con dential to protect commercial interests. This is also shown by the public availability of the study protocols: for 40 trials of our cohort, the original study protocol could be identi ed, 30 belonged to the IIT sub-cohorts and 10 to the IST sub-cohorts.
A Health Technology Report was conducted to evaluate the impact of Cochrane Reviews published by 20 Cochrane Review Groups, on health care, patient outcomes and value for money [40]. A random sample of 20 Cochrane Reviews and 40 selected reviews, more likely to have had an impact, provided by the Cochrane Review Groups were selected for further evaluation. "Overall, 40 of the 60 reviews had been cited in some form of clinical guidance and 15 had in uenced further primary research." We found that more than half of the trials are represented in systematic reviews and more than a quarter in guidelines. To further improve this knowledge transfer from research into practice, several issues have to be considered. The rst issue lies in why 17% of the trials have not been published. The second issue is how to improve the transfer of 28% of the trials that have been published but had no impact.
Typical reasons for not publishing trial results as presented in a systematic review are lack of time and/or resources, noncompletion of study, publication was not an aim, or only had low priority [41]. Possible explanations for published trial results not being included in systematic reviews are that no review related to the research question has been conducted or updated after the date of publication. Reasons reported for non-inclusion of published articles in the systematic reviews of our study cohort were that the eligibility criteria were not ful lled, e.g. wrong patient group, intervention, comparator, outcome measure, or study type.
For inclusion of trial results in guidelines, the same reasons as for systematic reviews could apply. However, in guidelines, in addition to publications, systematic reviews are also a relevant pathway for inclusion of trial results. A detailed investigation of the systematic reviews that have not been included in guidelines (56%) would be useful, e.g. to nd the reasons for lack of guidelines and to be able to further improve the transfer of important trial nding into medical practice.
Strengths and limitations of the study A strength of our study was that all trials were registered in study registries so that for all of them basic study information was available. Study characteristics were double-extracted independently in a pre-piloted extraction form following a written manual. All data extractors were trained. We captured all relevant information available in any study registry. Discrepancies in different sources were discussed and resolved. The identi cation of systematic reviews citing the original study report was conducted semi-automatically by using a self-developed program. The search for clinical guidelines was done manually following prede ned standardized rules.
Another strength was that we controlled for possibly biasing factors by design, i.e. by balancing important study characteristics to Public Germany as the reference sub-cohort.
A limitation arising from this was the limited number of studies in the sub-cohorts Public Germany and Commercial Germany. The number of trials meeting our inclusion criteria for the Public Germany gov (reference sub-cohort) was xed to 60. For the sub-cohort Commercial Germany, a balancing for non-drug trials was not fully possible: only 171 could be identi ed in DRKS and ClinicalTrials.gov registries instead of the pre-planned 200 studies per comparison sub-cohort. However, it is not expected that this difference of 29 trials have a relevant in uence on our results.
Our cohort was composed of trials that were included in study registries and, partially, also in databases maintained by funding organizations. Against the background that still not all trials are registered, our trial cohort might be a "positive" selection compared to those conducted worldwide. Therefore, there is a potential risk that our cohort is biased, resulting in a limited external validity of our project results.
Even though all studies were included in at least one study registry, for some studies information in registries was scarce and detailed study protocols were only rarely available. Therefore, for some trials it was di cult to nd out whether a published article corresponded to the trial. We also had to rely on the information reported in registries. Data of prospectively registered studies can include preliminary study information, for example information about study start and completion date. This may have in uenced our ndings.
We tried to assess actual data and included trials that started in 2005 or later and were completed by the end of 2016, for which we searched for corresponding publications in 2018/2019. For trials completed late during this time period, there might not have been enough time for publication and inclusion in systematic reviews and guidelines. Our results, however, indicate that this only concerns a few trials because since 1) compared to literature, the publication rate of our cohort was relatively high, and 2) the stacked probability plot (Figure 3) also indicates that only few rst publications are to be expected. Nevertheless, in such projects there will always be a compromise between presenting actual data with respect to the timeframe of included studies and leaving enough time for studies to be published and have an impact.
An important result of our study was that for 15% of the trials, results were solely available in study registries and were not published as journal article. In such cases, we could only search for guidelines citing the trial by using the registry identi cation number, but this was not possible for systematic reviews. To identify citing systematic reviews, we used the "cited by"-or "times cited"-functions of PubMed and WoS. These functions only consider journal articles, so that we were limited to the published journal articles.
The full text of some clinical practice guidelines from the United Kingdom identi ed via NICE or TRIP were only accessible to people located within the country, so that we were not able to verify the citation for those. Therefore, we did not consider them for our project.

Conclusion
In the trial cohort investigated, a considerable proportion of clinical trials was published and had an impact on clinical practice, whereas those proportions depend on speci c study characteristic. Study registries are an important alternative or complement to journal articles for publishing study results. For future, there is still the need to improve the transfer of knowledge generated in clinical research into practice, especially for those trials that are underreported. Proportion of published trials and type of publication per sub-cohort. Please also refer to Table 2.

Figure 3
Page 24/28 Cumulative incidence functions (Aalen-Johansen estimates) Figure 4 Kaplan-Meier estimates of the cumulative distribution function for time to publication of results, grouped by sub-cohort.

Figure 5
Page 25/28 Proportion of trials cited by systematic reviews.

Figure 6
Proportion of trials cited by guidelines, shown by type of publication. Impact on clinical practice. Total number of trials, published articles and systematic reviews (SRs) and guidelines, citing the published articles.

Figure 9
Publication and impact of trials Page 27/28

Figure 10
Impact of trials per sub-cohort.