Low Result Reporting Rates Of The Clinical Trial Of Regenerative Medicine For Neurological Diseases Registered On Clinicaltrials.Gov: A Pilot Study Using Four Neurological Diseases

doi:10.21203/rs.3.rs-1425336/v1

Download PDF

Research Article

Low Result Reporting Rates Of The Clinical Trial Of Regenerative Medicine For Neurological Diseases Registered On Clinicaltrials.Gov: A Pilot Study Using Four Neurological Diseases

https://doi.org/10.21203/rs.3.rs-1425336/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this older preprint version

Read the latest preprint version →

Sharing the methods and results of clinical trials with full transparency is an ethical obligation of those involved in clinical research. In our previous trend analysis of regenerative medicine clinical trials for stroke, the result reporting rate of trials on gene and/or cell therapy was suspected to be low. Here a pilot study was conducted, the scope of trials based on four neurological diseases using data registered in ClinicalTrials.gov was expanded, and factors affecting the result reporting rate by multivariate analysis were examined using trials on small-molecule compounds and biologics as controls. As a result, stroke as a disease and gene and/or cell therapy as a modality were identified as factors affecting the low reporting rate to ClinicalTrials.gov. However, none of these factors had an inhibitory effect on the journal reporting rate using PubMed as the index. Therefore, these results indicate that disease and modality affected the reporting rate to ClinicalTrials.gov. We believe that further detailed investigation on the effects of disease area and modality on study result reporting rates is warranted.

The need to share methods and results of clinical trials has been a long-standing issue, requiring those involved in research to report the results of registered clinical trials fully and transparently as part of their ethical obligations. Global clinical trial registries, such as the ClinicalTrials.gov,[1] EU Clinical Trial Register (EUCTR),[2] and International Clinical Trials Registry Platform (ICTRP),[3] require disclosure of trial results. Although disclosure is mandatory, the compliance rate is unsatisfactory, with only about half of results being disclosed. Furthermore, the ClinicalTrials.gov[1] and EUCTR[2] require reporting of results to the registry within 1 year of completion of the trial; however, the deadline is rarely met. [4, 5]

The trend of clinical trials in regenerative medicine has been analyzed from the perspective of a general regenerative medicine researcher who is not an expert in clinical trials, using information from databases, i.e., the ClinicalTrials.gov. Our analysis on the trends in gene and/or cell therapy (GCT) for stroke,[6] which was one of our efforts, increased our awareness on the problems associated with the result reporting rates of GCT, which may be significantly lower than that of conventional therapies, and thus, we decided to conduct a pilot analysis.

In this study, the target diseases included a total of four diseases: Parkinson’s disease (PD), spinal cord injury (SCI), and traumatic brain injury (TBI), in addition to stroke (STR), because their GCT clinical translation is in a relatively early stage and focuses on their efficacy and safety issues. PD is relatively common, and thus, many patients and drugs have been investigated. However, effective treatments for SCI and TBI have only recently been developed. As these four neurological diseases are not representative of the clinical applications of GCT in all disease areas, we believe that they can serve as models for various other disease areas currently at the early stages of GCT clinical application. Clinical trials investigating small-molecule compounds (LMWC) and biologics (BLG; excluding GCT) were used as comparators for GCT clinical trials for these four diseases. Based on the perspective of a general regenerative medicine researcher conducting a pilot analysis, the current status of clinical trial result disclosure rates was evaluated using ClinicalTrials.gov alone, a global clinical trial registry, and PubMed, a biomedical literature database. Multivariate analysis was also performed using these databases. Therefore, this study attempted to analyze factors affecting the result reporting rate and also discuss the issues involved.

Data collection

Data collection and processing of this study are summarized in Fig. 1. Trial data were downloaded from the ClinicatTrials.gov database in CSV format as of August 9, 2019, using the following keywords: “study-type” field to “interventional” and “condition/disease” field to “PD,” “stroke (STR),” “spinal cord injury (SCI),” or “traumatic brain injury (TBI).” Next, trials evaluating the targeted three different types of treatment modalities, namely, low-molecular-weight compound (LMWC), BLG, and GCT, were screened and classified. Trials as of August 9, 2019, or earlier in the “Primary Completion Date” field were further extracted. However, trials described as “withdrawn” in the “status” field were excluded. The resulting dataset, including 1731 clinical trials, was designated as 20190809. The dataset 20190809 was updated on July 12, 2021, about 2 years after the original data collection date, to gather possible collectible information. The reported status as of July 5, 2021, was reviewed in both the ClinicalTrials.gov registry and PubMed, and 34 trials whose reports were under review in ClinicalTrials.gov but not listed in PubMed were excluded. Then, four trials that are additionally withdrawn from the registry were also excluded. Furthermore, we considered trials as “actually completed trials” when the “Actual Primary Completion Date” field was filled, which yielded 1,376 trials. To extract the relevant subset, the “Actual Primary Completion Date” was narrowed from January 1, 2005, to June 30, 2019, and obtained 1298 trials. Lastly, all Phase II–IV trials were selected, for which the result reporting rate is mandatory, resulting in 1017 trials designated as 20210630 dataset.

Analysis Of Result Reporting

Tables 1 and 2 show the results of multivariate analysis regarding the result reporting rate of ClinicalTrials.gov and PubMed, respectively. Table 3 indicates the results of the multivariate analysis on the total result disclosure rates (either ClinicalTrials.gov or PubMed or both).

Table 1

Summary of the multivariate analysis on the result reporting rate to ClinicalTrials.gov
Variables	Trials with actual completion date	Reported to Clinical Trials.gov	Clopper-Pearson	Multivariate Analysis
Variables	Trials with actual completion date	Reported to Clinical Trials.gov	Proportion (%) (95% CI)	Crude odds ratio (95% CI)	P value	Adjusted odds ratio (95% CI)	P value
total	1017	440	43.3 (40.2–46.4)
Diseases:
PD	423	202	47.8 (42.9–52.6)	1.09 (0.69–1.71)	0.714	0.89 (0.52–1.50)	0.648
PD STR TBI	3	2	66.7 (9.4–99.2)	2.38 (0.21–27.19)	0.485	5.85 (0.14–254.31)	0.359
SCI	94	39	41.5 (31.4–52.1)	0.84 (0.47–1.51)	0.567	0.94 (0.49–1.82)	0.855
SCI STR	3	3	100.0 (29.2–100.0)	1.92E + 9 (0 - )	0.999	4.35E + 9 (0 - )	0.999
STR	394	145	36.8 (32.0–41.8)	0.69 (0.44–1.10)	0.117	0.45 (0.26–0.76)	0.003
STR TBI	8	7	87.5 (47.3–99.7)	8.33 (0.99–70.48)	0.052	2.21 (0.21–22.99)	0.506
TBI	92	42	45.7 (35.2–56.4)	Reference		Reference
completion year:
2005	43	3	7.0 (1.5–19.1)	0.11 (0.03–0.40)	< 0.001	0.03 (0.01–0.13)	< 0.001
2006	35	6	17.1 (6.6–33.6)	0.31 (0.12–0.85)	0.023	0.12 (0.04–0.35)	< 0.001
2007	51	2	3.9 (0.5–13.5)	0.06 (0.01–0.28)	< 0.001	0.03 (0.01–0.16)	< 0.001
2008	99	53	53.5 (43.2–63.6)	1.75 (0.95–3.23)	0.074	1.09 (0.55–2.19)	0.801
2009	71	37	52.1 (39.9–64.1)	1.65 (0.85–3.20)	0.137	1.10 (0.53–2.30)	0.799
2010	70	39	55.7 (43.3–67.6)	1.91 (0.98–3.71)	0.057	1.67 (0.79–3.53)	0.176
2011	73	32	43.8 (32,2–55.9)	1.18 (0.61–2.29)	0.615	0.84 (0.40–1.76)	0.636
2012	79	39	49.4 (37.9–60.9)	1.48 (0.78–2.82)	0.233	1.21 (0.59–2.48)	0.602
2013	84	41	48.8 (37.7–60.0)	1.45 (0.77–2.73)	0.254	1.09 (0.53–2.22)	0.820
2014	78	40	51.3 (39.7–62.8)	1.60 (0.84–3.05)	0.155	1.36 (0.66–2.80)	0.402
2015	86	44	51.2 (40.1–62.1)	1.59 (0.85–2.99)	0.150	1.55 (0.77–3.14)	0.224
2016	71	31	43.7 (31.9–56.0)	1.18 (0.61–2.28)	0.632	1.20 (0.58–2.51)	0.622
2017	73	29	39.7 (28.5–51.9)	Reference		Reference
2018	71	34	47.9 (35.9–60.1)	1.39 (0.72–2.70)	0.324	1.26 (0.60–2.65)	0.539
2019	33	10	30.3 (15.6–48.7)	0.66 (0.27–1.59)	0.353	0.80 (0.31–2.11)	0.658
status:
Completed	832	350	42.1 (38.7–45.5)	Reference		Reference
Terminated	173	87	50.3 (42.6–58.0)	1.39 (1.00–1.94)	0.048	1.17 (0.80–1.72)	0.417
Unknown	6	0	0.0 (0.0–45.9)	0 (0 - )	0.999	0 (0 - )	0.999
Active, not recruiting	6	3	50.0 (11.8–88.2)	1.38 (0.28–6.86)	0.696	0.76 (0.11–5.51)	0.786
phases:
II	430	162	37.7 (33.1–42.4)	0.49 (0.37–0.65)	< 0.001	0.59 (0.42–0.82)	0.002
III	364	201	55.2 (49.9–60.4)	Reference		Reference
IV	223	77	34.5 (28.3–41.2)	0.43 (0.30–0.60)	< 0.001	0.43 (0.29–0.64)	< 0.001
Sponsor:
BIG	268	172	64.2 (58.1–69.9)	3.07 (2.25–4.19)	< 0.001	4.17 (2.79–6.24)	< 0.001
SMALL	255	86	33.7 (27.9–39.9)	0.87 (0.64–1.20)	0.399	0.75 (0.52–1.08)	0.124
AC	494	182	36.8 (32.6–41.3)	Reference
Modality:
CC	808	354	43.8 (40.4–47.3)	Reference		Reference
BLG	167	83	49.7 (41.9–57.5)	1.27 (0.91–1.77)	0.164	1.41 (0.94–2.12)	0.100
GCT	42	3	7.1 (1.5–19.5)	0.10 (0.03–0.32)	< 0.001	0.10 (0.03–0.35)	< 0.001

Table 2

Summary of the multivariate analysis on the result reporting rate to PubMed
Variables	Trials with actual completion date	Reported to PubMed	Clopper-Pearson	Multivariate Analysis
Variables	Trials with actual completion date	Reported to PubMed	Proportion(%) (95% CI)	Crude odds ratio (95% CI)	P value	Adjusted odds ratio (95% CI)	P value
total	1017	405	39.8 (36.8–42.9)
Disease:
PD	423	132	31.2 (26.8–35.9)	0.81 (0.51–1.30)	0.385	0.48 (0.29–0.82)	0.007
PD STR TBI	3	3	100.0 (29.2–100.0)	2.89E + 9 (0 - )	0.999	1.27E + 9 (0 - )	0.999
SCI	94	17	18.1 (10.9–27.4)	0.40 (0.20–0.78)	0.007	0.30 (0.15–0.61)	< 0.001
SCI STR	3	2	66.7 (9.4–99.2)	3.58 (0.31–40.94)	0.306	2.78 (0.15–53.10)	0.497
STR	394	211	53.6 (48.5–58.6)	2.06 (1.29–3.30)	0.003	1.55 (0.94–2.58)	0.089
STR TBI	8	7	87.5 (47.3–99.7)	12.52 (1.48–106.18)	0.021	7.61 (0.76–75.98)	0.084
TBI	92	33	35.9 (26.1–46.5)	Reference		Reference
completion year:
2005	43	17	39.5 (25.0–55.6)	1.73 (0.78–3.85)	0.177	1.92 (0.81–4.57)	0.138
2006	35	11	31.4 (16.9–49.3)	1.22 (0.50–2.93)	0.665	1.19 (0.47–3.05)	0.711
2007	51	17	33.3 (20.8–47.9)	1.33 (0.61–2.88)	0.478	1.66 (0.72–3.84)	0.234
2008	99	42	42.4 (32.5–52.8)	1.95 (1.02–3.74)	0.044	2.52 (1.24–5.12)	0.010
2009	71	27	38.0 (26.8–50.3)	1.63 (0.81–3.28)	0.175	1.90 (0.89–4.05)	0.096
2010	70	32	45.7 (33.7–58.1)	2.23 (1.11–4.48)	0.024	2.64 (1.25–5.60)	0.011
2011	73	32	43.8 (32.2–55.9)	2.07 (1.04–4.13)	0.039	2.49 (1.18–5.23)	0.016
2012	79	38	48.1 (36.7–59.6)	2.46 (1.25–4.84)	0.009	3.35 (1.61–6.99)	0.001
2013	84	37	44.0 (33.2–55.3)	2.09 (1.07–4.08)	0.032	2.64 (1.28–5.45)	0.009
2014	78	37	47.4 (36.0–59.1)	2.39 (1.21–4.72)	0.012	2.52 (1.21–5.25)	0.014
2015	86	31	36.0 (26.0–47.1)	1.49 (0.76–2.94)	0.245	2.11 (0.999–4.46)	0.050
2016	71	30	42.3 (30.6–54.6)	1.94 (0.97–3.89)	0.063	2.23 (1.05–4.75)	0.037
2017	73	20	27.4 (17.6–39.1)	Reference		Reference
2018	71	26	36.6 (25.5–48.9)	1.53 (0.76–3.10)	0.237	1.93 (0.89–4.18)	0.094
2019	33	8	24.2 (11.1–42.3)	0.85 (0.33–2.19)	0.733	0.89 (0.32–2.49)	0.826
recruiting status:
Completed	832	355	42.7 (39.3–46.1)	Reference		Reference
Terminated	173	46	26.6 (20.2–33.8)	0.49 (0.34–0.70)	< 0.001	0.39 (0.26–0.59)	< 0.001
Unknown	6	2	33.3 (4.3–77.7)	0.67 (0.12–3.69)	0.647	1.24 (0.20–7.61)	0.817
Active, not recruiting	6	2	33.3 (4.3–77.7)	0.67 (0.12–3.69)	0.647	0.74 (0.11–4.88)	0.754
phase:
II	430	135	31.4 (27.0–36.0)	0.41 (0.31–0.55)	< 0.001	0.47 (0.34–0.65)	< 0.001
III	364	191	52.5 (47.2–57.7)	Reference		Reference
IV	223	79	35.4 (29.2–42.1)	0.50 (0.35–0.70)	< 0.001	0.50 (0.34–0.73)	< 0.001
Sponsor:
BIG	268	132	49.3 (43.1–55.4)	1.71 (1.26–2.31)	< 0.001	1.73 (1.20–2.50)	0.003
SMALL	255	94	36.9 (30.9–43.1)	1.03 (0.75–1.41)	0.866	1.17 (0.81–1.69)	0.401
AC	494	179	36.2 (32.0–40.6)	Reference
Modality:
CC	808	310	38.4 (35.0–41.8)	Reference		Reference
BLG	167	78	46.7 (39.0–54.6)	1.41 (1.01–1.97)	0.046	0.94 (0.63–1.42)	0.783
GCT	42	17	40.5 (25.6–56.7)	1.09 (0.58–2.06)	0.784	1.69 (0.82–3.47)	0.155

Table 3

Summary of the multivariate analysis on the total result disclosure rate
Variables	Trials with actual completion date	Total disclosure	Clopper-Pearson	Multivariate Analysis
Variables	Trials with actual completion date	Total disclosure	Proportion(%) (95% CI)	Crude odds ratio (95% CI)	P value	Adjusted odds ratio (95% CI)	P value
total	1017	629	61.8 (58.8–64.8)
Disease:
PD	423	254	60.0 (55.2–64.7)	0.84 (0.53–1.34)	0.468	0.62 (0.37–1.03)	0.067
PD STR TBI	3	3	100.0 (29.2–100.0)	9.04E + 8 (0 - )	0.999	5.94E + 9 (0 - )	1.000
SCI	94	45	47.9 (37.5–58.4)	0.51 (0.29–0.92)	0.026	0.43 (0.23–0.80)	0.008
SCI STR	3	3	100.0 (29.2–100.0)	9.04E + 8 (0 - )	0.999	9.86E + 9 (0 - )	1.000
STR	394	257	65.2 (60.3–69.9)	1.05 (0.65–1.69)	0.842	0.78 (0.47–1.31)	0.345
STR TBI	8	8	100.0 (63.1–100.0)	9.04E + 8 (0 - )	0.999	2.20E + 9 (0 - )	1.000
TBI	92	59	64.1 (53.5–73.9)	Reference		Reference
completion year:
2005	43	17	39.5 (25.0–55.6)	0.67 (0.31–1.44)	0.308	0.40 (0.17–0.91)	0.029
2006	35	15	42.9 (26.3–60.6)	0.77 (0.34–1.74)	0.530	0.48 (0.20–1.16)	0.102
2007	51	17	33.3 (20.8–47.9)	0.51 (0.25–1.08)	0.078	0.45 (0.21–0.996)	0.049
2008	99	71	71.7 (61.8–80.3)	2.61 (1.38–4.91)	0.003	2.40 (1.22–4.72)	0.011
2009	71	46	64.8 (52.5–75.8)	1.89 (0.97–3.69)	0.062	1.62 (0.79–3.29)	0.185
2010	70	52	74.3 (62.4–84.0)	2.97 (1.47–6.01)	0.003	3.00 (1.42–6.31)	0.004
2011	73	46	63.0 (50.9–74.0)	1.75 (0.91–3.39)	0.096	1.63 (0.81–3.28)	0.175
2012	79	55	69.6 (58.2–79.5)	2.36 (1.21–4.57)	0.011	2.42 (1.19–4.88)	0.014
2013	84	54	64.3 (53.1–74.4)	1.85 ( 0.98–3.51)	0.060	1.76 (0.89–3.48)	0.105
2014	78	56	71.8 (60.5–81.4)	2.62 ( 1.33–5.13)	0.005	2.41 (1.18–4.90)	0.016
2015	86	56	65.1 (54.1–75.1)	1.92 ( 1.01–3.63)	0.045	2.25 (1.13–4.45)	0.020
2016	71	46	64.8 (52.5–75.8)	1.89 (0.97–3.69)	0.062	2.04 (1.01–4.15)	0.048
2017	73	36	49.3 (37.4–61.3)	Reference		Reference
2018	71	47	66.2 (54.0–77.0)	2.01 (1.03–3.94)	0.042	2.26 (1.10–4.64)	0.026
2019	33	15	45.5 (28.1–63.6)	0.86 (0.38–1.95)	0.713	1.03 (0.43–2.47)	0.952
recruiting status:
Completed	832	522	62.7 (59.4–66.0)	Reference		Reference
Terminated	173	102	59.0 (51.2–66.4)	0.85 (0.61–1.19)	0.351	0.68 (0.47–0.99)	0.042
Unknown	6	2	33.3 (4.3–77.7)	0.30 (0.05–1.63)	0.162	0.37 (0.06–2.13)	0.262
Active, not recruiting	6	3	50.0 (11.8–88.2)	0.59 (0.12–2.96)	0.525	0.35 (0.05–2.34)	0.279
phase:
II	430	238	55.3 (50.5–60.1)	0.44 (0.32–0.59)	< 0.001	0.51 (0.37–0.72)	< 0.001
III	364	269	73.9 (69.1–78.3)	Reference		Reference
IV	223	122	54.7 (47.9–61.4)	0.43 (0.30–0.61)	< 0.001	0.42 (0.28–0.61)	< 0.001
Sponsor:
BIG	268	203	75.7 (70.2–80.8)	2.22 (1.59–3.09)	< 0.001	2.44 (1.64–3.63)	< 0.001
SMALL	255	137	53.7 (47.4–60.0)	0.82 (0.61–1.12)	0.211	0.79 (0.56–1.12)	0.180
AC	494	289	58.5 (54.0–62.9)	Reference
Modality:
CC	808	495	61.3 (57.8–64.6)	Reference		Reference
BLG	167	115	68.9 (61.2–75.8)	1.40 (0.98–2.00)	0.065	1.27 (0.84–1.92)	0.259
GCT	42	19	45.2 (29.8–61.3)	0.52 (0.28–0.98)	0.041	0.68 (0.34–1.34)	0.260

Diseases

TBI was selected as a reference for multivariate analysis according to disease type. Table 3 shows SCI had significantly lower rates of total result disclosure with crude and adjusted odds ratios (OR: 0.51 [95% confidence interval {CI}: 0.29–0.92]; p = 0.026) and (0.43 [0.23–0.80]; p = 0.008, respectively). Next, Table 2 shows low reporting rates from PubMed with both crude and adjusted ORs (0.40 [0.20–0.78]; p = 0.007), and 3.40 [0.15–0.61]; p < 0.001, respectively). Conversely, no difference was observed in those from ClinicalTrials.gov (Table 1). The lower reporting rate of SCI to PubMed was interpreted as a cause of reduction in the total disclosure rate. In PD, no significance in the reporting rates was observed in total disclosure or ClinicalTrials.gov, but those in PubMed were slightly lower and significant with adjusted OR of 0.48 (0.29–0.82); p = 0.007. In STR, no difference in the reporting rate was observed in the total disclosure, but that in ClinicalTrials.gov was lower and significant with adjusted OR of 0.49 (0.26–0.76); p = 0.003. In contrast, the PubMed reporting rate was significantly higher only for the crude OR (2.06 [1.29–3.30]; p = 0.003). In STR TBI (where both STR and TBI were evaluated in single trials), the crude OR for reporting to PubMed was significantly higher (12.52 [1.48–106.18]; p = 0.021) although the sample size was quantitively low.

Figure 2a shows the comparison of reporting rates by disease category. The reporting rate of STR trial results to Clinicaltrials.gov is the lowest among four neurological diseases, which are just < 40%, whereas the other neurological diseases have also not reached 50% of the reporting rate. Nevertheless, results of the STR trial were well published in journals, with a total disclosure rate exceeding 60%. The total result disclosure rate of PD and TBI also attained 60%. Conversely, the total disclosure rate of SCI trials was < 50%.

Primary Completion Date (Year)

For the analysis of the primary completion date, data for the year 2017 was used as reference. The total result disclosure rate was significantly lower with both crude and adjusted OR from 2005 to 2007, but was significantly higher with adjusted OR in 2008, 2010, 2012, 2014, 2015, 2016, and 2018. We also observed a significantly higher reporting rate to PubMed with adjusted ORs from 2008 and 2010 to 2016.

Recruitment Status

To analyze the recruitment status, trials with the field of “Recruiting Status” described as “Completion” were used as reference. Regarding the field described as “Terminated,” the reporting rate to PubMed was significantly low with crude and adjusted odds ratios (0.49 [0.34–0.70]; p < 0.001) and (0.39 [0.26–0.59)]; p < 0.001), respectively, whereas the total result disclosure rate was significantly low with adjusted OR of 0.68 (0.47–0.99); p = 0.042; however, the result reporting rate to ClinicalTrials.gov was significantly high with crude OR of 1.39 (1.00–1.94); p = 0.048.

Phases

In multivariate analysis, phase III was used as a reference, and both phases II and IV had significantly lower outcome reporting rates than phase III for any reporting destination (whether the results were posted to ClinicalTrials.gov or publication in a journal). In both phases II and IV, the reporting rate was significantly lower than that in phase III (whether the results were posted to ClinicalTrials.gov or publication in a journal).

Figure 2b shows the publication status of trial results classified by clinical phases. The total disclosure rate was highest for phase III trials and significantly lower for Phase II and IV trials.

Sponsor

The academic institution (AC) was set up as a reference. The big pharma (BIG) had significantly higher reporting rates to ClinicalTrials.gov and also to PubMed, whereas small pharma (SMALL) had a similar reporting rate to AC in all result reporting status.

Figure 2c illustrates the result reporting rate classified by the sponsor. Regarding the disclosure of trial results to ClinicalTrials.gov, SMALL and AC had low rates at 35%, while BIG had higher rate at > 60%. In addition, the reporting rate of BIG was also high in PubMed.

Treatment Modality

LMWC was set up as a reference for treatment modality. For BLG, no difference was confirmed in any result reporting rates to ClinicalTrials.gov and also total disclosure, but the reporting rate was slightly higher to PubMed with a significant crude OR. The reporting rate for both Clinicaltrials.gov and PubMed in GCT was significantly lower with a crude OR (0.52 [0.28–0.98]; p = 0.041), however, no significant difference was observed with the adjusted OR, and the reporting rate to PubMed was not significant either. Conversely, the reporting rate to ClinicalTrials.gov was revealed to be significantly lower (adjusted OR 0.10 [0.03–0.35]; p < 0.001). Therefore, trials that evaluated the targeted treatment modality, i.e., GCT, contributed somehow to be one of the factors that reduce the reporting rate of study results to Clinicaltrials.gov.

Figure 2d illustrates how the treatment modality affects the disclosure of trial results. The BLG trial had the highest rate of reporting trial results, followed by the LMWC. The GCT trial had a low reporting rate of trial results. The reporting rate for GCT results to Clinicaltrials.gov was remarkably low although there no significant difference was observed in PubMed. Consequently, the total disclosure rate was reduced correspondingly.

Next, we examined the time required from study completion to result in reporting by treatment modality. Figure 3 shows the decrement in the unreported rate of results using the Kaplan–Meier method over time from the primary completion date of the trials to the disclosure on ClinicalTrials.gov or PubMed, or total. In all cases, the unreported rate decreased over time but to various degrees. For example, when comparing the unreported rate at 60 months in total disclosure (Fig. 3c), the GCT group had more unreported results (56%) than the BLG (34%) and LMWC (43%) groups, a ranking that remained unchanged throughout the study period. The log-rank method showed a significant difference between BLG and GCT (p = 0.012); however, when the reporting destination was limited to PubMed (Fig. 3b), these differences were reduced and reversed to 53% for BLG, 63% for LMWC, and 57% for GCT, without significant differences among them. Conversely, when trials that reported results to the ClinicalTrials.gov registry were chosen (Fig. 3a), the difference significantly increased in reporting rates. The unreported rate for both BLG and LMWC was 57% and 61%, respectively, at 60 months after the primary completion date, whereas 95% of GCTs were unreported, a tendency that remained unchanged throughout the study period. The results of the log-lank method showed significant differences (p < 0.001) between BLG and GCT and between LMWC and GCT.

During our analysis on the trends in clinical trials of regenerative medicine evaluating STR,[6] low result reporting rates were suspected in GCT trials. This observation motivated us to expand the analysis to four neurological diseases and examined factors affecting the result reporting rate using multivariate analysis. In this analysis, to the best of our knowledge, the different effects in modality were considered as a factor affecting the result reporting rate in addition to the sponsor and phase, which have been previously reported.

We previously attempted to combine the ClinicalTrials.gov and ICTRP to investigate trends in regenerative medicine.[6] However, using multiple registry databases can be burdensome because of the need to extract duplicate trial data and then eliminate them. Moreover, the ICTRP does not require inclusion of an estimated “trial completion date” on the registration of individual clinical trials. Therefore, we decided that using the ICTRP database is unsuitable for this analysis; instead, we exclusively applied the ClinicalTrials.gov registry. The database used in this analysis was limited to journal articles from PubMed, carrying out a hands-on approach. Various reporting exemptions granted by ClinicalTrials.gov were also ignored.

In this study, STR was identified as one of the independent factors for the low reporting rate of results to ClinicalTrials.gov, because a significant difference in the adjusted OR was observed. We found that the reporting rate for STR results to Clinicaltrials.gov was the lowest among the four neurological diseases at <40%, while the reporting rate to Clinicaltrials.gov was also <50% for the remaining diseases. Although result reporting to PubMed was significantly different only in crude ORs, not in adjusted ORs, STR results were frequently reported in journal publications. Therefore, the combined reporting rate to Clinicaltrials.gov and PubMed exceeded 60%. Result reporting rates of PD and TBI also reached 60% when Clinicaltrials.gov and PubMed results were combined, whereas that for Clinicaltrials.gov of SCI was still <50%. Our findings indicate that some degree of bias may occur in favor of either Clinicaltrials.gov or PubMed in terms of result reporting depending on the disease type although the cause of this bias remains unknown. This may have contributed at least in part to the less reporting of STR to Clinicaltrials.gov. Nevertheless, these reporting rates are broadly similar to those previously reported by Zwierzyna et al.[4] (overall, neurology had a 40% reporting rate on Clinicaltrials.gov and <60% for both Clinicaltrials.gov and PubMed).

Regarding the primary completion date (year), the result reporting rates to ClinicalTrials.gov were low in 2005–2007 because Section 801 of the Food and Drug Administration Amendments Act became effective in 2007.[7] Furthermore, PubMed had significantly higher reporting rates from 2008 and 2010 to 2014 and 2016; however, its cause needs to be examined in more detail.

Regarding the field of “recruitment status,” a few errors were found that may have been due to data entry errors by the primary investigator, such as “unknown status” or “active, not recruiting,” despite the inclusion of the field of “Actual primary completion date.” “Terminated,” in which the trial was terminated before the estimated patient recruitment was completed for various reasons, was identified as a factor that decreases the result reporting rate in PubMed. The reason for “Terminated” may be related to the efficacy or safety issues of the study drug. In such cases, the results should be reported ethically. The results of this analysis show that the reporting result rate to ClinicalTrials.gov did not differ based on the number of completed trials, but the number of reports on publication in journals was lower. A more detailed analysis is needed for further discussion.

The reporting rate of clinical phases to ClinicalTrials.gov and PubMed was significantly lower for Phase II and IV trials when phase III trials were considered as reference. Phase III trials are high because the efficacy of the drug has been confirmed to some extent at an earlier phase, and thus, the result at the subsequent regulatory application is essential. Conversely, the reason for the low reporting rate of Phase II trials may be partly because the efficacy and safety are not yet sufficiently clear, and researchers tend to hesitate to report “unfavorable” results. A similar negative bias may occur for phase IV trials. Goldacre et al.[5] reported a similar trend of significantly higher reporting rates for phase III and lower reporting rates for phases II and IV using crude ORs although the trend was not statistically significant using adjusted ORs. Remarkably, similar trends were obtained despite the different enrollment databases and scope of data collection.

Regarding the result reporting, findings of lower reporting rates for SMALL and AC are consistent with the previous report by Zwierzyna et al.[4] Notably, Goldacre et al.⁵ who used the EUCTR as the registry database also reported a trend of higher result reporting rates from big industries and lower from universities (academic institutions: AC). DeVito et al.[8] showed that industry groups were more likely to comply with regulations if sponsors who initiated the development were industry groups compared to non-industry or non-US government sponsors. Conversely, Goldacre et al.[5] showed that non-major sponsors are more likely to be unaware of the reporting obligations or may lack the administrative procedures to point out violations and help researchers comply with the regulations. Mayo-Wilson et al.[9] noted that while big companies work to improve result reporting rates by assigning dedicated personnel to internal administrative departments for procedures such as registration and reporting, implementing such measures is not financially achievable for most non-major pharmaceutical institutions. These factors imply that clinical trial sponsors could affect the reporting of clinical trial results.

Finally, multivariate analysis of modalities showed that GCT contributed to as one of the factors that could reduce the rate of reporting trial results to Clinicaltrials.gov. The decrement curve of the unreported result rate over time by the Kaplan–Meier method (Fig. 3) significantly illustrates the pronounced influence of the modality and dispels the concern that only GCTs might have a shorter duration after primary completion compared to LMWC and BLG.

Overall, our multivariate analysis showed that STR as a disease and GCT as a modality were contributing factors to the lower reporting rate to ClinicalTrials.gov. The answer to our initial question was considered to be an additive effect of STR as a disease and GCT as a modality. In STR, the reporting rate to ClinicalTrials.gov was low, but was relatively high to PubMed (significant only in crude ORs), indicating that result disclosure in a journal publication was more preferred. Furthermore, the rate of reporting results in PubMed for GCT was not different from that for control LMWC, indicating that reporting results of GCT to ClinicalTrials.gov tended not to be favored, whereas journal publication was more preferred. In general, trial result reporting to ClinicalTrials.gov is considered to comply with higher standards and be more complete than publication in journals.[10,11] This may raise another ethical issue if more investigators avoid reporting results to ClinicalTrials.gov in favor of publishing results in journals, which is an easier option. If negative safety and efficacy issues occur during clinical trials and those are not disclosed, researchers may carry out unnecessarily duplicate human clinical trials in the future, which may raise another ethical issue. Therefore, we believe that some systematic approach is needed to further encourage reporting trial results not only to ClinicalTrials.gov but also to global registries.

In this study, from the perspective of mid-level regenerative medicine researchers who are not particularly experts, we applied solely ClinicalTrials.gov as a registry and searched for four neurological diseases relying on the ClinicalTrials.gov thesaurus followed by searching PubMed for article reports as we used a manual method of searching by NCT (national clinical trial) number and manually classifying the modalities to be examined. We are aware of the possibility of underestimates in journal publication due to this research method; however, we believe our pilot research remains significant. Although it is challenging and have high expectations to apply practical use of new modalities such as GCT, we need to be cautious on the risks ahead.

Disclosure of trial results could be an indicator of keeping transparency and safety issues; therefore, those involved in research should clarify the status of trial result reporting to meet their ethical obligations. If the disclosure rate of trial results remains poor, discussion should be stimulated to solve and determine the cause and take effective countermeasures. In particular, the field of hematologic oncology, which has rapidly advanced in practical usage in the past decade, evolves on a different scale than the field of neurological diseases; therefore, simple analogies cannot be made based on this study. Separate analysis will be needed. To address these issues, we would further expand the scope of our study to tackle the result reporting issues by disease areas and modalities.

Data collection

At first, data in the CSV format were downloaded from the ClinicatTrials.gov website on August 9, 2019, using the following keywords: “interventional studies” as the study type and “Parkinson’s disease,” “stroke,” “spinal cord injury,” or “traumatic brain injury” as the condition or disease. Next, trials evaluating the targeted treatment modalities (LMWC, BLG, or GCT) were screened and classified among these trials using reference databases, e.g., AdisInsight,[12] DrugBank,[13] and KEGG. [14] Other modalities such as devices, procedures, behaviors, radiopharmaceuticals, and diagnostics were excluded. Next, trials with a Primary Completion Date of August 9, 2019, or earlier were selected. We excluded studies that had no description for the “Primary Completion Date.” In cases where there was no date in the “Primary Completion Date” field, the date in the “Study Completion Date” was entered instead. Other trials without a description of “Primary Completion Date” were excluded. We excluded studies described as “withdrawn” in the “status” field. The dataset was subsequently updated on July 12, 2021 to follow up on the reported results of the selected trials above. We further selected trials with an “Actual Primary Completion Date” to ensure that we only completed trials were included for analysis. If the “Actual Primary Completion Date” field is not filled in, the date in the “Actual Primary Study Completion Date” field is used instead. Among all trials with an Actual Primary Completion Date from January 1, 2005, to June 30, 2019, we examined whether their results were registered in ClinicalTrials.gov or found in PubMed until July 5, 2021; thus, even if a trial was completed on June 30, 2019, a time lag of at least 2 years was allowed before the result is reported. Among clinical trials with “No Results Available” in the “Study Results” field, if they were described as “Results Submitted,” we excluded them from analysis because the studies are still under review and have not yet been reported and should not be counted as “No Results Available.” However, trials reported in PubMed were counted as PubMed publication.

Data Of Result Reporting

We considered a study to be “reported” if the field “Study Results” was filled with “Has Results” in the ClinicalTrials.gov registry, and the date “Results First Posted” was used as the disclosure date. For journal publications, articles that published results of clinical trials in two ways were reviewed: 1) in the “More Information” section of individual trials on ClinicalTrials.gov, by searching for links to “Publication of Results,” or by investigating the link “Publications automatically indexed to this study by ClinicalTrials.gov Identifier (NCT Number)” and 2) by searching the PubMed database using the NCT number assigned to each clinical trial as an index. For each article, whether it reported the results that corresponded to the primary outcome criteria was manually checked, and trials with confirmed corresponding results were judged to have been published in PubMed. Trials without corresponding results, with only the trial protocol (including design and plan) published, and those withdrawn were judged as “no results reported.” The PubMed publication date was used as the actual publication date of the article or the earlier electronic publication date if available. Articles published before the “Primary Completion Date” were excluded.

Time Lag To Date Of Result Reporting

Time course due to time lag in the no-result reporting rate were depicted using the Kaplan–Meier method. The event date was defined as the date when the results of the study were reported (“Results First Posted” in ClinicalTrials.gov, the publication date in PubMed, or the earlier of the two dates if both were reported). If none of the events were reported by the data collection date (July 5, 2021), this collection date was used as the censoring date, and the number of days from the actual primary completion date was calculated and analyzed.

Sponsor Classification

To investigate the relationship between result reporting and study sponsors, we classified the organizations responsible for studies as BIG, SMALL, or AC. The top 66 companies, with revenues exceeding $2 billion in the global pharmaceutical industry in 2018,[15] were classified as “BIG”; smaller companies were designated as “SMALL”; and non-commercial organizations, i.e., universities, hospitals, and government organizations, were designated “AC.”

Phase Classification

For convenience, we classified the trial phases as follows: Phase I as Phase I; Phase I/II and II as Phase II; Phase II/III and III as Phase III; and Phase IV as Phase IV.

Software

The downloaded CSV file was subjected to data cleaning and tagging using Microsoft Excel. IBM SPSS Statistics v28 was used for all statistical analyses. Sankey diagram was depicted using Origin Pro (LightStone Corp., Tokyo).

Statistical analysis

Multivariate analysis was performed with LCON = 0.000000001. The significance level was set to 0.05. To analyze the time lag to date of result reporting, we tested for significant differences in three combinations between the three groups (BLG, GCT, and LMWC) using the log-rank method and Bonferroni method. Multiple comparisons were made using the Bonferroni method (the significance level was changed to 0.0166).

Ethics approval and consent to participate

Not applicable (This study is an analysis of existing publicly available data.)

Consent for publication

Not applicable.

Availability of data and materials

Data that support the study findings are available from the corresponding author upon reasonable request.

Competing interests

The authors declare no conflicts of interest associated with this manuscript.

Funding

This work was supported by the Highway Program for Realization of Regenerative Medicine of The Japan Agency for Medical Research and Development (AMED) under Grant Number JP18bm0504009.

Authors' contributions

TN, HO, and AM conceived of the work. TN, SH, TA, and HO processed and analyzed the data. All authors contributed to create the manuscript and approved the final manuscript.

Acknowledgements

Not applicable

National Institutes of Health, US Department of Health and Human Services: Clinical trials registration and results information submission. Final Rule. Fed Regist 2016, 81:64981–5157.
Call for all sponsors to publish clinical trial results in EU database https://www.ema.europa.eu/en/news/call-all-sponsors-publish-clinical-trial-results-eu-database
Moorthy VS, Karam G, Vannice KS, Kieny MP: Rationale for WHO’s new position calling for prompt reporting and public disclosure of interventional clinical trial results.PLoS Med 2015, 12:e1001819.
Zwierzyna M, Davies M, Hingorani AD, Hunter J: Clinical trial design and dissemination: Comprehensive analysis of clinicaltrials.gov and PubMed data since 2005. BMJ 2018, 361:k2130.
Goldacre B, DeVito NJ, Heneghan C, Irving F, Bacon S, Fleminger J, Curtis H: Compliance with requirement to report results on the EU clinical trials register: Cohort study and web resource.BMJ 2018, 362:k3218.
Negoro T, Okura H, Maehata M, Hayashi S, Yoshida S, Takada N, Matsuyama A: Tends in clinical trials for stroke by cell therapy: data mining ClinicalTrials.gov and the ICTRP portal site.NPJ Regen. Med.2019, 4:20.
PUBLIC LAW 110–85—SEPT. 27, 2007 https://www.govinfo.gov/content/pkg/PLAW-110publ85/pdf/PLAW-110publ85.pdf
DeVito NJ, Bacon S, Goldacre B: Compliance with legal requirement to report clinical trial results on clinicaltrials.gov: A cohort study. Lancet 2020, 395:361-369.
Mayo-Wilson E, Heyward J, Keyes A, Reynolds J, White S, Atri N, Alexander GC, Omar A, Ford DE: Clinical trial registration and reporting: A survey of academic organizations in the United States.BMC Med 2018, 16:60.
Hartung DM, Zarin DA, Guise JM, McDonagh M, Paynter R, Helfand M: Reporting discrepancies between the ClinicalTrials.gov results database and peer-reviewed publications.Ann. Intern. Med. 2014, 160:477-83.
Riveros C, Dechartres A, Perrodeau E, Haneef R, Boutron I, Ravaud P: Timing and completeness of trial results posted at ClinicalTrials.gov and Published in Journals.PLoS Med. 2013, 10:e1001566.
AdisInsight: Home https://adis.springer.com/
DRUGBANK Online https://go.drugbank.com/
KEGG DRUG Database https://www.genome.jp/kegg/drug/
Ito K: List of the top 66 companies with revenues exceeding $2 billion in the global pharmaceutical industry in 2018. Nikkei Biotechnol. Nikkei Business 2019.

No competing interests reported.

Download PDF

Version 1

posted

You are reading this older preprint version

Read the latest preprint version →

Low Result Reporting Rates Of The Clinical Trial Of Regenerative Medicine For Neurological Diseases Registered On Clinicaltrials.Gov: A Pilot Study Using Four Neurological Diseases

Status:

Version 1

Abstract

Figures

Introduction

Results

Data collection

Analysis Of Result Reporting

Diseases

Primary Completion Date (Year)

Recruitment Status

Phases

Sponsor

Treatment Modality

Discussion

Methods

Data collection

Data Of Result Reporting

Time Lag To Date Of Result Reporting

Sponsor Classification

Phase Classification

Software

Statistical analysis

Declarations

References

Additional Declarations

Status:

Version 1