On January 30th, 2020, COVID-19 was declared a public health emergency of international concern and on March 11th, it was declared a pandemic. As of our initial search date, April 19th, 1,603,209 cases have been reported worldwide, 54,225 patients were at a serious or critical state, and 169,750 have died. These numbers were increasing by the hour as the world faced a race to mitigate the impact of the disease. Our results suggest that a considerable effort in clinical research has been early mobilized against COVID-19. The hundreds of trials being conducted worldwide are a motivation for hope. For maximal efficacy of research in infectious disease epidemics, research must be fast, flexible, and integrated with the frontline response.
Adaptive clinical trials,[14, 15] where sample sizes and allocation ratios can be refined, treatments or doses can be abandoned, and focus can move towards patients with a higher likelihood of benefit, may be particularly useful in the current pandemic. A few large trials, notably, the RECOVERY trial and the WHO Solidarity trial, are underway and use an adaptive design. Currently, preliminary results are available confirming the usefulness of this type of trial design in providing informative evidence, namely showing positive results for dexamethasone and lack of benefit for hydroxychloroquine and lopinavir/ritonavir. Another example is the Adaptive COVID-19 Treatment Trial, (ACTT), sponsored by the National Institute of Allergy and Infectious Diseases (NIAID) that lead remdesivir to receive a conditional marketing authorisation by EMA and an initial Emergency Use Authorization (EUA) followed by formal approval of the FDA.
However, with these few exceptions, the research we found seems largely insufficient to reach clear answers for core outcomes in the shortest possible time. Our results suggest that patients at the largest risk of death due to COVID-19 were not being prioritised in clinical trials. This is an important missed opportunity since the high baseline risk means that smaller treatment effects would be more easily detected. Therefore, participants with a higher likelihood of benefit should not only be the focus of clinical research from an ethical point of view, but would be the most efficient population to study in order to identify which treatments are worthwhile pursuing and which are not.
Although most treatment trials use a randomised controlled design, we found that 92 (15.9%) were single-arm trials. These trials are a matter for concern, namely as CFR change considerably over time, and trials using historical controls will likely lead to more false-positive findings. Also of concern is the fact that most treatment trials (58.3%) were not multicentric. It is well documented that evidence from single-centre trials is more prone to bias compared with multicentre trials, and tend to provide larger treatment effects. This is particularly clear in the critical care setting, where many positive single-centre trials have been contradicted by subsequent multicentre trials. Furthermore, few RCTs conducted in intensive care units and using mortality as a primary endpoint show a beneficial impact of the intervention on the survival of patients with critical illness.
Another essential aspect of trial design is the choice of endpoints. While we recognize the importance of surrogate endpoints, which allow faster results to be obtained when compared with core clinical endpoints, the lack of hard and more pragmatic endpoints such as death and length of hospitalisation are causes for concern. To eschew these more clinically relevant endpoints is a methodological mistake that is hard to understand. High-quality information, with as little indirectness as possible, will be key moving forward. For research to permit informed clinical decision-making, this will have to change, and trials must use uniform disease-related definitions. Furthermore, many treatment trials are a priori already deemed to fail. The median target sample size of assessed COVID-19 pharmacotherapy trials is 100 participants. This is manifestly insufficient to detect anything but an extremely large treatment effect. For example, regarding trials with mortality as a primary endpoint, only three trials were powered to detect a difference of 50% or more between treatment groups (assuming a baseline risk of 2%). Even assuming a baseline risk of 10%, only 38 trials were adequately powered to detect a difference of 50% or more between treatment groups. In fact, half of the trials evaluating mortality have a sample size of 112 participants or fewer, being only able to identify a treatment effect on mortality over 90% (irrespectively of the baseline risk), which is delusional (supplementary material). On the other hand, the planned trial duration is also relatively short (median of 184 days for overall treatment trials), given the uncertainty regarding the natural history of COVID-19.
Some of our concerns have been increased by the early termination of a clinical trial studying remdesivir, conducted in ten hospitals in Wuhan, China. This trial was to enroll 453 participants, but as the disease in the area was brought under control the number of eligible patients became too small, and recruitment was stopped at 236 participants. This led to a reduced power in the trial, of only 58%, while it was intended to be of 80%. This trial failed to demonstrate any difference in time to clinical improvement with remdesivir (HR 1.23 [95% CI 0.87–1.75]), while, later on, the ACTT trial, an adaptive, larger and adequately powered trial succeeded in showing the usefulness of remdesivir in hospitalized patients (rate ratio for recovery 1.32 [95% CI 1.12-1.55]).
We found that only 18.9% of treatment trials were industry-funded. Although trials done by the initiative of investigators are important, the absence of the efficient and experienced industry trials means that there is considerable room for improvement and effort by part of these multinational corporations. Although we are aware of important efforts by industry to maintain pharmacotherapies available during the crisis, these entities should not forget their essential role in conducting high-quality, high-output clinical research.
In mid-March 2020, the majority of COVID-19 cases worldwide were no longer from continental China, and as of April 19th, 2020, only around 4% of cases were from China. These figures are in clear contrast with the fact that so many treatment trials (47.6%) were conducted exclusively in China. Previous research has suggested that a large proportion of clinical trial data submitted to support new drug registrations in China may be considered to be incomplete or substandard. Therefore, despite the large push in research, we question if the coming flood of data do have enough quality to produce clear answers to critical questions at early stages and the necessary recruitment capability. We are also worried that considerable efforts were wasted in the 92 treatment trials and combined planned target samples of 18,892 participants studying the effect of TCM on COVID-19. We would urge that these resources be put to more promising use.
The main pharmacotherapies under investigation were chloroquine/hydroxychloroquine, antivirals, notably lopinavir/ritonavir, and monoclonal antibodies. We found no substantive methodological differences between trials evaluating these interventions. However, there are differences in patients’ clinical characteristics, with monoclonal antibody trials allowing the inclusion of more patients with several and critical illness, as well as participants with relevant comorbidities. Antiviral trials include relatively fewer patients with several and critical illness, and largely exclude participants with COVID-relevant comorbidities.
As of September 14th, 2020, six months after the pandemic declaration, only a minority of these early trials has provided results available to the public. Additionally, according to planned completion dates, 251 trials should have been completed by this date, while only 33 accomplished this commitment. This corresponds to much lower figures than anticipated, considering the trials planned follow-up. Moreover, the information on the recruitment status of the trials often seems to be not updated in the registries, rendering it difficult to interpret the real state of research on this topic. The current available overall results and status from these early 580 treatment trials reinforce our initial worries about the overall inadequacy of these trials to provide clinically relevant conclusions.
Our study has several limitations. First, the data presented focus exclusively on registered trials. Similarly, phase 1 trials may be underrepresented. Second, there is a significant amount of missing or unsubmitted data for certain data fields, which limits the completeness of the analyses and thus the interpretability of the results presented. Third, given the rush to conduct more research, there may be trials underway that had not yet been registered, an important issue given the common nature of retrospective registration. Fourthly, the quality of the available records was largely poor, with inconsistencies and errors throughout. We think that most of the issues have been resolved, though we cannot be certain that nothing was missed.
With the hundreds of trials enrolling thousands of people currently underway, a more efficient and useful approach would be for research bodies such as the WHO, NIH, Inserm, etc. to create a coordinated research response to face the pandemic. The European Medicines Agency has called for similar efforts. We understand that there are political, ethical, administrative, contractual, regulatory, logistic, economic, and societal factors that may hinder research, though these difficulties should be overcome in times of global crisis. Persisting on the path of isolated investigation will likely only lead to futile trials and more death on a global scale. Given the large numbers of people with COVID-19, and given the recent push for more real-world evidence, we consider there to be an urgent need for a global high-quality COVID-19 patient registry, which could be used to detect large beneficial effects and provide relevant evidence for health-care decision making.
Initiatives such as the WHO R&D Blueprint aim to tackle the challenge of generating new evidence during disease outbreaks. We believe that clinical research must be integrated as an essential element of coordinated international response to epidemics. As those are exceptionally difficult contexts for clinical research, tools such as adaptive protocols that could feasibly be integrated into clinical practice, as well as global research networks and platforms, may be of great help to produce informative research. Due to unpredictable features of new outbreaks, continued enrolment throughout different locations should be advocated, allowing to achieve sufficient participants and combine research efforts.