We conducted a comprehensive search on the impact of ivermectin for the management of patients with COVID-19 and observed that ivermectin does not have an effect in reducing mortality or mechanical ventilation in patients with COVID-19. Despite the low quality of evidence, this effect was consistent when comparing ivermectin vs. placebo, and ivermectin associated with SOC vs. SOC, as well as in sensitivity analysis. Additionally, there was very low quality of evidence of no increase in risk of adverse effects.
Despite not being recommended in current COVID-19 guidelines by WHO and IDSA [4, 29], the prevalence of self-medication during COVID-19 course was high, and ivermectin was one of the medications commonly used, as shown in a recent systematic review [55]. This may be related to self-medication, misinformation in the media, science denialism and low access to health services combined with the low cost of ivermectin, and the belief that it has a safe adverse effect profile [55, 56]. In Brazil, for example, the Ministry of Health included the medication in its COVID-19 guidelines. Up to August 2021, estimates from the Brazilian Parliamentary Commission of Inquiry showed that only one pharmaceutical company sold more than 83 million US dollars in ivermectin [57, 58].
Living systematic reviews may have changed this scenario, as they are supposed to incorporate all new relevant evidence as they become available [59]. Nonetheless, with the overwhelming number of studies published in COVID-19 pandemic area, keeping the living reviews updated is a challenge difficult to overcome. This challenge is even more complex when the living reviews propose to assess different comparisons with multiple drugs and evolving SOC. Ivermectin has been the subject of two systematic reviews. The British Medical Journal's living review was last updated not so recently, in April 2021, and suggested a possible reduction in mortality in patients who used ivermectin, when compared to standard of care (RR 0.31 95% CI 0.14-.072). The authors highlighted the fact that data was limited by extremely few events, leading to very serious imprecision, and serious risk of bias [27]. The other living review, by the Pan American Organization, has been recently updated in December 2021, and analyzed the evidence from 14 studies. The authors reported that pooled estimates suggested mortality reduction with ivermectin (RR 0.50 95% IC 0.29-0.87), an effect that was no longer apparent when a subgroup analysis of the three studies classified as low risk of bias was performed (RR 0.96 95%CI 0.58 to 1.59) [60]. These two living reviews were not updated after Elgazzar et al study was retracted. We reckon that another update must also take into account the fact that studies that compared ivermectin with hydroxychloroquine may now be clinically inappropriate, and therefore should not be kept in the pooled analysis studies, as evidence of harm with the use of hydroxychloroquine is now robust [37].
Differently from our results, a recent review has found substantial differences in the results of studies with or without important methodological limitations, highlighting that important benefits associated with ivermectin were based on potentially biased results [61]. Because this review was mainly interested in investigating "bias as a source of inconsistency", as stated in the review title, the authors included in the pooled analysis the aforementioned retracted trial, as well as studies in which ivermectin was compared to other drugs, such as hydroxychloroquine [62, 63]. This approach resulted in pooling data from 5592 patients, more than twice the number of patients included in our review. We strongly believe the best body of evidence now available should not include such studies.
In the present study, the certainty of the evidence on mortality and need for mechanical ventilation was ranked as low (GRADE) due serious concerns about risk of bias and imprecision. Methodological limitations were mainly due to lack of adequate blinding of patients and outcome assessors and high number of losses after randomization. Additional concerns included the fact that some studies were not pre-registered prior to enrolling patients, others had the protocol modified, and the majority did not report the funding source, although three of them were sponsored by pharmaceutical companies. However, as mortality and mechanical ventilation are hard endpoints, and our findings were negative, those sources of bias might not have had a great impact on these outcomes. Instead, it could have been influential for assessing adverse effects. In fact, studies usually assess overall adverse effects, without separating them according to severity, and this is a limitation addressing this outcome. For example, in Okumus et al, while patients who took ivermectin had serious neurological adverse effects which require drug discontinuation, the control group had only nausea, vomiting or two-fold increase in alanine transaminase. None of these side effects were severe enough to require termination of treatment in the control group [51].
Another limitation is the low event rate. Among the 20 studies that could be assessed for mortality, 12 did not have any events. COVID-19 severity varied among the different studies, but the majority of them included patients with mild to moderate disease. Therefore, mortality is expected to be very low in this context. The same applies for mechanical ventilation requirement.
One strength of the present review is to have applied strict methodological criteria, to have performed a broad search in several databases, and to be comprehensive, analyzing not only studies comparing the drug to placebo, but also those in which ivermectin associated with SOC was compared to SOC, in a stratified analysis. This is different from a recent Cochrane review, with the last search performed in May 2021, which selected only placebo-controlled studies. Only two studies were included in the pooled analysis to assess mortality and mechanical ventilation requirement [28]. As aforementioned, a recent network meta-analysis has shown how different statistical approaches (random vs. fixed) lead to different results on the effects of ivermectin on viral clearance [15]. This shows how misleading results may be when inappropriate methods are used. Random effects model is more appropriate in this context, as studies included patients with heterogeneous disease severity and the management was different among the different studies. This could be observed by the several definitions of standard of care. As we should not assume a common effect size to all studies included, and the goal of the analysis is to generalize to a range of scenarios, a random effect approach is recommended [64].
The urgent demand for treatment options for COVID-19 has created the need for randomized clinical trials. Scientists tested several approved drugs against the disease, "throwing every already-approved drug" [8], and the rush to conduct those trials led to conduct and publication of studies with varied quality and important methodological limitations. This "provided fertile ground for even poorly evidenced claims of efficacy to be amplified, both in the scientific literature and on social media" [20]. This is very problematic, as results from studies with high risk of bias were quickly widespread in clinical practice and public policy and SOC were also adopted in a rush in different countries. Even worse, different governments were reluctant to change their protocols after the evidence had shown that some drugs should not be used. In Brazil, for example, a huge polarization and politicization disseminated the SOC supported by the Brazilian President, known as "kit COVID-19" which included hydroxychloroquine, ivermectin and azithromycin [65]. Consequently, studies have included varied and clinically inappropriate options as they defined their comparators as SOC.
For example, months after evidence that hydroxychloroquine may increase the risk of death was available [66] a research protocol of Beltran-Gonzalez et al study was registered, in Brazil [48]. With regard to antibiotics as SOC in patients with COVID without evidence of bacterial pneumonia (for example doxycycline or azithromycin), the World Health Organization advised against the practice in May 2020; still, two studies which started recruitment at that month kept antibiotics in their definition of SOC [51, 67].
There are ethical considerations in this regard, exposing patients to harm and, in the case of antibiotics, contributing to the emergence of antibiotic resistance [68], which is a major issue worldwide. In a letter to the editor about Podder et al, Meneses pointed out important ethical issues. He observed that the authors mentioned approval of their study by the director of the health center, but apparently there was methodological and ethical evaluation by an institutional board [69].
Niaee et al, which included hydroxychloroquine in the comparison arms and was the main study which has shown benefits of ivermectin in clinical outcomes after the aforementioned retractions, has been recently questioned [64]. An editorial note was published in October 2021, reporting concerns about various aspects of the study, including possible problems in the randomization of participants [65]. This raises concerns that flawed evidence in studies with ivermectin or other drugs may impact in systematic reviews.
Our meta-analysis was innovative for using the Living Overview of Evidence database (L.OVE, issued by Epistemonikus) for a comprehensive search, in addition to the traditional search. L.OVE is a digital tool that compiles articles from several databases, including preprint databases, kept up to date through computational algorithms [70]. A previous analysis has shown that it may be more efficient than the traditional search [71]. Consequently, in terms of databases, our search was broader than other systematic reviews on the topic, and the tool made it easier to update the search regularly. Furthermore, we tried to minimize potential biases in the review process by following the methods recommended by the Cochrane Collaboration [30] and set out in our published PROSPERO protocol [31], which provides transparency in the review process. Additionally, we presented a summary of findings table with GRADE results and assessment, in accordance with the new standards required by Cochrane.
In a recent publication, the authors reflect that besides the retracted studies, several other studies which claim a benefit for ivermectin may be similarly fraught. They highlighted unexplainable mismatches between trial registry updates and published patient demographics and timelines that are not consistent with the veracity of the data collection [20]. Therefore, it is of utmost importance for authors of systematic reviews, before following strict methodological criteria, to keep updated with possible new study retractions. Our sensitivity analysis did not show any difference in the point estimates when individual studies were removed, so we do not expect large changes in point estimates.
Thousands of supporters, many of them anti-vaccine activists, have continued to vigorously campaign for ivermectin use, claiming that the real evidence is being ignored. In the context of misinformation infodemics, some sites have published systematic reviews with meta-analysis on the effectiveness of the use of ivermectin in COVID-19 outcomes (https://ivmmeta.com and Home - FLCCC | Front Line COVID-19 Critical Care Alliance (covid19criticalcare.com) Most reviews have not undergone peer review, do not show the criteria used in the selection of RCT's, do not present records and statistical criteria for evaluating the effect and heterogeneity between studies. According to Roman et al (2021), these sites contribute to misinformation of patients, their families, the general population and health professionals who cannot critically analyze scientific studies.
We believe this transparent and thorough summary may contribute to disseminate truthful evidence. Despite the limitation in the analysis of adverse effects, previous studies list some serious adverse effects, such as toxidermias, encephalopathies, confusional disorders [72]. Associated with the lack of clinical benefit, this should be considered when managing patients with COVID-19.
In conclusion, the evidence suggests that ivermectin does not reduce mortality risk and the risk of mechanical ventilation requirement. Although we did not observe an increase in the risk of adverse effects, the evidence is very uncertain regarding this endpoint.