This review sought to provide a detailed overview of the actual practice of the statistical analysis of safety data in the unique setting of drug trials for the preventions of malaria in pregnancy as reflected published literature. Our work has shown that there is limited reporting of statistical analyses of safety data, at the end of the trial, in these published reports. Our findings are useful to advance the development of standardized guidelines for safety data statistical analysis in analysis in antimalarial drugs in pregnancy trials and related fields. Such guidelines will not replace but rather complement the CONSORT guidelines that are general (i.e. not providing specific statistical methods in analysing harms in RCTs). To our knowledge, this is the first paper to review statistical methods for safety data in antimalarial drugs in pregnancy.
Descriptive methods were commonly used to summarize safety data. Our review found that each clinical trial used at least one descriptive method to summarize safety data. Univariate statistical methods such as Chi-square or Fishers exact tests were used in two thirds of the articles reviewed. Such descriptive statistics and univariate statistical inference ignored useful information such as variability in follow-up time, missing data and correlation (for those trials which had their multiple safety outcomes repeatedly measured). Hence there was inefficient data use during analysis that may lead to a loss of useful information for improved and informative conclusions. Although a third of the reviewed trials attempted to use crude incidence, they failed to adequately account for individual patient follow-up-time and potential confounders.
All trials dichotomized at least a single continuous clinical laboratory safety outcome (i.e. where AE was defined based on standard cut-off points for adult toxicity). Although this aids in providing time-specific drug safety status and easy interpretation, the dichotomized outcome may miss some information on the magnitude of the temporal changes, overtime during the trial. The information loss may lead to reduction in statistical power to detect safety signal if it exists. Valid longitudinal methods (used without restriction on cut-off points) can address the information loss by exploiting potential within-subject correlations for the repeated clinical laboratory measurements. (18–20). Furthermore, the longitudinal methods can provide the basis for developing improved cut-off points tailored to pregnant women in malaria-specific settings. To ensure improved uptake of such methods, future work needs to strive towards making the results from the longitudinal methods feasibly interpretable to the medical practitioners.
Only three studies appropriately used multivariable statistical methods. Since use of crude methods can lead to false findings, multivariable methods (e.g Poisson regression) help in adjusting for patient characteristics (e.g. confounders such as age). Of specific interest in this review, the Poisson model was more suitable in the context of rare AEs which usually have low event rates (20, 21). Since Poisson regression assumes a constant rate of occurrence of a rare event, it is not ideal for other multiple transient AEs that were common or recurred and would vary in occurrence overtime (22). Alternatively, mixed effects models could be considered to characterize the safety events over time since they capture patient-specific effects (23, 24). Whenever time to AE occurrence information is available, survival analysis models may also be preferred to characterize the time to AE occurrence. For recurrent safety events, that may induce dependence, methods that extend the Cox regression model may be preferred; such models include survival mixed effects models (e.g. frailty models) (25).
Almost half of the reviewed trials did not explicitly define the population on which the safety analysis was based. If per protocol analysis is used to address non-adherence there is potential selection bias since it destroys the balance due to randomization. Although CONSORT recommends ITT, as an alternative, for analysis of safety endpoints, non-adherence cannot be explicitly addressed with ITT approach since it ignores dropouts, withdraw or loss-to-follow up for various reasons including safety concerns; ITT-based inference ignores causal effect of the actual treatment received (26). Patient withdrawal or dropout due to adverse events can induce informative censoring useful in quantifying antimalarial drug safety. For example, if a patient withdraws due to vomiting after taking an antimalarial drug, their obstetric efficacy outcomes such as birth weight may appear as missing data. In the context of antimalarial drug for malaria prevention, even mild AE can lead to drug non-adherence. Since the patient has no disease symptoms, they would judge it less costly for them to discontinue the drug than continue experiencing AEs. Hence, inclusion of information on treatment/trial completion status in relation to antimalarial safety, would enrich development of the safety profile of antimalarial drug in pregnancy. Although study completion status, antimalarial drug safety and missing data may be interlinked, missing data received limited attention such that in the few trials that considered efficacy missing data did not explicitly explore the potential link. Studying such complex associations requires statistical methods that can appropriately estimate the pathway from the antimalarial chemoprevention to study completion. Advantageously, methods based on causal inference framework, such as mediation analysis (27–30) could be adapted/extended to assess the influence of the adverse events on non-adherence in RCTs.
Despite the about three quarters of the trials reporting p-values after comparing safety outcomes by treatment arms, only about half of the reviewed trials adhered to International Harmonisation Conference Guideline E9 in reporting of confidence intervals in quantifying the safety effect size (31). Use of confidence interval aids in interpretation of results by providing a measure of precision. Furthermore, graphical displaying of safety data to aid in summarizing of safety data was inadequate. Graphs on safety data have a greater ability to convey insight about patterns, trends, or anomalies that may signal potential safety issues compared to presentation of such data in tabular form only(32). For example, the graphs could help to visualize frequency and changes in adverse events over time by treatment arm. The graphs could further help in assessing assumptions for some statistical methods.
Over three-quarters of the reviewed trials were designed as superiority trials based on efficacy outcomes. Although the statistical approach for safety assessment was mainly on superiority hypotheses (for both the superiority and non-superiority trials), clinical and statistical justification of assessing safety based on superiority hypotheses may be invalid. Superiority hypotheses concentrate on the absence of difference in drug safety effect/risk between or across the treatment arms which may be challenging (15). For example, when comparing high AE incidences, non-significant difference (when using a superiority hypothesis) would not necessarily translate to a conclusion that a drug is safe and well-tolerated since sometimes all compared treatment arms may have high AE incidence. Perhaps, drug safety evaluation should strive to prove that there is no risk beyond a protocol-defined/hypothesized priori clinical safety margin (i.e. no excessive safety risk). Based on our findings, we encourage researchers to consider defining safety margins in safety assessment of antimalarial drugs. Since safety is mostly a secondary outcome, it is not straight forward on how to define a non-inferiority margin and the appropriate analysis population. Currently, it is still unclear and debatable how to implement this, such that further research is needed (4).
Interestingly, we observed that over half of the trials were open-label which may influence physician’s clinical safety assessment on a patient and patient`s reporting of AEs based on their expectations since they know the treatment assigned. Appropriate reporting of the AEs would be guided by DSMBs right from early stages of the trial. However, availability of DSMBs in over three-quarters of the trials did not translate to improved reporting and analysis. Therefore, it is important for DSMB member to understand improved analysis approaches for AEs since they influence on how safety data is analysed and reported.
This review agrees with other similar publications focusing on drug safety assessment in clinical trials that have noted the need for further improvement in the statistical analysis of the safety data (8, 33). This review concurs with a recent review that has noted that inappropriate handling of multiple test is prevalent, although their review focussed on four high impact journals, AE in general and a short time of review period (34). Issues raised in this review include time-dependence of AEs, informative censoring due to discontinuation of treatment because of AEs, safety graphs, and repeated occurrence of AEs and multivariate longitudinal structure of laboratory data that yields complex correlation. This is an ongoing work whereby further analysis will be explored to address the identified statistical issues above.
The application of the systematic review protocol in gathering the current practice in our context is more reliable since it exhaustively identified the published antimalarial drug clinical trials in pregnancy for studied period. However, our review covered only the last decade of publications and may have missed studies published in other languages or that did not appear in our search. Because the trials reported in the publications spanned for a decade, it was difficult to assess temporal trends. This review represents the most comprehensive review of safety data analysis practice for this important indication.