Network meta-analysis from a pairwise meta-analysis design: to assess the comparative effectiveness of oral care interventions in preventing ventilator-associated pneumonia in critically ill patients

In this research, we assessed the usefulness of network meta-analysis (NMA), in creating a hierarchy to define the most effective oral care intervention for the prevention and management of ventilation-associated pneumonia (VAP). We applied NMA to a previously published robust pairwise meta-analysis. Statistical analyses were based on comparing rates of total VAP events between intervention groups and placebo-usual care groups. We synthesized a netgraph, reported the ranking order of the interventions, and summarized output by a forest plot with a reference treatment placebo/usual care. The results of this NMA are from the low and high risk of bias studies, and hence, we strongly recommend not to use findings of this NMA for clinical treatment needs, but based on results of the NMA, we highly recommend for future clinical trials. With our inclusion and exclusion criteria for the NMA, we extracted 25 studies (4473 subjects). The NMA included 16 treatments, 29 pairwise comparisons, and 15 designs. Based on results of NMA frequentist-ranking P scores, tooth brushing (P fixed-0.94, P random-0.89), tooth brushing with povidone-iodine (P fixed-0.90, P random-0.88), and furacillin (P fixed-0.88, P random-0.84) were the best three interventions for preventing VAP. Any conclusion drawn from this NMA should be taken with caution and recommend future clinical trials with the results. NMA appeared to be an effective platform from which multiple interventions reported in disparate clinical trials could be compared to derive a hierarchical assessment of efficacy in VAP intervention.


Introduction
Oral care interventions (OCI) have been recognized as favorably impacting the risk and course of ventilator-associated pneumonia (VAP) in critically ill patients [1]. A range of preventive strategies has been suggested that include the use of topical (rinse) formulations of antimicrobial agents, such as chlorhexidine (CHX) and povidone-iodine (PI), and mechanical cleansing by healthcare providers [1][2][3][4]. Debate persists as to which tactic is most clinically and cost-effective. Several randomized trials (RCTs) have been completed to address this uncertainty [4]. In almost all cases, these RCTs have used a standard clinical trial pairwise design in which a placebo or best care was compared to a test agent or regimen. While this approach provides snapshot outcomes for a specific intervention, it cannot hierarchically assess or rank the efficacy of each in the context of all of the interventions studied.
To address this deficiency, we explored the utility of a novel approach in which network meta-analysis (NMA) was applied to a previously published comprehensive pairwise meta-analysis (PMA) [5]. NMA, also known as multiple treatment comparison or mixed treatment comparison, is a generalization of conventional pairwise meta-analysis. The network statistically combines direct and indirect evidence from trials [6] to yield inter-study intervention comparisons. Besides, NMA expresses the relative effectiveness of interventions among all trials and then rank orders them. Compared to the PMA, the NMA may be a less powerful tool, and conclusions drawn are never more robust than the PMA. We explored the utility of NMA as a means of comparing different OCIs to identify those most effective for mitigating VAP in critically ill patients.

Concepts of network meta-analysis
For clinical trials, conventional PMA typically focuses on pairwise comparisons of an active treatment vs placebo or usual care to assess the test agent's superiority vs a control. If the investigation seeks to compare multiple active agents simultaneously, the sample size must increase, leading to extended accrual times, extraordinary expense, and efficacy assessment challenges.
In contrast, NMA utilizes a multiple comparison methodology which enables the interventions of one trial to be contrasted with the active interventions of other trials, while maintaining the internal randomization of the direct and indirect comparisons. For example, when two active OCIs like chlorhexidine (CHX) and Toothbrushing (Tb) are independently compared for efficacy against a saline control in two different trials then randomised comparison in the trial 1--CHX and saline provides a direct estimate of the treatment effects of CHX and saline, measured on the scale as log odds ratio. We then denote this approach as θ CHX saline direct . Trial 2, provide information on the direct comparison between treatment Tb and saline, denoted by θ Tb saline direct . Then, NMA provide indirect evidence for the comparison of CHX and Tb from the treatment difference CHX and saline and Tb and saline as follows: θ CHX Tb indirect = θ CHX saline direct --θ Tb saline direct and the variance of this association is given by the Var (θ CHX Tb . So as to have the NMA combination for the direct and indirect comparisons, we are assuming that the trials 1 and 2 are independent, the underlying effects are consistent and any differences in the data are due to random error. The NMA now has a consequent network having its integer of total treatments, designs (a design refers to each combination of treatment), pairwise comparisons, and its subsequent statistical inferences of all the included studies.

PMA selection and description
We selected the PMA reported by Hua et al. [5] basis on which to build an NMA and assess its potential clinical meaningfulness. We believe that the report represents a current, comprehensive, and inclusive review of the topic (OCI and VAP) as it was screened from the Cochrane Oral Health

Inclusion and exclusion criteria
To assure consistency, we used the same inclusion and exclusion criteria as Hua et al. [5] Additionally, we excluded feasibility studies and cross-over randomized trials.VAP was defined as pneumonia developing in a critically ill patient who has received mechanical ventilation for at least 48 h and excluded studies in which patients were not critically ill and were not dependent on mechanical ventilation for less than 48 h, or if the patients had an acquired respiratory infection at baseline. We accepted study-described definitions for intervention (test) and control groups. Typically controls of a "placebo" were described as usual care or any oral hygiene intervention care. We accepted studies in which saline was included as a usual care/placebo component but did not include studies in which saline rinsing/swab was described as an active intervention versus placebo-usual care. We noted that saline was used as a most common oral rinse amongst hospitalized patients and so was included as a component of the usual care procedure. In contrast, in clinical trials, saline was used as the most common control drug. Since saline rinsing/swab as an active intervention might affect the NMA analysis and geometry saline-rinsing/ swab as a treatment was excluded. Chlorhexidine trials were stratified based on concentration (0.12%, 0.2%, 1%, and 2%) with each being considered a specific intervention and compared in the network along with other therapies. Careful adaptation of the studies from the pairwise meta-analysis for the indirect comparison in the network meta-analysis ensures the unbiased estimate of the relationships and a comparison of the differences in these two frameworks [7]

Data collection
We obtained data from studies that met our inclusion and exclusion criteria from the PMA [5] using a standardized data collection form. For the NMA data analysis, we calculated the treatment effects (TE) and standard error of the treatment effects (SeTE). Variable TE was determined by comparing the pairwise treatment effect of treatments treat1 (intervention) and treat2 (control) in each study with variable SeTE as the corresponding standard error. When dealing with the multiarm studies in which there were more than two treatment arms, we have included each multi-arm study in the dataset as a two-arm comparison series. Thus, with every comparator in the multi-arm, we have obtained treatment effects and the standard error of the treatment effects for each treatment on the other.

Statistical analysis
Frequentist comparative effectiveness approache for multiple treatment comparison was used [6,[8][9][10][11][12][13]. Statistical analyses were based on comparing rates of total VAP events between the intervention and placebo-usual care groups. For outcomes, odds ratios (OR) with 95% confidence intervals (CI) were calculated using pairwise meta-analysis format. The log odds ratio was used to calculate the TE and SeTE of all the included studies. We used the R package netmeta for the NMA analysis. We reported the random and fixed effects ranking order (P scores) of the treatment effectiveness. For ranking order of the interventions, we used the R package's net ranking function by computing the likelihood of one intervention being the best, second best, and so on for an intervention preventing VAP outcome. Total or generalized heterogeneity of NMA's whole network was quantified using Cochran's Q total statistics test. Cochran's Q total statistics test is the total sum of the heterogeneity and inconsistency statistics that represents the variability between the NMA direct and indirect comparisons. We used Q statistics heterogeneity decomposition function to determine the heterogeneity/inconsistencies between the NMA network designs. Finally, to compare several treatments to a common treatment was done by placing placebo-usual care as a reference treatment is represented with a forest plot. All statistical analyses were performed using R 3.5.1 (R Foundation for Statistical Computing, Vienna, Austria).

Description of the studies
From the Hua et al. study of 38 RCTs (6016 subjects), 25 studies (4473 subjects) met our inclusion criteria (Fig.1). In our cohort, 2254 subjects were randomly assigned to an active OCI and 2219 subjects who were randomly assigned to the placebo or usual care group. The characteristics of the studies are described in Table 1.
The results are based on the low risk and high risk of bias studies, and hence, these results are not recommended for clinical treatment needs. However, we highly recommend using this NMA for future clinical trial purposes. Our intention in combining eligible studies for the NMA was to create a hierarchical ranking order of the treatments. Selecting lowrisk bias studies would guarantee the best evidence--PMA, thus compared the only subset of the treatment comparison. However, it is impossible to determine whether other treatment yielded a more significant benefit. But the inclusion of all studies (low-risk and high-risk studies), we believe that

Evidence used in the NMA
After assuring the comprehensiveness of the studies included in the analysis, we included 25 trials (this comprises the total number of trials combined in the network), 16 treatments (number of total treatments compared in the network), and 29 pairwise comparisons (the pairwise is a combination of the individual trials in the two-arm and three-arm trials). There were 15 designs in the network. Figure 2 shows the graphical representation of the NMA. The nodes' size is proportional to the number of studies evaluating each intervention, and the width/thickness of the edges indicates inverse standard error of the direct treatment comparisons, and the shading indicates threearm study. For example, in Fig. 2, the effectiveness of three different chlorhexidine concentrations (CHX 0.2%, 1%, and 2%)-the difference in thickness/density of connecting edges suggest that CHX 0.2% has superior evidence than CHX 1% based on supporting study data. Importantly, this visual graphical representation of the thickness or density does not indicate the comparison's statistical significance. The most common comparator across all trials was the placebo or usual care arm, which appears the most common node. While most studies were two-arm trials, two, 3-arm trials were included in our network (shaded region in the netgraph). A forest plot (Fig. 3) shows the fixed effects model for each intervention having compared with a reference treatment placebo/usual care. In NMA, the forest plot's importance is to compare several treatments to a common comparator, also called reference or baseline treatment. We have taken placebo/usual care as the reference treatment for our readers to compare and contrast and comprehend the treatments significantly different from placebo/usual care.

Results of heterogeneity and consistency
The heterogeneity statistics of the NMA follows the chisquare distribution, and the chief prerequisite of assessing the variability is to pinpoint studies whose data differ significantly from what the model predicts. Our first aim was to identify the total or generalized heterogeneity of NMA's whole network using Cochran's Q total statistics test and second to determine the heterogeneity/inconsistencies between the NMA network designs.

Total heterogeneity statistics of NMA network
The heterogeneity statistics of the decompose function of netmeta package provided the generalized DerSimonian

Rank order of interventions
The relative effect estimates of the treatments' ranking according to the multiple comparisons are shown in Table 2. Numerals between 0 and 1, with mean 0.5, demonstrate the rank of treatment within the given assortment of competing treatments, where a score of 1 is linked to the best outcome, and a score of 0 is associated with the worst outcome. The hierarchical ranking order of the intervention being the best and worst is introduced by many authors in the Bayesian and frequentist methods [11,12]. Rucker and Schwarzer introduced the ranking order of interventions in the frequentist NMA as P scores, which are analogs to the Bayesian method, surface under the cumulative ranking curve [11]. These values are derived from the effect estimates and their variances. The P scores are based on the frequentist's method point estimates and the standard error of the network meta-analysis estimates under normality assumption and calculated as means of onesided P values [11,12,[14][15][16]. Numerous studies are using ranking order in NMA so as to display a ranking from the network, which is a better way to present the interventions in terms of the effect estimates [11,12,[14][15][16]. Most commonly the effect estimates might get affected with some ambiguity, and we will rarely know in placing a particular trial in the first order or second order. Hence, we classified the ranking first three interventions as best, second three-best interventions as next best, etc. Based on the ranking order, we found that tooth brushing was the most effective intervention for preventing VAP vs placebo or usual treatment which was the worst. The best three interventions were tooth brushing (P score fixed of 0.94, P score random of 0.89), tooth brushing with povidone-iodine (P score fixed of 0.91, P score random 0.88), and furacillin (P score fixed of 0.88, P score random 0.84). CHX of 0.2% concentrations (P score fixed of 0.65, P score random of 0.65) ranked as the second-best interventions in the network along with Biotene (P score fixed of 0.59, P score random 0.54) and potassium permanganate (P score fixed of 0.53, P score random 0.54). At the same time, chlorhexidine 0.2%, a recommended oral care product for preventing VAP in critically ill patients has a P score of 0.65 fixed and 0.65 random.

Discussion
We applied NMA to an existing and robust pairwise metaanalysis (results of multiple comparisons extracted from data first presented by one single meta-analysis) to assess this novel analytics' utility in defining a hierarchical comparison to determine the effectiveness of oral interventions in preventing VAP [5]. Our results suggest that the application of NMA to a conventional meta-analysis provides additional actionable information relative to preventing VAP by comprehensively comparing treatment options otherwise sequestered in pairwise comparisons.
These results have to be taken with caution as the assumptions are based on multiple comparisons. The NMA allows us to presume direct and indirect comparison performed in a structured statistical framework. Although the inferences are from low risk and unclear risk of bias RCTs, estimated network and ranking of treatment are liable to have distinctions as discussed in this NMA vs. pairwise meta-analysis [5]. A potential value of the method is its informative function relative to directing future studies and, in this case, a specific trial assessing preventive interventions for VAP in critically ill patients. The NMA is a comprehendible way of combinations that stem from consolidating a future trial from the network. Consequently, the NMA, when compared to pairwise metaanalysis, weighs the logical possibilities, even within the network while maintaining the internal randomization of the individual trails.
Compared with the published pairwise meta-analysis, the NMA showed a divergent finding concerning the ranking probabilities from the multiple comparisons [3][4][5]. This is the first NMA to report on comparative effectiveness research on oral care intervention for preventing VAP. In contrast to the standard of care where CHX is described as the best oral care intervention to prevent VAP, NMA demonstrated the superiority of tooth brushing or mechanical cleaning. This finding is especially significant, given the recent findings associated with CHX toxicity [17]. When combined with a mouthwash, we also determined that tooth brushing is superior to a mouthwash alone; tooth brushing with PI is superior to any other mouthwash or ranking second in the first threebest interventions. Our findings suggest the superiority of furacillin as a mouthwash in preventing the VAP. Furacillin belongs to the nitrofuran class and is a potent broad spectrum of organic antimicrobial that is effective when applied topically [18]. Given the sparsity of clinical data around its use, additional studies seem warranted.
The PMA showed weak evidence of the PI superior to saline in preventing VAP, and inadequate evidence of the tooth brushing preventing VAP in critically ill patients [5]. According to clinical comparative effectiveness research, the NMA shows tooth brushing alone or tooth brushing along with PI are the best interventions. Clinical trials in the VAP and non-VAP area suggest the mechanical cleaning/tooth brushing's effectiveness in mitigating the oral microbial pooling among hospitalized patients [19][20][21][22]. Conversely, the methods involved, and care provider's timing also plays a significant role [19][20][21][22]. Mechanical plaque removal is an effective way to reduce the oral microbial pooling in hospitalized patients. However, the evidence for the effectiveness of tooth brushing in reducing VAP or non-VAP is very minimal.
There is a lack of comparative effectiveness research and vagueness with regard to OCI in preventing VAP among critically ill patients, and NMA is never performed. While our results support the usefulness of NMA as a tool to optimize collective analyses of meta-analyses for comparative effectiveness research, it does have limitations. For justifying the rationality of findings and to minimalize error, NMA is designed methodically and conducted carefully. Transporting the high-quality systematic search and search results of the Hua et al. study [5], we established our own inclusion and exclusion criteria for building NMA network. We argue that this way, we pragmatically compared the PMA to the NMA and reflected on its comparative effectiveness research. While observing fewer research on OCI on preventing the VAP in critically ill patients after the Hua et al. study, using the Hua et al. research supplemented NMA construction, which defends the judicious literature search and assesses the risk of bias and quality of evidence. Nevertheless, challenges of the NMA persist when comparing the studies with low and unclear-risk biases. In summary, this research accomplishes to provide comparative effectiveness of OCIs in preventing VAP in critically ill patients when combining direct and indirect evidence by having a transitivity assumption that studies are independent and underlying effects are somewhat consistent.

Limitations
Compared to the Hua et al. study, authors have done the subgroup analysis based on different interventions, we have combined all the interventions in the NMA for hierarchical ordering--this might reduce the robustness of the NMA. However, we believe this way we combined different direct and indirect intervention in the NMA and those ranking order could be used judiciously in future clinical trials.

Conclusions
As meta-analysis is considered the epitome of the evidence-based clinical medicine, NMA is an extension positioned in this framework. Given the challenges of the proof of concept of existing oral care intervention in preventing VAP, and lack of head-to-head robust trials of the best available treatment modalities, this approach is exceptional. We followed stern assumptions and standardization, and our study cohort was based on the largest pairwise meta-analysis of oral care intervention in preventing the VAP. The transparency, reproducibility, and detailed documentation of our finding can be appropriately appraised. According to the NMA outcome, toothbrushing alone or toothbrushing along with a potent antiseptic mouthwash povidone-iodine was related to the highest response rate in preventing VAP in critically ill patients followed by furacillin and chlorhexidine 0.2%, respectively.

Declarations
Conflict of interest The authors declare that they have no conflict of interest.
Ethics approval and consent to participate NA Consent for publication NA Availability of data and materials NA Funding No funding received for this manuscript.