Application of Network Meta-Analysis to Assess the Comparative Effectiveness of Oral Care Interventions in Preventing Ventilator Associated Pneumonia in Critically Ill Patients.

Background In this research, we assessed the efficacy of a novel analytic, network metanalysis (NMA), in creating a hierarchy to define the most effective oral care intervention (OCI) for the prevention and management of ventilation-associated pneumonia (VAP). Methods We applied NMA to a previously published robust pairwise meta-analysis (PMA). Statistical analyses were based on comparing rates of total VAP events between intervention groups and placebo-usual care groups. We synthesized a netgraph, reported ranking order of the treatment and summarized our output by a forest plot with a reference treatment placebo/usual care. Results With our inclusion and exclusion critiera for the NMA we extracted 25 studies (4473 subjects). The NMA included 16 treatments, 29 pairwise comparisons and 15 designs. Based on the results of multiple comparisons with frequentist ranking probability P scores, tooth brushing (P score fixed of 0.9353, P score random of 0.8892), toothbrushing with povidone iodine (P score fixed of 0.9091, P score random 0.8801), and furacillin (P score fixed of 0.8798, P score random 0.8358) were the best three interventions for preventing VAP. of 0.5520. The results show that there is moderate heterogeneity in the NMA network, and considerably very less heterogeneity within designs and between designs.


Abstract
Background In this research, we assessed the efficacy of a novel analytic, network metanalysis (NMA), in creating a hierarchy to define the most effective oral care intervention (OCI) for the prevention and management of ventilation-associated pneumonia (VAP).
Methods We applied NMA to a previously published robust pairwise meta-analysis (PMA). Statistical analyses were based on comparing rates of total VAP events between intervention groups and placebo-usual care groups. We synthesized a netgraph, reported ranking order of the treatment and summarized our output by a forest plot with a reference treatment placebo/usual care.
Results With our inclusion and exclusion critiera for the NMA we extracted 25 studies (4473 subjects).
The NMA included 16 treatments, 29 pairwise comparisons and 15 designs. Based on the results of multiple comparisons with frequentist ranking probability P scores, tooth brushing (P score fixed of 0.9353, P score random of 0.8892), toothbrushing with povidone iodine (P score fixed of 0.9091, P score random 0.8801), and furacillin (P score fixed of 0.8798, P score random 0.8358) were the best three interventions for preventing VAP.
Conclusion NMA appeared to be an effective platform from which multiple interventions reported in disparate clinical trials could be compared to derive a hierarchical assessment of efficacy in the intervention of VAP. According to the NMA outcome, toothbrushing alone or toothbrushing along with a potent antiseptic mouthwash povidone iodine was related to the highest response rate in preventing VAP in critically ill patients followed by furacillin and chlorhexidine 0.2%, respectively.

Background
Oral care interventions (OCI) have been recognized as favorably impacting the risk and course of ventilator-associated pneumonia (VAP) in critically ill patients. [1] A range of preventive strategies have been suggested that include the use of topical (rinse) formulations of antimicrobial agents, such as chlorhexidine (CHX) and povidone iodine (PI), and/or mechanical cleansing by healthcare providers. [1][2][3][4] Debate persists as to which tactic is most clinically-and cost-effective. A number of randomized trials (RCTs) have been completed to address this uncertainty. [4] In almost all cases, these RCTs have used a standard clinical trial pairwise design in which a placebo or best care was compared to a test agent or regimen. While this approach provides snapshot outcomes for a specific intervention, it lacks the ability to hierarchically assess or rank the efficacy of each in the context of all of the interventions studied.
To address this deficiency, we explored the utility of a novel approach in which network meta-analysis (NMA) was applied to a previously published comprehensive pairwise meta-analysis (PMA). [5] NMA, also known as multiple treatment comparison or mixed treatment comparison, is a method of generalization of conventional pairwise meta-analysis whereby the network statistically combines direct and indirect evidence from trials [7] to yield inter-study intervention comparisons. In addition, NMA expresses relative effectiveness of interventions among all trials and then rank orders them. We explored the utility of NMA as a means of comparing different OCIs with the objective of identifying those most effective for mitigating VAP in critically ill patients.

Concepts Of Network Meta-analysis
For clinical trials, conventional PMA typically focus on pairwise comparisons of an active treatment vs. a placebo or usual care with the objective of assessing superiority of the test agent vs. a control. If the investigation seeks to compare multiple active agents simultaneously, the sample size must increase leading to extended accrual times, extraordinary expense and efficacy assessment challenges.
In contrast NMA utilizes a multiple comparison methodology which enables the interventions of one trial to be contrasted with the active interventions of other trials, while maintaining the internal randomization of the direct and indirect comparisons. [ figure 1]. For example, when two active OCIs like chlorhexidine ( CHX) and Toothbrushing (Tb) are independently compared for efficacy against a saline control in two different trials then randomised comparison in the trial 1, CHX and saline provides a direct estimate of the treatment effects of CHX and Saline, measured on the scale as log odds ratio. We then denote this approach as θ CHX  independent, the underlying effects are consistent and any differences in the data are due to random error. The NMA now has a consequent network having its integer of total treatments, designs (a design refers to each combination of treatment), pairwise comparisons and its subsequent statistical inferences of all the included studies.

PMA selection and description
We selected the pairwise PMA reported by Hua et al [5] basis on which to build an NMA and assess its potential clinical meaningfulness We believe that the report represents a current, comprehensive, and

Inclusion and exclusion criteria
To assure consistency, we used the same inclusion and exclusion criteria as Hua et al. VAP was defined as pneumonia developing in a critically ill patients who has received mechanical ventilation for at least 48 hours and excluded studies in which patients were not critically ill and were not dependent on mechanical ventilation for less than 48 hours, or if the patients had an acquired respiratory infection at baseline. We accepted study-described definitions for intervention (test) and control groups. Typically controls of a "placebo" were described as usual care or any oral hygiene intervention care. We accepted studies in which saline was included as a component of usual care/placebo but did not include studies in which saline rinsing/swab was described as an active intervention versus a placebo-usual care. We noted that amongst hospitalized patients, saline was used as a most common oral rinse and so was included as a component of the usual care procedure, while in clinical trials saline was used as a most common control drug. Since the use of saline rinsing/swab as an active intervention might affect the NMA analysis and geometry salinerinsing/swab as a treatment was excluded. We also excluded feasibility studies and cross over randomised design trials. Chlorhexidine trials were stratified based on concentration (0.12%, 0.2%, 1%, and 2%) with each being considered as a distinct intervention and compared in the network along with other therapies.

Data collection
We obtained data from studies which met our inclusion and exclusion criteria from the PMA [5] by a standardized data collection form. For the purpose of the NMA data analysis, we calculated the treatment effects (TE) and standard error of the treatment effects (SeTE). Variable TE, which was determined by comparing the pairwise treatment effect of treatments treat1 (intervention) and treat2 (control) in each study with variable SeTE as the corresponding standard error. When dealing with the multi-arm studies in which there were more than two treatment arms, we have included each multiarm study in the dataset as a series of two-arm comparison. Thus, with every comparator in the multiarm we have obtained treatment effects and the standard error of the treatment effects for each treatment on the other.

Statistical analysis
Frequentist methods of comparative effectiveness approach with multiple treatment comparison [6][7][8][9][10][11] were used. Statistical analyses were based on comparing rates of total VAP events between the intervention group and the placebo-usual care group.
For outcomes, odds ratios (ORs) with 95% confidence intervals (CIs) were calculated using pairwise meta-analysis format and log odds ratio was used to calculate the TE and SeTE of all the included studies. We used the R package netmeta for the NMA analysis.
We reported the random and fixed effects ranking order (P scores) of the treatment effectiveness. For ranking order of the interventions, we used net ranking function of R package by computing likelihood of one intervention being the best, second best and so on for an intervention preventing VAP  2] In our cohort there were 2254 subjects who were randomly assigned to an active OCI and 2219 subjects who were randomly assigned to the placebo or usual care group. The basic characteristics of studies are described in Table 1.  suggest that CHX 0.2% has superior evidence than CHX 1% based on supporting study data.
Importantly, this visual graphical representation of the thickness or density does not indicate the statistical significance of the comparison. The most common comparator across all trials was the placebo or usual care arm which appears as the network's most common node. While the majority of studies were two-arm trials, two, 3-arm trials were included in our network (shaded region in the netgraph). A forest plot [ Figure 4] shows the fixed effects model for each intervention having compared with a reference treatment placebo/usual care. In NMA, the forest plot's importance is to compare a number of treatments to a common comparator also called reference or baseline treatment. We have taken placebo/usual care as the reference treatment for our readers to compare and contrast and to comprehend the treatments are significantly different to placebo/usual care.

Results of heterogeneity and consistency
The heterogeneity statistics of the NMA follows the Chi -square distribution, and the chief prerequisite of assessing the variability is to pinpoint studies whose data differ significantly from what the model predicts. Our first aim was to identify the total or generalized heterogeneity of NMA's whole network

Rank order of interventions
The relative effect estimates of the ranking of the treatments according to the multiple comparisons are shown in the Table 2. Numerals between 0 and 1, with mean 0.5, demonstrate the rank of a treatment within the given assortment of competing treatments, where a score of 1 is linked to best outcome and a score of 0 is associated with worst outcome. Hierarchical ranking order of the intervention being the best and worst, are introduced by many authors in the Bayesian and frequentist methods. [10,11] Rucker and Schwarzer introduced ranking order of interventions in the frequentist NMA as P scores, which are analogues to the Bayesian method, surface under the cumulative ranking curve. [10] These values are derived from the effect estimates and their variances. The P scores are based on the frequentist's method point estimates and the standard error of the network meta-analysis estimates under normality assumption and calculated as means of onesided p-values. [10,11,13,14,15]. Numerous studies are using ranking order in NMA so as to display a ranking from the network, which is a better way to present the interventions in terms of the effect estimates. [10,11,13,14,15] Most commonly the effect estimates might get affected with some ambiguity and we will rarely know in placing a particular trial in the first order or second order. Hence, we classified the ranking first three interventions as best, second three-best interventions as next best, and so on. Based on the ranking order, we found that tooth brushing was the most effective intervention for preventing VAP vs. placebo or usual treatment which was the worst. The best three interventions were tooth brushing (P score fixed of 0.9353, P score random of 0.8892), tooth brushing with povidone-iodine (P score fixed of 0.9091, P score random 0.8801) and furacillin (P score fixed of 0.8798, P score random 0.8358). CHX of 0.2% concentrations (P score fixed of 0.6531, P score random of 0.6478) ranked as the second-best interventions in the network along with biotene (P score fixed of 0.5954, P score random 0.5400) and potassium permanganate (P score fixed of 0.5274, P score random 0.5446). While chlorhexidine 0.2%, a recommended oral care product for preventing VAP in critically ill patients has a P score of 0.6531 fixed and 0.6478 random.

Discussion
We applied NMA to an existing and robust pairwise meta-analysis to assess the utility of this novel analytic in defining a hierarchical comparison to determine the effectiveness of oral interventions in preventing VAP. [5] Our results suggest that the application of NMA to a conventional meta-analysis provides additional actionable information relative to preventing VAP by comprehensively comparing treatment options otherwise sequestered in pairwise comparisons.
These results have to be taken with caution as the assumptions are based on the results of multiple comparisons. This novel technique allows us to presume direct and indirect comparison performed in a structured statistical framework. Although the inferences are from low risk and unclear risk of bias RCTs, estimated network and ranking of treatment are thus liable to have distinctions as discussed in this NMA and previous pairwise meta-analysis. [5] A potential value of the method is its informative function relative to directing future studies and, in this case, a specific trial assessing preventive interventions for VAP in critically ill patients. The NMA is a comprehendible way of combinations which stem a possibility of consolidating a future trial from the network. Consequently, the NMA when compared to pairwise meta-analysis, weighs the logical possibilities, even within the network while maintaining the internal randomization of the individual trails.
In comparison with the published pairwise meta-analysis, the NMA showed a divergent finding with respect to the ranking probabilities from the multiple comparisons. [3][4][5] This is the first NMA in this regard to report on comparative effectiveness research on oral care intervention for preventing VAP.
In contrast to standard of care where CHX is described as the best oral care intervention to prevent VAP, NMA demonstrated the superiority of tooth brushing or mechanical cleaning. This finding is especially significant given the recent findings associated with CHX toxicity. [16] We also determined toothbrushing when combined with a mouthwash is superior compared to a mouthwash alone; toothbrushing with PI is superior to any other mouthwash or ranking second in the first three-best interventions. This is the first time showing the superior benefit of the furacillin as a mouthwash in preventing the VAP. Furacillin belongs to the nitrofuran class and is a potent antimicrobial organic compound. It is efficient against gram-positive bacteria and gram-negative bacteria. Studies shows furacillin effective against many bacterial and fungal entities when applied topically. [17] Although there aren't many studies on this intervention, this network warrants a possible pilot trial. The PMA showed weak evidences of the PI superior than saline in preventing VAP, and inadequate evidences of the toothbrushing preventing VAP in critically ill patients. [

Conclusions
As meta-analysis is considered epitome of the evidence-based clinical medicine, NMA is an extension positioned in this framework. Given the challenges of the proof of concept of existing oral care intervention in preventing VAP, and lack of head to head robust trials of the best available treatment modalities, this approach is exceptional. We followed stern assumptions and standardization and our study cohort was based on the largest pairwise meta-analysis of oral care intervention in preventing the VAP. The transparency, reproducibility and detailed documentation of our finding can be appropriately appraised. According to the NMA outcome, toothbrushing alone or toothbrushing along with a potent antiseptic mouthwash povidone iodine was related to the highest response rate in preventing VAP in critically ill patients followed by furacillin and chlorhexidine 0.2%, respectively.    Netgraph of the oral care intervention for the prevention of ventilation associated pneumonia.

Figure 4
Forest plot of the fixed effect network meta-analysis of the oral care intervention for the prevention of ventilation associated pneumonia, when placing the placebo-usual care as a reference treatment.

Supplementary Files
This is a list of supplementary files associated with this preprint. Click to download.