DOI: https://doi.org/10.21203/rs.3.rs23653/v1
Background In this research, we assessed the efficacy of a novel analytic, network metanalysis (NMA), in creating a hierarchy to define the most effective oral care intervention (OCI) for the prevention and management of ventilationassociated pneumonia (VAP).
Methods We applied NMA to a previously published robust pairwise metaanalysis (PMA). Statistical analyses were based on comparing rates of total VAP events between intervention groups and placebousual care groups. We synthesized a netgraph, reported ranking order of the treatment and summarized our output by a forest plot with a reference treatment placebo/usual care.
Results With our inclusion and exclusion critiera for the NMA we extracted 25 studies (4473 subjects). The NMA included 16 treatments, 29 pairwise comparisons and 15 designs. Based on the results of multiple comparisons with frequentist ranking probability P scores, tooth brushing (P score fixed of 0.9353, P score random of 0.8892), toothbrushing with povidone iodine (P score fixed of 0.9091, P score random 0.8801), and furacillin (P score fixed of 0.8798, P score random 0.8358) were the best three interventions for preventing VAP.
Conclusion NMA appeared to be an effective platform from which multiple interventions reported in disparate clinical trials could be compared to derive a hierarchical assessment of efficacy in the intervention of VAP. According to the NMA outcome, toothbrushing alone or toothbrushing along with a potent antiseptic mouthwash povidone iodine was related to the highest response rate in preventing VAP in critically ill patients followed by furacillin and chlorhexidine 0.2%, respectively.
Oral care interventions (OCI) have been recognized as favorably impacting the risk and course of ventilatorassociated pneumonia (VAP) in critically ill patients. [1] A range of preventive strategies have been suggested that include the use of topical (rinse) formulations of antimicrobial agents, such as chlorhexidine (CHX) and povidone iodine (PI), and/or mechanical cleansing by healthcare providers. [1–4] Debate persists as to which tactic is most clinically and costeffective. A number of randomized trials (RCTs) have been completed to address this uncertainty. [4] In almost all cases, these RCTs have used a standard clinical trial pairwise design in which a placebo or best care was compared to a test agent or regimen. While this approach provides snapshot outcomes for a specific intervention, it lacks the ability to hierarchically assess or rank the efficacy of each in the context of all of the interventions studied.
To address this deficiency, we explored the utility of a novel approach in which network metaanalysis (NMA) was applied to a previously published comprehensive pairwise metaanalysis (PMA). [5] NMA, also known as multiple treatment comparison or mixed treatment comparison, is a method of generalization of conventional pairwise metaanalysis whereby the network statistically combines direct and indirect evidence from trials [7] to yield interstudy intervention comparisons. In addition, NMA expresses relative effectiveness of interventions among all trials and then rank orders them. We explored the utility of NMA as a means of comparing different OCIs with the objective of identifying those most effective for mitigating VAP in critically ill patients.
For clinical trials, conventional PMA typically focus on pairwise comparisons of an active treatment vs. a placebo or usual care with the objective of assessing superiority of the test agent vs. a control. If the investigation seeks to compare multiple active agents simultaneously, the sample size must increase leading to extended accrual times, extraordinary expense and efficacy assessment challenges.
In contrast NMA utilizes a multiple comparison methodology which enables the interventions of one trial to be contrasted with the active interventions of other trials, while maintaining the internal randomization of the direct and indirect comparisons. [figure 1]. For example, when two active OCIs like chlorhexidine ( CHX) and Toothbrushing (Tb) are independently compared for efficacy against a saline control in two different trials then randomised comparison in the trial 1, CHX and saline provides a direct estimate of the treatment effects of CHX and Saline, measured on the scale as log odds ratio. We then denote this approach as θ _{CHX Saline} ^{direct}. Trial 2, provide information on the direct comparison between treatment Tb and Saline, denoted by θ _{Tb Saline} ^{direct}. Then NMA provide indirect evidence for the comparison of CHX and Tb from the treatment difference CHX and Saline and Tb and Saline as follows:
θ _{ CHX Tb } ^{ indirect } = θ _{CHX Saline} ^{direct}  θ _{Tb Saline} ^{direct} and the variance of this association is given by the Var (θ _{CHX Tb}^{indirect}) = Var(θ _{CHX Saline} ^{direct} ) + Var (θ _{Tb Saline} ^{direct} ). So as to have the NMA combination for the direct and indirect comparisons, we are assuming that the trial 1 and 2 are independent, the underlying effects are consistent and any differences in the data are due to random error. The NMA now has a consequent network having its integer of total treatments, designs (a design refers to each combination of treatment), pairwise comparisons and its subsequent statistical inferences of all the included studies.
We selected the pairwise PMA reported by Hua et al [5] basis on which to build an NMA and assess its potential clinical meaningfulness We believe that the report represents a current, comprehensive, and inclusive review of the topic (OCI and VAP) as it was screened from the Cochrane Oral Health’s Trials Register (to 17 December 2015), the Cochrane Central Register of Controlled Trials (CENTRAL) (the Cochrane Library, 2015, Issue 11), MEDLINE Ovid (1946 to 17 December 2015), Embase Ovid (1980 to 17 December 2015), LILACS BIREME Virtual Health Library (1982 to 17 December 2015), CINAHL EBSCO (1937 to 17 December 2016), Chinese Biomedical Literature Database (1978 to 14 January 2013), China National Knowledge Infrastructure (1994 to 14 January 2013), Wan Fang Database (January 1984 to 14 January 2013) and VIP Database (January 2012 to 4 May 2016).
To assure consistency, we used the same inclusion and exclusion criteria as Hua et al. VAP was defined as pneumonia developing in a critically ill patients who has received mechanical ventilation for at least 48 hours and excluded studies in which patients were not critically ill and were not dependent on mechanical ventilation for less than 48 hours, or if the patients had an acquired respiratory infection at baseline. We accepted studydescribed definitions for intervention (test) and control groups. Typically controls of a “placebo” were described as usual care or any oral hygiene intervention care. We accepted studies in which saline was included as a component of usual care/placebo but did not include studies in which saline rinsing/swab was described as an active intervention versus a placebousual care. We noted that amongst hospitalized patients, saline was used as a most common oral rinse and so was included as a component of the usual care procedure, while in clinical trials saline was used as a most common control drug. Since the use of saline rinsing/swab as an active intervention might affect the NMA analysis and geometry salinerinsing/swab as a treatment was excluded. We also excluded feasibility studies and cross over randomised design trials. Chlorhexidine trials were stratified based on concentration (0.12%, 0.2%, 1%, and 2%) with each being considered as a distinct intervention and compared in the network along with other therapies.
We obtained data from studies which met our inclusion and exclusion criteria from the PMA [5] by a standardized data collection form. For the purpose of the NMA data analysis, we calculated the treatment effects (TE) and standard error of the treatment effects (SeTE). Variable TE, which was determined by comparing the pairwise treatment effect of treatments treat1 (intervention) and treat2 (control) in each study with variable SeTE as the corresponding standard error. When dealing with the multiarm studies in which there were more than two treatment arms, we have included each multiarm study in the dataset as a series of twoarm comparison. Thus, with every comparator in the multiarm we have obtained treatment effects and the standard error of the treatment effects for each treatment on the other.
Frequentist methods of comparative effectiveness approach with multiple treatment comparison [6–11] were used. Statistical analyses were based on comparing rates of total VAP events between the intervention group and the placebousual care group.
For outcomes, odds ratios (ORs) with 95% confidence intervals (CIs) were calculated using pairwise metaanalysis format and log odds ratio was used to calculate the TE and SeTE of all the included studies. We used the R package netmeta for the NMA analysis.
We reported the random and fixed effects ranking order (P scores) of the treatment effectiveness. For ranking order of the interventions, we used net ranking function of R package by computing likelihood of one intervention being the best, second best and so on for an intervention preventing VAP outcome. Total or generalized heterogeneity of NMA’s whole network was quantified using Cochran’s Q _{total} statistics test. Cochran’s Q _{total} statistics test is the total sum of the heterogeneity and inconsistency statistics, that represents the variability between the NMA direct and indirect comparisons. And for determining the heterogeneity/inconsistencies between designs of the NMA network, we used Q_{statistics} heterogeneity decomposition function. Finally, to compare a number of treatments to a common treatment was done by placing placebousual care as a reference treatment is represented with a forest plot. All statistical analyses were performed using R Studio, Version 1.1.456 (RStudio: Integrated Development for RStudio. RStudio, Inc., Boston, MA).
From the Hua et al study of 38 RCTs (6016 subjects), 25 studies (4473 subjects) met our inclusion criteria. [figure 2] In our cohort there were 2254 subjects who were randomly assigned to an active OCI and 2219 subjects who were randomly assigned to the placebo or usual care group. The basic characteristics of studies are described in Table 1.
REFERENCE, YEAR  number of participants  Intervention  Control  Study type 

BELLISMORODRIGUES2009  133  Chlorhexidine (0.12%)  Placebo/usual  Twoarm 
BERRY2013  271  Bicarbonate rinse + Toothbrushing  Placebo/usual + Toothbrushing  Threearm 
BERRY2013  265  Listerine + Toothbrushing  Placebo/usual + toothbrushing  Threearm 
BERRY2013  260  Listerine + Toothbrushing  Bicarbonate rinse + Toothbrushing  Threearm 
CABOV2010  40  Chlorhexidine (0.2%)  Placebo/usual  Twoarm 
DERISO 1996  353  Chlorhexidine (0.12%)  Placebo/usual  Twoarm 
FENG2012  139  PovidoneIodine  Placebo/usual  Threearm 
FENG2012  136  Furacillin  PovidoneIodine  Threearm 
FENG2012  133  Furacillin  Placebo/usual  Threearm 
FOURRIER2000  58  Chlorhexidine 0.2%  Placebo/usual  Twoarm 
FOURRIER2005  228  Chlorhexidine (0.2%)  Placebo/usual  Twoarm 
GRAP2011  39  Chlorhexidine (0.12%)  Placebo/usual  Twoarm 
JACOMO2011  160  Chlorhexidine (0.12%)  Placebo/usual  Twoarm 
KOEMAN2006  257  Chlorhexidine (2%)  Placebo/usual  Twoarm 
KUSAHARA2012  96  Chlorhexidine (0.12%) + Toothbrushing  Placebo/usual  Twoarm 
LONG2012  61  Tooth brushing + PovidoneIodine  PovidoneIodine  Twoarm 
LORENTE2012  436  Chlorhexidine (0.12%) + Toothbrushing  Chlorhexidine (0.12%)  Twoarm 
MEINBERG2012  52  Chlorhexidine (2%) + Toothbrushing  Placebo/usual  Twoarm 
OZCAKA2012  61  Chlorhexidine (0.2%)  Placebo/usual  Twoarm 
PANCHABAI2009  171  Chlorhexidine (0.2%)  Potassium permanganate  Twoarm 
POBO2009  147  Chlorhexidine (0.12%) + Toothbrushing  Chlorhexidine (0.12%)  Twoarm 
SCANNAPIECO2009  146  Chlorhexidine (0.12%) + Toothbrushing  Placebo/usual  Twoarm 
SEBASTIN2012  86  Chlorhexidine (1%)  Placebo/usual  Twoarm 
SEGUIN2006  67  PovidoneIodine  Placebo/usual  Twoarm 
SEGUIN 2014  150  PovidoneIodine  Placebo/usual  Twoarm 
STEFANSCU2013  41  Biotene  Placebo/usual  Twoarm 
TANTIPONG2008  110  Chlorhexidine (2%) + Toothbrushing  Placebo/usual  Twoarm 
YAO2011  53  Tooth brushing  Placebo/usual  Twoarm 
ZHAO2012  324  Triclosan  Placebo/usual  Twoarm 
After assuring the comprehensiveness of the studies included in the analysis we included 25 trials (this comprises the total number of trials combined in the network), 16 treatments (number of total treatments compared in the network), and 29 pairwise comparisons (the pairwise is combination of the individual trials in the twoarm and threearm trials) and there were 15 designs in the network [figure 2]. Figure 3 shows the graphical representation of the NMA. The size of the nodes is proportional to the number of studies evaluating each intervention and the width/thickness of the edges indicates inverse standard error of the direct treatment comparisons, and the shading indicates threearm study. For example, Fig. 2 compares the effectiveness of three different chlorhexidine concentrations (CHX 0.2%, 1% and 2%). The difference in thickness/density of connecting edges suggest that CHX 0.2% has superior evidence than CHX 1% based on supporting study data. Importantly, this visual graphical representation of the thickness or density does not indicate the statistical significance of the comparison. The most common comparator across all trials was the placebo or usual care arm which appears as the network’s most common node. While the majority of studies were twoarm trials, two, 3arm trials were included in our network (shaded region in the netgraph). A forest plot [Figure 4] shows the fixed effects model for each intervention having compared with a reference treatment placebo/usual care. In NMA, the forest plot’s importance is to compare a number of treatments to a common comparator also called reference or baseline treatment. We have taken placebo/usual care as the reference treatment for our readers to compare and contrast and to comprehend the treatments are significantly different to placebo/usual care.
The heterogeneity statistics of the NMA follows the Chi square distribution, and the chief prerequisite of assessing the variability is to pinpoint studies whose data differ significantly from what the model predicts. Our first aim was to identify the total or generalized heterogeneity of NMA’s whole network using Cochran’s Q _{total} statistics test and second to determine the heterogeneity/inconsistencies between designs of the NMA network
The heterogeneity statistics of the decompose function of netmeta package provided the generalized DerSimonian estimator tau^{2} value of 0.2829, Higgins’ I^{2} value of 55.7%, CIs, 17.5%; 76.2%. The Cochran’s Q _{total} statistics showed value of 27.10 with a degree of freedom (dof) 12 and a P value of 0.0075.
Q_{statistics} heterogeneity within design showed value of 25.91 with a dof 10 and a P value of 0.0039, and between design heterogeneity/inconsistency value of 1.19 with a degree of freedom 1.19 and a P value of 0.5520. The results show that there is moderate heterogeneity in the NMA network, and considerably very less heterogeneity within designs and between designs.
The relative effect estimates of the ranking of the treatments according to the multiple comparisons are shown in the Table 2. Numerals between 0 and 1, with mean 0.5, demonstrate the rank of a treatment within the given assortment of competing treatments, where a score of 1 is linked to best outcome and a score of 0 is associated with worst outcome. Hierarchical ranking order of the intervention being the best and worst, are introduced by many authors in the Bayesian and frequentist methods. [10, 11] Rucker and Schwarzer introduced ranking order of interventions in the frequentist NMA as P scores, which are analogues to the Bayesian method, surface under the cumulative ranking curve. [10] These values are derived from the effect estimates and their variances. The P scores are based on the frequentist’s method point estimates and the standard error of the network metaanalysis estimates under normality assumption and calculated as means of onesided pvalues. [10, 11, 13, 14, 15]. Numerous studies are using ranking order in NMA so as to display a ranking from the network, which is a better way to present the interventions in terms of the effect estimates. [10, 11, 13, 14, 15] Most commonly the effect estimates might get affected with some ambiguity and we will rarely know in placing a particular trial in the first order or second order. Hence, we classified the ranking first three interventions as best, second threebest interventions as next best, and so on. Based on the ranking order, we found that tooth brushing was the most effective intervention for preventing VAP vs. placebo or usual treatment which was the worst. The best three interventions were tooth brushing (P score fixed of 0.9353, P score random of 0.8892), tooth brushing with povidoneiodine (P score fixed of 0.9091, P score random 0.8801) and furacillin (P score fixed of 0.8798, P score random 0.8358). CHX of 0.2% concentrations (P score fixed of 0.6531, P score random of 0.6478) ranked as the secondbest interventions in the network along with biotene (P score fixed of 0.5954, P score random 0.5400) and potassium permanganate (P score fixed of 0.5274, P score random 0.5446). While chlorhexidine 0.2%, a recommended oral care product for preventing VAP in critically ill patients has a P score of 0.6531 fixed and 0.6478 random.
Oral intervention  Pscore (fixed)  Pscore (random) 

Tooth brushing  0.9353  0.8892 
Tooth brushing with povidoneIodine  0.9091  0.8801 
Furacillin  0.8798  0.8358 
Chlorhexidine (0.2%)  0.6531  0.6478 
Biotene  0.6026  0.5582 
Potassium permanganate  0.5374  0.5515 
PovidoneIodine  0.5289  0.5474 
Chlorhexidine (2%)  0.5889  0.5261 
Chlorhexidine (0.12%) with toothbrushing  0.4620  0.4460 
Chlorhexidine (0.12%)  0.3605  0.3963 
Triclosan  0.3427  0.3563 
Chlorhexidine (1%)  0.2869  0.3141 
Chlorhexidine (2%) with toothbrushing  0.2641  0.2930 
Bicarbonate  0.2487  0.2802 
Listerine  0.2276  0.2635 
Placebo/usual  0.1725  0.2145 
We applied NMA to an existing and robust pairwise metaanalysis to assess the utility of this novel analytic in defining a hierarchical comparison to determine the effectiveness of oral interventions in preventing VAP. [5] Our results suggest that the application of NMA to a conventional metaanalysis provides additional actionable information relative to preventing VAP by comprehensively comparing treatment options otherwise sequestered in pairwise comparisons.
These results have to be taken with caution as the assumptions are based on the results of multiple comparisons. This novel technique allows us to presume direct and indirect comparison performed in a structured statistical framework. Although the inferences are from low risk and unclear risk of bias RCTs, estimated network and ranking of treatment are thus liable to have distinctions as discussed in this NMA and previous pairwise metaanalysis. [5] A potential value of the method is its informative function relative to directing future studies and, in this case, a specific trial assessing preventive interventions for VAP in critically ill patients. The NMA is a comprehendible way of combinations which stem a possibility of consolidating a future trial from the network. Consequently, the NMA when compared to pairwise metaanalysis, weighs the logical possibilities, even within the network while maintaining the internal randomization of the individual trails.
In comparison with the published pairwise metaanalysis, the NMA showed a divergent finding with respect to the ranking probabilities from the multiple comparisons. [3–5] This is the first NMA in this regard to report on comparative effectiveness research on oral care intervention for preventing VAP. In contrast to standard of care where CHX is described as the best oral care intervention to prevent VAP, NMA demonstrated the superiority of tooth brushing or mechanical cleaning. This finding is especially significant given the recent findings associated with CHX toxicity. [16] We also determined toothbrushing when combined with a mouthwash is superior compared to a mouthwash alone; toothbrushing with PI is superior to any other mouthwash or ranking second in the first threebest interventions. This is the first time showing the superior benefit of the furacillin as a mouthwash in preventing the VAP. Furacillin belongs to the nitrofuran class and is a potent antimicrobial organic compound. It is efficient against grampositive bacteria and gramnegative bacteria. Studies shows furacillin effective against many bacterial and fungal entities when applied topically. [17] Although there aren’t many studies on this intervention, this network warrants a possible pilot trial. The PMA showed weak evidences of the PI superior than saline in preventing VAP, and inadequate evidences of the toothbrushing preventing VAP in critically ill patients. [5] The NMA shows toothbrushing alone or toothbrushing along with PI are the best interventions according to the clinical comparative effectiveness research.
There are lack of comparative effectiveness research and vagueness with regard to OCI in preventing VAP among critically ill patients and NMA is never performed. While our results support the usefulness of NMA as a tool to optimize collective analyses of metaanalyses for comparative effectiveness research, it does have limitations. For justifying the rationality of findings and to minimalize error, NMA is designed methodically and conducted carefully. Transporting the highquality systematic search and search results of the Hua et al study, [5] we established our own inclusion and exclusion criteria for building NMA network. We argue that this way we pragmatically compared the PMA to the NMA and reflected on its comparative effectiveness research. Observing fewer research on OCI on preventing the VAP in critically ill patients after the Hua et al study and using the Hua et al research supplemented NMA construction, which defends the judicious literature search along with assessing risk of bias and quality of evidence. But challenges of the NMA persists when comparing the studies with low and unclearrisk biases. In summary, this research accomplishes to provide comparative effectiveness of OCIs in preventing VAP in critically ill patients when combining direct and indirect evidences by having a transitivity assumption that studies are independent and underlying effects are somewhat consistent.
As metaanalysis is considered epitome of the evidencebased clinical medicine, NMA is an extension positioned in this framework. Given the challenges of the proof of concept of existing oral care intervention in preventing VAP, and lack of head to head robust trials of the best available treatment modalities, this approach is exceptional. We followed stern assumptions and standardization and our study cohort was based on the largest pairwise metaanalysis of oral care intervention in preventing the VAP. The transparency, reproducibility and detailed documentation of our finding can be appropriately appraised. According to the NMA outcome, toothbrushing alone or toothbrushing along with a potent antiseptic mouthwash povidone iodine was related to the highest response rate in preventing VAP in critically ill patients followed by furacillin and chlorhexidine 0.2%, respectively.
Ventilation associated pneumonia
Oral care interventions
Network metaanalysis
Pairwise metaanalysis
Randomized controlled trials
Chlorhexidine
tooth brushing
Treatment effects
Standard error of the treatment effects
Povidone iodine
Tbrush  
Tooth brushing 
tbrush_povid  
Tooth brushing with Povidone Iodine 
Fura  
Furacillin 
chx_.2%  
Chlorhexidine 0.2% 
potas 
Potassium permanganate

biotene 
Biotene 
povid 
Povidone Iodine 
chx_2% 
Chlorhexidine 2% 
chx_.12% 
Chlorhexidine 0.12% 
chx_.12% 
Chlorhexidine 0.12% with tooth brushing 
tricl 
Triclosan 
chx_1% 
Chlorhexidine_1% 
chx_2%_toothbrushing 
Chlorhexidine_2%_toothbrushing 
bica 
Sodium Bicarbonate 
list 
Listerine 
placus 
Placebo or usual care 
Satheeshkumar PS: Study design, statistical analysis, data interpretation, manuscript drafting, revision, and critical evaluation
Sonis S: Study design, data interpretation, manuscript revision and critical evaluation.
Acknowledgements NA