Biomarkers in Diagnosis of Sepsis and Systemic Infection in Adult Patients: A Systematic Review and Bayesian Network Meta-Analysis

Sepsis is estimated to affect over 30 million people worldwide and to result in six million deaths every year. The denition of sepsis has been heavily revised in recent years, resulting in the need for a comprehensive comparison for clinicians to choose among the current biomarkers for sepsis. We conducted a systematic review and synthesized both direct and indirect evidence by using network meta-analyses with bivariate hierarchical random-effects arm-based model in a Bayesian framework. We applied the Quality Assessment of Diagnostic Accuracy Studies-2 criteria to assess the risk of bias, investigated heterogeneity using Bayesian network meta-regression models, and estimated optimal adjusted cutoff values for each included biomarker.


Introduction
Sepsis is estimated to affect over 30 million people worldwide and result in six million deaths annually.
Despite being a substantial public health burden worldwide, sepsis lacks a reliable gold standard for diagnosis. [1,2] Accordingly, there have been a total of three historical revisions made in the de nition of sepsis in the recent decades. [3][4][5] The contemporary 3.0 version sepsis (Sepsis-3) utilizes the change of Sequential Organ Failure Assessment (SOFA) score to obtain an operational diagnosis as an infectioninduced life-threatening organ dysfunction associated a with high mortality risk. The revision of sepsis also results in the need for a comprehensive comparison of biomarkers.
On the other hand, few studies exist to compare several biomarkers simultaneously for sepsis, which prevent an objective ranking of the performance of these biomarkers. Furthermore, con icting results and insu cient evidence comparing performance biomarkers exist. [6][7][8][9] With the advantages of the Bayesian network meta-analysis, we can rank the biomarkers and obtain more precision by incorporating indirect evidence with well-quanti ed relative accuracies and credible intervals between biomarkers. [10,11] In this systematic review and Bayesian network meta-analysis, we compared the performance of wellstudied biomarkers in the detection of systemic infection and sepsis among adult patients with suspected infection. We aimed to provide an up-to-date insight into the diagnostic value of biomarkers under different de nitions of sepsis in adult patients.

Methods
We reported our study according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension statement for network meta-analysis (PRISMA-NMA). [12] This study has been registered on the International Prospective Register 61 of Systematic Reviews (PROSPERO) database (number: CRD42018086545). [13] Data sources and Search strategy A comprehensive systematic search including published literature from their inception to May 2019 was conducted in the following databases: (1) PubMed, (2) EMBASE, and (3) Scopus. We used sets of keywords consisting of the names of biomarkers, as well as the keywords "diagnosis", "sepsis", and "adult" to search in these databases (Appendix pp. [3][4]. The references of the eligible papers were also screened to identify additional studies.

Study selection and data extraction
Two authors (CCW and HML) independently scrutinised and evaluated the studies according to the preidenti ed selection criteria. Studies that met the following criteria were included: (1) original articles; (2) adult patients (i.e., ≥ 18 years old); (3) patients with suspected infection; (4) studies containing diagnostic accuracy assessments of sepsis. We did not apply any language restriction, however, studies were excluded if any of the following conditions occurred: (1) not detecting systemic infection or sepsis; (2) non-diagnostic studies; (3) non-original articles; (4) duplicated search results; (5) lack of necessary parameters for diagnostic accuracy assessments (Appendix pp. 3-4); (6) only including burns patients; [14] (7) using only non-systemic in ammatory response syndrome (SIRS) patients as the reference group (extreme-control); (8) biomarkers with su cient numbers (more than three) of original studies to allow for meta-analyses. Finally, the included studies were classi ed into two groups according to the two reference standards for clinical diagnosis of sepsis: (1) the Sepsis-3 (sepsis) group to study the diagnostic performance of biomarkers for Sepsis-3, [5] including the former severe sepsis and septic shock; [3,4] (2) the Sepsis-1 (systemic infection) group to study how biomarkers performed in differentiating infectious from non-infectious SIRS. [3,4] We further extracted information from either the full texts or the published abstracts of each study, including the year of publication, country, clinical settings, criteria of sepsis, study design, sample sizes, targeted biomarkers, types of specimen, methods of measurement, sponsorships, cut-off values, proportions of patients with sepsis, sensitivities, and speci cities. All disagreements between authors were resolved by consensus meetings with the third clinical expert (KFC).

Quality assessment
For each included study, two authors (CCW and HML) independently assessed the quality of the study according to our modi cation to the Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) criteria (Appendix pp. [4][5]. [15,16] Only the studies that satis ed all the signaling questions were marked as "low risk". The degree of agreement between the two authors was measured using Cohen's Kappa statistic. [17] Any disagreement was resolved through consensus meetings. Additionally, the publication bias was tested by using Deek's funnel plot. [18,19] Data synthesis For Bayesian network meta-analyses, we applied the bivariate hierarchical random-effects arm-based model. [11,20,21] Unlike conventional methods, which only allow for pairwise comparison between two biomarkers, this model compares multiple biomarkers simultaneously. In brief, we incorporate direct evidence with indirect evidence through a third biomarker between the other two biomarkers (Fig. 1). We applied the preferred Bayesian framework for network meta-analysis with potential multiple comparison issues, using Hamiltonian Monte Carlo random sampling methods in Stan language with the rstan package (Version 2.18.2) and the NO-U-Turn sampler within in R (version 3.5.3). We reproduced the simulations for 50,000 iterations in four chains with 25,000 burn-in iterations until convergence, which was evaluated by time series plots of the parameters. The results of data synthesis were presented as a point estimate with a 95% credible interval (CrI). In order to test the validity of indirect comparison, we also calculated the inconsistency factor with the node-splitting model. [22] Heterogeneity and Bayesian network meta-regression In diagnostic studies, between-study heterogeneity can be caused by two effects, threshold and nonthreshold. For the potential threshold effect, we calculated the Spearman correlation coe cients between logit-transformed sensitivities and false-positive rates; for the potential non-threshold effect, we calculated the Chi-square statistics, Cochrane-Q test, and the I 2 metric. We further performed univariable Bayesian network meta-regression analysis to explore possible sources of variability by considering (1) regions of the study (Asian/Europe); (2) patient sources (ICU/ED); (3) patient characteristics (medical/surgical); (4) specimen types (plasma/serum/whole blood); (5) using non-infectious SIRS patients as the reference group; (6) sponsorship; (7) case-control study design; (8) risks of bias and applicability of study based on QUADAS-2; and (9) accessibility to the full text content. To adjust the diagnostic accuracies based on the variables accounting for signi cant heterogeneity from the univariable Bayesian network meta-regression, we performed the multivariable Bayesian network metaregression. In order to analyse the potential effect of the data obtained from non-full text and the outliers, as well as year of the study conducted, we performed sensitivity analyses with data removing these studies separately.

Optimal cutoff evaluation
We further estimated optimal cutoffs based on adjusted diagnostic performance metrics using multivariable Bayesian network meta-regression models (Appendix pp. 7). The optimal cutoffs were determined using locally weighted scatterplot smoothing plot based on the meta-regression-adjusted sensitivities and speci cities. We determined the optimal ranges of biomarkers by identifying cutoffs that result in the maximal differences between the adjusted sensitivities and false-positive rates (1speci city).

Data sharing
With the publication of this article, the full dataset and codes will be freely available online in Mendeley Data, a secure online repository for research data (DOI: 10.17632/cdftd22xgs.1). [23] Role of the funding source The funder of this study had no role in the study design, data collection, data analysis, data interpretation, writing of the report, or in the decision to submit this paper publication. The authors had full access to all the data, and the corresponding author was responsible for the decision to submit for publication.

Patient and public involvement
Our study contained no direct patient or public involvement; however, the research question was informed by front line health care providers who need accurate biomarkers to detect patients with systemic infection or sepsis.

Search results and characteristics of included studies
After searching and screening the literature, 336 unique studies were identi ed; seven biomarkers with su cient numbers (more than three) of original studies to allow for meta-analyses were included: procalcitonin, C-reactive protein (CRP), interleukin-6 (IL-6), presepsin (cluster of differentiation [CD] neutrophil marker 14 subtype), CD64, soluble triggering receptor expressed on myeloid cells 1 (sTREM-1), and lipopolysaccharide-binding protein (LBP). Among these, 134 studies comprising of 20,564 patients published from 1996 to 2019 met the inclusion criteria and were included (Fig. 2, Appendix pp. .  Overall, 33 studies were evaluated for the diagnostic performance of biomarkers for sepsis Publication bias Furthermore, we noticed signi cant publication biases in the studies using procalcitonin for Sepsis-1 (p < 0.01), and three outlier studies accounted for the majority of the bias. [75,102,135] Signi cant I 2 (> 75%) were found for all biomarkers under both de nitions, except for LBP for Sepsis-1 and IL-6 for Sepsis-3 (Appendix pp. 102). We did not nd any inconsistency of network between presepsin and procalcitonin for both Sepsis-3 (inconsistency factor (IF) for sensitivity: − 0.17-0.23, IF for speci city: − 0.1-0.44) and Sepsis-1 (IF for sensitivity: − 0.13 − 0.12, IF for speci city: − 0.12-0.19, Appendix pp. 102).

Meta-regressions
The Bayesian network meta-regressions model was used to adjust the effects of suboptimal quality, the study population, the types of serum specimen, and sponsorship. In the univariable Bayesian network meta-regression model for Sepsis-1, a signi cantly higher sensitivity was associated with Asian studies for CRP, whereas lower sensitivities were associated with sponsorship and applicability of patient selection in sTREM-1 (Appendix pp. 44-45). Meanwhile, signi cantly higher speci cities were associated with medical patients for CRP and Asian studies for CD64, while lower speci cities were associated with European studies for CD64, patients in ED for CRP, patients in ICU for CD64, serum specimens for procalcitonin and sponsorship for CRP. Furthermore, signi cantly higher speci cities were associated with sponsorship and risk of bias in the index test for IL-6 in Sepsis-3 (Appendix pp. 46-47).
After selecting variables using univariable Bayesian network meta-regression to solve the heterogeneity issue, the types of serum specimen, European studies, ICU, medical patients, sponsorship, and the applicability of patient selection for Sepsis-1, sponsorship, and risk bias of index test for Sepsis-3 were selected in the multivariable Bayesian network meta-regression (Appendix pp. 44-47). The pooled sensitivities and speci cities changed slightly after adjustment, after which CD64 showed the best pooled adjusted sensitivities and speci cities (Appendix pp. 48-49). The sensitivity analyses demonstrated that that the studies from data obtained from non-full text abstract and the outliers only had a small effect (Appendix pp. 105). However, we found that studies performed later would demonstrate poorer performance for presepsin and sTREM-1 (Appendix pp. 108).

Cutoffs recommendation
The optimal cutoffs were further determined using adjusted values plotted in the locally-weighted scatterplot smoothing plots with 800-900 pg/mL and 600-700 pg/mL for presepsin for the detection of Sepsis-3 and Sepsis-1, respectively (Fig. 5, Appendix pp. [9][10][11]. Similarly, we suggested a cutoff between 1.5-2.0 ng/mL and 1.0-1.5 ng/mL for the use of procalcitonin in the detection of Sepsis-3, and Sepsis-1, respectively (Fig. 5). Owing to the lack of a uni able measuring unit, we cannot provide optimal cutoffs for CD64.

Discussions
The application of Bayesian network meta-analyses in our search resulted in a large number of included studies, allowing for a full recognition of the diagnostic performance between sepsis biomarkers. Overall, presepsin exhibited the best pooled sensitivities, whereas CD64 showed the best pooled speci city for both Sepsis-1 and Sepsis-3 detection. After adjusting for study quality, study populations, types of specimens, year of the study conducted and sponsorship, we found that CD64 showed the best-pooled sensitivities and speci cities, and we provided suggested cutoffs for presepsin and procalcitonin.
This study has several strengths. First, it provides an up-to-date overview of the diagnostic performance of biomarkers in detecting either Sepsis-3 or Sepsis-1 by including a number of recent studies. In two of the previous systematic reviews and meta-analyses, presepsin was found to have a marginally superior sensitivity than procalcitonin in detecting Sepsis-1; however, the limited numbers of studies included in these meta-analyses prevent a conclusive comparison. [6,7] Second, our work covered not only wellstudied biomarkers, such as procalcitonin, but also some novel biomarkers such as CD64, thus contributing to the accumulating body of knowledge on their current utility. Third, the Bayesian network meta-analyses we applied allowed for the simultaneous pooling of sensitivities and speci cities and provided well-quanti ed credible intervals for relative diagnostic performance, which strengthened the evidence, with a superior reliability than conventional methods (Appendix pp. 6-7). In previous metaanalyses that applied different methods to evaluate the performance, the pooled sensitivities and speci cities were not stable and were sometimes contradictory (Appendix pp. [106][107][108]. Fourth, we applied rigorous criteria to verify the quality of the studies, and thoroughly examined their in uence on the diagnostic performance with Bayesian network meta-regressions. Therefore, we were able to evaluate the biomarkers more con dently. Lastly, an objective optimal cutoff was achieved after adjusting the heterogeneity with multivariable Bayesian network meta-regression. The optimal points were visualized as a locally weighted scatterplot smoothing plot.
In this study, we pooled together numerous studies evaluating the biomarkers for severe sepsis and Sepsis-3 to provide up-to-date evidence for clinicians to re-evaluate these biomarkers. The newly revised de nition of Sepsis-3 re ects a paradigm shift from implicitly diagnosing a systemic infection to explicitly identifying a severe infection associated with mortality. [5] The biomarkers that were found to be associated with systemic infections, therefore, are not necessarily predictive of Sepsis-3. Therefore, the diagnostic performance of current biomarkers, especially procalcitonin, should be re-evaluated. To date, only a few studies have started to re-evaluate the biomarkers for Sepsis-3. [27,132,135] Presepsin has been studied most extensively during the past decade with various degrees of investigation into biomarkers for the detection of sepsis. [106] By adding indirect comparisons, our Bayesian network meta-analysis indicates that presepsin has a signi cantly better sensitivity than procalcitonin for the detection of both Sepsis-1 and Sepsis-3. Furthermore, presepsin has similar speci cities to procalcitonin in terms of the detection of both Sepsis-1 and Sepsis-3, which could also be used for the guidance of antimicrobial therapies. [141,142] Presepsin also shows a greater sensitivity than the currently used screening tools, such as quick Sepsis-related Organ Failure Assessment (qSOFA) and the systemic in ammatory response syndrome (SIRS) criteria. In a systematic review and metaanalysis, the qSOFA score and SIRS criteria showed a pooled sensitivity of 0.51 and 0.29 for the detection of Sepsis-3, respectively. [143] Due to its extensively varied performance across the studies, the diagnostic value of procalcitonin for Sepsis-1 was undetermined. The use of different study selection criteria has prevented a homogeneous conclusion being drawn in previous meta-analyses of procalcitonin (Appendix pp. [106][107][108]. A univariable meta-analysis containing 2,097 patients in 2007 proclaimed the rather insu cient diagnostic capacity of procalcitonin for Sepsis-1, targeting the general population but excluding non-adult patients, in addition to a limited spectrum of patients. [8] On the other hand, a more recent bivariate meta-analysis containing 3,487 patients in 2013 found that procalcitonin had an excellent diagnostic capacity, targeting a more general but heterogeneous population by including paediatric patients and patients with septic shock, as well as studies with only non-infectious SIRS as the reference group. [9] In our study, we de ne an adult population without burn patients and use the subgroups of Sepsis-1 and Sepsis-3 to perform a better comparison of the performance of the biomarkers.
The potential confounding results in terms of the performance evaluation of these biomarkers have been extensively evaluated in our study. Suboptimal quality, study populations, types of serum specimens, and sponsorships were found to signi cantly bias the results. After adjusting these biases, through the application of the multivariable Bayesian network meta-regression, surprisingly, CD64 became the biomarker that performed best in the detection of both Sepsis-1 and Sepsis-3.
Neutrophil CD64 is an Fcγ receptor commonly expressed on monocytes, as well as occasionally on polymorphonuclear leukocytes. [144] With a remarkably higher speci city for differentiating infectious from non-infectious SIRS, the potential superiority of neutrophil CD64 compared to procalcitonin was implied for use in antimicrobial therapies. However, the lack of a uni able measuring unit for neutrophil CD64 means that the estimation of the optimal cutoff will needed to be determined in a future study. [145,146] The evaluation of the optimal cutoffs could assist clinicians in their decision making. In theory, a relevant diagnosis could be inferred during the synthesis of data from the cutoff parameters of the hierarchical summary receiver operating characteristic (HSROC) model, [147] or in other words, the back-transformed bivariate model. However, it was assumed that no covariates asymmetrically affected the overall diagnostic accuracies. [148] This assumption, in practice, could be violated by either including patients with different disease severities or non-SIRS controls. Another attempt was previously made from the previous meta-analysis, which suggested the use of 0.5-2.0 ng/mL as a cutoff range for procalcitonin detecting Sepsis-1 by nding the interquartile range (IQR) across studies. [9,147] In this study, we provided suggested cutoffs by visualization of the optimal points, with the adjusted values plotted on a locally weighted scatterplot smoothing plot, thereby avoiding the need for an assumption or an adjustment for the confounders.

Limitations
A few methodological issues in this study were noted. First, a gold standard for the diagnosis of sepsis does not currently exist, although both Sepsis-3 and Sepsis-1 do provide operative de nitions for the description of life-threatening organ failure status or systemic in ammatory response. However, with the current revised operational de nition of Sepsis-3, clinicians would be able to treat their patients more accurately. Nonetheless, attempts to validate biomarkers for Sepsis-3 are still needed to draw more reliable conclusions.
Second, while the large number of the studies included in this review contributed to a better con dence, it also resulted in a higher heterogeneity. Our Bayesian network meta-regression suggested that those strictly implementing random selection processes were generally associated with a decreased sensitivity. It is also compulsory to investigate whether the exclusion of patients with other conditions, including malignancy, status post-operation, kidney diseases, and pregnancy, is associated with a signi cant in uence on the diagnostic performance of biomarkers in the future.

Conclusion
In this study, we provided an accurate comparison and an up-to-date Bayesian network meta-analysis allowed for a more reliable comparison of biomarkers for diagnostic test accuracy studies for sepsis.
Presepsin and CD64 were found to outperform procalcitonin and other biomarkers in their use for the detection of both systemic infection and sepsis. After adjusting for study quality, study populations, types of specimens, year of the study conducted and sponsorship, CD64 showed the best-pooled sensitivities and speci cities. For presepsin and procalcitonin, 600-700 pg/mL and 1-1.5 ng/mL were suggested as the optimal cutoffs for detecting systemic infection, respectively. Nonetheless, owing to the lack of a uni able measuring unit, we cannot provide optimal cutoffs for CD64. However, further investigation of these biomarkers will be needed to identify the potential biases caused by suboptimal quality, the study population, the type of specimen, sponsorship, and the use of post-hoc cutoffs.
SUMMARY BOXES Section 1: What is already known on this topic The de nition of sepsis, a worldwide public health burden, has been heavily revised in recent years (Sepsis-3), resulting in the need for a comprehensive comparison for clinicians to choose among the current biomarkers for sepsis. Previous meta-analyses have mainly focused on the capacity of single biomarkers to detect the onset of systemic infection, previously de ned as Sepsis-1. Section 2: What this study adds This systematic review and Bayesian network meta-analyses included the largest number of studies, allowing for the veri cation of the diagnostic performance of presepsin compared to other current biomarkers to provide different de nitions of sepsis. In this analysis, the authors further adjusted the potential resources of heterogeneity and provided suggested cutoffs accordingly. CD64 and presepsin were found to outperform other biomarkers in detecting systemic infection and sepsis-3 without any signi cant inconsistency in the Bayesian network meta-analyses or meta-regression.

Declarations
Authors' contribution HML and KFC wrote the manuscript. HML and CCW performed the literature review. HML, CCW and KFC performed the statistical analysis. SHL, CHL, YKT and KFC revised the text. All authors read and approved the nal manuscript.

Financial Support
The study was supported by the Ministry of Science and Technology and Chang Gung Memorial Hospital in Taiwan (MOST 107-2314-B-182 -052 -MY2, CMRPG2H0322, CMRPG2H0312). The funder has no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Con ict of Interest Disclosure
The authors declare that they have no con icts of interest Acknowledgment Not applicable

Declaration of interests
The authors declare no con icts of interest.

Consent for publication
Not applicable.

Role of the funding source
The funder of this study had no role in study design, data collection, data analysis, data interpretation, writing of the report, or in the decision to submit for publication. The authors had full access to all the data, and the corresponding author was responsible for the decision to submit for publication.

Availability of supporting data
With the publication of this article, the full dataset and codes will be freely available online in Mendeley Data, a secure online repository for research data (DOI: 10.17632/cdftd22xgs.1).

Ethical statement
This meta-analysis study is exempt from ethics approval, since the study authors were collecting and synthesising data from previous clinical studies in which informed consent has already been obtained by the investigators.  Diagnostic performances of biomarkers compared to presepsin. Note: PCT: procalcitonin, CRP: C-reactive protein, IL-6: interleukin-6, CD64: neutrophil CD64, sTREM-1: soluble triggering receptor expressed on myeloid cells 1, LBP: lipopolysaccharide-binding protein.