This protocol is in adherence with the Preferred Reporting Items for Systematic reviews and Meta-Analyses for Protocols (PRISMA-P) 2015 (61,62).
Types of studies
We will include randomised clinical trials, irrespectively of publication status, publication type, publication year, or written language. Cluster randomised trials and the first part, before cross-over, of randomised cross-over trials will also be included. Quasi-randomised studies, and other studies that are not randomised clinical trials will be excluded.
Types of participants
We will include adults and children of all ages, including neonates, irrespectively of sex and comorbidities.
Types of interventions
The experimental intervention will be cerebral NIRS monitoring to guide clinical care, irrespectively of the length of the intervention period and clinical setting. The control intervention will be no access to cerebral NIRS monitoring. In some trials, participants in the control group will have received cerebral NIRS monitoring to collect data on cerebral oxygenation values during the trial, but where the oxygenation values were unavailable to the clinical staff. In such trials, the control intervention will also be defined as no access to cerebral NIRS monitoring, as this additional monitoring to collect data, was not a part of the control intervention.
Any co-interventions can be accepted but only if the co-intervention is planned to be delivered similarly in both the experimental and control group.
Outcome measures
Primary outcomes
- All-cause mortality at maximal follow-up.
- Moderate or severe, persistent cognitive or neurological deficit, significantly affecting daily life, at maximum follow-up (e.g. modified Rankin score of three or higher (63), Gross Motor Function Classification System level two or higher (64) or Bayley Scale of Infant Development score below minus two standard deviations at two years or later (65)).
- Proportion of participants with one or more serious adverse events. We will define a serious adverse event as any untoward medical occurrence that resulted in either death, was life-threatening, jeopardised the participant, was persistent, led to significant disability, hospitalisation, or prolonged hospitalisation (66). As we expect the trialists’ reporting of serious adverse events to be heterogeneous and not strictly according to the International Committee of Harmonization-Good Clinical Practice (ICH-GCP) recommendations, we will include the event as a serious adverse, if the trialist either: a) used the term ‘serious adverse event’ but not refer to ICH-GCP (66), or b) reported the proportion of participants with an event we consider to full-fill the ICH-GCP definition (e.g. myocardial infarction or hospitalisation). If several of such events are reported, then we will choose the highest proportion reported in each trial.
Secondary outcomes
- Mild, moderate or severe, temporary or persistent, cognitive or neurological deficit as defined by the trialists (e.g. postoperative delirium, postoperative cognitive decline, or Bayley Scale of Infant Development score below minus one standard deviations (65)).
- Quality of life defined as any validated continuous outcome scale used by the trialists at maximum follow-up.
- Any evidence of brain damage on imaging as defined by the trialists at maximal follow-up.
- Proportion of participants with one or more adverse events defined as an untoward medical occurrence that did not necessarily have had to have a causal relationship with the intervention, and which is also non-serious (66).
Exploratory outcomes
- Any evidence of a negative impact on the brain as defined by the trialists (including mild, moderate or severe, temporary or persistent cognitive or neurological deficits, evidence of brain damage on imaging, or evidence of brain damage on electrophysiological monitoring).
- Individual serious adverse events (66).
- Individual adverse events (66).
Electronic searches
Trials will be identified through systematic searches within the following databases: Cochrane Central Register of Controlled Trials (CENTRAL), EMBASE, MEDLINE and Science Citation Index Expanded. No language or time restriction will be applied. The reference lists of all relevant trials will be checked as well.
The following MEDLINE search strategy will be used and adapted to other databases:
(Near-infrared spectroscop*) OR (Near-infrared spectromet*) OR (Near infrared spectroscop*) OR (NIR spectromet*) OR (NIR spectroscop*) OR (NIRS) OR (Oxygenati*) OR (Oxygen saturation) OR (Oximetr*) OR (Cerebral perfusion) OR (Cerebral saturation) AND (Randomized controlled trial) OR (Randomised controlled trial) OR (Randomized clinical trial) OR (Randomised clinical trial) OR (Controlled clinical trial) OR (Clinical trial)
Searching other resources
As an additional search tool, we will search for unpublished and ongoing trials at clinicaltrials.gov and the following medical device companies websites: Medtronic, Minneapolis, MN, USA; NIRO, Hamamatsu, Hamamatsu City, Japan; CAS Medical, Branford, CT, USA; Nonin Medical, Plymouth, MN, USA; Masimo, Irvine, CA, USA; Enginmed, Suzhou, China; and Oxyprem, Zürich, Switzerland. Furthermore, we will also identify on-going relevant trials through trial registers in Europe and USA including clinicaltrials.gov.
For ClinicalTrials.gov we will conduct the following search:
Condition or disease: (Near-infrared spectroscop*) OR (Near-infrared spectromet*) OR (Near infrared spectroscop*) OR (NIR spectromet*) OR (NIR spectroscop*) OR (NIRS) OR (Oxygenati*) OR (Oxygen saturation) OR (Oximetr*) OR (Cerebral perfusion) OR (Cerebral saturation)
Other terms: (Randomized) or (Randomised)
Study type: Interventional Studies (Clinical trials)
Study results: all studies
Data collection and analysis
Selection of studies
All trials that are identified during the literature search will be uploaded to EndNote (Clarivate, Philadelphia, PA, US). Two authors (MLH and SHS) will screen the titles and abstracts of the identified studies. If a study is deemed potentially relevant by the two authors, the full text will be retrieved and assessed for eligibility by the same two authors. If ineligible, the reason for exclusion will be documented. If there is a disagreement between the two authors regarding eligibility or ineligibility, a third author (JCJ) will make the final decision of inclusion or exclusion. Eligible trials will be included in the systematic review and a Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) flow diagram will be included as well. Also, a table displaying the characteristics of excluded studies will be presented in the final systematic review (67). Analysis of cluster-randomised trials and the first part, before cross-over, of randomised cross-over trials, will be handled as depicted in The Cochrane Handbook for Systematic Reviews of Interventions (68).
Data extraction and management
Once the relevant trials have been included, data extraction will be conducted by the two authors MLH and SHS independently. Any disagreements will be discussed in the author group and a final decision will be made. The following data will be extracted from each study:
- General information: title, author(s), year of publication, language of publication, funding sources, potential conflicts of interest
- Methodology: study aim, study design, clinical setting, inclusion and exclusion criteria, type of interventions, cerebral NIRS monitoring unavailable for clinical staff in the control group, outcome measures, time of outcome assessment
- Sample size: number of participants meeting the criteria for inclusion
We will use specific data extraction forms designed for this purpose. If some of the relevant data is not available in the study report or publication, e.g. if the study does not report all of the prespecified outcomes, the trialists will be contacted and asked if they can provide such data. The correspondence with the trialists will be included in the systematic review as an appendix.
Assessment of risk of bias in included studies
Randomised clinical trials with certain methodological flaws carry an increased risk of bias (69–74). Such methodological flaws increase the likelihood that the trialists will come to the wrong conclusion by over- or underestimating effect sizes (75). Therefore, it is important to assess the risk of bias in trials included in a systematic review (67). Based on the Cochrane Risk of Bias tool – version 2 (RoB 2) (76) described in The Cochrane Handbook for Systematic Reviews of Interventions (77), we will assess the risk of bias for the following domains: 1) bias arising from the randomisation process, 2) bias due to deviation from intended interventions, 3) bias due to missing outcome data, 4) bias in measurements of outcomes, and 5) bias in selection of the reported results. Risk of bias assessment of the included studies will be conducted by the two authors MLH and SHS, who independently will transfer data into the Stata file. Any disagreement between their assessments will be discussed and, if necessary, a final decision will be made by a third author (JCJ).
Bias arising from the randomisation process
A trial will be considered at low risk of bias if the allocation sequence was adequately concealed (e.g. performed by an on-site locked computer, a central independent unit or sealed, identical envelopes), AND there are no baseline imbalances between the experimental and control group (if any appeared, they must be compatible with chance), AND the allocation sequence generation was adequate (e.g. generated by a computer random numbers generator, a random numbers table, tossing a coin, shuffling cards or drawing lots - the latter three methods will only be considered low risk of bias if the sequence generation was conducted by an independent person with no involvement in the trial), OR if a description of the method for allocation sequence generation is missing.
A trial will be considered of some concerns if the allocation sequence was adequately concealed AND there is a problem with the method of allocation sequence generation, OR baseline imbalances suggest a problem with the randomisation process, OR if the method for allocation concealment was not described for the trial, AND baseline imbalances across intervention groups appear to be by chance, OR if no information is available to answer any of the signalling questions.
A trial will be considered at high risk of bias if investigators were aware of the allocation sequence (70), OR if the method for allocation concealment was not described, AND if baseline imbalance suggest a problem with the randomisation process.
Furthermore, trials where the generation was not at random, or quasi-randomised, will be considered high risk of bias and excluded from the review (73,78)
Bias due to deviation from intended interventions
A trial will be considered at low risk of bias if participants and clinical staff – and parents in paediatric trials – were unaware of the group allocation during the trial, OR if they were aware of the group allocation during the trial, but any deviation from the intended intervention reflected normal clinical practice, OR if they were aware of group allocation, but any deviation from the intended intervention was unlikely to influence the outcomes, AND no trial participants were analysed on the basis of the received intervention, instead of on the basis of their randomised allocation group.
A trial will be considered of some concerns if participants and clinical staff – and parents in paediatric trials – were aware of group allocation, AND no information is available regarding deviations from normal clinical practice, which potentially could impact the outcomes AND the deviations from clinical practice were imbalanced between the intervention groups, OR some trial participants were analysed on the basis of the received intervention instead of on the basis of randomised allocation group, but it was deemed as insufficient to significantly alter the intervention effect estimate.
A trial will be considered at high risk of bias if participants and clinical staff – and parents in paediatric trials – were aware of group allocation (75,79), AND there were deviations from intended interventions which were unbalanced between the intervention groups, AND likely to affect the outcomes, OR some participants were analysed on the basis of the received intervention instead of on the basis of randomised group allocation, AND it was deemed as sufficient to significantly alter the intervention effect estimate.
Bias due to missing outcome data
A trial will be considered at low risk of bias if there is no missing outcome data, OR if the proportion of missing outcome data is similar between the intervention groups, AND the reasons for missing outcome data are similar, OR if there is evidence that the missing outcomes do not make an important difference to the estimate of the intervention effect (e.g. sensitivity analyses such as ‘best-worst, worst-best’ case scenario analysis).
A trial will be considered of some concerns if the amount of missing outcome data is unclear, OR there is unclear information regarding the proportion of missing data between intervention groups, AND reason for missing outcome data between intervention groups is unclear, AND there is no evidence that the missing outcome data do not make an important difference to the estimate of the interventions effect (e.g. lack of sensitivity analyses such as ‘best-worst, worst-best’ case scenario analysis).
A trial will be considered at high risk of bias if the amount of missing data is high (more than 5%), AND missing outcome data between the intervention groups differ, OR the reason for missing outcome data between intervention groups differ, AND there is no evidence that the missing outcome data do not make an important difference to the estimate of the interventions effect (e.g. lack of sensitivity analyses such as ‘best-worst, worst-best’ case scenario analysis) (73).
Bias in measurement of outcomes
A trial will be considered at low risk of bias if the outcome assessors were blinded to group allocation, OR if the outcome assessors were not blinded to group allocation, but it was deemed that knowledge of group allocation was unlikely to influence outcome assessment.
A trial will be considered of some concerns if there is no available information to evaluate whether outcome assessors were blinded to group allocation AND if such knowledge could influence outcome assessment.
A trial will be considered at high risk of bias if outcome assessors were not blinded to group allocation AND it is deemed likely that knowledge of group allocation was likely to influence outcome assessment (75,80).
Bias in selection of the reported result
A trial will be considered at low risk of bias if the outcome data reported are unlikely to have been selected based on the results of multiple outcome measurements (e.g. different scales to measure the outcome, multiple assessors of the outcome, different time points for assessment of the outcome) within the outcome domain, AND if the outcome data reported are unlikely to have been selected based on the results from multiple outcome analysis.
A trial will be considered of some concerns if it is uncertain whether the outcome data reported have been selected based on the results of multiple outcome measurements (e.g. different scales to measure the outcome, multiple assessors of the outcome, different time points for assessment of the outcome) within the outcome domain OR from multiple outcome analysis.
A trial will be considered at high risk of bias if the reported outcome data are likely to have been selected based on the results of multiple outcome measurements (e.g. different scales to measure the outcome, multiple assessors of the outcome, different time points for assessment of the outcome) within the outcome domain OR from multiple outcome analysis.
Overall risk of bias
The included trials will be considered as overall in low risk of bias or high risk of bias. A trial will be considered as overall low risk of bias if the trial is judged as low risk of bias in ALL the above domains. If the trial is considered at high risk of bias, or to be of some concern, in any of the above domains, the trial will be considered as overall in high risk of bias. Within each trial, each outcome result will be assessed for bias, based on the three domains ‘bias due to missing outcomes’, ‘bias in measurements of outcomes’, and ‘bias in selection of the reported result’. Thus, we will be able to assess not only risk of bias in each trial, but also for each outcome. Additionally, the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) assessment will be used to assess the quality of the body of evidence for all outcomes and summarised in a Summary of Findings table (81) (See the section “Summary of Findings table” for a description of the five considerations included in the GRADE assessment).
The primary conclusion will be based on the analysis of our primary outcome results in all trials assessed as having an overall low risk of bias (58).
Assessment of bias in conducting the systematic review
The systematic review will be conducted according to this protocol. Any deviation in the conduct will be reported in the section ‘Differences in the methodology between protocol and review’ in the systematic review.
Measures of treatment effect
Dichotomous outcomes
For dichotomous outcomes we will calculate risk ratios (RRs) with 95% confidence intervals (CIs) and Trial Sequential Analysis-adjusted CIs (58,82).
Continuous outcomes
For continuous outcomes, i.e. ‘Quality of Life’, we will calculate the standardised mean difference with a 95% CI and a Trial Sequential Analysis-adjusted CI (58,82).
Handling missing data
We will use the intention-to-treat data from the included trials for both dichotomous and continuous outcomes. For trials with missing or unclear outcome data, the trial authors will be contacted by MLH with JCJ as ‘cc’. The trial authors will be requested to provide missing outcome data or to elaborate on unclear outcome data. All correspondence will be attached to the systematic review in an appendix. If it is not possible to obtain missing outcome data, we will not impute the missing data for the primary analysis. Instead, this will be done in the sensitivity analyses.
Data synthesis
All data analyses will be conducted in STATA 16.1 (StataCorp LLC, College Station, Texas, USA). The meta-analysis will be conducted as recommended in The Cochrane Handbook for Systematic Reviews of Interventions (83). For outcomes where data is only available from one trial, the results will be narratively described. If one or more of the included trials reports on multiple intervention arms, we will only include the relevant arms. Furthermore, the population in the control group will be halved for such studies, if two of the comparisons are included in the meta-analysis. An eight-step procedure by Jakobsen et al. will be used to assess if thresholds for statistical and clinical significance are crossed (58).
Step one – meta-analysis
Fixed-effect and random-effects meta-analysis will be used to estimate the effect of the intervention (84,85). The most conservative point estimate from the two analyses will be used. If the point estimates are similar, the one with the widest CI will be used (58).
Step two – assessment of heterogeneity
Statistical heterogeneity will be evaluated by using I2 statistics, with a threshold for significant heterogeneity at p < 0.1 (86), and by visual inspection of forest plots. Clinical heterogeneity will be assessed by evaluating the characteristics of the included trials based on the PICO model (Participants, Interventions, Comparisons, Outcomes). Any signs of heterogeneity will be explored in the subgroup analyses.
Step three – accounting for multiplicity
Since we report on three primary outcomes, a p-value below 0.025 will be considered statistically significant for each of the primary outcomes (58).
Step four – Trial Sequential Analysis
To control the risks of type I and II errors (87), all primary outcomes will undergo Trial Sequential Analysis and the required meta-analysis information size as well trial sequential boundaries for benefit, harm, and futility will be established (88). If the required number of randomised participants to achieve sufficient power is not reached, the confidence interval for the point estimates will be adjusted accordingly by the Trial Sequential Analysis Program (82,88). A relative risk reduction of 20% will be used as the anticipated intervention effect for each primary outcome, an alpha of 2.5% will be used as the acceptable risk of type 1 errors and a beta of 10% will be used as the acceptable risk of type 2 errors.
For cumulative Z-scores that reach below 50% of the diversity-adjusted Trial Sequential Analysis required information size (or sample size), we will downgrade imprecision by two levels for the GRADE assessments (see section on ‘Summary of findings table’). For cumulative Z-scores that reach between 50% to 100% of the diversity-adjusted Trial Sequential Analysis required information size (or sample size), we will downgrade imprecision by one level for the GRADE assessments. For cumulative Z-scores that cross the monitoring boundaries for benefit, futility, or harm, we will not downgrade imprecision for the GRADE assessments.
Step five – Bayes factor
The Bayes factor (89) will be calculated for all primary outcomes and 0.1 will be used as threshold for significance (58). An anticipated risk reduction of 20% will be used when calculating the Bayes factor (58).
Step six – subgroup and sensitivity analysis
The following subgroup analyses will be conducted, if possible:
1) Comparison of the intervention effect between trials at overall low to high risk of bias.
2) Comparison of the intervention effect between trials assessing different clinical settings: neonatal intensive care, paediatric intensive care, children during surgery, adult intensive care, and adults during surgery.
3) Comparison of trials without support from the medical device industry compared to trials at risk of such support (71).
4) Comparison of trials where participants in the control group underwent cerebral NIRS monitoring, but where the oxygenation values were unavailable to the clinical staff, compared to trials where participants in the control group did not undergo cerebral NIRS monitoring at all.
To quantify the potential impact of missing outcome data, the following two sensitivity analysis will be conducted on the three primary outcomes.
1) 'Best-worst case' scenario: we will assume that all participants lost to follow-up in the experimental group either died, suffered from ‘moderate or severe persistent cognitive or neurological deficit’ or had ‘one or more serious adverse events’, while all participants lost to follow-up in the control group experienced these events.
2) 'Worst-best case' scenario: we will assume that all participants lost to follow-up in the experimental group suffered died, suffered from ‘moderate or severe persistent cognitive or neurological deficit’ or had ‘one or more serious adverse events’, while all participants lost to follow-up in the experimental group did not experience any of these events.
Step seven – assessment of risk of publication bias
If at least ten trials are included in the meta-analysis, we will create funnel plots and visually inspect them to assess any potential publication bias. As an additional measure, we will evaluate the funnel plot asymmetry by conducting the Harbord test (90) for dichotomous outcomes and the Egger test for continuous outcome (91).
Step eight – assessment of clinical significance
If the data analyses show statistically significant effects of the intervention, we will also assess whether the results are clinically significant. The assessment of clinical significance will be based on the definitions of minimal important differences (see ‘Trial Sequential Analysis’), the Summary of Findings Table (see section on ‘Summary of Findings table’) as well as a thorough evaluation of beneficial and harmful outcomes. Furthermore, we will calculate the number-needed-to-treat for all dichotomised outcomes (58).
Summary of findings table
To present the findings for the pre-specified primary, secondary and exploratory outcomes, we will create and include a ‘Summary of findings’ table in the systematic review as recommended in the Cochrane Handbook for Systematic Reviews of Interventions (92). The Grading of Recommendations, Assessment, Development and Evaluations (GRADE) approach will be used to assess and rate the quality of the body of evidence, i.e. the certainty in the range of an effect estimate for all pre-specified outcomes (81). The GRADE approach evaluates the body of evidence based on the five following considerations: 1) risk of bias assessment (76,93), 2) heterogeneity or inconsistency of results (94), 3) imprecision of the effect estimates due to wide CIs (95), 4) indirectness of evidence (96), and 5) publication and for-profit bias (97). Imprecision will be assessed using Trial Sequential Analysis (58). To assess for-profit bias, we will search for information regarding industry funding for each trial and trial author. If a trial, or an author, is sponsored by the industry, we will judge the trial as in high risk of for-profit bias. To conduct the rating, we will use the online tool GRADEpro software (www.gradepro.org). The reasons for any up- or down-grading of the certainty of evidence will be described and justified in details in the systematic review (92).
As we expect heterogeneity due to the various clinical settings and wide age span of participants in the included studies within this systematic review, we will also create summary of findings tables for the subgroup analysis within each of the different clinical settings (e.g. neonatal intensive care trials, paediatric intensive care trials, adult intensive care trials and perioperative care trials).