This protocol is reported according to the recommendations of PRISMA-P 2015 statement (13). The pertinent checklist can be found in additional file 1.
Eligibility criteria
The eligibility criteria for studies to be included in the systematic review and meta-analysis are as follows:
-
They include patients in whom a carcinoma of the esophagus, stomach, pancreas, or rectum was resected oncologically, i.e., with systematic lymphadenectomy.
-
They include at least one group of patients who underwent neoadjuvant therapy (chemotherapy, radiotherapy, or chemoradiotherapy) prior to surgery and one group of patients who underwent upfront surgery (surgery without prior neoadjuvant therapy).
-
They report the lymph node yield (number of resected lymph nodes) for study participants.
-
There is no limitation regarding study design if the criteria named above are met.
-
Abstract and full text of the study are available in English, German, Russian, Italian, Spanish or French.
Information sources
The electronic literature databases PubMed and Cochrane Library will be searched through their respective online search engines using a defined search strategy (see below). The search will be performed on studies published between database inception and the cutoff date October 8, 2020. Moreover, the reference lists of included articles will be manually searched.
Search strategy
The search strategy used for the databases can be found in additional file 2.
Data management
The abstracts of the publications identified with the search strategy will be uploaded into the web application Rayyan QCRI (14), in which study selection will be performed as described below. Data extracted from the single studies will be stored in a standardized spreadsheet and will subsequently be transferred into the review software RevMan 5.3. (15)
Selection process
Two independent reviewers will read the abstracts of the studies identified by the literature search. The reviewers use the abstract to determine whether the studies meet the inclusion criteria. If a final assessment is not possible based on the abstract alone, the assessment will be based on the full text of the publication. A study is included or excluded from the systematic review if both reviewers unanimously decide. If no agreement can be reached between the two independent reviewers, a third, independent reviewer will be used as arbiter in the selection process. Duplicates and multiple reports of the same study will be identified and excluded or collated, so that each study rather than each report will be the unit of interest in the review. The record selection process will be recorded in sufficient detail to complete a PRISMA flow diagram.
Data collection process
A standardized data collection form will be used for the collection of study characteristics and outcome data. The form will be piloted on at least one study included in the review. One review author will independently extract study characteristics and results from included studies. The following study characteristics and results will be extracted.
Data items
From the full texts of the selected publications, the following data will be collected for the overall study population and the defined subgroups, if available from the publication:
-
General information on the publication: title, authors, date of publication, status of publication, journal in which the manuscript was published, language of the publication, funding of the study.
-
Study design
-
Disease for which the study participants were treated (carcinoma of the esophagus, stomach, pancreas, or rectum)
-
Patient characteristics: sex, age, category in the ASA physical status classification system (16), ECOG Performance Status (17)
-
Pretherapeutic cTNM stage
-
Description of the surgical approach(es)
-
Possible neoadjuvant therapy:
Regimen and number of cycles as well as dosage of chemotherapeutic drugs
-
Lymph node yield during resection (total number of histopathologically identified lymph nodes in the resection specimen)
-
Positive lymph nodes (number of lymph nodes in the resection specimen with histopathological confirmation of tumor invasion)
-
Lymph node ratio (number of positive lymph nodes divided by the lymph node yield)
-
Duration of the operation
-
Postoperative complications (if available, according to the Clavien-Dindo classification (18))
-
pTNM stage (from resection specimen)
-
Overall survival time
-
Disease-free survival time
Outcomes and prioritization
A meta-analysis will be conducted for the following outcomes:
Primary outcome
Secondary outcomes
-
Positive lymph nodes
-
Lymph node ratio
-
Duration of the operation
-
Postoperative complications (if available, according to the Clavien-Dindo classification (18))
-
pTNM stage
-
Overall survival time
-
Disease-free survival time
Risk of bias in individual studies
The risk of bias of the individual studies will be estimated according to the respective study design.
For non-randomized studies, the ROBINS-I tool developed by the Cochrane Collaboration will be used. A full description of this tool can be found in the Cochrane Handbook (19). Prior to assessment, an emulated ideal randomized controlled trial aiming to answer the research question will be conceived. This trial will serve as a reference against which the actual studies are compared regarding their risk of bias in the single domains. The following domains of bias will be considered:
At-intervention domain
Post-intervention domains
-
Bias due to deviations from intended interventions
-
Bias due to missing data
-
Bias in measurement of the outcome
-
Bias in selection of the reported result
Specifically for the present meta-analysis, the following confounding domains will be addressed: pretherapeutic tumor stage, pretherapeutic physical status, age. These are supposed to predict whether a study participant undergoes neoadjuvant therapy or not. A specific co-intervention, which could be related to the intervention received and which is at the same time prognostic for the outcome of interest, is surgical approach. This will also be considered specifically as potential source of confounding bias.
Table 1
Overall risk-of-bias judgement | Interpretation | Criterion |
Low risk of bias | The study is comparable to a well-performed randomized trial. | The study is judged to be at low risk of bias for all domains for this result. |
Moderate risk of bias | The study appears to provide sound evidence for a non-randomized study but cannot be considered comparable to a well-performed randomized trial. | The study is judged to be at low or moderate risk of bias for all domains. |
Serious risk of bias | The study has one or more important problems. | The study is judged to be at serious risk of bias in at least one domain, but not at critical risk of bias in any domain. |
For each domain, the tool foresees ‘signaling questions’ whose response options are “yes” / “probably yes” / “probably no” / “no” / “no information”. Based on the responses, the risk of bias for each domain will be judged as ‘low’, ‘moderate’, ‘serious’, ‘critical’ or ‘no information’. From the risk of bias for the single domains, an overall risk of bias for the study will be ascertained according to Table 1.
For randomized trials, the RoB 2 tool developed by the Cochrane Collaboration will be used. A full description of this tool can be found in the Cochrane Handbook (20). The following domains of bias will be considered:
-
Bias arising from the randomization process
-
Bias due to deviations from intended interventions
-
Bias due to missing outcome data
-
Bias in measurement of the outcome
-
Bias in selection of the reported result
For each domain, the tool foresees ‘signaling questions’ whose response options are “yes” / “probably yes” / “probably no” / “no” / “no information”. Based on the responses, the risk of bias for each domain will be judged as ‘low’, ‘moderate’, ‘serious’, ‘critical’ or ‘no information’. From the risk of bias for the single domains, an overall risk of bias for the study will be ascertained according to Table 2:
Table 2
Overall risk-of-bias judgement | Criteria |
Low risk of bias | The trial is judged to be at low risk of bias for all domains for this result. |
Some concerns | The trial is judged to raise some concerns in at least one domain for this result, but not to be at high risk of bias for any domain. |
High risk of bias | The trial is judged to be at high risk of bias in at least one domain for this result. Or The trial is judged to have some concerns for multiple domains in a way that substantially lowers confidence in the result. |
The overall risk of bias determined for the single studies (low, moderate / some concerns, high / serious) will be used to include the respective study in subsequent sensitivity analyses (see below).
Data synthesis
The primary outcome (lymph node yield) will be reported separately for the intervention group (neoadjuvant therapy) and control group (upfront surgery) as weighted mean with standard deviation. If studies do not report mean and standard deviation, these will be calculated using the methods described by the Cochrane Collaboration and Hozo et al. (21, 22). The groups will be compared using the weighted mean difference (and relative difference of standard deviation) for which 95% confidence intervals will be calculated. A forest plot will be drawn. The same analysis of the primary outcome will be done for the defined subgroups of patients with neoadjuvant chemotherapy versus neoadjuvant radiotherapy versus neoadjuvant chemoradiotherapy and esophageal versus gastric versus pancreatic versus rectal cancer.
The secondary outcomes number of positive lymph nodes, lymph node ratio and duration of the operation will be assessed in the same way. The secondary outcome postoperative complications will be dichotomized (grade 1 and 2 versus grade 3a and higher according to the Clavien-Dindo classification). The incidence of severe complications (grade 3a and higher) per group will be determined and compared using the chi2−test and a forest plot. Rates for the secondary outcomes overall and disease-free survival at 1, 3, and 5 years will be compared using weighted rates and a forest plot. The histopathological tumor stage (pTNM) will be qualitatively described for the groups.
Sensitivity analyses will be conducted according to ascertained risk of bias as described above. For these, all studies with a high/serious risk of bias will be excluded and the analyses of the primary outcome, as described above, will be conducted.
The I² statistic, p value from chi2 test, and the between-study heterogeneity τ2 will be used to assess heterogeneity among the trials in each analysis. If substantial heterogeneity (greater than 50%) is identified, reasons for this will be sought by performing subgroup analyses considering the specified subgroups and the causes of heterogeneity. Heterogeneity will also be assessed by evaluating whether there is good overlap of confidence intervals. Any statistical heterogeneity will be taken into account when interpreting the results.
To assess possible publication bias, if the number of included studies is sufficient, we will create a funnel plot using the primary outcome and evaluate funnel asymmetry with Begg's and Egger's tests (23, 24) with respect to continuous data or Peters' test with respect to binary data (25).
Assessing the strength of the body of evidence
A 'Summary of findings' table will be created using the five GRADE considerations (study limitations, consistency of effect, imprecision, indirectness, and publication bias) to assess the quality of the body of evidence based on the studies that contributed data to the meta-analyses for each outcome, classifying it as high, moderate, low or very low. The methods and recommendations described in Chap. 14 of the Cochrane Handbook for Systematic Reviews of Interventions will be used. (26)