The Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocols (PRISMA-P) was used to guide the reporting of this protocol and can be found in Appendix A.29 The PRISMA statement will be utilized to guide the reporting of this review to ensure transparency; a completed and detailed checklist will be provided with the publication. 30 This systematic review will be conducted per Riley et al. and recommendations of the Prognosis Research Strategy (PROGRESS) Group. 31–34 The critical appraisal and data extraction for systematic reviews of prediction modelling studies (CHARMS) checklist was used to frame the research question.34,35
Data Sources and Search Strategy
We searched the following electronic databases from inception until August 2020: (a) Medline Epub Ahead of Print, In-Process and Other Non-Index citations, (b) Pubmed exclusive of Medline citations, (c) EMBASE, (d) CINAHL, and (e) Web of Science. We consulted an academic-affiliated librarian (RC) for the systematic literature search and developed a conservative search strategy for all prognostic factors reported in patients who experience a cardiac arrest, cognizant of the fact that search strategies focused on distinct factors or outcomes may overlook eligible articles. We included search terms informed by Haynes’ sensitive strategies for clinical prediction guides and etiology/risk, and by prior prognostic factor reviews.36 Text and MeSH terms for CPR and cardiac arrest will be utilized along with a combination of prognosis terms.34 A detailed search strategy can be found in Appendix B. Citation tracking will be conducted on all eligible studies to highlight any articles potentially missed by our search strategy. The search will be restricted to adults and non-animal studies. Further, we will exclude non-conference proceedings and conference abstracts as the limited methodological descriptions inhibit assessors from determining the risk of bias (ROB). Case studies and case series will also be excluded, given their lack of a comparison group.
Study Selection and Screening
We plan to include observational study designs (prospective and retrospective) that enrolled adults (≥ 90% of the sample aged 18 and older) who received CPR and reported an association between frailty and any of the outcomes mentioned above. Additionally, studies will be required to provide an explicit definition or description of the frailty instrument used for inclusion in this review. We elected not to limit our search to older adults, given that frailty can also be found in younger populations.37 We plan to exclude studies if: (a) they were purposed to examine the efficacy of a specific clinical treatment, (b) they did not specify the timing of outcome measurement, (c) frailty was defined using a single measure (e.g., laboratory finding, radiographic imaging, etc.), (d) studies that only measured outcomes beyond one year from the date of cardiac arrest data (unless time-to-event analysis was completed), and (e) studies that only evaluated patients receiving a particular clinical therapy (e.g., therapeutic hypothermia, extracorporeal membrane oxygenation, etc.) Studies will not be excluded based on language, sample size, or time of publication. Studies with missing or insufficient data to generate effect estimates or estimates of precision (95% confidence intervals) will be excluded from the meta-analysis if the necessary information cannot be obtained from the appointed corresponding authors. In this situation, the data will be reported narratively.
Titles and abstracts will be imported into Covidence software (Melbourne, Australia), where citations will be screened and duplicates removed. Title, abstract, and full-text screening will be conducted independently and in duplicate by four reviewers (FM, RS, RHC, DM). A standardized and piloted form was created and will be utilized during title and abstract screening; this form is shown in Appendix C. Cases of disagreement over titles and abstracts will be included for full-text review. Any discourse between reviewers regarding study inclusion following full-text review will be resolved through discussion, and if necessary, through independent adjudication by a third study reviewer (F.F). If a decision cannot be determined regarding the study eligibility, reviewers will consult a content expert (J.M). Inter-rater agreement of full-text screening will be reported as a kappa statistic.38
Data Extraction
Data will be extracted independently and in duplicate by four reviewers (FM, RHC, RS, DM). A standardized and pilot-tested data collection form will be created to ensure consistency of extraction. The following data from eligible studies will be extracted: author(s), year of publication, study design, single versus multisite, country of study, inclusion and exclusion criteria, recruitment time frame (months), follow-up length, total sample size, the proportion of adults ≥ 65 years of age, definition and timing of outcome, number of events, baseline demographics (e.g., age, sex, frailty status/score), the prognostic factors entered into the multivariable model, unit of change for continuous predictors, classification for categorical predictors, the unadjusted and adjusted point estimates of risk and lower and upper confidence intervals. All extracted data values will be rounded to two decimal places. Abstracted data will be recorded using Microsoft Excel.®
Data Synthesis and Analysis
Data will be synthesized using R and the ‘meta’ package.39 We plan on generating point estimates and their respective 95% confidence intervals using relative risks, odds ratios or hazard ratios, where appropriate. Study results will be pooled according to the specific type of frailty instrument used, with modified versions of scales included with the original version. Where more than one frailty measure is used in a study, each measure will provide specific information for that class of frailty instrument.
We plan to synthesize and report both univariable and multivariable estimates separately to determine if frailty is robust to the bias of confounding. If baseline risk estimates are not reported, the conversion is not possible, or we are unable to obtain this data from study authors, we will conduct subgroup analyses to compare studies based on the format of the effect estimate. Specifically, we will compare studies that provide information on baseline risk, where we can appropriately convert OR, RR, and HR to studies where conversion is not possible or appropriate. In the latter scenario, we will report the HR or OR. A hot-deck approach will be used to impute an associated variance for such variables.40 We will conduct a sensitivity analysis to determine the influence that data imputation has on pooled estimates. A random-effects model will be used for all statistical pooling,41 mindful of the fact that models of care and patient populations are likely to vary between regions.
Risk of Bias within Studies
We will examine the risk of bias of the individual studies using the Quality in Prognostic Studies (QUIPS) instrument.42 The risk of bias will be determined through the examination of six distinct domains: study participation, study attrition, prognostic factors, outcome measurement, study confounding, statistical analysis and reporting. The QUIPS tool will be modified to ensure better sensitivity in identifying overfit statistical models (e.g., < 10 events for each binary predictor). We will use the individual domains, rated as low or high risk of bias, to inform the overall risk of bias in each study. Five or six low-risk domains will result in a classification of low risk of bias, whereas two or more high-risk domains will result in a classification of ‘high’ risk of bias. Paired reviewers will independently assess each study using the modified QUIPS tool.
Sources of Heterogeneity
Statistical heterogeneity will be assessed through the visual inspection of forest plots, examining the consistency among point estimates and overlap among the associated confidence intervals, and the chi-square test for homogeneity. The inconsistency index (I2) measure will only be used if the majority of studies include < 500 patients, given that prior work has demonstrated a lack of variance in the I2 measure among studies with large sample sizes.43,44 Clinical and methodological heterogeneity will be examined to identify factors that may modify the association between frailty and the outcomes of interest and to determine the appropriateness of meta-analysis. Six effect modifiers are of interest: location of the arrest, etiology of the arrest, the use of advanced cardiac life support, the threshold used to define frailty within a measurement, adjustment for a minimum confounder set, and aspects related to the risk of bias within studies.
- Location. Effect estimates will be compared between studies that focus primarily on IHCA and those that focus on OHCA.
- Etiology of Arrest. Effect estimates will be compared between patients who arrest due to traumatic injury (e.g., blunt or penetrating trauma) versus those who arrest as a result of a medical condition (e.g., myocardial infarction).
- Basic versus Advanced Life Support. Effect estimates will be compared between those who receive basic life support (e.g., chest compressions and ventilation) and those who received additional measures for advanced cardiac life support (e.g., epinephrine administration, endotracheal intubation, etc.)
- Measurement. Effect estimates will be compared across studies that use the same instrument but differing thresholds to define frailty.
- Confounders. Effect estimates will be compared across studies that statistically adjusted for patient age, studies that adjusted for both patient age and the presence of a shockable rhythm (per current Advanced Cardiac Life Support guidelines), and finally for studies without adjustment for these factors.
- Risk of Bias. Effect estimates will be compared across studies with high and low risk of bias.
Confidence in Estimates
Overall confidence in estimates will be determined using the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) approach.43,45 Per GRADE recommendations, confidence will be rated as either high, moderate, low, or very low. In accordance with GRADE guidance for prognostic studies, observational studies will be considered high confidence until proven otherwise.43 For meta-analyses with more than ten studies, funnel plots will be produced to examine the distribution of positive and negative findings allowing for the detection of publication bias.43 An assessment of confidence will be given for each outcome individually. Congruency of pooled estimates between our sensitivity analysis (including imputed non-significant studies evaluating the predictor of interest) and our primary model (not including the imputed non-significant predictor) will be reported and taken into consideration when discussing our confidence in the estimates.