Bias Estimation In Study Design: A Meta-Epidemiological Analysis of Transcatheter Versus Surgical Aortic Valve Replacement
Objective: To estimate the bias associated with specific nonrandomized study attributes among studies comparing transcatheter aortic valve implantation with surgical aortic valve replacement for the treatment of severe aortic stenosis.
Data sources and study selection: We searched 7 databases from inception to June 2017: Medline, Medline In-Process/ePubs, Embase, Cochrane Central Register of Controlled Trials, Cochrane Database of Systematic Reviews, Scopus, and Web of Science. We included all RCTs and nonrandomized studies that reported outcomes of interest.
Data extraction and synthesis: We categorized studies according to study design, and evaluated 41 nonrandomized study attributes as potential sources of bias. We calculated odds ratios (OR) and other effect measures with 95% confidence intervals (CI) using random effects models.
Main outcomes: One month postoperative mortality, and length of stay. Bias was defined as the difference in estimates of treatment effects between nonrandomized studies and high quality (low risk of bias) RCTs, which were considered to provide “gold standard” estimates.
Results: We included 6 RCTs and 87 nonrandomized studies. Surgical risk scores were similar for comparison groups in RCTs, but were higher for patients having transcatheter aortic valve implantation in nonrandomized studies. Nonrandomized studies underestimated the benefit of transcatheter aortic valve implantation compared with RCTs. For example, nonrandomized studies without adjustment estimated a higher risk of postoperative mortality for transcatheter aortic valve implantation compared with surgical aortic valve replacement (OR 1.43 [95% CI, 1.26 to 1.62]) than high quality RCTs (OR 0.78 [95% CI, 0.54 to 1.11). Nonrandomized studies using propensity score matching (OR 1.13 [95% CI, 0.85 to 1.52]) and regression modelling (OR 0.68 [95% CI, 0.57 to 0.81]) to adjust results estimated treatment effects closer to high quality RCTs. Nonrandomized studies describing losses to follow-up estimated treatment effects that were significantly closer to high quality RCT than nonrandomized studies that did not.
Conclusion: Studies with different attributes produce different estimates of treatment effects. Study design attributes related to the completeness of follow-up may explain biased treatment estimates in nonrandomized studies, as in the case of aortic valve replacement where high-risk patients were preferentially selected for the newer (transcatheter) procedure.
Figure 1
Figure 2
Figure 3
This is a list of supplementary files associated with this preprint. Click to download.
Posted 24 Sep, 2020
On 23 Dec, 2020
Received 17 Oct, 2020
On 09 Oct, 2020
Invitations sent on 08 Oct, 2020
On 02 Oct, 2020
On 22 Sep, 2020
On 18 Sep, 2020
On 02 Sep, 2020
Bias Estimation In Study Design: A Meta-Epidemiological Analysis of Transcatheter Versus Surgical Aortic Valve Replacement
Posted 24 Sep, 2020
On 23 Dec, 2020
Received 17 Oct, 2020
On 09 Oct, 2020
Invitations sent on 08 Oct, 2020
On 02 Oct, 2020
On 22 Sep, 2020
On 18 Sep, 2020
On 02 Sep, 2020
Objective: To estimate the bias associated with specific nonrandomized study attributes among studies comparing transcatheter aortic valve implantation with surgical aortic valve replacement for the treatment of severe aortic stenosis.
Data sources and study selection: We searched 7 databases from inception to June 2017: Medline, Medline In-Process/ePubs, Embase, Cochrane Central Register of Controlled Trials, Cochrane Database of Systematic Reviews, Scopus, and Web of Science. We included all RCTs and nonrandomized studies that reported outcomes of interest.
Data extraction and synthesis: We categorized studies according to study design, and evaluated 41 nonrandomized study attributes as potential sources of bias. We calculated odds ratios (OR) and other effect measures with 95% confidence intervals (CI) using random effects models.
Main outcomes: One month postoperative mortality, and length of stay. Bias was defined as the difference in estimates of treatment effects between nonrandomized studies and high quality (low risk of bias) RCTs, which were considered to provide “gold standard” estimates.
Results: We included 6 RCTs and 87 nonrandomized studies. Surgical risk scores were similar for comparison groups in RCTs, but were higher for patients having transcatheter aortic valve implantation in nonrandomized studies. Nonrandomized studies underestimated the benefit of transcatheter aortic valve implantation compared with RCTs. For example, nonrandomized studies without adjustment estimated a higher risk of postoperative mortality for transcatheter aortic valve implantation compared with surgical aortic valve replacement (OR 1.43 [95% CI, 1.26 to 1.62]) than high quality RCTs (OR 0.78 [95% CI, 0.54 to 1.11). Nonrandomized studies using propensity score matching (OR 1.13 [95% CI, 0.85 to 1.52]) and regression modelling (OR 0.68 [95% CI, 0.57 to 0.81]) to adjust results estimated treatment effects closer to high quality RCTs. Nonrandomized studies describing losses to follow-up estimated treatment effects that were significantly closer to high quality RCT than nonrandomized studies that did not.
Conclusion: Studies with different attributes produce different estimates of treatment effects. Study design attributes related to the completeness of follow-up may explain biased treatment estimates in nonrandomized studies, as in the case of aortic valve replacement where high-risk patients were preferentially selected for the newer (transcatheter) procedure.
Figure 1
Figure 2
Figure 3