Background: A lot of studies have compared the ability of statistical methods to control for confounding. However, a majority of studies mistakenly assumed these methods estimate the same effect. The aim of this study was to use Monte Carlo simulations to compare logistic regression, propensity scores and instrumental variable analysis for estimating their true target odds ratios in terms of bias and precision in the absence and presence of unmeasured confounder.
Methods: We established the formula allowing us to compute the true odds ratio of each method. We varied the instrument’s strength and the unmeasured confounder to cover a large range of scenarios in the simulation study. We then use logistic regression, propensity score matching, propensity score adjustment and two-stage residual inclusion to obtain estimated odds ratios in each scenario.
Results: In the absence of unmeasured confounder, instrumental variable without direct effect on the outcome could produce unbiased estimates as propensity score did, but the mean square errors of instrumental variable were greater. When unmeasured confounder existed, no other method could produce unbiased estimation except instrumental variable, provided that the proposed instrument is not directly related to the outcome. Using the defined instrument, which affected the outcome directly, resulted in positive biased estimation of the treatment effect and this bias was greater compared to that from other methods.
Conclusions: Overall, with good implementation, instrumental variable can lead to unbiased results. However, the bias caused by violating the required assumptions of instrumental variable can overweigh the positive effect of its ability to control for unmeasured confounder.