Background
Despite consensus about the need to assess study design quality in systematic reviews, there remains a need for practical tools that can be used for quality assessment across diverse study designs.
Methods
We developed the Study Quality Assessment of Design (SQUAD) tool by combining and streamlining the Cochrane tool for grading randomized controlled trials and the risk of bias criteria developed for Effective Practice and Organisation of Care (EPOC) reviews. We validated the tool and refined it through a four-iteration pilot. We then used the tool to evaluate the quality of studies in a systematic review of the effects of interpersonal interventions on Quadruple Aim outcomes (i.e., population health, cost, patient and provider experience).
Results
During a pilot with 12 studies (8 randomized controlled trials and 4 observational studies), the ICC (1,1) improved from 0.41 to 0.89. In the systematic review of 77 studies (68 randomized control trials and 9 observational studies), the ICC (2,1) was 0.72.
Conclusions
SQUAD is a practical and reliable tool for assessing the quality of studies of various designs when synthesizing findings for systematic reviews. Standardized practices for quality assessment are critical to the reliability of systematic reviews. This pragmatic and standardized tool can facilitate efficient and high-quality assessments for a broad range of studies.