This study demonstrates how collider bias confounds cross-jurisdictional comparative research. As predicted, there was an inverse association between claim rate and disability duration. These served as proxies for compensability thresholds and injury severity, suggesting systematic baseline differences in cohorts. This makes it extremely difficult to differentiate a compensation systems cohort-shaping and outcome-shaping effects.
The association reversed when compensation systems were treated as fixed effects in a textbook example of Simpson’s Paradox, or more technically, Simpson’s Reversal (43). If the positive association can be considered the “true” relationship between claim rate and disability duration (i.e., occupational injury frequency and severity are positively associated in the real world), the results demonstrate how collider bias can mask real effects. To be clear, neither the inverse or positive association are inherently misleading or wrong on their own, though Simpson’s Paradox highlights the importance of matching analysis to the research question (44). In this study, the question was whether differences in compensability thresholds between compensation systems produce a spurious association between claim rates and disability duration. This makes the unnested approach the appropriate one, though the nested approach adds insight.
The clustered ordering of states/territories in Figure 3 provides additional evidence of collider bias due to compensability thresholds, in this case employer excess periods. Victoria and South Australia have the longest excess period in Australia at ten days/two weeks, which is twice the next longest. Correspondingly, both are situated in the upper-left quadrant of Figure 2, denoting fewer claims and longer durations. However, the remaining order of states/territories appears unrelated to employer excess: after Victoria and South Australia comes Western Australia, which has no employer excess period, followed by Tasmania (no employer excess period), Northern Territory (part of first day), New South Wales (one week), and Queensland (around one week). This suggests other compensability thresholds are at work.
The effects of left-censoring and potential for further bias
Left-censoring was applied to account for employer excess periods, a known type of compensability threshold. While the inverse association between claim rate and disability duration attenuated to non-significance with left-censored data, the direction of effect remained negative. However, when Statistical Areas were nested by state and territory, which treated compensability thresholds as fixed effects, the association between claim rate and disability duration was again positive, as it had been in uncensored analysis. As above, if this is the “true” association, it remained masked in unnested analyses even when data were left-censored to account for employer excess periods.
Survival analyses on simulated claims data revealed another potential for reversal of effects due to left-censoring. In a real world setting, this could happen where one system successfully resolves many of the low-severity, easy-to-resolve cases, while another delays their exit until after the censored period, a “depletion of the susceptibles” in reverse (45).
Adding an arbitrary low value to left-censored cases largely accounted for the effects of left-censoring, which is in line with the theoretical paper that proposed this approach (21). However, the simulated data include all censored cases, an advantage that real world data likely lacks. For instance, in real world data the arbitrary low value would be applied to medical-only claims, or those without any recorded time loss. However, we run into a similar problem as with time loss durations: such injuries are only recorded as claims if they are compensated for treatment. Medical care benefits also have compensability thresholds that vary across compensation systems. For instance, as of 2019 Victoria required employers to cover the first $707 of medical costs, while Queensland required employers to pay the first $1,527.80 of combined costs (10). Even for medical-only claims, compensation status is a source of collider bias.
Alternative approaches
There are other ways to test whether and how compensation systems improve or worsen injured worker outcomes. Randomised controlled trials avoid much of the problem of collider bias by randomly allocating exposures, theoretically balancing baseline differences between cohorts. However, these are often ethically or financially impractical and can lack external validity (2,46). Quasi-experiments may be better suited to answering questions about the impact of system settings. These include study designs such as interrupted time series and difference-in-differences, which compare outcomes before and after an event like legislated changes, and regression discontinuity, which use arbitrary cut-offs like wage replacement caps. They can also overcome logistical hurdles of randomised controlled trials through exogenous allocation of large populations to experimental/exposure and control conditions in ways that mimic randomisation (46,47). In some circumstances, quasi-experiments have better external validity than randomised controlled trials because they rely on population-level, real-world data (46).
However, quasi-experiments have important limitations. Events like legislative change are infrequent and may not modify policies of interest, or entirely change the compensation system, making it difficult to differentiate the effects of policy change from service disruption (48). Legislative change may also introduce a collider if it alters compensability thresholds. For instance, we previously found that when New South Wales restricted eligibility to its compensation system in 2012, the claim rate decreased while disability duration increased (49). Such an outcome would be consistent with both a change in the cohort towards more severe and complex injuries and a system that has increased the stressfulness of compensation. At the very least, analyses based on legislative change should examine whether the event affected claim rates (48,50) to pro-actively test for cohort-shaping effects as an indicator of collider bias.
Strengths and limitations
Study strengths include use of population-level claims data from workers’ compensation systems with near-universal coverage of the Australian workforce and the use of simulated data with known characteristics to demonstrate how collider bias can distort statistical associations. To my knowledge, this is the first work disability study to directly engage with the problem of collider bias.
Limitations include the inability to test the proposed mechanism of compensability thresholds directly. I used labour force denominators to estimate claim rates, which are not equivalent to covered worker estimates; variations in the proportion of the workforce who are insured against workplace injury could vary across Statistical Areas or compensation systems and bias claim rate estimates.