Background Before and after studies allow for the investigation of population-level health interventions and are a valuable study design in situations where randomisation is not feasible. The before and after study design involves measuring an outcome both before and after an intervention and comparing the outcome rates in both time periods to determine the effectiveness of the intervention. These studies do not involve a contemporaneous control group and must therefore take into account any underlying secular trends in order to separate the effect of the intervention from any pre-existing trend. Neglecting this important step can lead to spurious results.
Methods To illustrate the importance of accounting for underlying trends, we performed a before and after study assessing 30-day mortality in hip fracture patients without any actual intervention, and instead designated an arbitrarily-chosen time-point as our ‘intervention’. We did this to ensure that we were basing our results exclusively on the underlying trend throughout the studied period and also to enable us to show that even an intervention of nothing may be spuriously interpreted to have an effect if the before and after study is incorrectly analysed.
Results We found a secular trend in our data showing improving 30-day mortality in hip fracture patients in our institution. We then demonstrated that disregarding this underlying trend showed that our intervention of nothing ‘resulted’ in a significant decrease in mortality, from 6.7% in the ‘before’ period to 3.1% in the ‘after’ period (p<0.0008). This apparent impact on mortality disappeared when we accounted for the underlying trend in our analysis (IRR of 0.75, 95% CI 0.32 – 1.78; p=0.5). In the context of declining 30-day mortality following hip fracture, failure to consider the existing underlying trend lead us to believe that it was our ‘intervention’ that ‘caused’ the decrease in mortality in the ’after’ period compared to the ‘before’ period when our results clearly show that mortality was decreasing irrespective of any intervention.
Conclusion Our study highlights the importance of appropriate measurement and consideration of underlying trends when analysing data from before and after studies and illustrates what can happen should researchers neglect this important step.