Which interactions matter in economic evaluations? A systematic review and simulation study
Background: We aimed to assess the magnitude of interactions in costs, quality-adjusted life-years (QALYs) and net benefits within a sample of published economic evaluations of factorial randomised controlled trials (RCTs), evaluate the impact that different analytical methods would have had on the results and compare the performance of different criteria for identifying which interactions should be taken into account.
Methods: We conducted a systematic review of full economic evaluations conducted alongside factorial RCTs and reviewed the methods used in different studies, as well as the incidence, magnitude, statistical significance, and type of interactions observed within the trials. We developed the interaction-effect ratio as a measure of the magnitude of interactions relative to main effects. For those studies reporting sufficient data, we assessed whether changing the form of analysis to ignore or include interactions would have changed the conclusions. We evaluated how well different criteria for identifying which interactions should be taken into account in the analysis would perform in practice, using simulated data generated to match the summary statistics of the studies identified in the review.
Results: Large interactions for economic endpoints occurred frequently within the 40 studies identified in the review, although interactions rarely changed the conclusions.
Conclusions: Simulation work demonstrated that in analyses of factorial RCTs, taking account of all interactions or including interactions above a certain size (regardless of statistical significance) minimised the opportunity cost from adopting treatments that do not in fact have the highest true net benefit.
Figure 1
This is a list of supplementary files associated with this preprint. Click to download.
Posted 15 Apr, 2020
On 12 Apr, 2020
On 11 Apr, 2020
On 11 Apr, 2020
On 01 Apr, 2020
Received 01 Apr, 2020
On 01 Apr, 2020
Invitations sent on 12 Mar, 2020
On 11 Mar, 2020
On 10 Mar, 2020
On 10 Mar, 2020
On 14 Feb, 2020
Received 29 Jan, 2020
On 12 Dec, 2019
Received 26 Oct, 2019
On 09 Oct, 2019
Invitations sent on 25 Sep, 2019
On 28 Aug, 2019
On 28 Aug, 2019
On 26 Aug, 2019
On 21 Aug, 2019
Which interactions matter in economic evaluations? A systematic review and simulation study
Posted 15 Apr, 2020
On 12 Apr, 2020
On 11 Apr, 2020
On 11 Apr, 2020
On 01 Apr, 2020
Received 01 Apr, 2020
On 01 Apr, 2020
Invitations sent on 12 Mar, 2020
On 11 Mar, 2020
On 10 Mar, 2020
On 10 Mar, 2020
On 14 Feb, 2020
Received 29 Jan, 2020
On 12 Dec, 2019
Received 26 Oct, 2019
On 09 Oct, 2019
Invitations sent on 25 Sep, 2019
On 28 Aug, 2019
On 28 Aug, 2019
On 26 Aug, 2019
On 21 Aug, 2019
Background: We aimed to assess the magnitude of interactions in costs, quality-adjusted life-years (QALYs) and net benefits within a sample of published economic evaluations of factorial randomised controlled trials (RCTs), evaluate the impact that different analytical methods would have had on the results and compare the performance of different criteria for identifying which interactions should be taken into account.
Methods: We conducted a systematic review of full economic evaluations conducted alongside factorial RCTs and reviewed the methods used in different studies, as well as the incidence, magnitude, statistical significance, and type of interactions observed within the trials. We developed the interaction-effect ratio as a measure of the magnitude of interactions relative to main effects. For those studies reporting sufficient data, we assessed whether changing the form of analysis to ignore or include interactions would have changed the conclusions. We evaluated how well different criteria for identifying which interactions should be taken into account in the analysis would perform in practice, using simulated data generated to match the summary statistics of the studies identified in the review.
Results: Large interactions for economic endpoints occurred frequently within the 40 studies identified in the review, although interactions rarely changed the conclusions.
Conclusions: Simulation work demonstrated that in analyses of factorial RCTs, taking account of all interactions or including interactions above a certain size (regardless of statistical significance) minimised the opportunity cost from adopting treatments that do not in fact have the highest true net benefit.
Figure 1