Effectiveness of a Health Technician-delivered Brief Intervention for alcohol A Bayesian reanalysis of a clinical trial

doi:10.21203/rs.3.rs-1455486/v1

PDF

Research Article

Effectiveness of a Health Technician-delivered Brief Intervention for alcohol A Bayesian reanalysis of a clinical trial

https://doi.org/10.21203/rs.3.rs-1455486/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

Objective:

To reanalyze a clinical trial on the effectiveness of a Brief Intervention (BI) delivered by non-professionals to reduce risky alcohol drinking. Our previous null-hypothesis test of the effects of the BI yielded a ‘non-significant’ p-value, yet remained uninformative. Here we use the Bayesian paradigm which allows for expressing the probability of different effect sizes to better inform public policy decisions.

Results:

The posterior probability of the odds of risky drinking at follow-up favored a marked effect of the BI, with 96% of the probability mass being less than OR = 1, and 84% being less than OR = 0.8. Our findings show that there is a high probability that the BI delivered by health technicians lowered risky alcohol use. The posterior distributions of the BI’s effects are presented to help contextualize the evidence for policy making in Chile.

Alcohol brief interventions

non-professionals

Bayesian analysis.

Excessive alcohol drinking is the leading cause of mortality and disability in the Chilean population[1]. To address this issue, a nation-wide preventive program in primary care offers detection and brief interventions (BI) for excessive drinking, provided mainly by Health Technicians (HTs)[2]. These health workers have two and a half years of education and training, and perform routine healthcare tasks such as medical check-ups and measuring patients’ vital signs. As alcohol BIs are delivered by physicians in high income countries[3], less is known about BIs’ effectiveness when provided by non-professional health workers. In this context, we conducted a randomized controlled trial (RCT) to inform policy makers regarding the potential impact of the national program[4]. Our main finding was an OR of 0.6 (CI 0.34–1.05) for risk reduction with a p-value of 0.07, from which we concluded that there was no observed effect attributable to the BI, and thus, the program could make better use of resources by not providing one-to-one HT-led interventions.

This conclusion followed the convention that p-values over 0.05 indicate that a null effect cannot be ruled out; however, this convention is problematic for a number of reasons[5], including that point estimates are very sensitive to single data points, failing to acknowledge that a range of effect sizes cannot be ruled out, and not incorporating previous scientific knowledge and contextual factors for decision-making. Moreover, dichotomizing the evidence as if only two categories existed, efficacious or not efficacious, is not fully informative for policy making, particularly when done based on an arbitrary cutoff. On the contrary, this public health decision should rely on comprehensive criteria that incorporates a nuanced understanding of the possible effects of the brief intervention in order to justify its nation-wide implementation. In fact, the program is being implemented in Chile, so the decisions on maintaining, intensifying, or stopping the program would rather rely on the likelihood that marked effects are being obtained.

To answer these questions, the null hypothesis test paradigm (NHTP) offers little help, since it typically dichotomizes the evidence instead of presenting the whole picture. These difficulties in analyzing and communicating our results motivated us to redo the analyses from a Bayesian perspective. This paradigm allows for consideration of different effect sizes by estimating their compatibility with the observed data[6]. Here we present the results of a reanalysis of data from this clinical trial using Bayesian inference.

Trial summary

The clinical trial was conducted in five primary care facilities in the city of Santiago – Chile and compared two interventions in people with risky alcohol use (i.e., Alcohol Use Disorders Identification Test (AUDIT) total score between 8 and 15)[7]. One group received an informative pamphlet (n = 168), and the other received the same pamphlet plus a HT-delivered BI based on the contents from the pamphlet (n = 174). The BI was based on motivational interview and is similar in contents and duration to a standard intervention for risky alcohol use.

Our outcomes were the AUDIT risk category (primary outcome), and the AUDIT-total and AUDIT-C scores (i.e., sum of items one to three), measured 6 months after enrollment. For practical reasons, the only outcome measurement was a one-year retrospective AUDIT. This minimal outcome collection allowed protocol implementation without changing much the way the program operates in the real world.

Previous analysis

All analysis were conducted on participants that completed follow-up (n = 294) using Chi-squared and T-tests accordingly. Then, mixed-effects regressions were used to adjust for demographic variables as fixed-effects (age, sex, and educational level) and the health center as random effects. See the main paper for a detailed description of the trial[4].

Current analysis

We used Bayesian inference to estimate the same mixed-effects regression models specified in the original analysis. The alcohol risk status (i.e., primary outcome), was modeled with a Bernoulli distribution with fixed effects for all the covariates, and random intercepts for health centers. The model for the primary outcome is presented in Eq. 1.

Risky drinking ~ Bernoulli ( q )

\(\text{log}\left(\frac{q}{1-q}\right)\) = \(\beta\)₁ + \(\beta\)₂ Group + \(\beta\)₃ Sex + \(\beta\)₄ Education(complete) + \(\beta\)₅ Education(superior) + \(\beta\)₆ Age + C Center

\(\beta\) _[1−6] ~ Normal ( 0, 1 )

C ~ Normal ( 0, sigma_c )

sigma_c ~ Normal ( 0, 1 )

AUDIT-total and AUDIT-C scores (i.e., secondary outcomes), were modeled with normal distributions with fixed effects for all covariates and random intercepts for health centers. The model for the secondary outcomes is presented in Eq. 2.

AUDIT-score ~ Normal ( mu, sigma )

mu = \(\beta\)₁ + \(\beta\)₂ Group + \(\beta\)₃ Sex + \(\beta\)₄ Education(complete) + \(\beta\)₅ Education(superior) + \(\beta\)₆ Age + \(\beta\)₇ AUDIT-score(baseline) + C Center

\(\beta\) _[1−7] ~ Normal ( 0, 1 )

sigma = Normal ( 0, 1 )

C ~ Normal ( 0, sigma_c )

sigma_c ~ Normal ( 0, 1 )

Hamiltonian Markov chain Monte Carlo was used for Bayesian inference. For each model, 50,000 iterations were run with 25,000 warm-up iterations in four chains. The analyses were run using R with R-Stan version 2.21.0 on a Mac mini M1.

The marginal posterior probability distribution for the fixed effect OR of risky drinking is shown in Fig. 1. As can be seen, a strong majority of the distribution is located to the left of the null (i.e., OR = 1), with 96% of the probability mass being less than OR = 1, and 84% being less than OR = 0.8. The mean of the distribution is located at OR = 0.63. Similarly, Fig. 2 shows the marginal posterior probability distributions for the AUDIT scores, where the probability that the difference in means is less that 0 was 98% for the AUDIT-total and 97% for the AUDIT-C. The mean of these distributions are located at MD = 0.74 for the AUDIT-total and at MD = 0.4 for the AUDIT-C. Table 1 shows the posterior probabilities of different effect sizes for both primary and secondary outcomes.

Table 1

Cumulative probabilities of possible intervention´s effects
	Posterior probability ^a
Risk reduction ^b
odds-ratio < 1	0.96
odds-ratio < 0.8	0.84
odds-ratio < 0.6	0.48
AUDIT´s total score ^c
any reduction	0.98
at least 0.5 points	0.74
at least 1 point	0.24
AUDIT - C ^c
any reduction	0.97
at least 0.25 points	0.77
at least 0.5 point	0.33
a. Proportion of the posterior probability distribution. b. Primary outcome. The proportion of participants with low risk (i.e., AUDIT-total < 8), at 6 months follow-up. c. Secondary outcomes. AUDIT scores at 6 months follow-up.

We used Bayesian analysis to reanalyze a clinical trial on the effectiveness of a BI to reduce risky alcohol drinking and found that there was marked evidence suggesting that the intervention reduced risky alcohol use. In concordance with most evidence on BIs[3, 8], this effect was small but clear; the probability that risky drinking was less in the BI-group compared to the pamphlet group six months post-treatment was 96%. Furthermore, the probability that the OR comparing the two groups was less than 0.8 was 84%. Our findings support the effectiveness of this BI. In the following paragraphs we put these results in the context of our previous analysis and other factors relevant to decision-making for public policy.

First, we address an important difference with respect to scientific inference from the original analysis and this reanalysis. Our current results show the degree to which the data from the trial support the BIs effectiveness, indicating a 96% probability of a positive effect. However, our previous analyses following a NHTP framework obtained a p-value of 0.07; that is, in a hypothetical world where there is exactly no difference between exposure to the BI or the pamphlet, the probability of observing the data from the trial was 7%. Conventionally, this means that a null effect cannot be ruled out, since only extreme observations yielding p-values less than 0.05 are regarded as strong evidence under this paradigm[9]. This usually implies that interventions are ruled ineffective, despite the data supporting a range of effect sizes different from the null. Even though this way of thinking has its merits in protecting us from type 1 error, it also hinders scientific inference when taken as the only criterion to decide if there is an important effect of the intervention. In fact, inference cannot rely only on a statistical model, as recently cautioned by the American Psychology Association[10].

With the Bayesian analysis conducted herein, we obtained a probability distribution over the effects of the BI (in comparison to the pamphlet), allowing us to assess the evidence in a continuous rather than dichotomic manner. This makes it possible to interpret the results in context of existing evidence, specific decision-making contexts, and other available interventions. In our models, we used normal priors centered around the null which encode a conservative view of the effects of the intervention before the data was consulted. Bayesian inference entails updating this prior assumption with the data, migrating to the posterior-probability distributions summarized in Table 1 and shown in Figs. 1 and 2. This means that our effect estimates have been ‘pulled’ towards the null, reducing the risk of spurious findings and reducing the influence of single data points[11]. The posterior distributions show how compatible different effect estimates are with the data collected in the trial, and how unlikely it is that the BI had no effect, or had the effect of increasing risky drinking (probability < 4%).

Another factor for inference is some degree of inaccuracy in our measurements: as this trial was conducted in real world conditions, we refrained from using more precise but demanding instruments. In fact, we used the AUDIT with a one-year retrospective window, which is less sensitive to changes occurred during the last six months (i.e., the study timeframe). However, despite using a one-year retrospective questionnaire, our data strongly indicate that the intervention had an effect.

Altogether, our analyses indicate effectiveness of the BI, and so, the use of HTs as providers is supported by this evidence. Based on this evidence, non-professional health care personal could remain delivering BIs in Chilean primary care: this strategy uses the available health workers to implement a crucial preventive program that otherwise would not be feasible. A related question remains on how these HT-delivered BIs compare in effectiveness to a physician or nurse-delivered BI. Most of the evidence on effectiveness and cost-effectiveness of BIs derives from professional-delivered BIs, so a comparison among both delivery methods would be informative for low- and middle-income countries.

A health-technician delivered BI reduced risky drinking after six months as compared with the delivery of a pamphlet. The entire posterior distribution of this effect is presented to help contextualize the evidence for policy making in Chile.

A more informative analysis could be perused by incorporating previous knowledge on Brief Interventions. We used less informative assumptions to remain conservative and comparable to our previous analysis.

AUDIT

Alcohol Use Disorders Identification Test

Brief Intervention

Health Technician

NHTP

Null hypothesis test paradigm

Odds ratio

Mean difference

Ethics approval and consent to participate

The study protocol was approved by the Scientific Ethics Committee of the School of Medicine of the Pontificia Universidad Católica de Chile, the Scientific Ethics Committees of the South East Metropolitan Health Service, and the Central Metropolitan Health Service. All participants gave written informed consent.

Availability of data and material

The datasets generated and/or analysed during the current study are available in the Open Science Foundation repository, https://doi.org/10.17605/OSF.IO/AGBH9.

Funding

This study was funded by the Chilean National Agency for Research and Development (ANID)

Authors' Contributions

NB and FP designed and carried out the original trial. NB, FP, LB, and MB conceptualized this new analysis and made meaningful insights from the results. MB and NB performed the Bayesian analysis. All authors edited the manuscript and approved its final version.

Acknowledgements

Not applicable

Consent for publication

Participants authorized the publication of anonymized data and analyses.

Competing interests

Authors have nothing to declare

Ministerio de Salud Chile. Estudio de carga de enfermedad y carga atribuible. Ministerio de Salud de Chile; 2008. http://www.repositoriodigital.minsal.cl/handle/2015/602.
Ministerio de Salud Chile. Intervenciones Breves para reducir el consumo de Alcohol de riesgo. 2011. https://mfr.osf.io/render?url=https://osf.io/b5gry/?direct%26mode=render%26action=download%26mode=render.
Kaner EF, Beyer FR, Muirhead C, Campbell F, Pienaar ED, Bertholet N, et al. Effectiveness of brief alcohol interventions in primary care populations. Cochrane Db Syst Rev. 2018;24:1680.
Barticevic NA, Poblete F, Zuzulich SM, Rodriguez V, Quevedo D, Sena BF, et al. A Health Technician-delivered Brief Intervention linked to AUDIT for reduction of alcohol use in Chilean primary care: a randomized controlled trial. Addict Sci Clin Pract. 2021;16:39.
Nuzzo R. Scientific method: Statistical errors. Nature. 2014;506:150–2.
Bendtsen M. A Gentle Introduction to the Comparison Between Null Hypothesis Testing and Bayesian Analysis: Reanalysis of Two Randomized Controlled Trials. J Med Internet Res. 2018;20:e10873.
Babor TF, Higgins-Biddle JC, Saunders JB, Monteiro MG. The Alcohol Use Disorders Identification Test. Guidelines for use in primary care. Ginebra: World Health Organization; 2001.
Beyer F, Lynch E, Kaner E. Brief Interventions in Primary Care: an Evidence Overview of Practitioner and Digital Intervention Programmes. Curr Addict Reports. 2018;5:265–73.
Benjamin DJ, Berger JO. Three Recommendations for Improving the Use of p-Values. Am Stat. 2019;73:186–91.
Wasserstein RL, Lazar NA. The ASA’s Statement on p-Values: Context, Process, and Purpose. Am Stat. 2016;70:129–33.
Bendtsen M. An Electronic Screening and Brief Intervention for Hazardous and Harmful Drinking Among Swedish University Students: Reanalysis of Findings From a Randomized Controlled Trial Using a Bayesian Framework. J Med Internet Res. 2019;21:e14420.

PDF

Editorial decision: Minor revision
19 Apr, 2022
Reviews received at journal
01 Apr, 2022
Reviewers invited by journal
24 Mar, 2022
Editor invited by journal
23 Mar, 2022
Editor assigned by journal
19 Mar, 2022
First submitted to journal
15 Mar, 2022

You are reading this latest preprint version

Effectiveness of a Health Technician-delivered Brief Intervention for alcohol A Bayesian reanalysis of a clinical trial

Status:

Version 1

Abstract

Objective:

Results:

Figures

Introduction

Methods

Results

Discussion

Conclusion

Limitations

Abbreviations

Declarations

Ethics approval and consent to participate

Availability of data and material

Funding

Acknowledgements

Consent for publication

Competing interests

References

Status:

Version 1