DOI: https://doi.org/10.21203/rs.3.rs-1289313/v1
Background: The increasing volume of research on electronic health (eHealth) smoking cessation interventions presents a challenge for policymakers and service providers when they try to decide which strategies to select for health promotion programs. This study aimed to explore the perceptions of Australian policymakers and service providers regarding the effectiveness and quality of eHealth and mHealth interventions for smoking cessation against the evidence in a recent systematic review.
Methods: This cross-sectional study invited 38 Australian public health policymakers and service providers in smoking cessation to participate in an online survey assessing their knowledge of the current available evidence-base. Descriptive statistics were used to determine the results of the survey against the published review.
Results: Eighteen participants completed the survey. Comparison identified that the majority of participants failed to correctly identify the effects and the quality of the evidence of popular smoking cessation interventions. They lacked knowledge of the usefulness and quality of some eHealth and mHealth interventions for smoking cessation.
Conclusions: The finding of this study indicates that the prior perceptions and beliefs held by Australian policymakers and service providers about the effects of eHealth and mHealth interventions for smoking cessation are significantly detached enough from the evidence-base to affect the provision of effective interventions to smokers who want to quit. This has potentially widespread implications for improving health promotion and
communication between policymakers, service providers, and the current smoking
population.
Tobacco use is one of the leading causes of preventable diseases and death
worldwide, accounting for over seven million deaths each year (1). These deaths include an estimated 890,000 non-smokers who were indirectly exposed to second-hand smoke (1).The latest estimate suggests smoking kills two in three persistent users in Australia (2). In previous decades, the number of fatalities caused by tobacco use has been vast. In the 50 years from 1960–2010, it was estimated that smoking killed around 821,000 Australians (2). Even today, smoking is still directly responsible for the deaths of nearly 19,000 Australians annually (2). In 2011, tobacco smoking was estimated to be responsible for 80% of the lung cancer burden and 75% of the chronic obstructive pulmonary disease (COPD) burden in Australia (3). Reducing the burden of disease is a priority of public health, as is the prevention of the uptake of smoking and the cessation of smoking by current users. It is the responsibility of those in health policy and service to gather a reliable evidence base so they can ensure they are both accurate and effective in their response and strategies to reduce smoking rates, and thus the burden of disease (4).
The digital support approach, or the electronic health (eHealth) approach is considered a new opportunity for prevention of health-risk behaviours, and for tackling the global burden of smoking by expanding the accessibility of cessation programs to all smokers (5). eHealth interventions have unique advantages, including affordability, efficient delivery, easy accessibility and a wide reach of large segments of the population. Therefore, recent studies suggest that digitally support approaches (5), such as eHealth and mobile (mHealth) interventions, could hold significant promise to improve smoking cessation. Previous systematic reviews at the time investigated the effects of eHealth and mHealth interventions upon smoking cessation outcomes (6–9), however these reviews were limited in scope. A blanket acceptance and implementation of eHealth cessation programs is not only unhelpful, but potentially harmful in the face of a lack of thorough review of their effectiveness. A discerning and reliable evidence base from which policymakers can operate is imperative to achieve this important goal in population health.
Within broad scope of eHealth smoking cessations interventions, some strategies can help people quit; however, several barriers may affect their implementation by policymakers and service providers. The first barrier is the sheer quantity and differences in quality of the published primary studies on eHealth interventions for smoking cessation. The clutter makes it difficult to identify effective and reliable interventions for adoption. It is clear that policymakers and service providers have little time to conduct complex literature searches, screen through results, and then interpret the plethora of results of primary studies (10,11).
Policymakers and service providers have a responsibility to implement eHealth programs demonstrated through research as effective. Systematic reviews provide summary evidence from many research studies that have been risk assessed, and are useful for evidence informed decision-making. Generally, previous systematic reviews in smoking cessation have focused on the effects of particular devices on the outcome of smoking cessation or on a specific age group or population (6–9). However, until recently, no comprehensive systematic review has compares the effects of various eHealth platforms and the effects of modifiers among the published eHealth platform interventions.
Little research exists as to what policymakers and providers know and believe about eHealth strategies for smoking cessation, and how these compare to the evidence from well conducted studies. We hypothesised that policymakers and service providers could hold perceptions about the effectiveness of interventions that may be contrary to the evidence-base derived from advocacy, promotion, and word-of-mouth (12,13). We sought to identify the gap between policymakers’ knowledge and existing evidence to determine the risks that may be present as a result of flawed perceptions that influence policy and practice. In doing this, there is simultaneously an opportunity to actively engage policymakers and service providers with the current best evidence. The current best evidence for smoking cessation is a recent systematic review conducted by Do et al.,(5) which evaluated and compared the effectiveness of several eHealth interventions for smoking cessation, including mobile phone based, computer based and web-based programs. The review was wider is scope, investigated factors that affected effect sample size and included meta-regression. A summary of the findings on the effects of eHealth smoking cessation interventions based on Grading of Recommendations Assessment Development and Evaluation (GRADE) guidelines are described in the publication (5). The key findings and considerations for public health practice in plain language are described in a user friendly the Health EvidenceTM summary (5). The review is “strong evidence” as independently assessed by healthevidence.org (rated 10 out of 10). A Cochrane review of similar scope to Do 2018 was published a year following conduct of our study (14), The findings are of significance and trustworthy for public health practice. Given the strength of the review, the findings are worthy of consideration by policy and decision makers. By providing systematic review evidence of the effectiveness of eHealth programs, policymakers and service providers can potentially avoid advocating for interventions that have little or no effect. The referral by policymakers to programs that are not evidence-based could result in wasting the already limited resources and introducing extra burdens on excessively demanding health services. The current study explores the perceptions of Australian policymakers and service providers involved in smoking cessation programs. This study seeks to identify whether or not their prior perceptions about eHealth interventions are in contradiction with the best available evidence of effectiveness.
An online survey with Qualtrics was conducted among public health policymakers and service providers who were identified as responsible for smoking cessation programs in Australia. This study design was selected based upon previous experience, as we determined that policymakers were more likely to participate in an anonymous survey than a face-to-face interview.
A total of 38 Australian key policymakers and service providers were invited to participate in the study. A list of individuals was compiled through contact details on web pages of service providers (www.health.nsw.gov.au, www.cancer.org.au, www.cancerwa.asn.au) and the researchers’ own knowledge of senior policymakers and stakeholders in smoking cessation from Queensland, New South Wales and Western Australia. An email invitation containing an anonymous electronic survey link was sent to the identified individuals in October 2018. A follow up email reminder was sent two weeks later. No sample size calculation was undertaken as we invited the participation of all persons known to be in policy making roles in the three states.
The survey was constructed using the findings of the new systematic review (at the time in press) (5). The survey consisted of four subsections designed to assess the following: (a) the participants’ general characteristics (profession and organisation); (b) the participants’ perceptions and knowledge of the quality and effectiveness of six different eHealth and mHealth interventions described in the review (i.e. web based interventions, tailored web-based interventions, web based interventions with pharmacotherapy, mobile based interventions, high-frequency versus low-frequency text messages and computer based interventions); (c) the participants’ use and knowledge of systematic reviews; (d) the participants’ interest in receiving the findings of a new systematic review and their preferred methods of communication. The reliability of the survey was evaluated by carrying out internal consistency measurements (i.e. Cronbach’s alpha). The results of the test indicated the internal consistency of the all survey items was acceptable (0.803).
All analysis was performed using the SPSS Version 25 (15). Descriptive statistics were used to determine the frequencies and percentages of study variables. No statistical testing was undertaken due to the small sample size.
Of the total of 38 Australian policymakers and service providers who were invited to participate in the survey 18 participants returned complete response forms, which demonstrated a response rate of 47%. Table 1 provides a summary of the characteristics of the respondents. Half of the respondents identified as policymakers (50%), while the other half were service providers (27.8%), researchers (5.6%) and other professions (16.7%), such as cancer prevention advisors, policy influencers and program coordinators.
In general, the result of the survey showed mixed agreement of the participants’ perceptions with the review’s findings on the effectiveness of eHealth and mHealth smoking cessation interventions. Table 2 identifies the proportions of participant agreement with the review’s findings. The comparison shows the majority of participants consistently failed to correctly identify the effects of the smoking cessation interventions included within the review. The results of the analysis showed only one of the total 18 participants (5.6%) could correctly identify the evidence contained in the review regarding what works and what does not. Only two (11.1%) participants correctly responded to the effect of the “mobile-based interventions versus the non-active control group” (significant increase). Similarly, two participants (11.1%) correctly identified the effect of “high-frequency SMS messages versus low-frequency SMS messages” (little or no increase for high frequency messaging). Conversely, 13 (72.2%) and 8 (44.4%) total participants correctly identified the effect of web-based interventions and combined web-based and pharmacotherapy interventions, respectively. Similarly, 12 participants correctly identified the effect of computer-based interventions (66.7%).
A similar finding was observed regarding the quality of the evidence. Only 5 (27.8%), 4 (22.2%) and 5 (27.8%) of the Australian policy and decision makers correctly identified the quality of evidence for the web based, tailored web based and combined web based and pharmacotherapy interventions, respectively. Further, only 2 (11.1%) and 3 (16.7%) participants were able to correctly identify the quality of the evidence for mobile based interventions and the high-versus low-frequency SMS interventions, respectively. Finally, 5 (27.8%) participants correctly identified the evidence quality of computer based intervention. Table 3 presents the participants’ perceptions of the evidence quality of six different smoking cessation approaches, denoting the agreement of their views with the review’s findings.
To more effectively deliver the finding of the new review to end users developing policy and managing services, it was important to understand their preferred methods of communication. Therefore, participants were surveyed about whether or not they were interested in receiving the new review of eHealth and mHealth interventions and their preferred methods of receiving it. The majority of the policy and decision makers surveyed (77.8%) expressed interest in receiving the new evidence, while 22.2% were somewhat interested. Of note, when participants were offered the opportunity to receive an update on new research evidence on smoking cessation through email, only one participant sent an email of request to the principle researcher.
Regarding preferred methods of communication, email was the most preferred method of communication for the participants (88.9%), webinars 50%, and Health Evidence summaries 44.4%. Videos, podcasts, workshops and meetings with knowledge brokers were the least preferred methods of communication. Table 4 presents findings from analyses of respondents’ preferences regarding several possible research information distribution methods.
The results showed that of those surveyed, all (n=18) had prior knowledge of systematic reviews as a source of evidence. Most (88.9%) of them reported they were more likely to use research evidence to inform decision making in their field.
When participants were asked how likely they were to use the findings of a new systematic review on eHealth and mHealth interventions for smoking cessation to develop new policies, 83.3% of them demonstrated a preference for the latest or newer reviews.
This study explored the perceptions of a sample of Australian policymakers and service providers of the effectiveness and quality of different eHealth and mHealth interventions for smoking cessation. These perceptions were then compared with the findings of a new systematic review on the effectiveness of eHealth and mHealth smoking cessation programs. The study found that the participants held generally negative perceptions regarding the quality of available eHealth and mHealth interventions for smoking cessation, showing they were somewhat skeptical or unaware of existing evidence. Even though 44% of the study’s participants were to some extent confident about their knowledge of various eHealth and mHealth interventions for smoking cessation, their responses regarding the effects of each intervention suggest their knowledge was inadequate and their confidence unfounded. This was exemplified in that 88.9% of the participants were unaware that mobile-based health interventions significantly promote smoking cessation. Furthermore, the majority of the participants were unaware that tailored web based (94.4%) and high-versus low-frequency SMS message based (88.9%) have little or no additional impact on cessation. In addition, around half of the participants did not correctly identify that web based integrated with pharmacotherapy (55.6%) can moderately increase cessation.
On a positive note, the analysis found the majority of the participants correctly identified that web-based health interventions compared to non-active control can have a moderate impact on increasing cessation. Of concern in terms of resource allocation is the lack of implementer’s knowledge, which has the potential for the development of intervention policy and practice that has little or no effect. For example, the survey results showed that 61.1% of participants thought high-frequency SMS messages have a moderate effect on increasing cessation over low frequency (once a week), 16.7% of the participants felt they have a significant impact on cessation. On the contrary, high frequency messaging has little or no additional effect on increasing cessation. Similarly, 77.8% of the participants overestimated the effects of tailored web-based interventions (compared to a control with both groups receiving NRT) and thought these strategies have a moderate effect on increasing cessation, while they actually have little or no effect on cessation. The investment in ineffective programs without consideration of the evidence could lead to a waste of limited health resources and perhaps frustrate participants by their lack of effect.
A similar finding was also observed regarding the quality of the evidence. The survey results showed that participants failed to correctly identify the quality of the evidence. More than half of the participants were unaware that the quality of the evidence of the web-based approach (72.2%) and tailored web-based approach (77.8%) was moderate.Moreover, more thantwo-thirds of the participants were unaware that the quality of the mobile health approach (88.9%), high- versus low-frequency SMS messages (83.3%) and the computer-based approach (72.2%) was low. The lack of awareness among policymakers of the quality of the evidence of each interventions provided may affect their ability in decide which intervention to recommend. This high degree of uncertainty of the quality of evidence may indicate that policymakers are routinely assessing the quality of the evidence for decision making. An important finding of this study is that research evidence from systematic reviews needs to be communicated with decision makers and service providers to modify their perceptions and beliefs when in contradiction of current evidence.
To increase the participants’ knowledge of current evidence, as well as change previously held misperceptions of the usefulness and quality of findings, it is important to understand the participants’ preferred methods of communicating updates on research. The majority of the respondents indicated that email is their method of communication (88.9%). This result should, however, be treated with caution due to potential bias as it is the way they received the recruitment emails, and they use email as a daily method of communication. The second most preferred method of communication was webinars (50%), followed by Health Evidence summaries (44.4%). Videos, podcasts, workshops and meetings with knowledge brokers were the least preferred methods of communication, consistent with an earlier study identifying the electronic communications channel, in particular emails, as most preferred (10). Respondents also stated they were interested in accessing relevant research using the internet and recommended that summaries should be distributed through a public health professional organisational website. Furthermore, some participants expressed interest in face-to-face interactions with researchers to discuss research findings and their potential implication into practice. It is uncertain whether or not the participants understood the potential role of a knowledge broker as they are less familiar in Australia than in other parts of the world. Overall, the study indicated that audience centred approaches and technologies are important tools for engagement (16).
To overcome these barriers and support the use and uptake of research evidence into policy and practice, previous studies have indicated the review authors should conduct systematic reviews that are actionable and relevant to the need and the preference of those who will or could use their reviews, as well as making these reviews more accessible (16). Moreover, review authors should interact more with their target population and develop targeted strategies to inform them with their findings, including the use of review advisory groups (17) and realist reviews (18).
There are a number of strengths to this research. Firstly, the timing of this study reflects native existing perceptions because the information from the systematic review by Do et al.(5) was not available at the time of conducting this study and in doing so identified what views translation strategies are opportunities for targeting with new review findings. Secondly, the findings of this systematic review are considered potentially implementable, as they original from a source high on the evidence pyramid for evidence informed decision making (19). Thirdly, this study provides indications of the perceptions of the sampled Australian policy and decision makers around the effect and quality of eHealth and mHealth interventions for smoking cessation, and provides better understanding of their attitudes toward the use of research evidence in policy and decision making. Finally, this study identified the preferred methods of communication for Australian policy and decision makers, which can result in more effective future communication.
The results of this study should be treated with caution, as there are also a number of limitations. First, the relatively small sample size of the study can limit the generalisability of the results, although the number of senior policymakers in Australia is limited. Second, there is a risk of selection bias due to the low response rate among Australian policymakers (47%). The sample was not representative of all Australian policymakers, as it included only three states. These three states were chosen as the contact details of the Australian policymakers in these states were published online by the health department. But for the other states, no list was readily available. Based upon previous experience, we determined that policymakers were more likely to participate in an anonymous survey than a face-to-face interview.
Issues of non-response bias can arise from low response rate, particularly when the characteristics of the non-responders differ from the responders. In this study, it is unknown which of the identified key decision makers did and did not complete the survey, as the response survey es were anonymous and their IP addresses were not recorded. Furthermore, it is understood by those who have previously worked in state government that senior policymakers are significantly risk adverse and may be hesitant to participate in research about knowledge and attitudes. We also recognise that policymakers receive high volumes of email, and an invitation to a survey may be a low priority. To address the low response rate among the Australian policy and decision makers, a reminder email was sent to participants two weeks after the first email was sent, but no additional responses were received. This may be because the Australian sample was not from a pre-existing sampling frame, but rather were senior people identified as contacts on web pages and known networks in Queensland, New South Wales and Western Australia.
In addition to a focus on policymakers, future research is needed to augment these efforts, by exploring the attitudes and perceptions of smokers towards eHealth and mHealth interventions. Furthermore, the response rate for the present study’s survey suggests it is better to conduct research on perceptions amongst more identifiable policymakers. Given the important role of policy and decision makers in the provision of smoking cessation interventions, it is important to identify the positions of these policy and decision makers because some of them may influence the application of these programs on a large scale. During the COVID-19 pandemic in Australia, there was no new evidence showing increased engagement with digital interventions, nor increased policy maker awareness of such innovations.
This study indicates that the prior perceptions and beliefs held by Australian policymakers surveyed about the effects of eHealth and mHealth interventions for smoking cessation are significant enough to affect the provision of such interventions to smokers who want to quit. It would be very concerning if these views are widespread across decision makers. There is some uncertainty due to the small sample size, and risk to external validity as not all states and territories were included in the survey. While the results are not generalizable, they are indicative of a possible and important trend that requires further examination. It is apparent that policy and decision makers must know the available evidence to avoid advocating ineffective interventions, which could result in wasting limited resources and placing additional burdens on overstretched health services. To ensure findings of a new systematic review are used by those Australian policymakers, the existence of the review and its key findings must be delivered through the end users’ preferred methods of communication, which are email, webinars and health evidence summaries consistent with . Further research is needed to understand how policymakers in smoking cessation identify, access, utilise, adapt and adopt systematic review evidence within the context of knowledge translation frameworks.
eHealth: electronic Health; mHealth: mobile Health; LMIC: Low- and Middle-Income Countries; HIC: High Income Countries; COPD: Chronic Obstructive Pulmonary Disease; NRT: Nicotine Replacement Therapy; GRADE: Grading of Recommendations Assessment Development and Evaluation.
Ethical statement
The Human Research Ethics Committee at Queensland University of Technology approved the study (Approval number: 1800000956).
Conflict of Interest Statement
The authors declare no conflict of interest.
Consent for publication
Not applicable
Availability of data and materials
Not applicable
Funding
Alaa Subahe received a Kingdom of Saudi Arabia Scholarship to undertake her Masters of Public Health studies.
Authors’ contributions
JC assisted with the structure, design, writing, and editing of the document. She also revised aspects of the overall thesis of the article, including edits to generate emphasis on the case for the research, the findings, and the analysis. AS undertook this study as her Masters dissertation, conceiving and designing with her supervisor PB. She led the project undertaking analysis and writing. PB was the supervising academic who with AS conceived and designed the study. He contributed extensively to the writing, review and revision of the manuscript.
Acknowledgements
Special thanks toHuyen Do the author of the systematic review on “Which eHealth interventions are most effective for smoking cessation “for providing me with the results of the systematic review before it is been published.Appreciation is also extended to Clarence Baxter and Tifaniya for their feedback on the draft.
Table 1 Characteristic of the Survey Participants
|
Demographic Characteristics |
Australian Policy- and Decision-Makers N (%) |
|
Organization Government Not for profit Other
|
11 (61.1) 6 (33.3) 1 (5.6) Statutory Organisation |
|
Role / Job Descriptions
Researcher Service provider Other |
N (%)
9 (50) 1 (5.6) 5 (27.8) 3 (16.7)
|
Table 2 Proportions of Agreement of the Participants to the Review
|
Effect of different eHealth/mHealth interventions |
Don’t know N (%) |
Little or no increase N (%) |
Moderate increase N (%) |
Significant increase N (%) |
|
|
|
|
|
|
|
Web-based smoking cessation interventions.
|
1 (5.6) 0 (0.0)
2(11.1)
|
4(22.2) 1(5.6)‡
2(11.1)
|
13(72.2)‡ 14(77.8)
8(44.4)‡
|
0 (0.0) 3(16.7)
6(33.3)
|
|
mHealth smoking cessation interventions
|
0 (0.0) 2(11.1) |
4(22.2) 2(11.1)‡ |
12(66.7) 11(61.1) |
2(11.1)‡ 3(16.7) |
|
Computer-assisted smoking cessation intervention
|
1(5.6) |
12(66.7)‡ |
5(27.8) |
0 (0.0) |
‡Finding from the systematic review of Do et al.7
Table 3 Participants’ Perceptions of the Quality of the Evidence for Six Different Smoking Cessation Approaches
|
The quality of the evidence of different eHealth/mHealth interventions |
Don’t know N (%) |
Very low N (%) |
Low N (%) |
Moderate N (%) |
High N (%) |
|
Web based smoking cessation interventions.
|
7(38.9) 7(38.9)
7(38.9) |
2(11.1) 1(5.6)
1(5.6) |
4(22.2) 5(27.8)
3(16.7) |
5(27.8)‡ 4(22.2) ‡
5(27.8) ‡ |
0 (0.0) 1(5.6)
2(11.1)
|
|
mHealth smoking cessation interventions
|
7(38.9) 7(38.9) |
1(5.6) 2(11.1) |
2(11.1) ‡ 3(16.7) ‡ |
7(38.9) 6(33.3) |
1(5.6) 0 (0.0) |
|
Computer assisted smoking cessation intervention
|
8(44.4) |
2(11.1) |
5(27.8) ‡ |
3(16.7) |
0 (0.0) |
‡Finding from the systematic review of Do et al. 7
Table 4 Participants’ Preferred Methods of Receiving Research Information
|
Methods of Communication* |
Australian Policy- and Decision-Makers N (%) |
|
Emails |
16 (88.9%) |
|
Webinars |
9 (50%) |
|
Health Evidence.org summaries |
8 (44.4%) |
|
Podcast |
5 (27.8%) |
|
Workshops |
4 (22.2%) |
|
Videos |
2 (11.1%) |
|
Phone calls |
0 (0.0%) |
|
Meeting with knowledge brokers |
1 (5.6%) |
Each respondent could choose multiple responses.