Identifying Australian Policy Makers’ Perceptions of eHealth Interventions

Background: The increasing volume of research on electronic health (eHealth) smoking cessation interventions presents a challenge for policymakers and service providers when they try to decide which strategies to select for health promotion programs. This study aimed to explore the perceptions of Australian policymakers and service providers regarding the effectiveness and quality of eHealth and mHealth interventions for smoking cessation against the evidence in a recent systematic review. Methods: This cross-sectional study invited 38 Australian public health policymakers and service providers in smoking cessation to participate in an online survey assessing their knowledge of the current available evidence-base. Descriptive statistics were used to determine the results of the survey against the published review. Results: Eighteen participants completed the survey. Comparison identied that the majority of participants failed to correctly identify the effects and the quality of the evidence of popular smoking cessation interventions. They lacked knowledge of the usefulness and quality of some eHealth and mHealth interventions for smoking cessation. Conclusions: The nding of this study indicates that the prior perceptions and beliefs held by Australian policymakers and service providers about the effects of eHealth and mHealth interventions for smoking cessation are signicantly detached enough from the evidence-base to affect the provision of effective interventions to smokers who want to quit.


Background
Tobacco use is one of the leading causes of preventable diseases and death worldwide, accounting for over seven million deaths each year [1]. These deaths include an estimated 890,000 non-smokers who were indirectly exposed to second hand smoke [1]. Around 10% of all deaths from a non-communicable disease (including cardiovascular diseases, respiratory diseases, cancer and diabetes) are related to tobacco use, and most of these occur in low-and middle-income countries (LMIC) [2]. By 2040, the prevalence of tobacco use globally is expected to decrease to less than 5%, but because of the steady population growth, combined with the tobacco industry's focus on LMICs, the number of smokers is likely to increase in these countries [3]. The factors associated with smoking in LMICs include poor education, low-income level, unemployment and being male [3]. Similarly, despite the overall decline in smoking prevalence in high-income countries (HICs) over the last few decades, the rate of smoking remains disproportionately high in disadvantaged priority populations [4].
The latest estimate suggests smoking kills two in three persistent users in Australia [5]. In previous decades, the number of fatalities caused by tobacco use has been vast. In the 50 years from 1960-2010, it was estimated that smoking killed around 821,000 Australians [5]. Even today, smoking is still directly responsible for the deaths of nearly 19,000 Australians annually [5]. In 2011, tobacco smoking was estimated to be responsible for 80% of the lung cancer burden and 75% of the chronic obstructive pulmonary disease (COPD) burden in Australia [6]. Reducing the burden of disease is a priority of public health, as is the prevention of the uptake of smoking and the cessation of smoking by current users. It is the responsibility of those in health policy and service to gather a reliable evidence base so they can ensure they are both accurate and effective in their response and strategies to reduce smoking rates, and thus the burden of disease [7].
Public health policymakers and service providers in smoking cessation programs are increasingly expected to base their decisions on the best available research evidence in order to assist informed and progressive decision-making [7]. Systematically incorporating evidence-based research into policy formulation and health policy-making supports the delivery of high quality, effective and e cient health services [7]. Furthermore, it ensures that reasonable human and nancial investments are made into public health and healthcare [7]. However, in practice, intervention decisions are often based on the short term bene ts, lacking systematic planning and reviewing of the best avialable research evidence regarding effective approaches hence resulting in slow uptake of research evidence into practice [8].
This failure of optimally using research evidence in health policy and practice has resulted in a gap between the large body of research evidence produced and its uptake into health policy and practice [13].
As result of this knowledge-to-action gap, most countries population cannot gain optimal healthcare and promotion bene ts, which leads to decreased productivity and poorer quality of life. Also, there is a possibility for 20-30% of individuals in the various healthcare systems receive potentially harmful or not needed care, as well as, have limited uptake of effective treatments [14].
Previous research has documented the poor or inadequate use of research evidence by policy-and decision-makers due to several factors related to individuals, organisations and the research itself [15].
Several factors relating to individual action presents a signi cant barrier to the use of research evidence by policy and key decision-makers. Importantly, the individual decision-makers' perceptions, values and beliefs regarding the research evidence are considered major challenges to its use in the policy-making process [15,16]. Furthermore, the lack of access to health policy research and inadequate skills in knowledge translation and management limit the policymaker's ability to access and synthesise information on which to base decisions [15,16].
The digital support approach, or the electronic health (eHealth) approach is considered a new opportunity for prevention of health-risk behaviours, and for tackling the global burden of smoking by expanding the accessibility of cessation programs to all smokers [17]. eHealth interventions have unique advantages, including affordability, e cient delivery, easy accessibility and a wide reach of large segments of the population. Therefore, recent studies suggest that digitally support approaches, such as eHealth and mobile (mHealth) interventions, could hold signi cant promise to improve smoking cessation. Previous systematic reviews have investigated the effects of eHealth and mHealth interventions upon smoking cessation outcomes [18][19][20][21], however they are limited in scope. A blanket acceptance and implementation of mHealth cessation programs is not only unhelpful, but potentially harmful in the face of a lack of thorough review of their effectiveness. A discerning and reliable evidence base from which policymakers can operate is imperative to achieve this important goal in population health.
Within broad scope of eHealth smoking cessations interventions, some strategies can help people quit; however, several barriers may affect their implementation by policymakers and service providers. The rst barrier is the sheer quantity and differences in quality of the published primary studies on eHealth interventions for smoking cessation. The clutter makes it di cult to identify effective and reliable interventions for adoption. It is clear that policymakers and service providers have little time to conduct complex literature searches, screen through results, and then interpret the plethora of results of primary studies [15,16].
Policy makers and service providers have a responsibility to implement eHealth programs demonstrated through research as effective. Systematic reviews provide summary evidence from many research studies that have been risk assessed, and are useful for evidence informed decision-making. Generally, previous systematic reviews in smoking cessation have focused on the effects of particular devices on the outcome of smoking cessation or on a speci c age group or population [18][19][20][21]. However, until recently, no comprehensive systematic review has compares the effects of various eHealth platforms and the effects of modi ers among the published eHealth platform interventions.
Little research exists as to what policymakers and providers know and believe about eHealth strategies for smoking cessation, and how these compare to the evidence from well conducted studies. We hypothesised that policymakers and service providers hold perceptions about the effectiveness of interventions that may be contrary to the evidence-base derived from advocacy, promotion, and word-ofmouth. We sought to identify the gap between policymakers' knowledge and existing evidence to determine the risks that may be present as a result of awed perceptions that in uence policy and practice. In doing this, there is simultaneously an opportunity to actively engage policymakers and service providers with the current best evidence. The current best evidence for smoking cessation is a recent systematic review conducted by Do et al., [17] which evaluated and compared the effectiveness of several eHealth interventions for smoking cessation, including mobile phone based, computer based and web based programs. A summary of the ndings on the effects of eHealth smoking cessation interventions based on Grading of Recommendations Assessment Development and Evaluation (GRADE) guidelines are described in the publication [17]. The key ndings and considerations for public health practice in plain language are described in a user friendly the Health Evidence™ summary [17]. The review is "strong evidence" as independently assessed by healthevidence.org (rated 10 out of 10). The ndings are of signi cance and trustworthy for public health practice. Given the strength of the review, the ndings are worthy of consideration by policy and decision makers. By providing systematic review evidence of the effectiveness of eHealth programs, policymakers and service providers can potentially avoid advocating for interventions that have little or no effect. The referral by policymakers to programs that are not evidence-based could result in wasting the already limited resources and introducing extra burdens on excessively demanding health services. The current study explores the perceptions of Australian policymakers and service providers involved in smoking cessation programs. This study seeks to identify whether or not their prior perceptions about eHealth interventions are in contradiction with the best available evidence of effectiveness.

Study design
An online survey with Qualtrics was conducted among public health policymakers and service providers who were identi ed as responsible for smoking cessation programs in Australia.

Sample and recruitment
A total of 38 Australian key policy makers and service providers were invited to participate in the study. A list of individuals was compiled through contact details on web pages of service providers (www.health.nsw.gov.au, www.cancer.org.au, www.cancerwa.asn.au) and the researchers' own knowledge of senior policymakers and stakeholders in smoking cessation from Queensland, New South Wales and Western Australia. An email invitation containing an anonymous electronic survey link was sent to the identi ed individuals in October 2018. A follow up email reminder was sent two weeks later.

Instrument
The survey was constructed using the ndings of the new systematic review (at the time in press) [17].
The survey consisted of four subsections designed to assess the following: (a) the participants' general characteristics (profession and organisation); (b) the participants' perceptions and knowledge of the quality and effectiveness of six different eHealth and mHealth interventions described in the review (i.e. web based interventions, tailored web-based interventions, web based interventions with pharmacotherapy, mobile based interventions, high-frequency versus low-frequency text messages and computer based interventions); (c) the participants' use and knowledge of systematic reviews; (d) the participants' interest in receiving the ndings of a new systematic review and their preferred methods of communication. The reliability of the survey was evaluated by carrying out internal consistency measurements (i.e. Cronbach's alpha). The results of the test indicated the internal consistency of the all survey items was acceptable (0.803).

Data analysis
All analysis was performed using the SPSS Version 25 [22]. Descriptive statistics were used to determine the frequencies and percentages of study variables. No statistical testing was undertaken due to the small sample size.

Sample demographics
Of the total of 38 Australian policymakers and service providers who were invited to participate in the survey 18 participants returned complete response forms, which demonstrated a response rate of 47%. Table 1 provides a summary of the characteristics of the respondents. Half of the respondents identi ed as policymakers (50%), while the other half were service providers (27.8%), researchers (5.6%) and other professions (16.7%), such as cancer prevention advisors, policy in uencers and program coordinators. In general, the result of the survey showed mixed agreement of the participants' perceptions with the review's ndings on the effectiveness of eHealth and mHealth smoking cessation interventions. Table 2 identi es the proportions of participant agreement with the review's ndings. The comparison shows the majority of participants consistently failed to correctly identify the effects of the smoking cessation interventions included within the review. The results of the analysis showed only one of the total 18 participants (5.6%) could correctly identify the evidence contained in the review regarding what works and what does not. Only two (11.1%) participants correctly responded to the effect of the "mobile-based interventions versus the non-active control group" (signi cant increase). Similarly, two participants (11.1%) correctly identi ed the effect of "high-frequency SMS messages versus low-frequency SMS messages" (little or no increase for high frequency messaging). Conversely, 13 (72.2%) and 8 (44.4%) total participants correctly identi ed the effect of web-based interventions and combined web-based and pharmacotherapy interventions, respectively. Similarly, 12 participants correctly identi ed the effect of computer-based interventions (66.7%). 3.3 Participants' perceptions on the quality of the evidence A similar nding was observed regarding the quality of the evidence. Only 5 (27.8%), 4 (22.2%) and 5 (27.8%) of the Australian policy and decision makers correctly identi ed the quality of evidence for the web based, tailored web based and combined web based and pharmacotherapy interventions, respectively. Further, only 2 (11.1%) and 3 (16.7%) participants were able to correctly identify the quality of the evidence for mobile based interventions and the high-versus low-frequency SMS interventions, respectively. Finally, 5 (27.8%) participants correctly identi ed the evidence quality of computer based intervention. Table 3 presents the participants' perceptions of the evidence quality of six different smoking cessation approaches, denoting the agreement of their views with the review's ndings.  Regarding preferred methods of communication, email was the most preferred method of communication for the participants (88.9%), webinars 50%, and Health Evidence summaries 44.4%. Videos, podcasts, workshops and meetings with knowledge brokers were the least preferred methods of communication. Table 4 presents ndings from analyses of respondents' preferences regarding several possible research information distribution methods.

Participants access and use of systematic reviews
The results showed that of those surveyed, all (n = 18) had prior knowledge of systematic reviews as a source of evidence. Most (88.9%) of them reported they were more likely to use research evidence to inform decision making in their eld.
When participants were asked how likely they were to use the ndings of a new systematic review on eHealth and mHealth interventions for smoking cessation to develop new policies, 83.3% of them demonstrated a preference for the latest or newer reviews.

Discussion
This study explored the perceptions of a sample of Australian policymakers and service providers of the effectiveness and quality of different eHealth and mHealth interventions for smoking cessation. These perceptions were then compared with the ndings of a new systematic review on the effectiveness of eHealth and mHealth smoking cessation programs. The study found that the participants held generally negative perceptions regarding the quality of available eHealth and mHealth interventions for smoking cessation, showing they were somewhat skeptical or unaware of existing evidence. Even though 44% of the study's participants were to some extent con dent about their knowledge of various eHealth and mHealth interventions for smoking cessation, their responses regarding the effects of each intervention suggest their knowledge was inadequate and their con dence unfounded. This was exempli ed in that 88.9% of the participants were unaware that mobile-based health interventions signi cantly promote smoking cessation. Furthermore, the majority of the participants were unaware that tailored web based (94.4%) and high-versus low-frequency SMS message based (88.9%) have little or no additional impact on cessation. In addition, around half of the participants did not correctly identify that web based integrated with pharmacotherapy (55.6%) can moderately increase cessation.
This lack of policymaker and service providers knowledge in some important aspects of intervention effectiveness could limit the provision of effective interventions to smokers. These ndings are concerning if they are representative of widely held beliefs of key Australian policymakers and service providers. On a positive note, the analysis found the majority of the participants correctly identi ed that web-based health interventions compared to non-active control can have a moderate impact on increasing cessation. Of concern in terms of resource allocation is the lack of implementer's knowledge, which has the potential for the development of intervention policy and practice that has little or no effect. For example, the survey results showed that 61.1% of participants thought high-frequency SMS messages have a moderate effect on increasing cessation over low frequency (once a week), 16.7% of the participants felt they have a signi cant impact on cessation. On the contrary, high frequency messaging has little or no additional effect on increasing cessation. Similarly, 77.8% of the participants overestimated the effects of tailored web-based interventions (compared to a control with both groups receiving NRT) and thought these strategies have a moderate effect on increasing cessation, while they actually have little or no effect on cessation. The investment in ineffective programs without consideration of the evidence could lead to a waste of limited health resources and perhaps frustrate participants by their lack of effect.
An important nding of this study is that research evidence from systematic reviews needs to be communicated with decision makers and service providers to modify their perceptions and beliefs when in contradiction of current evidence. The review [17] and its summaries has the capacity to support the uptake of its ndings by decision makers and service providers as its formatting features align with those identi ed by Tricco 2016 [23]. These formatting features include a plain language summary, a summary of ndings table, distribution through a clearing house and graded format.
To increase the participants' knowledge of current evidence, as well as change previously held misperceptions of the usefulness and quality of ndings, it is important to understand the participants' preferred methods of communicating updates on research. The majority of the respondents indicated that email is their method of communication (88.9%). This result should, however, be treated with caution due to potential bias as it is the way they received the recruitment emails, and they use email as a daily method of communication. The second most preferred method of communication was webinars (50%), followed by Health Evidence summaries (44.4%). Videos, podcasts, workshops and meetings with knowledge brokers were the least preferred methods of communication, consistent with an earlier study identifying the electronic communications channel, in particular emails, as most preferred [24]. Respondents also stated they were interested in accessing relevant research using the internet and recommended that summaries should be distributed through a public health professional organisational website. Furthermore, some participants expressed interest in face-to-face interactions with researchers to discuss research ndings and their potential implication into practice. It is uncertain whether or not the participants understood the potential role of a knowledge broker as they are less familiar in Australia than in other parts of the world.
These ndings are consistent with evidence showing interactive methods to engage end users is a promising approach that can help in improve the use and uptake of research evidence into policy and practice. Moreover, such methods can help increase policymakers' and healthcare providers' capabilities to perform research or be involved in the research process, and better prepare researchers to produce relevant and useful [15,25,26]. Audience centred approaches and technologies are important tools for engagement [26].
The nding of this study has also provided meaningful insight into the use of systematic reviews in decision making by Australian policymakers and service providers. Although only a small sample of key decision makers and service providers were surveyed, the ndings of this study suggest that participants reported a positive attitude towards the use of research evidence in policy. This was exempli ed by the fact that all surveyed participants had prior knowledge of systematic reviews as a source of evidence. However, the ndings are limited in that we did not assess whether or not their understanding of systematic review is correct. This was consistent with earlier ndings of Dobbins, Cockrill & Barnsley study [15] that revealed 63% of Ontario public health decision makers had positive attitude towards the use of the systematic reviews in decision making. Over a two-year period, the study's participants indicated they based their decisions on at least two systematic reviews. A person's position is an important predictor for systematic reviews use, and accordingly the level of decision determines the type of information needed [27].
Conversely, the ndings from several studies indicates that policy and decision makers rarely use the research evidence to inform their decision making due to a range of individual, organisational and research factors [15,16]. For example, Campbell et al [28]. reveal in their study that less than one third of Australian policy and decision makers had used research to inform more than 75% of the policies they had developed in the previous 12 months. The di culty of accessing valuable research, the absence of relevant research and the time pressure were the most reported barriers to use the research evidence in policy by the policymakers in this study. The same barriers were also reported by drug policymakers in another study [29]. As result of these barriers, the majority of drug policymakers reported consulting an external expert during their most recent decision making process rather than using research evidence. Some of them relied on this method exclusively.
To overcome these barriers and support the use and uptake of research evidence into policy and practice, previous studies have indicated the review authors should conduct systematic reviews that are actionable and relevant to the need and the preference of those who will or could use their reviews, as well as making these reviews more accessible [26]. Moreover, review authors should interact more with their target population and develop targeted strategies to inform them with their ndings [26].

Strengths and Limitations
There are a number of strengths to this research. Firstly, the timing of this study re ects native existing perceptions because the information from the systematic review by Do et al. [17] was not available at the time of conducting this study and in doing so identi ed what views translation strategies are opportunities for targeting with new review ndings. Secondly, the ndings of this systematic review are considered potentially implementable, as they original from a source high on the evidence pyramid for evidence informed decision making [30]. Thirdly, this study provides indications of the perceptions of the sampled Australian policy and decision makers around the effect and quality of eHealth and mHealth interventions for smoking cessation, and provides better understanding of their attitudes toward the use of research evidence in policy and decision making. Finally, this study identi ed the preferred methods of communication for Australian policy and decision makers, which can result in more effective future communication.
The results of this study should be treated with caution, as there are also a number of limitations. First, the relatively small sample size of the study can limit the generalisability of the results, although the number of senior policymakers in Australia is limited. Second, there is a risk of selection bias due to the low response rate among Australian policymakers (47%). The sample was not representative of all Australian policymakers, as it included only three states. It did not include any participants working in the federal government.
Issues of non-response bias can arise from low response rate, particularly when the characteristics of the non-responders differ from the responders. In this study, it is unknown which of the identi ed key decision makers did and did not complete the survey, as the responses were anonymous and their IP addresses were not recorded. Furthermore, it is understood by those who have previously worked in state government that senior policymakers are signi cantly risk adverse and may be hesitant to participate in research about knowledge and attitudes. We also recognise that policymakers receive high volumes of email, and an invitation to a survey may be a low priority. To address the low response rate among the Australian policy and decision makers, a reminder email was sent to participants two weeks after the rst email was sent, but no additional responses were received. This may be because the Australian sample was not from a pre-existing sampling frame, but rather were senior people identi ed as contacts on web pages and known networks in Queensland, New South Wales and Western Australia.
This study provides insights into the future research needed to explore the attitudes and perceptions of smokers, towards eHealth and mHealth interventions for smoking cessation to identify which interventions are acceptable to those who want to quit and what they expect from these interventions.
Furthermore, the response rate for the present study's survey suggests it is better to conduct research on perceptions amongst more identi able policymakers. Given the important role of policy and decision makers in the provision of smoking cessation interventions, it is important to identify the positions of these policy and decision makers because some of them may in uence the application of these programs on a large scale.

Conclusion And Implications
This study indicates that the prior perceptions and beliefs held by Australian policymakers surveyed about the effects of eHealth and mHealth interventions for smoking cessation are signi cant enough to affect the provision of such interventions to smokers who want to quit. It would be very concerning if these views are widespread across decision makers. It is apparent that policy and decision makers must know the available evidence to avoid advocating ineffective interventions, which could result in wasting limited resources and placing additional burdens on overstretched health services. To ensure ndings of a new systematic review are used by those Australian policymakers, the existence of the review and its key ndings must be delivered through the end users' preferred methods of communication, which are email, webinars and health evidence summaries. Further research is needed to understand how policymakers in smoking cessation identify, access, utilise, adapt and adopt systematic review evidence.