This study explored the perceptions of a sample of Australian policymakers and service providers of the effectiveness and quality of different eHealth and mHealth interventions for smoking cessation. These perceptions were then compared with the findings of a new systematic review on the effectiveness of eHealth and mHealth smoking cessation programs. The study found that the participants held generally negative perceptions regarding the quality of available eHealth and mHealth interventions for smoking cessation, showing they were somewhat skeptical or unaware of existing evidence. Even though 44% of the study’s participants were to some extent confident about their knowledge of various eHealth and mHealth interventions for smoking cessation, their responses regarding the effects of each intervention suggest their knowledge was inadequate and their confidence unfounded. This was exemplified in that 88.9% of the participants were unaware that mobile-based health interventions significantly promote smoking cessation. Furthermore, the majority of the participants were unaware that tailored web based (94.4%) and high-versus low-frequency SMS message based (88.9%) have little or no additional impact on cessation. In addition, around half of the participants did not correctly identify that web based integrated with pharmacotherapy (55.6%) can moderately increase cessation.
On a positive note, the analysis found the majority of the participants correctly identified that web-based health interventions compared to non-active control can have a moderate impact on increasing cessation. Of concern in terms of resource allocation is the lack of implementer’s knowledge, which has the potential for the development of intervention policy and practice that has little or no effect. For example, the survey results showed that 61.1% of participants thought high-frequency SMS messages have a moderate effect on increasing cessation over low frequency (once a week), 16.7% of the participants felt they have a significant impact on cessation. On the contrary, high frequency messaging has little or no additional effect on increasing cessation. Similarly, 77.8% of the participants overestimated the effects of tailored web-based interventions (compared to a control with both groups receiving NRT) and thought these strategies have a moderate effect on increasing cessation, while they actually have little or no effect on cessation. The investment in ineffective programs without consideration of the evidence could lead to a waste of limited health resources and perhaps frustrate participants by their lack of effect.
A similar finding was also observed regarding the quality of the evidence. The survey results showed that participants failed to correctly identify the quality of the evidence. More than half of the participants were unaware that the quality of the evidence of the web-based approach (72.2%) and tailored web-based approach (77.8%) was moderate.Moreover, more thantwo-thirds of the participants were unaware that the quality of the mobile health approach (88.9%), high- versus low-frequency SMS messages (83.3%) and the computer-based approach (72.2%) was low. The lack of awareness among policymakers of the quality of the evidence of each interventions provided may affect their ability in decide which intervention to recommend. This high degree of uncertainty of the quality of evidence may indicate that policymakers are routinely assessing the quality of the evidence for decision making. An important finding of this study is that research evidence from systematic reviews needs to be communicated with decision makers and service providers to modify their perceptions and beliefs when in contradiction of current evidence.
To increase the participants’ knowledge of current evidence, as well as change previously held misperceptions of the usefulness and quality of findings, it is important to understand the participants’ preferred methods of communicating updates on research. The majority of the respondents indicated that email is their method of communication (88.9%). This result should, however, be treated with caution due to potential bias as it is the way they received the recruitment emails, and they use email as a daily method of communication. The second most preferred method of communication was webinars (50%), followed by Health Evidence summaries (44.4%). Videos, podcasts, workshops and meetings with knowledge brokers were the least preferred methods of communication, consistent with an earlier study identifying the electronic communications channel, in particular emails, as most preferred (10). Respondents also stated they were interested in accessing relevant research using the internet and recommended that summaries should be distributed through a public health professional organisational website. Furthermore, some participants expressed interest in face-to-face interactions with researchers to discuss research findings and their potential implication into practice. It is uncertain whether or not the participants understood the potential role of a knowledge broker as they are less familiar in Australia than in other parts of the world. Overall, the study indicated that audience centred approaches and technologies are important tools for engagement (16).
To overcome these barriers and support the use and uptake of research evidence into policy and practice, previous studies have indicated the review authors should conduct systematic reviews that are actionable and relevant to the need and the preference of those who will or could use their reviews, as well as making these reviews more accessible (16). Moreover, review authors should interact more with their target population and develop targeted strategies to inform them with their findings, including the use of review advisory groups (17) and realist reviews (18).
4.1 Strengths and Limitations
There are a number of strengths to this research. Firstly, the timing of this study reflects native existing perceptions because the information from the systematic review by Do et al.(5) was not available at the time of conducting this study and in doing so identified what views translation strategies are opportunities for targeting with new review findings. Secondly, the findings of this systematic review are considered potentially implementable, as they original from a source high on the evidence pyramid for evidence informed decision making (19). Thirdly, this study provides indications of the perceptions of the sampled Australian policy and decision makers around the effect and quality of eHealth and mHealth interventions for smoking cessation, and provides better understanding of their attitudes toward the use of research evidence in policy and decision making. Finally, this study identified the preferred methods of communication for Australian policy and decision makers, which can result in more effective future communication.
The results of this study should be treated with caution, as there are also a number of limitations. First, the relatively small sample size of the study can limit the generalisability of the results, although the number of senior policymakers in Australia is limited. Second, there is a risk of selection bias due to the low response rate among Australian policymakers (47%). The sample was not representative of all Australian policymakers, as it included only three states. These three states were chosen as the contact details of the Australian policymakers in these states were published online by the health department. But for the other states, no list was readily available. Based upon previous experience, we determined that policymakers were more likely to participate in an anonymous survey than a face-to-face interview.
Issues of non-response bias can arise from low response rate, particularly when the characteristics of the non-responders differ from the responders. In this study, it is unknown which of the identified key decision makers did and did not complete the survey, as the response survey es were anonymous and their IP addresses were not recorded. Furthermore, it is understood by those who have previously worked in state government that senior policymakers are significantly risk adverse and may be hesitant to participate in research about knowledge and attitudes. We also recognise that policymakers receive high volumes of email, and an invitation to a survey may be a low priority. To address the low response rate among the Australian policy and decision makers, a reminder email was sent to participants two weeks after the first email was sent, but no additional responses were received. This may be because the Australian sample was not from a pre-existing sampling frame, but rather were senior people identified as contacts on web pages and known networks in Queensland, New South Wales and Western Australia.
In addition to a focus on policymakers, future research is needed to augment these efforts, by exploring the attitudes and perceptions of smokers towards eHealth and mHealth interventions. Furthermore, the response rate for the present study’s survey suggests it is better to conduct research on perceptions amongst more identifiable policymakers. Given the important role of policy and decision makers in the provision of smoking cessation interventions, it is important to identify the positions of these policy and decision makers because some of them may influence the application of these programs on a large scale. During the COVID-19 pandemic in Australia, there was no new evidence showing increased engagement with digital interventions, nor increased policy maker awareness of such innovations.