This paper reports on a feasibility study using mixed methods including post-study survey and descriptive analysis of responses. As mentioned, it is a study nested within a larger study developing an intervention.
Participants and recruitment
The sample size was determined based on previous studies of asynchronous remote community research, with studies ranging from 13 participants to 48 participants (21-25). As the secret Facebook group takes place over a longer period of time, a larger group of participants can be accommodated in comparison to an in-person focus group or an online synchronous focus group (25). The in-person interviews previously conducted provided an initial source of participants, as the participants in this group consented to being contacted for future research. Thus, the majority of participants (16/21) had previously had face-to-face or telephone interviews with the researchers from June to September 2018. New participants (5/21) were recruited from a Facebook group that is specifically for women with current or previous gestational diabetes. The researchers received permission from the Facebook group administrator to post a flyer advertising the study. Interested members reached out to a study-specific email address to receive further information and access to an online information sheet and consent form.
All participants were living in the UK during the study but were geographically dispersed.
Content development
The Facebook group content was developed to address steps 2 through 4 of the 6SQuID framework: to clarify modifiable and causal factors of the problem; develop a theory of change and begin to develop a theory of action for a physical activity intervention (3). A theory of change is the process by which change comes about for individuals, groups and communities – it asks: how are we going to change this behaviour? To create a theory of change, we first performed a situation analysis (commencing in the previous in-person interview portion) to understand contributing factors to physical inactivity for this group and developed a fishbone diagram with participants to explain key barriers and facilitators to physical activity in their daily lives.
The current study built on the information gained during the interviews to understand modifiable and important factors to understand which factors can be changed with the greatest scope for improvement in their physical activity. The researcher presented the fishbone diagram of barriers to physical activity to the secret Facebook group participants to understand which factors they felt were the most important and significant barriers in their lives. This allowed the researchers to able to understand which factors needed to be modified to change their physical activity. The researcher then analysed this information to begin to develop a theory of how to change their behaviour: potential intervention ideas, settings, timing and other components were presented back to participants for views of acceptability and feasibility.
Running the workshop
The workshop took place over a 15-day period in May to June 2019, the length of which was based on previous research (26). To action the 6SQuID steps, the lead researcher created posts for the Facebook page that sequentially and cumulatively led to the steps above being completed. Both the lay advisory group and co-authors reviewed the drafts of the posts. The lead researcher aimed to use several different formats of Facebook posts including text, photo series, polls and embedded videos for variety to keep participants engaged. Emojis (digital icons to display an idea or emotion) were used by the researcher when posting in the group in a similar manner to participants; the researcher mimicked the style of writing by participants to encourage an informal atmosphere. Two lay advisory group members were also involved as participants and co-facilitators to encourage discussion if posts did not have any responses. The researcher posted one to two times per day, with two days in the middle of the workshop that were ‘catch-up’ days (Day 8 and Day 10). Messages and content were typically posted between 15:30 and 17:30 GMT, as suggested by the lay advisor group, as mothers would be returning from work and may have a short break before their evening meal. The researcher checked the group several times each day during the workshop to ask follow-up questions and clarifications and moderate if necessary. Participants were able to comment on the researcher posts with their opinions and also were able to generate their own posts. Participant posts had to be approved by the researcher to ensure they were appropriate to the group.
Ethical considerations
The British Psychological Society published ethical guidelines for internet-mediated research in 2017 encompassing 4 main principles: 1) respect autonomy, privacy and dignity; 2) maintain scientific integrity; 3) social responsibility and 4) increase the benefits whilst reducing the harms (27). Confidentiality, anonymity and identity verification were specific ethical issues considered in developing the online workshop. The nature of social media existing online holds inherent confidentiality risks. Facebook ultimately possesses the information on the website, and participants were informed of this in the consent process. Their data is not innately private because of the platform, but in terms of privacy of what one posts on social media being viewed by others, a secret Facebook group is clearly a private space – participants should feel able to write freely without concern that what they are saying is viewable to those who are not in the group. In contrast to confidentiality involving how the researcher manages and uses private information, anonymity involves obscuring identifiable information for participants and can be used to maintain confidentiality of identifying information (28, 29). The part of qualitative analysis of social media that involves a significant risk to anonymity is in disseminating the results (27). For research using open and online internet forums, the risks to anonymity are higher, as participant quotations can be traced back to the original source and can potential identify the participant. However, this is less of a risk in a secret Facebook group: A secret group is not searchable on Facebook or any search engine, and details given in the group are only viewable by group members. As such, anonymity is preserved barring any data breaches through participants sharing or Facebook security lapses. When participants were first added to the secret Facebook group (after providing informed consent), they were required to read through the ‘Group Rules’, in which one of them provided explicit privacy instructions: “Respect Everyone's Privacy and Confidentiality. Being part of this group requires mutual trust. Everyone is welcome to share sensitive information, thoughts and feelings in this group, but what's shared in the group should stay in the group”, to ensure that participants understood the privacy and confidentiality implications of the research.
Identity verification holds two primary considerations in social media research: 1. Are participants who they say they are? 2. Does the person that a participant portrays in a Facebook group accurately represent their real self? As “Facebook profile page amounts to a blank canvas on which each user has free reign to construct a public or semi-public image of him- or herself”, it is possible the participants that the researcher has not met before may not fit the eligibility criteria (p. 213) (30). However, this research is not concerned with who the participant actually is, but rather with each participant’s opinions and perceptions on intervention ideas. It is possible that people may say they approve or disprove of something that they do not actually approve or disprove of (in real life) – thus harming the data collected – but this is a risk in any research with people (10).
Informed consent was taken online in a Qualtrics survey, as studies have shown that an online consent form provides equal comprehension compared to a written consent form (31). Participants were emailed a link to a survey that laid out the same information as a written participant information sheet and consent form, but clicked ‘agree’ after each consent statement to indicate consent, and then provided identifying details to verify their identity including year of birth, year diagnosed with gestational diabetes, name, and Facebook profile name and link. Participants were informed during the consent process prior to the workshop commencing how to adjust their privacy settings of their Facebook profile – this allowed them to restrict what other participants in the secret Facebook group could see of their profile.
The online study was approved by the University of Edinburgh Health in Social Science ethics committee.
Evaluation of the online workshop
An evaluation plan was developed to explore the feasibility and acceptability of this method to generate a theory of change and theory of action for the intervention.
The key components of a feasibility study include acceptability, practicality, demand, implementation, adaptation, integration, expansion and limited efficacy, based on Bowen and colleagues’ research around important aspects of feasibility studies (32). Acceptability and practicality are most relevant for this study, and how they were assessed is described in detail in Table 1. Demand can be assessed using the Facebook analytics because it shows actual use. Implementation will be based on analytics and the researcher’s own experiences of executing the plan. Practicality, integration and limited efficacy were assessed based on the overall results of the workshop, whereas adaptation and expansion should be explored in future research.
To analyse the data, participants were asked to suggest their own pseudonyms at the end of the survey for analysis and report-writing. Descriptive analyses were performed for the survey questions, as there were no comparison groups. Simple counts were performed to determine the number of comments in total and for each post. For the purposes of developing an intervention, the researcher conducted thematic analysis based on Clarke and Braun’s thematic analysis methodology (not reported in depth here) (33).
Aspects and design
|
Aim of evaluation
|
Details
|
Acceptability
Survey –quantitative qualitative
|
Are recipients and deliverers satisfied with the method? Does it feel appropriate?
|
Survey questions involve assessing participant enjoyment, if they felt it was a safe environment to share thoughts, the timing and quantity of posts, among other questions.
|
Practicality
Survey –quantitative qualitative
|
To what extent can the workshop be carried out with intended participants using existing resources (what they already have?)? Are they able to carry out intervention activities?
|
Implementation
Memo-ing and overall success of workshop
|
1. Can it be successfully delivered to participants in some defined but not fully controlled context?
2. What kind of resources are needed?
3. What factors affect implementation?
4. What is speed & quality of implementation?
|
Researcher’s own experiences (daily memo-ing and reflecting on process)
|
Demand
Facebook analytics
|
1. How likely is this method to be used?
2. What content generated the most and least interaction?
|
Actual use of the workshop by participants. SocioGraph to measure: most commented posts; most reacted posts; average words per comment; ratio of ‘seen by’ per ‘ number of comments’
|
Table 1. Evaluation plan of the secret Facebook group workshop. Based on Bowen and colleagues 2010 How we define feasibility studies paper (32).