Trial Design
This study utilised a parallel convergent mixed method design including two phases; (i) a pilot cluster RCT, and (ii) a parallel qualitative process evaluation. [13]
Ethical approval and trial registration
Ethical approval was received from the Office of Research Ethics Northern Ireland and was prospectively registered on clinicaltrials.gov on 14/8/2018, registration number NCT03629496.
Recruitment
Phase 1
A formal sample size calculation was not conducted as this is not appropriate for feasibility or pilot studies, as the objectives do not include hypothesis testing to establish the effectiveness of an intervention. The report of statistically significant results in pilots studies tends to be opportunistic, in that the study was not initially designed to establish effectiveness. This calls into question the validity of the effect, as the study was not designed for the purpose of hypothesis testing, and therefore would not have the rigour of a definitive RCT. [14] Feasibility studies should instead focus on the acceptability of trial processes or an intervention, including the willingness of participants to be randomised, the time needed to collect and analyse data and response rates to outcome measures. [15-17]
There is little consensus on the appropriate sample size for a feasibility study, with guidance ranging from 12 per arm to 50 per arm. [18] The justification for larger sample sizes in feasibility studies is to obtain narrow standard deviations on outcome measures to maximise precision in a future power calculation. [19] However, whether this is an appropriate objective at the feasibility and piloting stage is debatable, as to obtain a narrow standard error for a precise power calculation the sample within the feasibility study would need to approach the size of a fully powered RCT.[20] This would in turn increase the likelihood of identifying a statistically significant effect during the feasibility stage, which can reduce the likelihood of a follow-up RCT. [21] A sample size of 30 is recommended by the NIHR’s Research Design Service for the estimation of a parameter, such as sample size, recruitment or attrition rate, for a definitive randomised controlled trial. [22] Due to the increased risk of identifying a statistically significant result with larger sample sizes, the objectives of the study, and the practical limitations of a small, single centre study, a sample size of 30 was selected for the pilot cluster RCT. A statistician at the Centre for Public Health at Queen’s University Belfast (HMcA), was consulted who confirmed this was an appropriate sample size to meet the objectives.
A convenience sample was used from an in-centre haemodialysis unit in Northern Ireland. Patients were screened and identified by clinical gatekeepers, and were approached by the primary researcher (CC) who explained the study in detail to the participant. Participants that expressed interest in participating were provided with a participant information sheet to take home and read. Patients were provided with a period of at least 48 hours to decide if they wished to participate. [23] Those that agreed were provided with a consent form by CC. Reasons for ineligibility and for non-participation were captured in screening logs throughout the recruitment process. The recruitment process lasted approximately a month, from September to October 2018, during which time all eligible participants within the unit (n=98) were approached and offered the chance to participate.
Phase 2
Participants for the process evaluation included both patients who had been recruited into phase 1 of the study, and healthcare professionals working on the haemodialysis unit during the implementation of the intervention.
Eligibility criteria for healthcare professionals:
- A member of the multidisciplinary team, including nurses, healthcare support workers, doctors, dietitians, social workers and counsellors.
- Have had experience with the intervention, meaning they had been present on the unit during implementation of at least one session of the intervention.
- Have worked in a clinical renal setting for more than three months.
During data collection for phase 1 patients were offered the opportunity to participate in the process evaluation. They were approached for participation once the implementation phase of the study had finished but before the three-month data collection time point. A purposive sampling strategy was used, with participants who would provide the richest data in the experimental group recruited. Participants in the control group were included to provide additional data on the trial processes and were purposively selected to provide a variety of experiences on random allocation and insight into the experience of the control condition.
Healthcare professionals were recruited for the process evaluation by purposive sampling. This sample technique was the most appropriate approach as it ensured the participants had the experience necessary to inform the research question. During the pilot cluster RCT the researcher was aware of what healthcare professionals were present during implementation of the intervention and used this to guide their sampling strategy. The ward manager acted as a gatekeeper and screened healthcare professionals to ensure that they met the inclusion criteria and sought permission for the researcher to approach them. Due to managerial and social hierarchies within healthcare, healthcare professionals may have felt pressure to participate when approached by their manager; therefore, the researcher provided the healthcare professionals with the participant information sheet and offered them a cooling-off period to consider participation, typically the time between the initial approach and their next shift on duty. This gave participants a minimum of 24 hours to read through the information sheet and consider whether they wanted to participate in the process evaluation, in line with best practice. [23] Informed consent was then collected at the start of each interview.
Randomization
Due to the highly interactive and collaborative nature of the intervention cluster randomisation was used to reduce the risk of contamination of control participants. All participants were informed that the study would involve random allocation to either engage in arts activities (experimental group) or be placed on a waiting list (control group). The initial randomisation procedure involved clustering participants according to the days of the week they attended haemodialysis. [14] However, due to differences in recruitment across these shift patterns, the clustering procedure was changed to ensure similar numbers in both intervention and control groups. Clusters were changed to the time of day participants attended for haemodialysis treatment.
The randomisation procedure was performed by a researcher not connected to the study. This involved flipping a coin that determined which group would receive the intervention, the allocation was placed in a sealed envelope which was then stored in a locked filing cabinet at the University site, along with the trial management file. The primary researcher (CC) opened the envelope once baseline data collection had been completed. Participants who attended the AM shifts were randomly allocated to the control group, and the participants who attended the PM shifts were randomly allocated to the intervention group.
Intervention
The intervention was developed using the Arts in Health framework, [24] guided by an interdisciplinary PPI advisory group, [24] a scoping review, [25] realist synthesis [6] and consultations with patients and healthcare professionals. [26]
The intervention was developed based on the psychological theory of flow, which posits the existence of a ‘flow state’, a state of optimal experience that results from complete absorption in a task. The hallmark experience of a flow state is ‘tachypsychia’, an altered perception of time where typically time feels like it is passing faster than it actually is. In order to induce a flow state, the task must present a challenge to the individual that they can overcome through skill development. [7] Qualitative literature has suggested arts-in-medicine programmes can induce the hallmark experiences of a flow state in patients who participate, such as an altered perception of time and reduction in rumination and anxiety [26].
The intervention consisted of six one-hour, one-to-one facilitated arts sessions at the participant’s bedside during their haemodialysis treatment, and implemented by CC. The facilitator was a registered mental health and amateur artist who was involved in the development of the intervention. The development of the intervention, and intervention manual, have been described elsewhere. [26] These sessions happened twice a week for a period of three weeks for each participant. Participants had a choice between creative writing or visual art during each session, as these could be safely implemented within the clinical constraints of haemodialysis treatment, and participants could choose from a selection of art materials and prompts.
Adherence, fidelity and dose of the intervention was monitored using activity logs completed by the facilitator, documenting the time spent implementing each art session, the activities the participants engaged in, any instances of non-participation and reasons, general feedback from patients during the sessions, instances of contamination between the control and experimental group and any adverse effects experienced by participants.
Control
Participants from the control group were asked not to participate in any active arts activities during their haemodialysis sessions throughout the study, but that once data collection was completed they would have an opportunity to take part in art sessions and receive a pack of art supplies. This was not a form of delayed entry as no data was collected from the control group once the follow-up time period had finished. The provision of the arts pack following the study was recommended by the interdisciplinary advisory group to promote retention of participants, and the waiting list design was recommended by the OREC NI ethics panel to ensure all participants had the opportunity to receive guidance and instruction on how to use the arts materials.
Outcomes
Phase 1 – Feasibility measures
The main feasibility outcome of interest was the recruitment rate of participants. Assessing the ability to recruit participants is a common issue explored in feasibility trials. [17,18, 28] Screening, approach and recruitment logs were kept during the recruitment phase of the study to capture the proportion of patients eligible and who consented to participation, and reasons for non-participation or ineligibility.
Nephrology and research involving patients with end-stage diseases experience high attrition rates having a detrimental impact on the quality of evidence available in these fields. [28] Therefore, the ability to retain participants is an important consideration prior to a definitive RCT. The attrition rate of participants over a three-month period (from baseline to final follow-up) was captured during data collection and intervention implementation. Reasons for withdrawal were documented in participant’s case report forms (CRF) to identify any modifiable factors that contributed to attrition.
Other outcomes captured during phase 1 included the acceptability of randomisation and clustering method according to differences in demographics and attrition rates between groups; acceptability, adherence and fidelity of the arts-based intervention according to adherence, completion rate and time participants engaged in arts sessions as recorded in the activity logs, and acceptability of clinical outcome measures according to completion rates and proportion of missing data.
Phase 1 – clinical outcome measures
Clinical outcome measures were administered to explore the acceptability and appropriateness of these outcomes within a definitive trial, as opposed to evaluating the effectiveness of the intervention. Baseline demographic and clinical data included age, gender, ethnicity, education, dialysis vintage, dialysis access, frailty as measured by the clinical frailty scale and number of self-reported co-morbidities. [29] These data were collected to explore potential demographic barriers to participation and to explore differences between groups to evaluate the acceptability of the clustering procedure in a definitive trial.
Arts-based interventions can improve depression and have also been found to improve QoL in observational studies of patients receiving haemodialysis. Therefore the Kidney Disease Quality of Life Short Form 36 (KDQoL-SF36), [30] and the Hospital Anxiety and Depression Scale (HADS) [31] were administered at baseline, immediately post-intervention, at six-week and three-month follow-up. Arts-based intervention research has faced criticism due to lack of longitudinal follow-up, which would be necessary to identify whether any benefits are sustained following cessation of the intervention. [9] Participants who are lost to follow-up in longitudinal RCTs concerning complex healthcare interventions tend to be older, diagnosed with a chronic illness and have higher levels of co-morbidity, [32] common demographic factors in patients with ESKD. [33] To establish the feasibility of follow-up within a definitive RCT and establish attrition rates over a longer period of time, participants were follow up at 6-weeks and 3-months post-baseline.
Phase 2 – Qualitative process evaluation
This phase of the study explored acceptability of the arts-based intervention and the trial processes for both patients and HCPs using semi-structured interviews. The semi-structured interviews used interview guides consisting of open questions informed by the RE-AIM QuEST framework. [34] The RE-AIM framework outlines that the reach, effectiveness, adoption, implementation and maintenance of an intervention should be explored with both qualitative and quantitative measures, in order to identify any necessary modifications to improve future implementation, both for replication in research and translation into clinical practice.[34,35] The interview guides used open ended questions to ensure participants could express and explore perspectives that they considered relevant to implementation, but the questions themselves were targeted and specific to the RE-AIM framework to ensure the responses were relevant to informing a larger trial.
Semi-structured interviews were conducted with nine participants from the intervention group, four from the control group, and nine HCPs who had experienced the intervention. Interviews were conducted until data saturation was reached across all three subgroups of participants. The interviews with HCPs and participants from the control group were conducted by CC, whilst interviews with participants from the intervention were conducted by HN and CMcK to reduce bias.
Statistical analysis
Quantitative Data
Data analysis was conducted using the Statistical Package for the Social Sciences (SPSS v 24). Descriptive statistics were used to present baseline demographic and clinical data. Categorical data was presented as frequencies and percentages, while continuous data was presented as means and standard deviations. Recruitment, participation and retention rates were reported and presented in a CONSORT flow diagram. [14] Exploratory inferential statistics were conducted, but no conclusions on the effectiveness of the intervention were made from the results and therefore these were not reported.
Qualitative Data
The semi-structured interviews were recorded and transcribed verbatim, and data were managed using NVivo Version 11. Inductive thematic analysis was used to identify overarching themes and sub-themes relating to the objectives of the process evaluation [36] and the RE-AIM framework. [34,35] The transcripts and identified themes were reviewed, revised and finalised collaboratively by CC, HN, JR and IW.
Progression criteria
Progression to a definitive RCT was determined by recruitment rates and the acceptability of the intervention. [10] The progression criteria for the primary outcomes of recruitment and attrition were based on similar feasibility studies on arts-based interventions in other clinical populations [37] and were confirmed as suitable for the objectives of the study by the statistician at the Centre for Public Health at Queen’s University Belfast (HMcA):
- 75–100% of the target sample size recruited from a single site would result in progression to a definitive RCT
- More than 20% attrition rate from the recruited sample will result in revision of the protocol and data from the process evaluation, and appropriate amendments will need to be made to address barriers to retention of participants, prior to progression to a full trial.[38]
Integration of quantitative and qualitative results
The results were integrated following analysis in order to adequately address the objectives and over all aim of the study. This was achieved by constructing a table that displayed the main quantitative and qualitative findings relating to the key objectives. Quantitative findings were presented and supporting and contradicting qualitative data were identified. [39] Where appropriate other quantitative data was also provided to support or contradict a finding. Whilst the quantitative data were used as a reference point for comparison this was not a consequence of the primacy of the quantitative data, instead the qualitative data provided more nuanced and complex insights that enabled the identification of supportive and contradicting points.