Study design
An observational study design was used to assess the fidelity and quality of delivery of the Activate intervention. The self-reported delivery of the content, duration and dose was examined using checklists and the observed fidelity and quality of nurses’ delivery was examined using audio-recordings of intervention consultations. Nurses’ beliefs towards delivering the intervention were assessed using questionnaires which were filled out during the intervention at multiple time points.
The Activate trial
Details of the Activate trial have been reported in detail previously [11]. In short, the Activate trial is a two-armed cluster-randomised controlled trial aiming at enhancing physical activity in primary care patients at risk for CVD. A total of 195 patients divided over 31 general practices (as clusters) throughout the Netherlands participated in the trial, of which 15 were allocated to the intervention group (n=93 patients; n=20 primary care nurses) and 16 were allocated to the control group (n=102 patients; n=16 primary care nurses). Patients in the intervention group received the Activate intervention and patients in the control group received care as usual according to the healthcare standards. The primary outcome was change from baseline at 6-months of follow-up in the number of minutes of physical activity in the moderate to vigorous category, measured with an accelerometer (personal activity monitor; Pam AM300) [20]. Secondary outcomes were sedentary behaviour measured with the accelerometer, self-efficacy for physical activity, patient activation for self-management and health status measured with questionnaires. Outcome data were collected at baseline, at 3-months of follow-up and at 6-months of follow-up.
The intervention was developed using the Behaviour Change Wheel [15] and included a comprehensive behavioural analysis of 1. what hinders and facilitates patients to increase their physical activity and 2. which behaviour is needed from nurses in order to deliver the Activate intervention adequately. The application of the Behaviour Change Wheel for patients’ behaviour resulted in a selection of 17 behaviour change techniques (BCTs), described in the BCT Taxonomy V1 [21]. These 17 BCTs were integrated in four nurse-led consultations during a 3-month period. Consultations were delivered at week 1, 3, 7 and 12 in patients’ own general practice. The first consultation aimed to last for 30 minutes, and the following three consultations for 20 minutes.
The first consultation aimed to enhance patients’ awareness of their behaviour and health consequences and to discuss patients’ motivation towards increasing their physical activity. The second, third and fourth consultation aimed to discuss patients’ level of goal attainment and (re)set a personal action plan. The third and fourth consultation also focused on relapse prevention.
The application of the Behaviour Change Wheel for nurses’ behaviour resulted in a selection of 21 BCTs, described in the BCT Taxonomy V1 [21]. These 21 BCTs were incorporated in a standardised comprehensive training programme for nurses to equip them with the necessary competencies to deliver the intervention. This training programme consisted of a one-day skills training supplemented with two individual coaching sessions held by a health psychologist, instructional videos with examples of how to apply the BCTs in the consultations, a scripted handbook of the content of each of the consultations, and checklists (what to do when).
Participants
The study sample consisted of all primary care nurses (n=20) from 16 general practices situated throughout the Netherlands who participated in the Activate trial and were allocated to the intervention group [12]. In total, these nurses delivered the intervention to 93 patients in 334 consultations.
Data collection
1. Self-reported delivery of the Activate intervention
The self-reported fidelity of delivery of the intervention was assessed by filling out checklists of the discussed content, the consultation duration and dose (n=279 consultations of 86 patients). These checklists (what to do when) are developed by the research team. The intervention content was structured in terms of prescribed intervention components for each of the four consultations separately; see Additional file 1. Nurses were asked to rate each prescribed component as “discussed” or “not discussed” directly after each consultation. Nurses were also asked to administer the consultation duration in minutes and the number of consultations attended (dose).
2. Observed delivery of the Activate intervention
To observe whether nurses delivered the intervention as intended, nurses were asked to randomly audiotape one of each of the four intervention consultations. Prior to each recording of a consultation, patients were asked to verbally consent on the recording. The audio-recordings (n=44 taped by 16 nurses) were used to evaluate whether nurses delivered the prescribed subsequent intervention components and applied corresponding BCTs using a self-developed coding list. Each intervention component and BCT was rated as “discussed” or “not discussed”; see Additional file I. The audio recordings were used to register the duration (in minutes:seconds) of the consultations.
3. Quality of delivery of the Activate intervention
The audio-recordings of the consultations were used to assess the quality of delivery. The quality of nurses’ counselling was assessed using the Behavior Change Counseling Index (BECCI) [22] and an additional scoring list for communication skills that was developed for this study; see Additional file 2. The BECCI is a validated scale to score practitioners’ use of behaviour change counselling in consultations. The BECCI consists of 11 items, which are rated on a five-point Likert scale (0=not at all to 4=a great extent) [22]. Two items were excluded (“Practitioner invites the patient to talk about behaviour change” and “Practitioner demonstrates sensitivity to talk about other issues”) since these items were not applicable. The mean score per item indicates the extent to which nurses applied behaviour change counselling while delivering the Activate intervention.
To assess the nurses’ communication skills that were conditional to deliver the intervention and not covered by the BECCI, a scoring list was developed by members of the research team; see Additional file 2. The scoring list was checked for face validity by all members of the research team and a health psychologist. The scoring list includes five items that cover communication skills that were integrated in the one-day training. A five-point Likert scale was used to provide an indication of the extent to which the trained communication skills were applied (0=not at all to 4=a great extent).
4. Beliefs towards delivery of the Activate intervention
To explore the nurses’ beliefs towards the delivery of the Activate intervention, their beliefs towards their capability, motivation, confidence to the deliver the Activate intervention and beliefs about the effectiveness of the Activate intervention were assessed using a questionnaire. This questionnaire was developed for this study by members of the research team and checked for face validity by all members of the research team and a health psychologist. The questionnaire includes four statements about beliefs towards delivering the intervention and the specific BCTs: 1. “I am capable of [delivering the intervention/BCT]”; 2. “I am motivated to [deliver the intervention/BCT]”; 3. “I am confident that I can [deliver the intervention/BCT]”; and 4. “I am convinced [delivering the intervention/BCT] is effective to enhance physical activity”; see Additional file 3. Nurses rated each statement using a seven-point Likert scale (1=completely disagree to 7=completely agree). Nurses were asked to fill out this questionnaire at four consecutive time points: at the start of the one-day training, directly after the one-day training, after their first individual coaching session and after they finalised the intervention (n=72).
Data analysis
1. Self-reported delivery of the Activate intervention
The checklists of the discussed intervention content were analysed by calculating the proportion of delivered intervention components for each of the four consultations separately and for the intervention as a whole. Consensus criteria were used to constitute adherence to the intervention content, in which <50% constitute low fidelity, 51-79% moderate fidelity and 80-100% high fidelity [7,23]. The self-reported duration of the consultations was analysed using the median and range in minutes. The dose was analysed by the number and percentage of patients who attended the consultations.
2. Observed delivery of the Activate intervention
The audio-recordings were transcribed verbatim. Two researchers independently coded the delivered content of the intervention in each of the consultations using the coding list. After coding every four to six audio-recordings, the researchers compared their findings to ensure consistent application of the coding list. Discrepancies were resolved through discussions. The inter-rater reliability was calculated using Cohen’s kappa [24] and percentage agreement [25]. The observed fidelity was analysed by calculating the proportion of delivered components and BCTs for each of the four consultations separately and for the intervention as a whole. Consensus criteria were used to constitute fidelity of the intervention content [7,23]. The observed duration of the consultations was analysed by the median and range in minutes:seconds.
3. Quality of delivery of the Activate intervention
Two researchers independently scored the BECCI and the scoring list covering communication skills for each of the audio-recorded consultations. After scoring every four to six audio-recordings, the researchers compared their findings to ensure consistent application of the scoring lists. Discrepancies were resolved through discussions. The inter-rater reliability was calculated using Cohen’s kappa [24] and percentage agreement [25]. The BECCI score and observed communication skills were descriptively analysed using the mean score and standard deviation (SD) per item for each of the consultations separately and for the intervention as a whole.
4. Beliefs towards delivery of the Activate intervention
To determine whether nurses’ beliefs about their capability, motivation, confidence and effectiveness of delivering the Activate intervention and applying the BCTs changed over time, the median score and corresponding interquartile range (IQR) per belief over time were calculated.
All analyses were performed using the Statistical Package for the Social Sciences (SPSS version 21; Chicago, IL, USA).