Fidelity of primary care nurses’ delivery of a behaviour change intervention to enhance physical activity in chronic patients

The effectiveness of a nurse-led intervention to enhance physical activity, the Activate intervention, was evaluated in primary care patients at risk for cardiovascular diseases in a cluster-randomised controlled trial (n = 195 patients, 31 general practices). To contribute to an accurate interpretation of the trial results, understanding of how the intervention works and enabling reproducibility, this study aimed to evaluate the delity of delivery of the Activate intervention by assessing: 1. self-reported delity of delivery; 2. observed delity of delivery; 3. quality of nurses’ delivery of the Activate intervention and 4. nurses’ beliefs about their capability, motivation, condence and effectiveness towards delivering the Activate intervention, including behaviour change techniques.

intervention design showed that the use of the Behaviour Change Wheel [15], as theoretical framework for developing the Activate intervention and use of a cluster-RCT design allowed for addressing the research questions [11]. The evaluation of the provider training showed that the standardised comprehensive training programme for nurses facilitated nurses to acquire essential competences to deliver the intervention, although adopting these competences was challenging [11,13]. Moreover, the evaluation of the treatment receipt and enactment showed that patients understood the intervention and used taught skills well and patients performed the intervention and taught skills in their daily lives [14].
To provide insight into the actual delivered content of the intervention, the delity of delivery of the Activate intervention needs to be assessed, which is often considered the core dimension of delity [3,6,16]. In addition to the delity of delivery, evaluating the quality of delivery is recommended as this has shown to in uence the effectiveness of behaviour change interventions [17]. Furthermore, the extent to which nurses are engaged to deliver the intervention according to the protocol is in uenced by their beliefs of their capability, motivation, con dence and effectiveness of the intervention [18,19]. Therefore, the speci c objectives of this study were: 1. To assess the delity of delivery of the Activate intervention by examining nurses' self-reported delity according to the content, consultation duration and dose of the intervention.
2. To assess the observed delity of delivery by examining the content and duration of nurses' delivery of the Activate intervention. 3. To evaluate the quality of nurses' delivery of the content of the Activate intervention. 4. To gain insight into nurses' beliefs about their capability, motivation, con dence and effectiveness towards delivering the Activate intervention.

Study design
An observational study design was used to assess the delity and quality of delivery of the Activate intervention. The self-reported delivery of the content, duration and dose was examined using checklists and the observed delity and quality of nurses' delivery was examined using audio-recordings of intervention consultations. Nurses' beliefs towards delivering the intervention were assessed using questionnaires which were lled out during the intervention at multiple time points.
The Activate trial Details of the Activate trial have been reported in detail previously [11]. In short, the Activate trial is a twoarmed cluster-randomised controlled trial aiming at enhancing physical activity in primary care patients at risk for CVD. A total of 195 patients divided over 31 general practices (as clusters) throughout the Netherlands participated in the trial, of which 15 were allocated to the intervention group (n=93 patients; n=20 primary care nurses) and 16 were allocated to the control group (n=102 patients; n=16 primary care nurses). Patients in the intervention group received the Activate intervention and patients in the control group received care as usual according to the healthcare standards. The primary outcome was change from baseline at 6-months of follow-up in the number of minutes of physical activity in the moderate to vigorous category, measured with an accelerometer (personal activity monitor; Pam AM300) [20]. Secondary outcomes were sedentary behaviour measured with the accelerometer, self-e cacy for  physical activity, patient activation for self-management and health status measured with questionnaires. Outcome data were collected at baseline, at 3-months of follow-up and at 6-months of follow-up.
The intervention was developed using the Behaviour Change Wheel [15] and included a comprehensive behavioural analysis of 1. what hinders and facilitates patients to increase their physical activity and 2.
which behaviour is needed from nurses in order to deliver the Activate intervention adequately. The application of the Behaviour Change Wheel for patients' behaviour resulted in a selection of 17 behaviour change techniques (BCTs), described in the BCT Taxonomy V1 [21]. These 17 BCTs were integrated in four nurse-led consultations during a 3-month period. Consultations were delivered at week 1, 3, 7 and 12 in patients' own general practice. The rst consultation aimed to last for 30 minutes, and the following three consultations for 20 minutes.
The rst consultation aimed to enhance patients' awareness of their behaviour and health consequences and to discuss patients' motivation towards increasing their physical activity. The second, third and fourth consultation aimed to discuss patients' level of goal attainment and (re)set a personal action plan. The third and fourth consultation also focused on relapse prevention.
The application of the Behaviour Change Wheel for nurses' behaviour resulted in a selection of 21 BCTs, described in the BCT Taxonomy V1 [21]. These 21 BCTs were incorporated in a standardised comprehensive training programme for nurses to equip them with the necessary competencies to deliver the intervention. This training programme consisted of a one-day skills training supplemented with two individual coaching sessions held by a health psychologist, instructional videos with examples of how to apply the BCTs in the consultations, a scripted handbook of the content of each of the consultations, and checklists (what to do when).

Participants
The study sample consisted of all primary care nurses (n=20) from 16 general practices situated throughout the Netherlands who participated in the Activate trial and were allocated to the intervention group [12]. In total, these nurses delivered the intervention to 93 patients in 334 consultations.
Data collection 1. Self-reported delivery of the Activate intervention The self-reported delity of delivery of the intervention was assessed by lling out checklists of the discussed content, the consultation duration and dose (n=279 consultations of 86 patients). These checklists (what to do when) are developed by the research team. The intervention content was structured in terms of prescribed intervention components for each of the four consultations separately; see Additional le 1. Nurses were asked to rate each prescribed component as "discussed" or "not discussed" directly after each consultation. Nurses were also asked to administer the consultation duration in minutes and the number of consultations attended (dose).

Observed delivery of the Activate intervention
To observe whether nurses delivered the intervention as intended, nurses were asked to randomly audiotape one of each of the four intervention consultations. Prior to each recording of a consultation, patients were asked to verbally consent on the recording. The audio-recordings (n=44 taped by 16 nurses) were used to evaluate whether nurses delivered the prescribed subsequent intervention components and applied corresponding BCTs using a self-developed coding list. Each intervention component and BCT was rated as "discussed" or "not discussed"; see Additional le I. The audio recordings were used to register the duration (in minutes:seconds) of the consultations.

Quality of delivery of the Activate intervention
The audio-recordings of the consultations were used to assess the quality of delivery. The quality of nurses' counselling was assessed using the Behavior Change Counseling Index (BECCI) [22] and an additional scoring list for communication skills that was developed for this study; see Additional le 2.
The BECCI is a validated scale to score practitioners' use of behaviour change counselling in consultations. The BECCI consists of 11 items, which are rated on a ve-point Likert scale (0=not at all to 4=a great extent) [22]. Two items were excluded ("Practitioner invites the patient to talk about behaviour change" and "Practitioner demonstrates sensitivity to talk about other issues") since these items were not applicable. The mean score per item indicates the extent to which nurses applied behaviour change counselling while delivering the Activate intervention.
To assess the nurses' communication skills that were conditional to deliver the intervention and not covered by the BECCI, a scoring list was developed by members of the research team; see Additional le 2. The scoring list was checked for face validity by all members of the research team and a health psychologist. The scoring list includes ve items that cover communication skills that were integrated in the one-day training. A ve-point Likert scale was used to provide an indication of the extent to which the trained communication skills were applied (0=not at all to 4=a great extent).

Beliefs towards delivery of the Activate intervention
To explore the nurses' beliefs towards the delivery of the Activate intervention, their beliefs towards their capability, motivation, con dence to the deliver the Activate intervention and beliefs about the effectiveness of the Activate intervention were assessed using a questionnaire. This questionnaire was developed for this study by members of the research team and checked for face validity by all members of the research team and a health psychologist. The questionnaire includes four statements about beliefs towards delivering the intervention and the speci c BCTs: 1. "I am capable of [delivering the intervention/BCT]"; 2. "I am motivated to [deliver the intervention/BCT]"; 3. "I am con dent that I can [deliver the intervention/BCT]"; and 4. "I am convinced [delivering the intervention/BCT] is effective to enhance physical activity"; see Additional le 3. Nurses rated each statement using a seven-point Likert scale (1=completely disagree to 7=completely agree). Nurses were asked to ll out this questionnaire at four consecutive time points: at the start of the one-day training, directly after the one-day training, after their rst individual coaching session and after they nalised the intervention (n=72).
Data analysis 1. Self-reported delivery of the Activate intervention The checklists of the discussed intervention content were analysed by calculating the proportion of delivered intervention components for each of the four consultations separately and for the intervention as a whole. Consensus criteria were used to constitute adherence to the intervention content, in which <50% constitute low delity, 51-79% moderate delity and 80-100% high delity [7,23]. The self-reported duration of the consultations was analysed using the median and range in minutes. The dose was analysed by the number and percentage of patients who attended the consultations.

Observed delivery of the Activate intervention
The audio-recordings were transcribed verbatim. Two researchers independently coded the delivered content of the intervention in each of the consultations using the coding list. After coding every four to six audio-recordings, the researchers compared their ndings to ensure consistent application of the coding list. Discrepancies were resolved through discussions. The inter-rater reliability was calculated using Cohen's kappa [24] and percentage agreement [25]. The observed delity was analysed by calculating the proportion of delivered components and BCTs for each of the four consultations separately and for the intervention as a whole. Consensus criteria were used to constitute delity of the intervention content [7,23]. The observed duration of the consultations was analysed by the median and range in minutes:seconds.

Quality of delivery of the Activate intervention
Two researchers independently scored the BECCI and the scoring list covering communication skills for each of the audio-recorded consultations. After scoring every four to six audio-recordings, the researchers compared their ndings to ensure consistent application of the scoring lists. Discrepancies were resolved through discussions. The inter-rater reliability was calculated using Cohen's kappa [24] and percentage agreement [25]. The BECCI score and observed communication skills were descriptively analysed using the mean score and standard deviation (SD) per item for each of the consultations separately and for the intervention as a whole.

Beliefs towards delivery of the Activate intervention
To determine whether nurses' beliefs about their capability, motivation, con dence and effectiveness of delivering the Activate intervention and applying the BCTs changed over time, the median score and corresponding interquartile range (IQR) per belief over time were calculated.
All analyses were performed using the Statistical Package for the Social Sciences (SPSS version 21; Chicago, IL, USA).

Results
All participating nurses were female (n=20), had a mean age of 46.9 years (SD 10.7) and had 6.8 years (SD 4.2) of working experience with patients at risk for CVD ( Table 1). The majority of nurses received additional training in coaching techniques (n=16; 80%) prior to the Activate intervention.  Table 2 and Additional le 1. Nurses' overall self-reported delity to delivery of the Activate intervention was high: 87.6% (48.5%-100%); see Table 2. Overall, all components were delivered; however, nurses less frequently reported discussing the use of prompts and cues (48.5%-75.9%). The median self-reported duration of all consultations was 20 minutes (10-50 minutes). The median duration of the consultations aligned with the prescribed duration (consultation 1: 30 minutes; consultation 2, 3 and 4: 20 minutes). However, the duration of all consultations showed a wide range; see Table 3.
The median observed duration of the consultations was 18:29 minutes (11:11-37:39). The median duration was lower than the prescribed duration and decreased as the intervention progresses; Table 3.

Beliefs towards delivery of the Activate intervention
All nurses lled out the rst questionnaire, the second questionnaire was lled out by 15 (75%) nurses and the latter by 17 nurses (85%).
After nurses followed the one-day training, they felt they were capable, motivated and con dent to deliver the intervention. The nurses were positive about the effectiveness of the Activate intervention to improve patients' level of physical activity. Nurses' beliefs towards delivering the Activate intervention did not substantially change over time (Table 6).
Generally, nurses' beliefs about their capability (median 4-5) and con dence (median 5-6) to apply the BCTs were moderate at the start of the one-day training and consistently improved afterwards. Nurses' motivation to apply the BCTs and their beliefs about the effectiveness of the BCTs were considerably high and consistent over time (median 5-7); see Additional le 3. Nurses' beliefs about their capability, motivation, con dence of applying the BCTs and its effectiveness tend to slightly uctuate over time, as scores slightly decreased after their rst individual coaching session (measurement 3) and stabilised or increased after nalising the intervention (measurement 4).

Discussion
This study assessed the delity of delivery of the Activate intervention by primary care nurses. The selfreported and observed delity of delivery of the prescribed subsequent components of the intervention was high. The observed delity of the BCTs constituted moderate delity. The overall observed quality of delivery was su cient as nurses frequently applied most communication skills and behaviour change counselling skills. Nurses felt capable, motivated and con dent to deliver the intervention. They considered the intervention, including the BCTs, to be effective in improving patients' level of physical activity. Nurses' beliefs regarding their capability and con dence improved consistently after nurses received the training. Nurses' beliefs about the effectiveness of the BCTs intervention and their motivation to deliver the intervention remained high over time.
The high self-reported and moderate to high observed delity of delivery of the Activate intervention was comparable [26] and higher [9,27,28.] than observed in other behavioural interventions including BCTs.
These differences might illustrate the inconsistency in the way behaviour change interventions are implemented [27]. The high delity of the intervention components and moderate delity of the BCTs might be explained by numerous reasons. First, the results of our qualitative study about nurses' perceptions towards intervention delivery revealed that the comprehensive training programme, including a one-day training, training tools and individual coaching, equipped nurses in acquiring the competences to deliver the intervention and boosted their delivery [13]. Given the complexity of behaviour change and nurses' tendency to easily relapse into traditional habits, adding training tools and coaching, additionally to a one-day training, are considered to be necessary and recommended to increase the intervention delity [10]. Furthermore, the use of training tools that can be easily used in practice is likely to increase delity [19]. Second, nurses were instructed to adhere to the protocol to increase the delity. Our qualitative study revealed that they tried to adhere to the consultation structure although there were challenges in delivering the intervention according to the protocol, such as distraction by patients who initiated discussion of other topics [13]. Third, the qualitative study showed that nurses were engaged to acquire skills in behaviour change support as they felt a need to improve their support. Patients' success of the intervention strengthened their engagement towards delivering the intervention and aligned with their intrinsic drive of being a nurse [13]. Furthermore, nurses' engagement towards intervention delivery was con rmed by the results of nurses' beliefs towards delivering the intervention. The delity of delivery was highest at the rst consultation and the subsequent consultations had slightly lower delity. This variation of delivery across consultations might be explained by the fact that nurses rarely discussed restructuring the physical or social environment, the use of prompts/cues and past successes, which were included in the second, third and fourth consultation. Despite the fact that they rarely applied these components and corresponding BCTs, our results showed that nurses considered themselves as capable, motivated and con dent to deliver these components and BCTs and considered them to be effective.
Furthermore, nurses overestimated their delivery of these components and BCTs compared to those observed. Nurses could have reported these components as being discussed while they only discussed a small proportion or only slightly touched upon these components and BCTs. Probably nurses did not recognise the value and content of discussing these components and BCTs as they were not used to discuss these components and apply these BCTs in their routine practice. Moreover, these components and BCTs might not have been relevant to the needs and concerns of all patients. However, nurses strictly adhered to deliver the core components and BCTs of the intervention, such as goal setting, action planning, review on behavioural goal(s), feedback on behaviour and self-monitoring. Tailoring the intervention to patients' individual circumstances is inherent to behavioural interventions and might result in applying only the prescribed core components and BCTs in one consultation and thus in not achieving 100% delity of delivery [7,29]. Therefore, strict adherence to the core components and BCTs of the intervention could be regarded as successful delivery of the intervention [30].
To optimise nurses' performance and delity, nurses received two individual coaching sessions after they started the Activate intervention and delivered several second consultations (coaching session one) and several third consultations (coaching session two). Our qualitative study revealed that nurses highly valued these coaching sessions as these enhanced their perceived quality of delivery [13]. The coaching primarily focused on their delivery of the core components and BCTs of the intervention, which might account for the lack of their delivery of the less essential components and BCTs.
The self-reported and observed delity showed some discrepancies as nurses regularly rated a lower or higher adherence than observed. Such discrepancy is commonly reported in studies as it is di cult to re ect on one's own performance and underpins the importance of observing the delity of delivery [8,9,26].
Interventions with higher levels of delity of delivery are associated with the effectiveness of such interventions [31]. Given the high delity of the Activate intervention, it is unlikely that the effectiveness of the intervention is underestimated. However, the delity of delivery of the BCTs was moderate and therefore showed room for improvement. In routine care, nurses insu ciently focused on behaviour change support and rarely applied BCTs [32]. Therefore, one could argue whether nurses were able to apply the BCTs correctly, despite the comprehensive training. Furthermore, while most nurses highly adhered to the protocol, their quality of delivery showed room for improvement as nurses easily tend to relapse into their own consultation style of closed-questioning, giving advice and lling in for patients, and nurses' tendency to adjust the intervention to their own beliefs and feelings of comfort [13].
Mastering complex interventions, such as the Activate intervention, requires tailored training tools, regular practice opportunities and ongoing coaching [13,18]. The su cient quality of delivery might have contributed to the lack of signi cant improvement of patients' level of physical activity. Using a validated comprehensive scoring list might have enhanced the assessment of the quality of delivery. However, to the best of our knowledge, such a scoring list is lacking and therefore a scoring list to measure the quality of delivery was developed.
Furthermore, the delity and quality of delivery varied within and across nurses, which is consistent with routine care [32] and might also have in uenced the real delivery of the intervention [33]. Moreover, despite our expectations and efforts, nurses submitted only a low number of audio-recordings across the intervention period, which might have over-estimated the delity of delivery as these consultations are likely to represent a 'best case' scenario [27]. Given all methodological factors in uencing the assessment of the real delity of delivery, it is likely to assume that these factors diluted the effectiveness of the Activate intervention.

Strengths and limitations
This study has several strengths. First, the interrater reliability between observers of the intervention content was almost perfect. All audio-recordings were independently coded by two researchers. One of the researchers was independent from the trial, suggesting that coding is likely to re ect actual performance without in uences of knowledge related to the nurses. The assessed intervention content was highly speci c due to the detailed protocol and the use of the taxonomy to code the applied BCTs [21], allowing consistent and systematic coding. These aspects suggest that the observed delity is reliable. Second, by de nition, delity of delivery refers to the extent to which the core intervention components are delivered as intended, which is distinguished from how components are delivered, such as quality of delivery [7]. The addition of the quality assessment is recommended [10] as this has been shown to in uence the effectiveness of behaviour change interventions [17]. Furthermore, the assessment of nurses' quality of delivery and beliefs of delivery of the intervention and the BCTs deepened our understanding of how and what nurses delivered.
Some limitations need to be addressed. First, nurses reported their discussed components of each of the consultations, but some nurses did not report non-delivery of speci c components by leaving these components un lled. Afterwards, nurses con rmed these components as non-delivered. Furthermore, nurses were not required to report the applied individual BCTs. Requiring nurses to self-report their adherence in a detailed level of BCTs might increase the accuracy but decrease adherence to their selfreports [26]. The high probability of inaccuracy and incompleteness of the self-reported data might have resulted in an overestimation of the self-reported delity, which we were not able to verify due to the relatively low number of audio-recordings. This suggests that delity of delivery should be observed rather than rely on self-reported delity of delivery [9,26,27].
Second, the 44 analysed audio-recordings represent 13.2% of the total n = 334 delivered consultations, which is lower than the 20% minimum recommended [30]. The qualitative evaluation among nurses revealed that nurses perceived recording of consultations as uncomfortable and felt being judged knowing that their performance was being analysed [13]. Despite that nurses were instructed to randomly audio-tape their consultations, the prudency of nurses towards audio-recording their consultations might have introduced selective inclusion of recorded consultations. Recording all delivered consultations might have reduced nurses' reluctance towards recording their consultations; however, this would probably have led to non-participation of nurses and patients in the trial. By using self-reports we strived to gain good insight into the delity of delivery. However, the comparison between the self-reported and observed delity urged caution due to this low number of audio-recordings. Although the self-reported delity was based on 279 of all 334 (83.5%) of intervention consultations and the observed delity showed similar high delity, the audio-recordings might not re ect all consultations delivered by nurses. Furthermore, we were not able to speci cally compare the self-reported versus the audio-recorded consultations as the audio-recordings were depersonalised.
Third, some nurses did not ll out the checklists or audio-record their consultations. These nurses might have shown lower delity of delivery.
Fourth, consultation duration varied across consultations. We did not assess whether consultation duration was associated with the degree of delity. Therefore, we could not evaluate whether (in)su cient time to deliver all subsequent components of the intervention and BCTs has contributed to the degree of delity of the prescribed intervention content.

Conclusions
Nurses delivered the prescribed components of the Activate intervention with high delity and applied the BCTs with moderate delity. The quality of delivery was su cient. Nurses felt capable, motivated and con dent to deliver the intervention and BCTs and considered the intervention, including BCTs, to be effective in enhancing patients' level of physical activity. Several methodological factors and nurses' variation in complex behaviour change delivery might have diluted the quality of delivery and therefore might have diluted the effectiveness of the Activate intervention. Availability of data and materials The datasets analysed during the current study are available from the corresponding author upon reasonable request.