Implementation fidelity of a transition program for adolescents with congenital heart disease: the STEPSTONES project

doi:10.21203/rs.3.rs-850443/v1

Background

Although transition programs have been evaluated for adolescents with chronic conditions, these have rarely involved process evaluations investigating the implementation process. The aim of this study was to evaluate the extent to which a transition program for adolescents with congenital heart disease was delivered as intended. Research questions were 1) to what level of fidelity was the program delivered? and 2) what potential moderating factors affected the delivery of the program and overall fidelity?

Methods

A mixed methods design was used, where a process evaluation was embedded in the STEPSTONES randomized controlled trial in Sweden. The implementation fidelity framework by Carrol (2007) and Hasson (2010) was used to design, collect and analyze data. Quantitative data consisted of intervention records on adherence and were analyzed with descriptive statistics. Qualitative data on moderators affecting fidelity were collected through interviews, log-books and focus group interviews with healthcare professionals implementing the intervention and participatory observations of the implementation process. Data were analyzed with deductive content analysis. Triangulation was used to integrate quantitative and qualitative data within the fidelity framework.

Results

Six out of eight components of the transition program were delivered to an extent that adhered to the program theory or achieved a high level of fidelity. However, components involving peer support had a low attendance by the participating sample (32.2%), and the joint transfer meeting was challenging to implement, despite achieving high adherence. Moderators affecting the implementation process were the adolescent’s and healthcare professional’s engagement in the intervention, contextual factors and a lack of standard operating procedures for all components in the program.

Conclusion

Barriers and facilitators for a future implementation of transition programs have been illuminated in this study. The use of an implementation fidelity framework in the process evaluation proved successful in providing a comprehensive evaluation of factors affecting the implementation process. However, implementation fidelity must be considered in relation to adaptations to the local and personal prerequisites in order to create interventions that can achieve fit.

Cardiac & Cardiovascular Systems

Pediatrics

Process evaluation

Implementation fidelity

Mixed methods

Randomized controlled trial

Adolescent

Chronic disease

Transition of care

To facilitate the transition to adulthood and transfer to adult care for adolescents with chronic conditions (CC) transition programs have been developed and evaluated through randomized controlled trials (RCT) (1). Transition programs are complex interventions. Consequently they target multiple organizational levels (i.e. pediatric and adult care), consist of several interacting components and require behavior change by those delivering and receiving the intervention. In addition, the causal pathways leading to outcomes are hidden and affected by several factors in the implementation process and surrounding context (2, 3). However, the implementation processes of transition programs have been scarcely evaluated (3). Such evaluations are considered process evaluations, where the implementation process, change mechanisms and contextual impact are explored in order to understand how and why outcomes are created. Implementation can both describe the process of translating evidence-based knowledge into clinical practice and implementation within an effectiveness evaluation, such as an RCT (2). If not assessing how complex interventions are delivered and implemented, perfect implementation is assumed, which is rarely or never the case (4).

Within process evaluations, implementation fidelity is investigated (5–7). This concept refers to the degree to which an intervention is delivered according to protocol (5). The relationship between fidelity and trial outcomes are well known (5). However, evaluations of fidelity have been debated, as interventions aiming to be tailored and adapted to the local context have proven to be more successful than interventions strictly adopted from protocol (8). Thus, in addition to assessing fidelity, it is pivotal for evaluations of complex interventions to include investigation of the implementation process, exploring how the intervention was delivered in order to understand whether a successful trial was due to successful implementation, or if a failed trial was due to poor implementation (4).

This study is part of the STEPSTONES (Swedish Transition Effects Project Supporting Teenagers with chrONic mEdical conditionS) project which aims to design, evaluate and implement a transition program for adolescents with congenital heart disease (CHD) in Sweden (9, 10). The aim of this study therefore was to evaluate to what extent the transition program was delivered as intended. Research questions related to the aim were 1) to what level of fidelity was the program delivered? and 2) what potential moderating factors affected the delivery of the program and overall fidelity?

A mixed methods design was used in this process evaluation study in order to provide a comprehensive evaluation of factors affecting implementation fidelity. The embedded design was chosen, as qualitative and quantitative data on implementation were embedded in the STEPSTONES RCT (11). A protocol describing the full extent of the process evaluation has been previously published (10). The Good Reporting of A Mixed Methods Study (GRAMMS), along with the Consolidated criteria for reporting qualitative research (COREQ) checklist was used in the design and reporting of the manuscript (Additional file 1).

The STEPSTONES transition program

The transition program was developed through intervention mapping to ensure an evidence-based and theoretically rigorous intervention in line with the stakeholders’ needs (12, 13). The RCT was carried out in seven outpatient pediatric cardiology units in Sweden between 2016–2021. The primary outcome was patient empowerment (14). Empowerment is an asset related to the individual’s capacity to manage their health and care, involving control and influence over their condition (15). Based on the primary outcome, a sample size of 210 patients were targeted in a three-armed design (Fig. 1) in order to handle potential contamination of the intervention to usual care (9). As the intervention was delivered at two of the seven centers, only participants and healthcare professionals (HCPs) enrolled in these two centers were part of the process evaluation.

The transition program consisted of eight components delivered through five implementation steps (Fig. 2). However, the components were not strictly delivered from step 1–5, but aimed to be tailored to the adolescents’ needs. Person-centered care (PCC) along with knowledge and skills in adolescent health were the theoretical foundations on which the intervention was built (9, 10). Furthermore, behavior change techniques to achieve patient empowerment were applied based on the needs assessment of the target population from previous studies (13, 16–19).

Methodological framework

The implementation fidelity framework by Carrol et al. (2007), further developed by Hasson (2010), (5, 6) was used to guide qualitative and quantitative data collection, analysis and integration (Fig. 3).

The evaluation of implementation fidelity is focused on the concept of adherence, which is defined as whether ‘a program service or intervention is being delivered as it was designed or written’ (20). Adherence includes the subcategories’ content, frequency, duration and coverage, which refers to the dose of the intervention received by participants (5, 21). Table 1 provides an overview of the constructs of implementation fidelity with related data sources in the present study.

Table 1

Overview of constructs of implementation fidelity, potential moderators, and data sources investigating these
	Adherence				Potential moderator
	Dose	Content	Frequency	Duration	Quality of delivery	Participant responsiveness	Facilitation strategies	Recruitment	Context
Quantitative data sources
Transition plan	X	X	X	X	X
Intervention implementation forms	X	X	X	X	X
Qualitative data sources
Paired interviews with transition coordinators		X			X	X	X		X
Log-books of transition coordinators		X			X	X	X	X	X
Focus group interview with ACHD nurses		X			X	X			X
Participatory observations		X			X	X	X		X

Fidelity is also proposed to be moderated by six factors (5, 6), five of which are investigated in this study and presented in Table 1. Quality of delivery is the manner in which the HCPs deliver the program in relation to the theoretical ideal and program content (21). Participant responsiveness relates to how participants respond to or are engaged by an intervention. It refers to both the individuals receiving the intervention (i.e. adolescents and parents) and those delivering it (i.e. HCPs). Facilitation strategies are approaches used to enable implementation and can be the use of manuals, guidelines, and training, along with monitoring and feedback for those delivering the intervention (5). In the present study, we expanded this concept by also investigating what strategies HCPs used to overcome obstacles in delivering the intervention. Recruitment focuses on procedures to approach and retain participants to the study, and finally, context refers to if and how the intervention was adapted to the local prerequisites. Context takes into account the social, organizational, and cultural events in the assessment of implementation (6) (Table 1). Comprehensiveness of policy description was not investigated as it was beyond the scope of this study.

Study sample and data collection

Quantitative data

Adolescents randomized to the transition program at the two intervention centers (n = 67) were sampled and inclusion criteria followed those of the RCT (9, 10). Quantitative data were collected longitudinally in parallel with the effectiveness evaluation. Two primary data sources were used: 1) transition plans (i.e. intervention records) of the included patients which provided data on adherence, dose and quality of delivery, and 2) intervention implementation forms, which provided detailed data on content and dose for each individual encounter (i.e. face-to-face, telephone, text message) in the transition program (Table 1) (Additional file 2). Adverse event reports were also included to ensure patient safety (9, 10). To ensure validity of data they were cross-checked with the enrolment form of the RCT before analysis.

Qualitative data

A multilevel nested sampling was employed where the qualitative sample was nested within the quantitative sample of the intervention study (22). This sample included participating adolescents and parents in the transition program along with HCPs. HCPs included the transition coordinators (TC) responsible for delivering the intervention, adult and pediatric HCPs (i.e. nurses, physicians, counselors and physiotherapists) and two patient representatives involved during the information day for participating adolescents and parents. Four qualitative data sources evaluated implementation and potential moderating factors (Table 1) and were collected by the first author (MS), which was only involved in the process evaluation of the RCT.

Paired interviews with TCs

TCs (n = 2) working at each intervention center were interviewed together four times throughout the intervention study. Semi-structured interviews (Additional file 3) were carried out in 6-12-month intervals, where they were asked to reflect on their experiences of working as TCs in the transition program along with barriers and facilitators in program implementation. Paired interviews allowed for their joint perspectives to be explored, and deepened the understanding of similarities and differences in these experiences (23). The interviews ranged between 63–68 minutes (m = 66) and were audio-recorded and transcribed verbatim by the first author (MS).

Log-books of TCs

Throughout the intervention study, the TCs were asked to keep log-books where they reflected on the experiences of working in the transition program along with barriers and facilitators to program implementation. These log-books allowed for individual reflection and for the TC to write about aspects of the transition program that might be lost in the interviews due to recall bias.

Participatory observations

To explore how the transition program was implemented in practice, participatory observations of the outpatient visits with TCs (n = 17) and information day for adolescents and parents (n = 3) were carried out longitudinally. In addition, informal interviews were carried out with the TCs if needed to verify or discuss the observation. In the observations of the outpatient meetings, sampling strived for variation (24) in terms of intervention center, visit observed (i.e. visit one, two or three), adolescents’ sex, and disease complexity of the CHD. During the observations, the observer (MS) did not interact with the participants if not approached. Participants in the observations were asked beforehand if they consented to the observation, with the option to decline. No participants declined observation. At the time of observation, the observer introduced himself and received written and oral informed consent from the participants. The same procedures were followed for the observations of the information day for participating adolescents and parents, except that additional written informed consent was not retrieved from the participants during these meetings. The field notes were handwritten and transcribed in close connection to the observations to ensure trustworthiness (25).

Focus group interview with Adult Congenital Heart Disease (ACHD) nurses

A focus group investigating the experiences of the transition program was performed with nurses (n = 4) working at the ACHD clinics at both intervention centers. The focus group lasted 55 minutes and was held in December 2020 through the video conferencing software Zoom ™ as the intervention centers were located in different cities and due to the Covid-19 pandemic (26). The focus group was audio-recorded and transcribed verbatim by the first author (MS).

Data analysis

Quantitative data

To estimate the level of adherence, data on dose, content, frequency and duration for the components and implementation steps were analyzed together, corresponding to the components of the transition program. Data were analyzed using descriptive statistics in the Statistical Package for Social Sciences (SPSS) v.24 and presented in absolute numbers and proportions. To estimate the level of fidelity, we created four categories based on each quartile of the percentages from 0-100 (i.e. low, somewhat low, somewhat high and high).

Qualitative data

Qualitative data on potential moderating factors to implementation fidelity were analyzed using deductive content analysis according to Kyngäs and Kaakinen (2020), using the five domains of implementation fidelity moderators as a coding matrix (27). The analyses were performed by two authors (MS & ELB) in NVivo v.12 and discussed among the co-authors to ensure trustworthiness. First, qualitative data from the four sources were read through to get a grasp of the whole. Second, coding of meaningful units of text related to the aim was performed and sorted into the five domains of the fidelity framework. Third, codes were compared and merged together within the framework’s domains.

Mixed method integration

Qualitative and quantitative data were collected and analyzed separately. However, to increase the understanding of implementation fidelity and moderating factors, method triangulation was used to integrate the findings (11, 28). Triangulation served the purpose of validating findings between different sources (e.g. using findings from participatory observations to validate findings from interviews) and to emphasize different aspects of the implementation process. In addition, data from qualitative sources were used to explain and understand findings from the quantitative sources and give answers to potential pitfalls in implementation (4). Qualitative and quantitative data have therefore been presented together in the results according to the conceptual framework of implementation fidelity.

The characteristics of the participating sample in the intervention group (i.e. process evaluation sample) and participants in the RCT is described in Table 2. From the 67 adolescents randomized to the intervention group, eight dropped out before the first implementation step, meaning only the remaining 59 adolescents were included in the analysis and results.

Table 2

Demographic characteristics of process evaluation sample in relation to RCT-sample
Characteristics	Process evaluation sample n = 59 (%)	RCT sample n = 128 (%)
Female sex	26 (44.1)	59 (45.7)
CHD complexity*
Mild	9 (15.3)	12 (9.4)
Moderate	31 (52.5)	81 (63.3)
Severe	19 (32.2)	35 (27.3)
Comorbidities	18 (30.5)	28 (21.9)
Centre
Centre 1	44 (74.6)	91 (71.1)
Centre 2	15 (25.4)	37 (28.9)
*CHD complexity categorized according to 2018 AHA/ACC Guideline for Management of Adults with Congenital Heart Disease.

Adherence and moderating factors on implementation fidelity

Table 3 describes adherence to the components of the transition program along with the categories of fidelity. Below, adherence is presented together with quality of delivery.

TABLE 3 HERE

Quality of delivery and content delivered

Quality of delivery investigated the manner in which the intervention components were delivered and is presented according to the eight components of the transition program.

Transition coordinator

There was high attendance by the adolescents to the three outpatient visits with the TC (Table 3). According to the observations, the TCs followed the SOPs to a high extent, yet stayed true to the cornerstones of PCC. Moreover, during the interviews, the TCs expressed enthusiasm for the intervention and for being able to meet the adolescents in this format, which served as a motivation for the TC.

Written person-centered transition plan

The implementation of PCC components varied. Goal setting, identification of supportive networks and participation in care were carried out to a high extent during the first and second outpatient visit (Table 3). The transition plan was jointly documented by the adolescent and TC during the outpatient visits, a central part of the adolescent’s participation in the plan being to formulate, write and carry out goals for their health in transfer and transition. In all observations of the outpatient visits with the TC (n = 17), the adolescent along with the TC formulated goals. However, the TC sometimes needed to guide the adolescent in goal setting by using the adolescent’s personal narrative as a facilitator. The TCs expressed that goal setting was difficult for adolescents who were perceived as disengaged or who had few symptoms of their CHD. Throughout the course of the trial, the TCs found it easier to support adolescents in goal setting as they found strategies to deal with this (Table 4).

Table 4

Illustration of domains of the fidelity framework with quotes and excerpts from field notes.
Domain	Sub-domain	Quote
Quality of delivery and content delivered	Written person-centered transition plan	“I do have a large number with goals that aren’t at all related to their heart defect but that they feel very motivated about, so I try to find what they want to work on because I think it isn’t a question of...of us wanting to achieve empowerment so maybe it doesn’t matter if they manage to learn about their heart condition inside out or if they manage to get a pass in maths. It might have more to do with feeling capable and that I’ve succeeded at something – whatever it is. So trying to find something we can improve so they can feel they are developing in it.” - Paired interview no. 2 with Transition Coordinators
	Information and education about the condition and information and contact with the adult clinic	“They’re still discussing how the heart defect affects social aspects. The youngster says her classmates don’t know about her heart defect. The reason is she wants to keep her condition to herself and is scared that things will change in class if they find out about it. She then says one person in the class knows about her heart defect – “It depends on how close you are to that person”, she says. The TC confirms that the youth can decide for herself whether to tell the others in her class about her heart defect or to keep it to herself.” - Field note, observation 2, visit 1, center 2
		“The TC starts explaining the anatomical heart model and a colorful picture of the heart. She first explains normal physiology, before going on to describe by drawing what kind of heart defect he had from birth. And then the TC feeds back that this is complex knowledge and that he doesn’t have to understand everything now and it will come gradually. Then she explains what operations have been done. The youth says he didn’t actually know a number of things the TC is explaining. The TC then takes out more pictures of what the heart looks like now, after the operation. The TC keeps reiterating that this is very difficult to understand and that you have take it step by step. She asks if his head is spinning – a bit, says the youth. The TC then asks him what he thinks is the best way to learn this. Bit by bit, replies the youth. The TC also explains that it depends on how much you want to learn. They both conclude that this is the goal – to learn about the heart defect from the beginning. The TC now asks how he thinks he wants to learn. He wants to read up on it to learn. The TC explains she has prepared educational materials for other patients and emphasizes that he doesn’t need to know everything by next time – it isn’t a test.” Field note, observation 9, visit 1, center 2
Participant responsiveness	Moderators of adolescents’ responsiveness	“There was a girl I met on the first visit... I can’t remember when she was born but she was suffering badly with her mental health from the start and we weren’t focusing on the transition or heart defect at all – we couldn’t approach that. So the conversation wasn’t at all as we had planned. But when she came for the next visit, even though the last conversation had been of a different character it meant we had built some kind of trust – I was familiar with her situation and had a completely different starting point from which to approach these questions that I had been meaning to ask in our first meeting” - Paired interview no: 4 with Transition Coordinators
		“The TC goes on to ask the youth what she knows about her heart defect. This makes the youth a bit nervous. It feels like the conversation is starting to take a turn here – the youth isn’t as responsive but just gives shorter and shorter answers. Then the youth says to herself: I should know this. The TC replies: it isn’t a test.” - Excerpt from field note from observation no: 6, first visit in the transition program at Center 2.
		“Because I have some I’d like to meet every three months to work more intensively on certain things... medication management, for example, quite often it can be a year...that’s kind of too long! And for some it is...” - Paired interview no: 3 with Transition Coordinators
	Moderators of the TCs’ responsiveness	“We have quite a few with ADHD… And how to reach them as well as everything going on in their heads. How they say their heads are boiling over and it’s really hard to take in what it should be like when their heads are just fizzing and they say that one of their mates usually smokes some weed and then they get much calmer... “Yeah, okay, how do you reason with that? So it would have been really exciting to speak to someone who meets these youngsters more often and simply understand what the scenarios are and stuff. Because these are things you get stumped by. Not being taken aback and being able to carry on and have a good conversation, because these are things they can’t really tell their parents either, even if they have a good relationship with them. But it’s exciting... and a bit like, ‘oh’” - Paired interview no: 2 with Transition Coordinators
		“I think it’s so much fun to put them in control – they’re not used to it. And I think it’s especially cool in the second meeting when they arrive and have that attitude and start talking, and it feels like they... that you can give them that – that they are allowed to speak up about what they want and we can give them that. I think that’s cool, and it feels like that’s not always the case in other care I’ve administered, but it’s been: this is what you need and we’ve decided what you need. It’s fun to turn it around.” - Paired interview no: 2 with Transition Coordinators
	ACHD nurses’ responsiveness	“Nurse 2 from Center 1: Yes, just like you say – sometimes, you’d just like to have a template for what issues you’re expected to raise, because we just wing it really – and then there are some things, like with studies, and when you’re supposed to be here, how you’re feeling... I mean the usual things, you deal with those anyway. But if there is something specific – then it would have been good... Nurse 1 from Center 1 = Yes Nurse 2 from Center 2 = Yes, I agree with that” - Excerpt from focus group with ACHD nurses
Facilitation strategies	HEADDS and communication tools – facilitating the partnership	“The TC starts the next speech bubble, which is sexuality and contraception. She doesn’t say the word, just points at the bubble. The youth says she hasn’t thought about it much before. The TC tells this youth she has to get in touch with a gynecologist if she wants the pill due to increased risk of blood clots with this specific heart defect. Then the TC carries on talking about pregnancy – that it’s not an obstacle for her but that she would probably need more monitoring during a pregnancy. The youths says she has thought about this a bit and didn’t think she could have children and wonders if there’s a risk her heart defect is hereditary. The TC explains it isn’t and that she can get pregnant. The youth seems to be happy about that but explains she had thought about adopting anyway as there are so many children in the world who need a parent. The TC gives her affirmation for this and says she is free to choose what she wants to do.” - Excerpt from field note from observation no: 2, first visit in the transition program at Center 2.
Context	The impact of social context on implementation	“If you’re not allowed that freedom to think about what you want to do... they describe quite strict restrictions on leisure activities and choice of profession and so on... then you don’t reflect on your limitations either if someone else is in the driving seat. You’re not even allowed to reflect on it. It’s not until you face a problem that you reflect on the solution, if someone else is always in the driving seat.” - Informal interview with TC after participatory observation no: 6 in Center 2.

The most difficult part of goal setting was not formulating a goal, but finding the resources the adolescent felt they needed in order to reach their goal (Table 3). From the interviews with TCs it was evident this was a recurring challenge, and the observations confirmed this as a barrier, with both the adolescents and TCs struggling to grasp these aspects. As a result, goal setting was not always performed, as the TCs did not want the adolescents to lose interest or disengage. Additionally, goal setting was not always performed during the final outpatient visit because many adolescents wanted to start in adult care on a ‘clean slate’ (Table 3).

In their interviews and logbooks the TCs said that the written personal narrative helped them become acquainted with the adolescents. However, personal narrative was implemented to a low degree before the transfer to adult care, with 5.1 % of adolescents completing this narrative. Potential reasons were that the adolescent forgot to write and send in the narrative before the visit, that the adolescents felt it was awkward to write a personal narrative to someone they did not know, and that the ACHD clinic did not have any way of storing the transition plan as an electronical medical file.

Information and education about the condition and Information and contact with adult clinic

The psychosocial interview guide HEADDS was used to a high extent in the first visit, but to a lesser extent during the second and third visit (Table 3). The TCs stated that performing HEADDS during the second and third visit was sometimes repetitive. But as time went by, the TCs built on their experience and found ways to facilitate the use of HEADDS in the visits. The observations showed that the visits began with a familiarization phase between the TC and adolescent through the use of HEADDS. By covering all domains in HEADDS, the TC made the adolescents feel relaxed and talkative, which facilitated the establishment of a partnership. Hence, when the TC brought up challenging issues, such as the adolescents’ CHD and its impact on daily life, most of the adolescents felt comfortable talking about these issues (Table 4).

From the quantitative data and observations it was evident the adolescents generally received tailored information, with the first visit being more informative than the latter in terms of level of fidelity (Table 3). In order to achieve tailored information and learning, behavior change techniques such as active learning, goal setting, modeling and verbal persuasion were used to a high extent during the outpatient visits and information day (Table 4, example of active learning).

Availability by telephone and email

In between the visits, the amount of contact between the adolescent and TC was low. Of the available mediums, text message was most commonly used (Table 3). The observations showed that the TC informed the adolescents about the option to contact them between the visits if they had questions or concerns. The logbooks and interviews also revealed that the TCs perceived this as an extra important feature of the intervention for some adolescents, while for some it was unnecessary.

Guidance for parents

The implementation of this component varied. A high proportion of adolescents had a parent, caregiver or sibling present during the first visit (81.4 %) but a low proportion of parents sought extra support from the TC (6.8 %). The observations showed that parents were present during the first 10–15 minutes of the outpatient visits. During this time, the TC opened up for discussion on the transition program, addressing the shift in responsibility from the parent to the adolescent.

Meeting with peers during the information day for adolescents and their parents

During the information day, information on the transfer to adult care and the opportunity to meet others in the same situation was delivered to the participating adolescents and parents by HCPs from adult care (i.e. nurses, physicians, counselors and physiotherapists), the TC, and two young adults from the patient organization of CHD. Three information days were held throughout the course of the trial, two in center 1 and one in center 2, where a total of 32.2 % of the adolescents in the transition program participated.

Transfer to adult care

A high proportion of adolescents participated in the transfer meeting and were satisfied with the arrangements made for the transfer. However, medical referral was sent to a low extent prior to this meeting (Table 3). The qualitative data showed that a potential explanation for this was that the transfer meeting in the transition program was not always synchronized with the medical transfer in usual care.

Participant responsiveness

Moderators of adolescents’ responsiveness

Four factors were identified from the qualitative data that moderated the adolescents’ responsiveness to the transition program. The first factor was their level of maturity. The TCs struggled to engage adolescents who were perceived as immature when implementing components such as discussing health behaviors, goal setting and initiating conversations about the transfer and transition. In addition, recruitment in the later stages of the RCT involved patients that were 15 years old at recruitment. The TCs experienced that these patients were harder to engage in relation to their age. The second factor was mental health issues along with neuropsychiatric variations. As the intervention involved discussions on aspects related to everyday life, these issues were considered a barrier towards engagement (Table 4). The third factor was disease complexity and the perceived need of the transition program. The TCs found it challenging to engage adolescents with a mild CHD, as it had little impact on their daily life. In contrast, patients with a complex CHD generally had more symptoms and therefore a greater need of the intervention. However, there were adolescents with mild CHD who expressed worries regarding their CHD and transfer to adult care and therefore, for some individuals, the perceived need of the intervention was a greater influence. The fourth factor, identified by the observations, was the adolescent’s willingness to talk and engage in their CHD. When the TC directed the conversation to their CHD or the transfer to adult care, these adolescents became disengaged (Table 4). Additionally, the time between the implementation steps (i.e. outpatient visits) was sometimes considered a barrier, as one year between the visits in the transition program could be too long for some adolescents to be comfortable and find meaningful output (Table 4).

Moderators of the TCs responsiveness

Three factors were identified. First, it was essential for the TC to become accustomed to the new role. Early in the implementation of the transition program, the TCs were still finding their professional role in terms of boundaries, responsibilities and managing challenging patient situations. However, by building experience and sharing these with each other, the TCs gained coping strategies. Second, lack of knowledge sometimes gave a feeling of inadequacy. This was related to meeting patients with mental health issues and neuropsychiatric variations, for whom the TCs felt their training and role was sometimes insufficient in meeting the patients’ needs (Table 4). The third factor was the TCs, ability to shift from working strictly to the intervention protocols to tailoring the intervention to the adolescent. This entailed letting the adolescent control the conversation and find aspects important for them to work with, which had positive effects on the TCs’ experiences (Table 4).

ACHD nurses’ responsiveness

The ACHD nurses perceived the transition program as an important intervention. They valued being able to participate during the information day for adolescents and parents, as it gave them the opportunity to bring up important issues in adult care and adult life before the transfer. The transition program also gave the nurses assurance that the adolescents were prepared and informed. Nevertheless, there were challenges when it came to changing the current way of working and a need to safeguard the pre-existing transfer protocol. Center 2 had a transfer meeting between the physicians from pediatric and adult care which the ACHD nurses felt conflicted with the transfer meeting in the transition program, even though they felt the content of the two meetings differed. In addition, several shortcomings were identified in the joint transfer meeting between the TCs and nurses. Generally, the nurses felt unprepared for the meeting, lacking a structure, purpose and role. The observations showed that the nurses often expected the TC to lead the conversation. Moreover, the nurses were unprepared for which patient they were going to meet because it was not part of the standard operation procedures (SOP) for the TC to report any patient information prior to the meeting. To facilitate future implementation of this component the nurses requested more information or a protocol on which topics were to be covered (Table 4).

Moderators of responsiveness during the information day

Four moderators of the participants’ engagement during these information days were identified. First, the use of movies enhanced the group discussions. The movies dealt with experiences of transfer and transition, Q&A of common medical issues, and social, vocational and financial issues when living with a CC. The movies thus served as icebreakers and facilitated the discussion by making the adolescents comfortable in communicating with the HCPs. Second, the use of patient representatives created an informal atmosphere. The representatives candidly shared their experiences and acted as conversation starters. The third moderator was the HCPs’ level of comfort in sharing power and meeting the participants on their terms. Two examples were identified. In example one, the HCPs tailored the information to the adolescents’ needs, which facilitated the discussion. In example two, the information was medically focused with use of complex terminology. This resulted in a group of disengaged adolescents. Fourth, the use of separate group discussions facilitated a safe space. For parents, this meant they could discuss worries and fears regarding the transfer and transition with parents in the same situation. For the adolescents, they could discuss sensitive topics (e.g. sex, alcohol and substance use) without parents present.

Facilitation strategies

Standard operation procedures and protocols (SOP)

SOPs were developed to guide the TCs in their work. Part of the SOP was to perform a medical chart review and consult the physician before each visit with the adolescent in the transition program. These procedures gave the TCs important medical information that prepared them and gave an understanding of the adolescents’ and families’ history. They also ensured patient safety, as the TCs were prepared for which information they could provide the adolescent within their field of responsibility. However, there were limitations in the SOPs related to the transfer meeting where the TCs lacked structure.

HEADDS and communication tools –facilitating the partnership

The HEADDS tool was appreciated by the TC in getting to know the adolescent as it covered aspects related to their everyday life rather than focusing on the illness. HEADDS was also effective in engaging adolescents who had low responsiveness by initiating topics related to their CHD and care in a natural way from the adolescents’ point-of-view.

To help the adolescents communicate their needs during the outpatient visits, communication tools were used. First, a shared decision-making model was used with the adolescent and parents to discuss the shift of responsibility from the parent to the adolescent during the transition to adulthood. Second, a concept map containing topics related to the CHD, care and adolescent life was used during the visits. The TCs experienced this tool as effective for adolescents who were untalkative or had cognitive challenges. Observations identified that these tools facilitated a more equal conversation, as the adolescents could choose topics to discuss. In addition, discussion of sensitive subjects (e.g. sex, drugs and alcohol) were facilitated (Table 4). Third, a linear analog scale was used to assess the adolescent’s quality of life and health status. The TCs stated that this enabled screening for both problems and resources and for some adolescents this was a starting point to formulate goals.

Utilizing the team in difficult situations

The TCs experienced that meeting the adolescents sometimes revealed mental, social and family issues that were beyond the scope of their professional role. The TCs handled this by utilizing the expertise of the surrounding team at the pediatric clinic (i.e. counselors, psychologists and physicians). However, safeguarding the adolescent’s confidentiality was sometimes considered a barrier to utilizing the team as the TCs felt obliged to protect this confidentiality.

Tailoring the intervention to the adolescents’ circumstances

The final facilitation strategy was related to tailoring the components of the intervention to achieve fit. As several components were focused on information and education through PCC, the TC used several strategies to achieve implementation of these components, such as engaging the adolescents by starting from their point-of-view, being flexible towards the adolescents’ needs and using information technology (i.e. smartphone Health-application). Moreover, as the TCs grew more confident in their role, they used these strategies when meeting adolescents who were disengaged or had other difficulties in expressing their needs.

Recruitment

The recruitment for the RCT spanned from June 2016 – December 2018. Three barriers and five strategies to recruitment and retention were identified. First, lack of engagement from the adolescents was most common according to the TCs. Second, problems with the postal services resulted in missed information to presumptive participants. Third, difficulties in reaching the participants were prevalent (e.g. no phone number registered in the adolescents medical file, adolescents living with separated parents, or that the information in their medical file was insufficient to recruit).

Strategies were used to handle these barriers. First, offering an incentive (i.e., cinema ticket) was perceived as encouraging the participants to be more engaged and continue study participation. Second, in some cases, using the parents as gatekeepers and contacting them first proved effective. Third, calling and texting was convenient as the adolescents were used to this form of communication and responded more promptly. Fourth, study information was adapted to each individual’s needs after reading about the adolescent in the medical file. And fifth, coordinating recruitment and outpatient visits in the intervention with the medical check-ups was an effective way for retention, as the patient was already at the clinic.

Context

The impact of social context on implementation

Social context surrounding the adolescent was a factor that affected implementation. Here, parents were considered to be both facilitators and barriers. Both interviews with the TCs and observations confirmed that some parents had difficulties in letting their adolescent manage their own care. On the other hand, the TCs considered most parents as resources to their child in the transition program. The TCs also felt the transition program gave insight into conflicts between the adolescent and parent that had to be considered by the TC (Table 4).

The impact of organizational context on implementation

As the transition program was delivered in addition to usual care, the TCs had to coordinate this. Three challenges were identified. First, the TCs had challenges in reaching the physician and finding a time for consultation before the outpatient visits. The TCs described having to be flexible and adjusting their schedule according to the physician’s needs, which was more commonly the case in center two than center one. The second challenge was related to the physical context, where center two had a designated office for the TC but center one had difficulties in finding a private space for the visits due to shortage of rooms in the outpatient pediatric cardiology clinic. The third challenge concerned attitudes from HCPs in usual care towards the intervention. Right from the start of the trial, the TCs described the HCPs as being skeptical towards them in their role. As the TCs had to safeguard the intervention’s content to avoid possible contamination of the intervention to usual care, they felt that this affected the general attitude. However, throughout the course of the trial these attitudes gradually changed towards a more positive outlook on the intervention and TCs (Table 4).

Our results show that the components of the STEPSTONES transition program were delivered to a high extent in a manner that adhered to the program theory. In addition, important moderators specific to transition programs (e.g. level of maturity of adolescents, cooperation between adult and pediatric care) and complex interventions for people with CCs have been illuminated. From our findings we identified two components that the participants found particularly difficult to engage in and implement - the information day for adolescents and parents and the transfer meeting with adult care. Indeed, only a third of adolescents in the transition program participated during the information day. Potential reasons for this are that adolescents with a CC might feel uncomfortable in engaging with others in the same situation and spend excessive time thinking about their illness (29, 30). Within patient empowerment, the ability to enable others is a core construct (14, 15). However, empowerment entails the individual being seen as a capable person with the ability to make informed decisions concerning their health (31). Arguably, choosing not to participate in an information day can be explained by the fact that some adolescents are comfortable in their illness identity and do not feel the need to engage with others to be empowered (32, 33). Nevertheless, when designing components involving peer support in complex interventions, these results emphasize the importance of considering the diverse needs of people with CCs and working with PCC to tailor components to achieve fit.

It was difficult to achieve a high level of fidelity for the transfer meeting to adult care. Reasons identified were the lack of SOPs concerning the content of the meeting, that adult care did not work with PCC, and the difficulties integrating the transition program transfer meeting with the transfer routines in usual care. These findings highlight important barriers to overcome for future implementation of transition programs and raise the issue of how to achieve fit for complex interventions into the complex social systems which healthcare systems exemplify (34). Although complex, the STEPSTONES transition program did not involve components targeting the organizational levels within pediatric and adult care (9, 10, 13). Arguably, this could be an additional reason for the difficulties in implementing this component and implementation step of the program.

Although implementation challenges and moderators of fidelity have been identified in this study, it is worth asking whether high fidelity and its assessment is important in process evaluations of complex interventions. When comparing interventions implemented strictly according to protocol with interventions that were tailored to the local context and patient population, the tailored interventions proved to be more effective in producing change (8). Furthermore, person-centered interventions promotes active patient participation and adaptation of the intervention to the person’s preferences in a partnership between the patient and HCP (35). Are evaluations of implementation fidelity redundant in these types of complex interventions? Perhaps, as the value of assessing implementation fidelity is to fully describe which components of the intervention were delivered and what the potential pitfalls and barriers to implementation were (4–6). However, low fidelity from quantitative assessments does not always indicate poor implementation or inadequacies in the delivery of the program; instead it may indicate a high level of tailoring to the contextual and individual circumstances. Carrol et al. (2007) and Hasson’s (2010) have therefore proposed moderating factors that were explored qualitatively and shown to be important in nuancing the quantitative findings. As process evaluations tend to be extensive in the amount of collected data (4), the use of frameworks can also aid in defining the research question.

Some methodological caveats must be raised when interpreting these findings. First, the process evaluation of the STEPSTONES project covered several aspects of the implementation, context and potential mechanisms of impact of the transition program (10). Consequently, there is a risk that the process evaluation and researcher performing the evaluation affected the effectiveness evaluation by being present during several implementation steps and by performing interviews with HCPs responsible for implementing the intervention (4, 36). However, the researcher performing the data collection was not part of the analysis and interpretation of findings from the effectiveness evaluation. Second, sampling procedures aimed to capture a broad variety of perspectives by observing different outpatient visits at different points in time. However, only 17 observations were performed across a total of 167 outpatient visits, so these observations cannot provide a complete picture of the implementation of the program in practice. Nevertheless, this limitation was handled by triangulating findings from the observations with data from interviews, logbooks and the quantitative sources to ensure trustworthiness and achieve a deeper understanding of the implementation process and the moderators (11). Moreover, as the process evaluation was performed longitudinally alongside the RCT, our findings have a high level of trustworthiness. Finally, as the data collection and analysis were performed before knowing the outcomes of the effectiveness evaluation we were unable to analyze which components were essential to outcomes (5). However, this can be considered a strength as it may avoid biased interpretation of the effectiveness evaluation (4).

The STEPSTONES transition program was delivered in a way that adhered to the program theory. Identified moderators affecting the implementation process were both the adolescents’ and HCPs’ responsiveness, contextual factors in pediatric and adult care, and the lack of comprehensive SOPs for all the implementation steps of the program. As a result, barriers and facilitators for a future implementation of transition programs into usual care have been illuminated. The use of an implementation fidelity framework to design and perform a process evaluation study proved successful in providing a comprehensive evaluation of factors affecting the implementation process. However, implementation fidelity must be considered in balance with adaptations to the local and personal prerequisites to create interventions that can achieve fit.

ACHD

Adult congenital heart disease

CHD

Congenital heart disease

CC

Chronic condition

COREQ

Consolidated criteria for reporting qualitative research

GRAMMS

Good Reporting of A Mixed Methods Study

HEADDS

Home, Education, Activities, Depressions, Drugs, Sexuality

HCP

Healthcare professionals

LAS

Linear analog scale

PCC

Person-centered care

RCT

Randomized controlled trial

SOP

Standard operation procedures

SPSS

Statistical package for social sciences

STEPSTONES

Swedish Transition Effects Projects Supporting Teenagers with Chronic Medical ConditionS

TC

Transition coordinator

Ethics approval and consent to participate

Ethical approval was received from the Regional ethical review board in Gothenburg, Sweden (No. 931-15). The study was performed according to the Declaration of Helsinki (37). Written informed consent was retrieved from all participating adolescents and parents in the RCT after they had received written and verbal information about the study. Participating adolescents in the qualitative data collection received additional written and verbal information about the qualitive study, along with retrieval of written informed consent. Participating HCPs and patient representatives gave verbal informed consent after receiving information about the study from the first author (MS) responsible for the data collection.

Authors’ contributions

MS, LW, PM, HG and ELB planned the study and were responsible for the design and draft of the manuscript. MS was responsible for data collection, analysis and first draft of the manuscript. MS and ELB performed the analyses. All authors critically revised and approved the manuscript before submission.

Acknowledgements

The authors would like to thank Kamila Bojanowska for helping with the data collection for the adolescent day and the international panel of experts and advisory board of the STEPSTONES project.

Availability of data and materials

All data are available from the authors upon request.

Competing interests

The authors declare that they have no competing interests.

Consent for publication

Not applicable for this publication.

Funding

The work presented in this paper was supported by research grants from the Swedish Heart-Lung Foundation (grant 20150535); Swedish Research Council for Health, Working Life and Welfare-FORTE (grant STYA-2015/0003); Swedish Children’s Heart Association; Swedish Research Council (grant 2015–02503); and the Institute of Health and Care Sciences, University of Gothenburg. However, the funding bodies have not participated in study design, collection and analysis of data or writing the manuscript.

1. Campbell F, Biggs K, Aldiss SK, O'Neill PM, Clowes M, McDonagh J, et al. Transition of care for adolescents from paediatric services to adult health services. Cochrane Database Syst Rev. 2016;4.

2. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. Int J Nurs Stud. 2013;50(5):587 − 92.

3. Le Roux E, Mellerio H, Guilmin-Crepon S, Gottot S, Jacquin P, Boulkedid R, et al. Methodology used in comparative studies assessing programmes of transition from paediatrics to adult care programmes: a systematic review. BMJ Open. 2017;7(1):e012338.

4. Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions: Medical Research Council guidance. Bmj. 2015;350:h1258.

5. Carroll C, Patterson M, Wood S, Booth A, Rick J, Balain S. A conceptual framework for implementation fidelity. Implement Sci. 2007;2:40.

6. Hasson H. Systematic evaluation of implementation fidelity of complex interventions in health and social care. Implement Sci. 2010;5(1).

7. Bragstad LK, Bronken BA, Sveen U, Hjelle EG, Kitzmuller G, Martinsen R, et al. Implementation fidelity in a complex intervention promoting psychosocial well-being following stroke: an explanatory sequential mixed methods study. BMC Med Res Methodol. 2019;19(1):59.

8. Sundell K, Beelmann A, Hasson H, von Thiele Schwarz U. Novel programs, international adoptions, or contextual adaptations? Meta-analytical results from German and Swedish intervention research. J Clin Child Adolesc Psychol. 2016;45(6):784 − 96.

9. Acuna Mora M, Sparud-Lundin C, Bratt EL, Moons P. Person-centred transition programme to empower adolescents with congenital heart disease in the transition to adulthood: a study protocol for a hybrid randomised controlled trial (STEPSTONES project). BMJ Open. 2017;7(4):e014593.

10. Saarijärvi M, Wallin L, Moons P, Gyllensten H, Bratt E-L. Transition program for adolescents with congenital heart disease in transition to adulthood: protocol for a mixed-method process evaluation study (the STEPSTONES project). BMJ open. 2019;9(8):e028229.

11. Creswell JW, Plano Clark V. Designing and conducting mixed methods research. 3rd ed. Los Angeles: Lose Angeles: SAGE publications; 2017.

12. Bartholomew LK MC, Ruiter R, et al. Planning health promotion programs: an intervention mapping approach. 4th ed. San Francisco: Jossey-Bass; 2016.

13. Acuna Mora M, Saarijarvi M, Sparud-Lundin C, Moons P, Bratt EL. Empowering young persons with congenital heart disease: Using intervention mapping to develop a transition program - the STEPSTONES project. J Pediatr Nurs. 2020;Jan 1;50:e8-17.

14. Acuna Mora M, Luyckx K, Sparud-Lundin C, Peeters M, van Staa A, Sattoe J, et al. Patient empowerment in young persons with chronic conditions: Psychometric properties of the Gothenburg Young Persons Empowerment Scale (GYPES). PLoS One. 2018;13(7):e0201007.

15. Small N, Bower P, Chew-Graham CA, Whalley D, Protheroe J. Patient empowerment in long-term conditions: development and preliminary testing of a new measure. BMC Health Serv Res. 2013;13(1):263.

16. Bratt E, Burström Å, Hanseus K, Rydberg A, Berghammer M. Do not forget the parents—Parents' concerns during transition to adult care for adolescents with congenital heart disease. Child: Care Health Dev. 2018 Mar;44(2):278 − 84.

17. Burström Å, Bratt E-L, Frenckner B, Nisell M, Hanséus K, Rydberg A, et al. Adolescents with congenital heart disease: their opinions about the preparation for transfer to adult care. Eur J Pediatr. 2017:1–9.

18. Burström A, Ojmyr-Joelsson M, Bratt EL, Lundell B, Nisell M. Adolescents With Congenital Heart Disease and Their Parents: Needs Before Transfer to Adult Care. J Cardiovasc Nurs. 2016;31(5):399–404.

19. Asp A, Bratt EL, Bramhagen AC. Transfer to Adult Care–Experiences of Young Adults with Congenital Heart Disease. J Pediatr Nurs. 2015;30(5):e3-10.

20. Mihalic S. The importance of implementation fidelity. Emotional and Behavioral Disorders in Youth. 2004;4(4):83–105.

21. Dusenbury L, Brannigan R, Falco M, Hansen WB. A review of research on fidelity of implementation: implications for drug abuse prevention in school settings. Health Educ Res. 2003;18(2):237 − 56.

22. Teddlie C, Yu F. Mixed methods sampling: A typology with examples. J Mixed Methods Res. 2007;1(1):77–100.

23. Wilson AD, Onwuegbuzie AJ, Manning LP. Using paired depth interviews to collect qualitative data. Qualitative Report. 2016;21(9):1549-73.

24. Palinkas LA, Horwitz SM, Green CA, Wisdom JP, Duan N, Hoagwood K. Purposeful Sampling for Qualitative Data Collection and Analysis in Mixed Method Implementation Research. Adm Policy Ment Health. 2015;42(5):533 − 44.

25. M. Roper J, Shapira J. Ethnography in Nursing Research. London: SAGE Publications Ltd; 2000.

26. Saarijärvi M, Bratt EL. When face-to-face interviews are not possible: tips and tricks for video, telephone, online chat, and email interviews in qualitative research. Eur J Cardiovasc Nurs. 2021;20(4):392-6.

27. Kyngäs H, Kaakinen P. Deductive Content Analysis. The Application of Content Analysis in Nursing Science Research. Springer; Cham; 2020. 23–30.

28. Dickson VV, Page SD. Using mixed methods in cardiovascular nursing research: Answering the why, the how, and the what's next. Eur J Cardiovasc Nurs. 2021;20(1):82 − 9.

29. Saarijärvi M, Wallin L, Moons P, Gyllensten H, Bratt E-L. Factors affecting adolescents’ participation in randomized controlled trials evaluating the effectiveness of healthcare interventions: the case of the STEPSTONES project. BMC Med Res Methodol. 2020;20(1):205.

30. Sheridan R, Martin-Kerry J, Hudson J, Parker A, Bower P, Knapp PJT. Why do patients take part in research? Overview Syst Rev Psychosoc Barriers Facilit. 2020;21(1):1–18.

31. Aujoulat I, d'Hoore W, Deccache A. Patient empowerment in theory and practice: polysemy or cacophony? Patient Educ Couns. 2007;66(1):13–20.

32. Castro EM, Van Regenmortel T, Vanhaecht K, Sermeus W, Van Hecke A. Patient empowerment, patient participation and patient-centeredness in hospital care: A concept analysis based on a literature review. Patient Educ Couns. 2016;99(12):1923-39.

33. Saarijärvi M, Wallin L, Moons P, Gyllensten H, Bratt EL. Mechanisms of impact and experiences of a person-centred transition programme for adolescents with CHD: the Stepstones project. BMC Health Serv Res. 2021;21(1):573.

34. Moore GF, Evans RE, Hawkins J, Littlecott H, Melendez-Torres GJ, Bonell C, et al. From complex social interventions to interventions in complex social systems: Future directions and unresolved questions for intervention development and evaluation. Evaluation. 2019;25(1):23–45.

35. Ekman I, Swedberg K, Taft C, Lindseth A, Norberg A, Brink E, et al. Person-centered care—Ready for prime time. Eur J Cardiovasc Nurs. 2011;10(4):248 − 51.

36. Landsberger HA. Hawthorne Revisited: Management and the Worker, Its Critics, and Developments in Human Relations in Industry. 1958.

37. World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects. JAMA. 2013;310(20):219

Due to technical limitations, Table 3 is only available as a download in the supplemental files section

Implementation fidelity of a transition program for adolescents with congenital heart disease: the STEPSTONES project

Status:

Journal Publication

Version 1

Abstract

Background

Methods

Results

Conclusion

Figures

Background

Methods

The STEPSTONES transition program

Methodological framework

Study sample and data collection

Data analysis

Results

Adherence and moderating factors on implementation fidelity

Discussion

Conclusion

Abbreviations

Declarations

References

Table 3

Supplementary Files

Status:

Journal Publication

Version 1