Characteristics of implementation sites
Of 36 organisations initially expressing interest, two local authorities subsequently reported changes in priorities, two home care providers were insufficiently resourced, and 13 NHS organisations were seeking funded involvement in research studies. Nineteen organisations engaged in detailed discussion. Five NHS organisations did not proceed due to perceived insufficiency of resources and staffing. Therefore, 14 organisations developed and agreed an implementation plan. These were 10 NHS organisations, three home care businesses and one local authority. Following staff training, one of the home care businesses failed to implement the intervention, and another withdrew due to an unexpected change in the entire senior management team. In addition, the legal department of the local authority objected to the wording of study documents required by the Health Research Authority (HRA), making it impossible to proceed; the HRA, through ethical review, protects and promotes the interests of people involved in health and social care research in England. This left 11 organisations, 10 NHS services and one home care business, that participated in the implementation and evaluation based on agreed implementation plans. Within the NHS organisations, the services involved were multi-disciplinary community teams providing secondary care. Most said they provided some form of post-diagnostic support, although in some cases this was acknowledged to be limited. All expressed an intention to increase the range of post-diagnostic support provided. Four of the organisations participated in Wave 1 and seven in Wave 2. During Wave 2, the implementation was disrupted by the COVID-19 pandemic. See Figure 1 for a flowchart summarising identification and engagement of partner organisations.
(((Figure 1 near here)))
Ninety-four staff in the 11 organisations received the two-day foundation-level training, and 88 provided immediate feedback, which was uniformly positive with some constructive suggestions for enhancement. The majority of trainees were occupational therapists (OTs; n=54), OT assistants or OT technicians (n=7). Other staff groups trained were clinical psychologists (n=7), assistant psychologists (n=3), nurses (n=6) and support workers or healthcare assistants (n=14); the professional background of three practitioners was not recorded. A small number of managers and supervisors, who were not expecting to deliver the intervention but wished to understand it in order to support their practitioners, additionally attended somee training sessions. Forty-one practitioners went on to provide the intervention.
Characteristics of service users receiving the intervention
Eighty-four people with dementia agreed to receive the intervention. Twenty-one did not proceed; reasons were not available in all cases, but included death, illness, ineligible diagnosis, being unhappy about a previous intervention, and cessation of services due to the COVID-19 pandemic. Sixty-three people with dementia had an initial therapy goal recorded. Nine did not complete the course of CR; reasons, available for 6, were illness, carer illness, lack of support from carer, lack of motivation, perceived unsuitability, and too much paperwork. Fifty-four people (average age 76.02, 55% male) completed a course of GREAT CR, typically consisting of six hour-long sessions. Those who completed a course of CR were mostly white British, and nearly one-third had no formal educational qualifications. The most common diagnosis was Alzheimer’s disease. There were 39 carers who contributed; they were predominantly White British, with an average age of 66 years and a broad range of educational backgrounds. They were mostly female, and either spouses (58.6%) or children (31.0%) of, and mostly (72%) co-resident with, the person with dementia. Characteristics of the people with dementia and carers are summarised in Additional Table 1.
Implementation outcomes
Of the 11 organisations, nine set targets for the number of practitioners providing the intervention (71 in total, an average of seven per organisation, range 3-14) and 10 set targets for the number of people with dementia receiving the intervention (266 in total, an average of 26 per organisation, range 12-44). Other organisation-specific targets included increasing the focus on providing psychosocial interventions [12], or demonstrating the potential of such interventions to commissioners. Achievements in relation to targets are summarised in Table 2. The number of practitioners attending foundation-level training was 94, but only 41 subsequently provided CR. The number of people with dementia who received CR (indexed by having a therapy goal recorded) was 63. The discrepancy between target and actual numbers was partly attributable to the impact of the COVID-19 pandemic. Courses of CR were reportedly provided outside the evaluation for some people with dementia unwilling to participate in the research element of the project, and some practitioners reported incorporating elements of CR into other work.
(((Table 2 near here)))
For Wave 1 organisations, training took place between July and September 2018. CR delivery started following training and continued for up to 59 weeks, completing between August and December 2019. In one organisation (#5) there was a six-month delay between training and the start of CR delivery due to major staffing changes, and adaptations were made by the study team to accommodate this. The Wave 2 organisations received training between September and November 2019 and were expected to continue therapy provision for up to 52 weeks, but due to the COVID-19 pandemic, all had to cease intervention delivery in early March 2020. Two (#9 and #15) were able to resume provision in November 2020 and continue until March 2021. For the three Wave 1 organisations that had the full 12-month delivery period, achievement of targets ranged from 60-100% for practitioners providing, and from 47-83% for people receiving, the intervention. For the three organisations that had approximately half of the expected period, achievement ranged from 36%-57% for number of practitioners providing, and 19-27% for number of people receiving, the intervention. The five organisations whose intervention delivery period was 18 weeks or less had limited opportunity to demonstrate outcomes in relation to targets; however, three each managed to deliver one course of CR.
Due to the COVID-19 pandemic, we were unable to conduct mid-way reviews with Wave 2 organisations or a follow-up exploration of sustainability. However, nine practitioners completed all elements of the foundation-level training and attended advanced-level training. Four subsequently completed all the advanced-level requirements. Over 30 practitioners participated in the inaugural community of practice meeting; following this the group was set up on LinkedIn and 17 practitioners joined.
Implementation processes
Information about implementation processes was gathered from records of mid-way meetings and from the survey responses and interviews with steering group members and practitioners, and considered at organisational and practitioner levels.
Organisational level
A senior staff perspective on the implementation was explored in mid-way review meetings and through telephone interviews with six steering group members from five organisations conducted after the end of the intervention delivery period; five interviewees were senior clinicians with leadership roles and one was a research manager. They represented organisations with varying degrees of success in achieving implementation targets (#4, #8, #9, #11 and #14; see Table 2). All described positive impacts for their organisations, while acknowledging that the outcomes could have been better. They thought CR encouraged a more person-centred and creative approach in planning and delivering interventions, leading to better outcomes for people with dementia and carers, and improved morale and confidence among staff:
“We had some discussion…just saying how much we’d learnt by actually delivering the intervention … and maybe our confidence …is better than it was….It had some …additional benefits…in terms of our team cohesion and our purpose in the roles we were in.”
“[Carer] said how beneficial it had been, because you get given a diagnosis, and then you're led to believe there is absolutely nothing you can do...Other than expect a decline in that person. And for her, it was important that she could see that there was lots of things that her husband could still do and achieve.”
However, it took considerable effort to develop a sense of ownership among senior decision-makers:
“It took some time to get senior managers interested in or to make them aware of this approach that we wanted to put to them as a viable option for people with dementia after the diagnosis….”
For some organisations service priorities specified by health commissioners, such as focusing on diagnosis and crisis support, meant it was hard to integrate cognitive rehabilitation:
“Our commissioners’ key performance indicators… are still about diagnostic rates.”
Nevertheless, some interviewees perceived indications that the implementation was beginning to translate into system and process changes, for example through inclusion in research and audit discussions. All expressed enthusiasm to continue providing CR and some had made arrangements for delivery to continue while attempts were made to secure support and resources for long-term sustainability. The following quote summarises both the perceived benefits and the challenges:
“I think cognitive rehabilitation would be a very good investment in preventing deterioration in independence and functionality that will mean more care from the social side of it to go into the future. So if you invest in a particular individual to keep themselves independent and to plan and be a step ahead of the deterioration, that will mean less care needed or less firefighting and crisis and all these services that are very expensive, but it’s an investment that we have to do at the beginning and it will cost to train the professionals and it will cost to hire the professionals to develop a system, a programme and to deliver those interventions. So it might be quite expensive at the beginning, but in the long run it’s going to save lots of money.”
Practitioner level
Training and supporting practitioners to deliver CR was a core element of the implementation strategy. The practitioner perspective was explored through the online survey and follow-up interviews. Twenty-four practitioners completed the survey. Their characteristics are summarised in Additional Table 2; they were mostly white females with a background in occupational therapy. Responses to the closed questions are shown in Additional Figure 1, and the analysis of open-ended responses, with example quotes, is summarised in Additional Table 3. Practitioners felt the training prepared them well, felt well-supported in providing the intervention, and found the skills they gained useful. The majority thought the intervention was beneficial for people with dementia, and would readily recommend it:
“This approach should be offered to all those people with dementia that are suitable as it's a powerful restorative experience for them.”
However, fewer than half believed their organisation could sustain delivery; the remainder were either unsure, or thought this unlikely:
“Greater commitment should, perhaps, be sought from the practitioner's organisation about scope and willingness to embed this approach in their service.”
We intended to interview practitioners in partner organisations with a range of implementation success, but in practice we interviewed practitioners from all four Wave 1 organisations, seven in total. We were not able to interview practitioners in Wave 2 organisations due to COVID-related disruption. Most practitioners thought involvement had allowed them to gain skills and use their expertise to prevent problems and improve quality of life rather than managing crises; for some, this contrasted markedly with the focus of their usual work:
“It’s not seen as high priority as things like carers going into complete crisis, social breakdown… I know as a service the idea is that we are looking at people’s well-being, but sometimes I don’t know if I really feel that in my day to day job.”
Practitioners emphasised the importance of having sufficient time to work with service users and gain confidence in delivering a high-quality personalised intervention. A summary of the qualitative data, with example quotes, is provided in Additional Table 4.
Barriers and facilitators
Information about barriers and facilitators impacting on the implementation, and fidelity to the implementation strategy, was gathered from records of mid-way meetings and from the survey responses and interviews with steering group members and practitioners. The information was grouped under three levels: organisational (external and internal influences), practitioner-related, and intervention-related.
Organisational
The majority of barriers were organisational. The primary external influence was local health service commissioning, which did not always support provision of psychosocial interventions.
Within organisations, contextual factors related to culture, structure, organisational change and resources. In organisations where nihilistic views about dementia were prevalent or there was strong adherence to a medical model, it was difficult to secure resources and referrals for psychosocial interventions. Across all organisations, dementia care focused primarily on complex needs and responding to crises, and preventive or rehabilitative work was not part of practitioner remits. In addition, there were high levels of staff absence and job vacancies, and frequent reports of organisational change disrupting referral pathways:
“There was good motivation in our group of practitioners, but something we couldn’t have predicted is that things changed in our service.”
“We had to have a number of discussions about whether or not after the [project] finished whether or not the [NHS organisation] would be left with any liability or responsibility to continue to deliver the interventions because we’d established some sort of precedent and expectation… It was crazy, from our point of view, because it was an intervention that was in the NICE guidelines…but I think it reflects…that our services are currently only commissioned or mainly commissioned to provide a very narrow service based on diagnosis and medical and medication follow-up.”
“We did quite a lot of education in our own teams and for anyone who was sort of potentially in a referrer position…We had some colleagues who just weren’t interested.”
These contextual characteristics meant that, contrary to the agreed implementation plans, practitioners were required to deliver CR on top of their usual caseload, and hence had limited time available:
“My colleagues were highly motivated but then swamped by their own work… Although the managers accepted they needed the time to do the work and they agreed to it, my colleagues’ time wasn’t protected. There was a disconnect between the managers and what was actually happening in the team.”
Implementation was also hindered where senior managers did not provide sufficient leadership, or where there was a lack of readiness to take ownership:
“They have asked us to come up with a business case for CR… I have been left with this to take forward, and the [NHS organisation] hasn't owned it yet in a sense.”
These barriers made it difficult to embed this work within existing structures and to identify people who might benefit, despite extensive problem-solving efforts. The main facilitators were engagement of motivated managers and committed staff champions who ensured that CR was kept on the agenda in their organisations and informed managers and colleagues about it, for example by giving presentations at meetings and using case examples:
“I felt very supported by the immediate service managers, who absolutely recognised that this was an intervention that they wanted to be available to people living with dementia.”
This helped to ensure support and stimulate referrals. Careful planning and preparation were valuable at all stages and promoted sustainability:
“It was also helpful that we put some time in to setting up our own internal systems… that allowed other practitioners to refer to us… As we’re actually resuming now to deliver CR anyway outside of [the project], having that system set up is really, really helpful.”
Practitioner-related
Reflecting the organisational barriers outlined above, lack of time was the greatest barrier for practitioners. This made it difficult to get to know the person with dementia sufficiently to deliver a personalised intervention. For some, it also led to low attendance at supervision sessions provided by the external facilitator. The other main barrier for practitioners related to self-efficacy:
“There was some confidence issues I would say, with the staff that were trained and actually using it. I don't know if that held people back from actually delivering it.”
Practitioners needed to develop confidence through experience of delivering CR, and then access further training to enhance their skills and capacity to support others, but getting started could be difficult, and was especially challenging if there was a gap between attending training and first use of the approach. This concern was addressed in Wave 2 by asking practitioners to identify potential recruits prior to training; this proved effective in enabling CR delivery to start immediately. Practitioners found the evaluation component burdensome due to the additional record-keeping involved.
The main facilitators for practitioners were acquiring skills and confidence through delivering the intervention and participating in supervision, teaming up with another practitioner or small group for peer support, and being able to involve assistants or technicians in providing some of the sessions:
“Hopefully, we are providing a better service with the skills that we have acquired”
Intervention-related
The intervention itself presented few barriers. Since it was, as yet, unfamiliar to service users, it was suggested that increasing awareness among people with dementia and carers would be helpful as they could advocate for its provision.
Facilitators were the adaptability of the approach, which provided the flexibility to accommodate individual preferences and take account of the wider situation of both the person with dementia and the carer, and the legitimacy resulting from the perceived strength and quality of the evidence-base.
Fidelity to the implementation strategy
Both sets of interviews suggested that fidelity to the implementation strategy was good in many respects, aided by the collaborative nature of the planning process and by the research team allowing flexibility to make adaptations. Despite expressed intentions, however, managerial leadership for the initiative was sometimes perceived as insufficient, and in some organisations failure to allow practitioners sufficient time meant they were unable to benefit fully from the supervision and support provided; external facilitators noted low attendance at supervision in some organisations despite offering as much flexibility as possible in scheduling sessions. The intention was for organisations to start taking responsibility for supporting practitioners in the later stages of the 12-month implementation process, but this proved difficult to achieve.
Intervention outcomes
All participants with dementia for whom information was available worked on one goal, and 15 (28%) worked on a second. The average increase in ratings of goal attainment on the BGSI-S 0 -10 scale was 4.20 points for people with dementia, 4.37 for carers and 4.80 for CR practitioners; this degree of improvement was greater than in the GREAT trial where the corresponding indicators were 2.89, 3.17 and 3.61 points respectively [7]. Almost all participants made some progress toward achieving their first goal, and more than half fully achieved it. Goals mainly related to managing everyday activities, tasks and situations or to using appliances, devices and the internet. Intervention outcomes and details of the targeted goals are summarised in Table 3. No harms or unintended effects were identified. Fidelity to delivering the core components of the intervention was examined through review of identified goals and rehabilitative strategies applied; the goals addressed were consistent with trial data and study guidance, and therapists employed the expected range of rehabilitative strategies. Case reports submitted for completion of foundation-level and advanced training requirements demonstrated good fidelity.
Survey responses (see Table 3) were received from 41 of the people with dementia who received a course of CR, and 35 carers; 93% of people with dementia and 100% of carers found CR very or somewhat useful, and almost all said they would recommend it to others. Responses to open-ended questions are summarised in Additional Table 5. People with dementia indicated that CR supported their everyday functioning and increased confidence and independence:
“I have become more independent and will now go out on my own regularly. I have attempted volunteering in a charity shop and will now get shopping on my own.”
Carers appreciated the opportunity to learn strategies for supporting their relative and saw how achieving a specific goal could lead to wider benefits:
“[Mother] is much more confident in planning daily tasks and finding information. This has taken pressure off me as her first point of contact.”
(((Table 3 near here)))
Practitioners’ survey and interview responses indicated that they thought the intervention both achieved specific goals and promoted wider benefits such as restoring confidence, facilitating independence, instilling hope, and reducing carer burden, while acknowledging that for some people complex needs and circumstances, especially physical ill-health, could impact on effectiveness. They attributed benefits to the person-centred and individualised nature of the intervention, the effectiveness of the techniques, the development of self-management capability, and positive engagement of carers:
“The person-centred approach means that it can make a big difference to daily life, as the goals are meaningful to the person.”
Evaluation of costs
We calculated the per-person cost of NHS OTs delivering a six-session course of GREAT CR under our implementation strategy. We modelled four scenarios involving different combinations of input by qualified OTs and OT support workers. As the time and costs of travel to conduct home visits are likely to differ locally, we present results both including and excluding travel costs. The scenarios, assumptions, calculation of unit costs and cost estimates are summarised in Table 4. Excluding travel time, the total cost of six sessions provided by a qualified OT was £349, which reduced to £239 if the first and last sessions were provided jointly with an OT assistant who conducted the intervening four sessions.
(((Table 4 near here)))
Further developments
In the later stages of the project, we undertook further development work to support sustainability and scaling-up. To promote awareness of the intervention we commissioned a short explanatory animation suitable for diverse audiences [28]. To make the approach accessible to people with dementia, we worked with a group of people living with dementia to co-produce a self-management guide based on CR principles which could be used either as a stand-alone tool or following a course of GREAT CR. The resulting resource, My Life, My Goals, can be accessed and downloaded via the Living with Dementia Toolkit [29] or the Alzheimer’s Society website [30].
The availability of sufficient practitioners with the skills and experience to provide GREAT CR is essential to future implementation. We developed an online version of the foundation-level training and trained 31 more practitioners. These were 23 additional practitioners from the partner organisations, and eight practitioners from other organisations. Five of the 31 practitioners completed all stages of the foundation-level training and one progressed to advanced-level training. We adapted the foundation-level training course into an e-learning format, incorporating educational videos prepared in collaboration with NHS Education for Scotland, and made this available to NHS practitioners via the NHS Learning Hub. We also explored prospects for sustaining the community of practice. As the majority of practitioners were OTs, we developed a dedicated support group within Royal College of Occupational Therapy structures.