The protocol for our process evaluation follows a systematic approach for the design and conduct of the evaluation [18].
Planning the process evaluation
Working with intervention developers and implementers
A process evaluation requires a degree of independence to appraise the intervention team’s delivery of the trial [18]. Thus, we delegated oversight of the evaluation to an expert not involved in the day-to-day conduct of the RCT but on the wider SENSE-Cog team (CA). Regular process evaluation reports will be made at Trial Steering Committee meetings (see [16]), and specific meetings with the study chief investigator (IL), the methodologist (EF) and the process lead (CA).
Overlap of the process and outcomes, and cost-effectiveness evaluations
Due to the complexity of the study, we have embedded the process evaluation within the daily conduct of the outcomes and cost-effectiveness evaluations. Thus, data for all three purposes are being collected concurrently, and some measures may be used for both process and outcomes’ evaluations. During team training, emphasized the multi-purposing of data and the need to maintain ‘researcher equipoise’.
Description of the intervention
We have previously detailed the SI components and implementation [12]. In summary, the SI, which is delivered as 10-session programme by a trained ‘sensory support therapist’ (SST) and audiologists/optometrists, is outlined in Figure 1. It involves several components, notably: (1) Identifying and correcting any vision or hearing impairment; (2) supporting adherence with the hearing and/or vision devices, through advice and training in correct use and care; (3) enhancing communication between the PwD and their companion; (4) demonstrating environmental aids and sensory devices; and (5) accessing relevant support services and social networks. The non-intervention group receives ‘care as usual’ (CAU) accesses the services and interventions normally available to PwD and their companions in their respective countries and sites.
Causal assumptions about how change will be produced
To understand fully the impact of the intervention on the outcomes, and to generalise the findings in the pan-European context of SENSE-Cog, an exploration of causal mechanisms and unanticipated pathways is needed [19]. We will do this by extracting pre-specified mediating variables and qualitative data (from a sub-sample of 30 participant dyads), and evaluating adverse events and unexpected consequences of the intervention. We will also undertake semi-structured interviews with the SSTs at each of the five study sites.
The principal aim of the intervention is to improve quality of life and functional ability by improving sensory function through devices and behavioural change. Dementia-related quality of life comprises the domains of daily activities (activities of daily living and self-care), physical health and wellbeing, cognitive functioning and social relationships [22]. To influence these domains, we have adopted the COM-B component of the Behaviour Change Wheel [23] as our framework for how the intervention might work. According to this model, behavioural change (‘B’) results from: capability (’C’), the individual’s psychological and physical capacity to engage in the activity concerned; opportunity (‘O’), the external factors that support behavioural change; and motivation (‘M’), the conscious and sub-conscious processes that direct decision making [23]. In Table 1 and Figure 1, using a logic model, we outline how each COM-B element aligns with specific components of the SI, and how these might hypothetically lead to improvements in quality of life, our primary outcome.
Briefly, for capability, the sensory aids (i.e. hearing aids, glasses, and sensory environment modification in the home) and SST adherence support will improve hearing and vision (physical capability), which will enhance sensory-cognitive function, overall functional ability, and reduce neuropsychiatric symptoms [24-28]. The SST will train communication skills and improve knowledge of dementia and sensory impairment (psychological capability). This will increase opportunity by decreasing dependency on companions, enhancing social interactions and reducing loneliness. Adherence support for PwD with sensory devices will enhance hearing [29], vision [30], or both [15]. Furthermore, social opportunities will be enhanced through signposting outside the home, thus addressing social isolation, improving social relationships, and providing respite for companions. Regarding motivation, higher sensory-cognitive function and improved neuropsychiatric symptoms will improve self-efficacy, self-esteem, and mental wellbeing [31, 32]. However, motivation may be reduced in dementia, particularly if apathy is present. Thus, the SI will address this through goal setting. We will measure apathy to take account of motivation as a potential moderator, or even mediator, of the intervention’s impact. Greater independence and communication ability in the PwD will reduce companions’ burden and stress, which will also impact positively the overall wellbeing. Attitudes and knowledge training will support change maintenance and relationship quality [33].
Table 1 Theoretical basis and logic model for how the Sensory Intervention may impact the person with dementia
Theoretical COM-B domain
|
Component of the Sensory Intervention
|
Behaviour change (B) due to the intervention
|
Immediate impact of the behaviour change
(2o outcomes of interest)
|
Intermediate
impact of the behaviour change
(2o outcomes of interest)
|
1o outcome of interest
|
Capability (C)
(the individual’s psychological and physical capacity to engage in the activity concerned)
|
Correct visual / auditory impairment
|
• Hear and see better (HHIE-25;VA-LVVFQ-20)
• Uptake and adherence improves
(HASK)
• Greater understanding /insight of impairments
• Dyadic interaction improves (RSS)
• Enhanced cognitive stimulation
|
• Enhanced cognitive functioning (MoCA)
• Reduced neuropsychiatric symptoms (NPI-12)
• Less loneliness in dyad
• Enhanced social interactions
• Greater independence for person with dementia
• Greater companion respite
|
• Enhanced self-efficacy
• Improved communication
• Greater functional ability (BADLS)
• Reduced apathy (NPI-12)
• Relationship satisfaction is higher (RSS) in both person with dementia and companion
• Companions health and well-being (SF-12; GHQ-12; FCS; HADS)
|
• Quality of life in PwD
(DEMQoL)
|
Training in correct use of hearing aid /glasses with dyad
|
Communication Training with dyad
|
Home-based functional assessment and tailored interventions to address support care needs enabled
|
Opportunity (O)
(external factors which impact on the individual’s ability to ‘live well’ with dementia)
|
Provision of supplementary sensory devices
|
• Hear and see better (HHIE-25;VA-LVVFQ-20)
• Appropriate levels of support from companion
|
Referral to health and social care services
|
• Uptake of health and social care services
• Uptake of social/hobby/ interest opportunities
• Enhanced cognitive stimulation
• Achievement of set goals
|
Referral to social/hobby/interest activities
|
Motivation (M)
(In dementia, apathy and loss of motivation is prevalent and may make behavioural change difficult)
|
Individualised goal setting
|
Referral to social/hobby/ interest activities
|
• Enhanced uptake and adherence of sensory aids and suggested social opportunities
• Uptake of meaningful and enjoyable social/hobby/interest activities
|
Key: BADLs: Bristol Activities of Daily Living Scale [34]; DEMQoL: Dementia Quality of Life [35]; FCS: Family Caregiving Role Scale [36]; HADS: Hospital Anxiety and Depression Scale [37]; HASK: Hearing Aid Skills and Knowledge Test [38]; HHIE-25: Hearing Handicap Inventory for the Elderly [39]; GHQ-12: General Health Questionnaire [40]; MoCA: Montreal Cognitive Assessment [41]; NPI-12: Neuropsychiatric Inventory [42]; RSS: Relationship Satisfaction Scale [43]; SF-12: Short Form Health Survey [44]; VA LV‐VFQ-20: Veterans Affairs Low vision Visual Functioning Questionnaire [45].
Identification of key uncertainties, and developing a framework for the process evaluation
In Table 2, we identified key uncertainties to address for each SI. We will use a mixed method approach to capture the data for the evaluation. This will involve a variety of instruments, as outlined in Table 3.
Table 2 Identifying key uncertainties to address in the process evaluation for each component of the sensory support intervention
Sensory support intervention component
|
Key uncertainty to address
|
Correct visual / auditory impairment
|
Is vision/hearing impairment actually corrected?
|
Training in correct use of hearing aid / glasses
|
How well are the devices used by the PwD?
|
Communication Training
|
Is the companion utilising the techniques?
|
Home-based functional assessment
|
What are the types and extent of functional impairment identified by the Sensory Support Therapist?
|
Referral to health and social care services
|
What types of services are available at each site?
How many referrals were made and actioned?
|
Provision of supplementary sensory devices in the home environment
|
How many of these were supplied and how often and effectively were they used?
|
Referral to social/hobby/interest activities
|
How many referrals were made and actioned?
|
Individualised goal setting
|
What is the number and types of goals set with the participant?
How many goals were achieved?
What is the level of apathy present (ascertained only after final assessment data collected?
|
Table 3 Measurement instruments used to collect quantitative and qualitative data to inform the process evaluation.
Measurement tool
|
Data captured by whom? About whom / what?
|
Timing of data capture
|
Process evaluation (Capability ´C´, Opportunity ´O´, Motivation ´M´) and measurement instruments (details)
|
1a)
|
Sociodemographic information
|
Researcher about dyad
|
Screening
|
To assess characteristics of companion and participant - (Covariates of age, gender, living conditions influencing outcomes).
|
1b)
|
Medical History
|
Researcher about dyad
|
Screening
|
Dementia sub-type and medication may influence uptake and effectiveness of intervention - (Eliciting the type of memory impairment and the current medication).
|
1c)
|
HearCheck
PEEK acuity, visual field confrontation test
|
Researcher about PwD
|
Screening
|
To establish the baseline for ‘C’ and ‘O’ for intervention/optimisation of function - (Screening for severity of visual and hearing loss of participant).
|
1d)
|
MoCA
|
Researcher about PwD
|
Screening
|
To ascertain impact of sensory optimisation on cognitive ability as intermediate step (‘C’, and ‘M’,) leading to improved QoL - (Assessing cognitive level of participant scale (score ≥10) also decides, whether the participant is appropriate candidate for the study).
|
1e)
|
DEMQoL
|
Researcher about PwD
|
Baseline
|
The 10 outcome of the behavioural change process, resulting from intermediate impacts - (Assessing quality of life of PwD by addressing all parts of the process evaluation: ´C´, ´O´, ´M´).
|
1f)
|
BADLs
|
Companion about PwD
|
Baseline
|
To ascertain impact of sensory optimisation on functional ability as intermediate step (‘C’, and ‘M’,) leading to improved QoL - (Assessing functional ability of PwD).
|
1g)
|
VA LV-VFQ-20
|
Researcher about PwD and companion
|
Baseline
|
Improvement in sensory input affects ‘C’, ‘O’ and ‘M’, all impacting on intermediate outcomes leading to overall improvement in QoL - (Assessing for vision- and hearing-related functional ability).
|
1h)
|
HHIE-25
|
Researcher about PwD and companion
|
Baseline
|
1i)
|
NPI-12
|
Researcher and companion about PwD
|
Baseline
|
To ascertain impact of sensory optimisation on neuropsychiatric function as intermediate step (‘C’, ‘O’ and ‘M’,) leading to improved QoL - (Assessing behavioural and psychological symptoms like anxiety, agitation etc.).
|
1j)
|
RSS
|
Researcher about PwD [companion completes on its own]
|
Baseline, week 18, week 36
|
Improved dyadic relationship optimises ‘O’, thus leading to improved QoL - (Capturing current level of relationship satisfaction between the PwD and study companion, reduce feelings of loneliness).
|
1k)
|
EQ-5D-5L
|
Researcher about PwD and companion
|
Baseline
|
To measure uptake of health services, enhancing ‘O’ - (Assessing health resource utilisation and capturing the amount and type of support that is in place).
|
1l)
|
RUD-Lite
|
Researcher and companion about PwD
|
Baseline, week 18, week 36
|
1m)
|
PwD full visual and / or hearing assessment
|
By optometrist and / or audiologist about PwD
|
Following randomisation
|
To establish the baseline for ‘C’ and ‘O’- (Assessing the degree of the PwD’s visual and/or hearing loss).
|
2)
|
Delivered devices record
|
By researcher / therapist about PwD
|
Following receipt of glasses and / or hearing aids
|
To ascertain the ‘dose’ of the intervention - (Capturing the number and type of corrective sensory devices prescribed by the optometrist and/or audiologist).
|
3a)
|
PwD Diary (acceptability and tolerability of therapy)
|
By PwD about their own experiences
|
Throughout sensory intervention
|
To ascertain the utility of the ‘correction’ component of the intervention from the PwD’s perspective - (Capturing the PwD’s ratings on acceptability and tolerability of the SI and acceptability across 5 indices (helpfulness, fatigue, effort, understanding and motivation) as well as the PwD’s view about their corrective sensory devices - also informing ´C´).
|
3b)
|
PwD Diary (acceptance of devices)
|
By PwD about their own experiences
|
Throughout sensory intervention
|
3c)
|
Companion diary (acceptability of therapy)
|
By companion about PwD’s experiences
|
Throughout sensory intervention
|
To ascertain the utility of the ‘correction’ component of the intervention from the companion’s perspective - (Capturing companions' ratings on the PwD’s response to the session across 5 indices of therapy acceptability (interest, autonomy, motivation, emotional responses, and mastery) recorded after each session).
|
3d)
|
Companion diary (adaptation to devices)
|
By companion about PwD’s experiences
|
Throughout sensory intervention
|
To ascertain uptake/adherence with the corrective devices; to assess level of inter-site variability - (Capturing companions’ ratings on the PwD’s adaptation to, management and use of the corrective sensory devices).
|
3e)
|
Companion diary (companion’s confidence)
|
By companion about their own experiences
|
Throughout sensory intervention
|
To capture the companion training component of the intervention; to assess level of inter-site variability - (Capturing companions’ confidence in supporting the PwD’s use of their corrective sensory devices).
|
4a)
|
SST Logbook (visit record)
|
By therapist about SI visits
|
Throughout sensory intervention
|
To ascertain dose of the intervention; to assess level of inter-site variability by:
- Measuring the dose of the SI (number of sessions, duration of each session and frequency).
- Quantifying the PwD’s capability, opportunity and motivation to use their glasses/hearing aids.
-Capture adherence to the protocol – i.e. which aspects of the SI were delivered by the SST.
|
4b)
|
SST Logbook (response to corrective devices)
|
By therapist about PwD
|
Throughout sensory intervention
|
4c)
|
SST Logbook (delivered components record)
|
By therapist about SI visits
|
Throughout sensory intervention
|
4d)
|
Bangor Goal Setting Inventory
|
By therapist about PwD
|
End of sensory intervention
|
To measure little changes within ´M´ - (Capturing the number and type of goals set by the PwD, and their progress towards these during the SI - address support care needs).
|
4e)
|
Hearing aid skills and knowledge test and / or Glasses / vision skills and knowledge test
|
By therapist about PwD
|
Following receipt of hearing aids / glasses and at end of sensory intervention
|
To enhance self-efficacy both in PwD and companion: ´C´ and ´O´- (Capturing the change in the PwD and companion’s skills and knowledge in using their corrective sensory devices through receiving the SI).
|
4f)
|
GHABP
|
By audiologist and therapist about PwD
|
At audiology assessment and at end of sensory intervention
|
To ascertain improved dyadic communication outcomes: ´C´- (Capturing self-reported gains from hearing aid use).
|
5)
|
Dyadic Experience of SI (interview)
|
By researcher about PwD and companion
|
Intervention end
|
To ascertain how the interviews with the dyads were experienced - (Reflecting upon the positive and negative experiences of intervention for the dyads, perceived benefits/short-comings, suggestions for improvement).
|
6)
|
Battery of Outcome Measures
|
By researcher about PwD
|
Baseline, Week 18, Week 36
|
The 2° outcome of the support intervention focusing on behavioural changes - (Assessing outcomes for participants and companions).
|
7a)
|
SST knowledge and skills checklist
|
By therapy supervisor about each therapist
|
Prior to and post SST training
|
To assess level of inter-site variability as to the SST by:
- A 13-item self-rated questionnaire to identify existing knowledge and skills necessary for the SST role and training needs pre and post- SST training).
- Verifying that the SST has received training in the delivery of the SI components).
- Verify that the SST is competent to undertake the role
- Capturing the number, duration and essence of supervision provided by the Senior SST;
- Assessing positive and negative aspects of intervention, emerging adaptations, suggestions for improvement.
|
7b)
|
SST training log
|
By therapy supervisor about each therapist
|
Prior to trial commencement
|
7c)
|
SST competency checklist
|
By therapy supervisor about each therapist
|
Prior to trial commencement
|
7d)
|
SST supervision log & reflective diary
|
By each therapist and therapy supervisor about themselves
|
Throughout trial intervention
|
Key: MoCA: Montreal Cognitive Assessment [41]; DEMQoL: Dementia Quality of Life [35]; BADLs: Bristol Activities of Daily Living Scale [34]; VA LV‐VFQ-20: Veterans Affairs Low vision Visual Functioning Questionnaire - 20 items [45]; HHIE-25:Hearing Handicap Inventory for the Elderly ‐ 25 items [39]; NPI‐12: Neuropsychiatric Inventory - 12 items [42]; RSS: Relationship Satisfaction Scale [43]; EQ‐5D‐5L: 5‐level EuroQol 5‐dimension [47]; RUD‐Lite: Resource Utilization in Dementia‐Lite [48]; GHABP: Glasgow Hearing Aid Benefit Profile [49].
Exploration of delivery
To ascertain whether the SI is delivered (i.e. ‘how’) and enacted (i.e. ‘what’) as intended [18], we will examine the fidelity and dose (i.e. duration, number and frequency of SI visits) of the delivered intervention. Due to potential burden on participants, we have chosen not to include an external evaluation of fidelity (i.e. independent observer during sessions). Instead, we will rely on the proxy measure of ‘fidelity’ as determined by rigour of SST training and supervision, use of therapist manual, and SST logbook recordings of sessions. We will document the nature of the support offered by the intervention, including the type of corrective devices, the environmental changes to support sensory function, the number and types of referral or signposts to extra-trial services. These data will be captured in through participant diaries (the PwD and their companion), and the SST logbook.
Specifically, the PwD diaries will contain Likert style ratings [50] of acceptability and tolerability of SI visits, including measures of helpfulness, effort, fatigue, understanding and motivation; and how acceptable the corrective sensory devices are. The companions´ diaries will capture data relating to how the PwD engages with the visit, how the PwD is adapting to their sensory aids, and how confident the companion feels in supporting the PwD in using the aids. The SST logbooks will contain details of each visit, the components of the SI delivered, participant response to the intervention and skill in managing their aids. Additionally, the SST logbooks will detail how the SI is specifically tailored to the dyad.
We will assess reach through the representativeness of the sites, the recruitment process (refusal rate, attrition rate) and the representativeness of the study population according to the target population [51].
Evaluation of contextual issues
The SENSE-Cog RCT will take place in several different countries and involve three languages (English, French and Greek). Thus, contextual issues, which are external to the intervention itself, need to be carefully considered. These include differences in language, culture, access to services, and the health and social care economy. Context may influence the SST’s ability to foster change in the participant dyad’s circumstances. For example, for social isolation, the SST may recommend attendance at a local lunch club; however, if transportation is not suitable for individuals with sensory and cognitive impairment, the opportunity to take up the offer will be hampered. Likewise, communication training with companions may be differently received in diverse cultural and linguistic contexts. Thus, the same intervention may have divergent outcomes according to the setting in which it is delivered [18]. The dyadic relationship (between the PwD and their companion) should also be considered because the level of support and quality of relationship may vary among dyads. To capture contextual data, we will collect information from the demographic and outcome measures, the participant dyad diaries, the SST logbook, and in-depth qualitative interviews of a sub-sample of participant dyads (n=30 dyads across the sites), as detailed in the SENSE-Cog Trial protocol [16].
Sampling and timing of data collection
We will collect characteristics of each participant dyad, including gender, age and support structure at the screening visit, and at baseline, week 18 and week 36 (Table 3 & Table 4). Following each SI visit (for the active arm), participant diaries and SST logbooks will be completed. The sub-sample qualitative interviews will take place within two weeks following the SI (details described in [16]). Training logs for the SSTs were collected prior to study start. SST supervision logs and fidelity checks of the SST logbooks are being collected throughout the trial. SST interviews will be held within two weeks after the last intervention visit of the last randomised dyad in each site. Briefly, these interviews will explore the experience of having received the intervention, from the perspective of each member of the dyad. The sample size for this sub-sample was selected to achieve theoretical and data saturation. The interviews from all sites will be analysed by using conventional qualitative content analysis [52] and a grounded theory approach [53].
Table 4 Participant assessment and follow-up visit schedule for the SENSE-Cog Trial (*) W (week)
|
Screening
|
Baseline0
|
Randomisation
|
Intervention/Care as usual period
|
Follow-up visit0
|
Follow-up
|
Final visit0
|
|
W-2/W0
|
W2/ W0
|
W0
|
W0-W7
|
W8
|
W9-W18
|
W18-W20
|
W26
|
W36-W38
|
Allowable visit window
|
|
|
max.3 days after baseline
|
|
|
+/-2 W
|
+/-2 W
|
+/-2 W
|
+/-2 W
|
Signed informed consent
|
ü
|
|
|
|
|
|
|
|
|
Clinical examination1
|
ü
|
|
|
|
|
|
|
|
|
For the PwD (regardless of randomised allocation): assessments of QoL, functional ability, Global cognitive functioning, mental well-being2
|
|
ü
|
|
|
|
|
ü
|
|
ü
|
For the PwD (regardless of randomised allocation): cost effectiveness assessment2
|
|
ü
|
|
|
|
|
ü
|
|
ü
|
For the companion (regardless of randomised allocation): experience, relationship with the PwD, depression and anxiety2
|
|
ü
|
|
|
|
|
ü
|
|
ü
|
Intervention group only3:
Multi-component Sensory Intervention
|
|
|
|
|
|
|
|
|
|
Intervention group only: Qualitative interview5
|
|
|
|
ü
|
|
|
|
Care as Usual group only: Care as Usual
|
|
|
|
|
|
|
|
|
|
Surveillance of clinical AEs (phone call at W8 and W26)
*and for the SI group only, during each SST visit
|
|
|
|
*
|
ü
|
*
|
ü
|
ü
|
ü
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
0Baseline, follow‐up and end visits may be split into two visits, occurring within a maximum of a two‐week window according to the PwD’s needs.
1Clinical examination including assessment of vision impairment (using the PEEK tool), hearing impairment (using the HearCheck device), level of cognitive impairment and other eligibility criteria, and medical diagnostic of depression
2 Baseline (except for the MoCA scale which will be performed at screening), W18 and W36 (around 2 to 2.5 hours)
3The number of sessions may vary from participant to participant but the maximum number of visits will be 10 at an average rate of 1 per week. The order and duration of each component may vary according to the participants’ needs, as determined in collaboration with the researcher, the PwD and the companion.
The remaining weeks until week 18 will allow the SST to revisit and recap previous components, based on the participant’s individual needs and goal‐attainment
4In case of dual impairment, the hearing assessment will be done first, followed by the vision assessment, if possible.
5A qualitative interview will be consecutively proposed to each dyad:
‐ who experienced completed SI (all intervention visits scheduled by the SST, in addition to the full hearing and vision assessments)
‐ at the therapist discretion based on their clinical impression of the person be able to provide meaningful feedback
‐ and who are willing to
‐ until to reach the participation of 60 people (30 interviews), 6 dyads per site: 4 dyads with single impairment (2 hearing only and 2 vision only if possible) and 2 dyads with dual impairment.
This qualitative interview will take place within two weeks of SI completion. One additional interview, if possible, will be undertaken with a dyad which did not complete the full SI.
Analysis
To avoid biased interpretation, as recommended by the MRC’s guidance, we will analyse and explore process data arising from the qualitative interviews and contextual factors before the unblinded trial outcomes are known [18]. We will use the process data to generate specific hypotheses (pre-trial explanation before trial outcomes are revealed) regarding factors that moderate and/or mediate the effect of the SI on outcomes, notably quality of life. This will minimise the risk of “fishing” for relationships and falsely significant findings due to multiple testing.
Moderator analyses will be undertaken, with appropriate caution, to investigate any influence of the baseline characteristics of the dyads (e.g. age, gender, type of sensory impairment, level of cognitive impairment, type of companion), and country/site effect on the strength and/or direction of the relationship between the SI and the outcomes. We will undertake mediation analyses to assess the degree to which the impact of the SI on the stated outcomes is a direct effect, or is indirect via the hypothesised mediating factors.
We will conduct the moderator/mediator analysis only after the final RCT analysis has been completed and the dataset has been un-blinded. We will do this regardless of whether the SI has a significant direct impact on the primary outcome.
We will apply a regression framework, using newer methods and statistical models [54, 55] that improve on traditional approaches (e.g. [56]). These models can become complex, particularly when controlling for multiple covariates. Thus, depending upon the number and complexity of the hypotheses to be tested, we will assess whether it will be better to analyse each factor separately, or to combine sets of moderators and/or mediators into a ‘conditional process analysis’ [54]. We will conduct separate analyses for those variables available in both trial arms, and those available in the intervention group only (e.g. related to the SI). The latter analyses will help to identify process measures that are part of the SI and may moderate its efficacy (for e.g., number of SST visits, SST experience, and fidelity), using appropriate techniques [57].