Design
The Work Package 1 protocol is published elsewhere (15). An iterative, multi-methods, consensus approach was chosen as it provides pragmatic information where empirical evidence is limited and in exploring ambiguous or controversial topics (16). The iterative nature allowed new insights to be incorporated into later stages, creating a dynamic and practical design, using the best available evidence at each point.
Participants
Participants involved in each stage are shown in Figure 1.
Procedure
The consensus process had three stages, shown in Figure 1.
Stage 1: Evidence gathering and generation of candidate standards
Systematic review and scoping survey
A systematic review and scoping survey of TMCDs in the UK supported the concept that TMCDs are effective and provided a picture of current TMCD service delivery. Results are published elsewhere (5).
Interviews and Focus Groups
Sixty interviews and nine focus groups were conducted with nine TMCDs. The sample size was based on the concept of information power (17). The interviews and focus groups had a broad aim, there was little existing theory regarding this topic, and a cross-case analysis was conducted, so a larger sample size than considered typical was necessary, as these factors reduce potential information power. However, the sample (See Figure 1) was specific, and the quality of dialogue was high due to researchers having appropriate experience and having undertaken pilot interviews. These factors increased the information power and ensured the sample size was sufficient. The interviews and focus groups explored how the team was set up, the services offered/available, operational factors considered important for effective working, and gathered examples of best practice, (or practice that had not gone as well), . Participants are shown in Figure 1. Carers and people with dementia were not required to be dyads so that carers of people with more severe dementia, who were themselves unable to participate, were able to participate. TMCDs were selected from a pool of teams that had expressed interest during the scoping survey (5), chosen to be demographically and geographically diverse, reflecting different models of crisis intervention provision. Data were analysed by the first two authors using thematic analysis, following Braun and Clarke’s six stages (see Figure 2). The analysis was combined with evidence from the systematic review and scoping survey (5) and used to develop 165 standards that captured the essence of effective TMCD working. Standards were developed by identifying where themes occurred within the data and selecting the component of TMCD service provision being described. These were documented, with new components added each time another was identified, until no further components were found.
Standards detailing similar or related aspects of practice were grouped together using themes identified to create categories that represented distinct aspects of working practice or service provision, and these were refined throughout the process.
Stage 2: Prioritisation and selection of standards
Consultation groups (1)
Two consultation groups with 14 stakeholders reduced the number of standards to a manageable amount prior to the consensus conference. Stakeholders were not considered participants, but represented critical friends to the project and were clinicians from TMCDs or other health and social care services, and members of the Patient and Public Involvement (PPI) group. Stakeholders represented a range of disciplines and expertise (e.g. a consultant geriatrician, an occupational therapist, and a person with dementia) and were drawn from local research and health service communities. The groups reviewed each category of standards in turn, facilitated by the research team. Each group reviewed half of the original 165 standards. Stakeholders categorised each standard as: ‘highly important’, ‘moderately important’, or ‘not important’ to TMCD working. Some standards were rated ‘Undecided’ if a decision could not be made and these were prioritised for discussion at the consensus conference. Items deemed highly important by the majority of the group were retained for inclusion in the consensus conference. Items deemed not important by the majority of the group were discarded from further inclusion. Items considered moderately important by the majority of the group were combined with other items or modified. The research team provided contextual evidence from the systematic review, scoping survey, and qualitative work and ensured that standards aligned with the evidence after changes were made. The initial 165 standards were reduced to 95 standards.
Consensus Conference
The consensus conference aimed to further refine and reduce the statements to a Best Practice Model that could be taken forward to field-testing. The consensus conference was a one day event involving 39 participants. Participants were selected via local and national research, practice, and PPI communities through contacts developed by the research team during the earlier stages of the research. Participants are shown in Figure 1. All participants had a working knowledge of crisis in dementia through personal or professional experience and represented expert viewpoints. The process used was similar to a consensus development panel, which involves organised meetings of experts in a given field from a variety of disciplines (16). Unlike nominal group techniques, the consensus conference approach is not anonymised, nor does it rely on standards having to reach a particular threshold of agreement to be retained. The face-to-face interactive aspect of the consensus conference provides a means to synthesise the best available evidence by encouraging interactions between people, drawing on and expressing multidisciplinary perspectives, with experts taking ownership of material on topics that directly impact them. It is an iterative, systematic, practical approach, enabling consensus to be reached in a day.
Participants were allocated to one of five discussion groups, each with approximately six participants. Groups were facilitated by members of the research team and the PPI group. Participants were allocated to groups on their experience of components of TMCD service delivery contained in each category, and where possible included a person with dementia and a family carer. Prior to the consensus conference all facilitators were trained in facilitation skills to moderate discussions and ensure everyone had equal opportunity to participate. Participants received the standards and their group allocation before the consensus conference. Groups considered one or two categories of standards (depending on the size of the category) but were encouraged to cross reference standards in other categories.
Participants received a workbook containing the 95 standards. Each standard was detailed in full and presented with quotes from the qualitative work that provided an evidence-base and contextual background. Presenting data from previous stages of the process enabled decisions to be reliant on the evidence rather than solely the personal experience of an individual. The facilitators of each group guided discussions by following the decision-making process outlined in Figure 3. Consensus was achieved when the whole group agreed on the inclusion and wording of each standard. Standards not reaching consensus in groups were considered by the whole conference, with further ideas identified until agreement was reached.
The consensus conference reduced the number of standards to 50. The research team reviewed these standards to ensure no duplication or missing elements and that, based on the knowledge gained from the systematic review, scoping survey, and qualitative work, the standards fit with current practice.
Modified Delphi process
Consensus conference participants acknowledged and agreed that not all standards could be considered equal in their contribution to delivering best practice. This was because some standards underpin others, and consequently standards must be weighted. A modified Delphi approach (16) was used to conduct a points allocation task, where stakeholders were invited to allocate a total of 100 points to the 50 standards by giving each standard a score of 1, 2, 3, or 4. Twenty-three stakeholders participated in round one of the points allocation task. Stakeholders consisted of participants who had attended the consensus conference, stakeholders from the consultation groups, members of our PPI group, and academics and were invited to participate by email. Scores from round one were collated and averaged to produce a score from one to four for each standard. In round two, the points allocation task was returned to the stakeholders with the average score shown next to each standard. Seventeen stakeholders were available to participate in round two, and scored the standards again taking account also of the average score. Scores were again collated, averaged, and allocated to each standard.
Stage 3: Refining and operationalising standards
Consultation groups (2)
A third consultation group (Figure 1) to determine the type and availability of evidence required to demonstrate fidelity to the standards, and how this could be collected. Standards developed earlier in the consensus process that were not retained in the final 50 were discussed and refined for use, where relevant, as potential indicators of evidence for each standard.
The research team created scoring sheets for use when reviewing TMCDs, based on feedback from consultation groups, and drawing inspiration from the CORE study (14), which used a similar review process to the present study (see Figure 4 for an example).
Field-testing
The field-testing process was based on the process used in the CORE study (14). Review days were held with 12 TMCDs and five non-crisis teams. Non-crisis teams included Community Mental Health Teams for older people (CMHT-OPs). Some CMHT-OPs were in areas where a TMCD also operated, and there was little overlap in working practices or service provision between these teams and the local TMCD, but others operated in areas without TMCD provision. Prior to the review day, the research team contacted the TMCD/CMHT-OP to explain the process, and support the teams in making arrangements for the review process. On the review days, a reviewing team comprising two members of the research team, a member of staff from a different TMCD, and a member of the PPI group visited each team with the scoring sheets (Figure 4). Reviewers began the day with a tour of the team base, and then conducted all other activities to fit with the needs of the team, either in pairs or individually, and some evidence was collected at a later date. The reviewers met to complete the scoring sheet (see figure 4) and provided feedback to the team. Draft reports were provided to teams, allowing for clarifications, further evidence to be collected, and any changes to the fidelity score were agreed by the reviewing team. Scores between TMCD and non-crisis teams were compared to determine discriminant validity of the measure, to assess for floor or ceiling effects, and (in conjunction with feedback from team managers) to determine face validity of the Best Practice Model.