Study Design
The research presented in this manuscript is part of the RECOVER-E project, a European research project with five patient-randomized trials on the implementation of community-based mental healthcare for people SMI (23). In each of the five sites, the study was designed as a clinical and health-economic evaluation on the basis of a hybrid effectiveness-implementation trial, which assesses both implementation outcomes and patient health outcomes. Each of the five hybrid trials is conducted as pragmatic randomised trial in two parallel groups with measurements among service users and healthcare providers (members of the CMHTs within each site).
The present study reports the findings from a paper-based survey among all team members of the CMHTs in each project site at local start of the study, after having completed a one-week training session on the concept of CMHT and the principles of the recovery-oriented practice approach. The local research teams were responsible to distribute the survey to the CMHT members after the training.
Study setting
Community mental health teams were established and implemented in Croatia, Zagreb; Montenegro, Kotor; Romania, Siret; North Macedonia, Skopje; and Bulgaria, Sofia (Table 1). The full rationale and selection criteria for these sites is described in a prior publication (23).
Table 1: Characteristics of the study settings/project sites
Participants
Healthcare providers were eligible for joining the CMHT and then participating in the study if they were over the age of 18 working with people with SMI in the five project sites. Additionally, peer workers (persons with lived experience of a SMI). over the age of 18 were asked to join the CMHT and to participated in this study. Each team had to consist of at least one nurse, psychiatrist, psychologist, social worker, and a peer worker. Healthcare professional and peer worker were excluded if they did not meet the inclusion criteria, did not give consent to participate in this study, or were not able to give consent to participate. Each team member of each project site participated on a voluntary basis and gave consent prior to the start of the study. Sampling and recruitment were organised by the local research team in each project site individually.
Sampling and recruitment
Zagreb, Croatia
During a daily meeting at the department of psychiatry at the University Hospital Centre Zagreb (ZUHC) the RECOVER-E project and its goals were presented. Healthcare professionals who were interested in this topic could approach the local principal investigator and decide if they want to participate. All healthcare professionals who accepted to become a part of the CMHT participated in this study, and were employees of the ZUHC. Peer workers were recruited from patients that were treated at the ZUHC due to SMI and were recovered during psychosocial treatment afterwards.
Kotor, Montenegro
In Montenegro CMHT members were employed by the Special Psychiatric Hospital in Kotor. They have been selected by the director of the clinic based on their commitment to work, their previous work results and on their own motivation and interest to work in this kind of programme and their awareness of the need to provide more than just hospital-based services for service users. The directors explained the project and the nature of the study and asked if they want to participate on a voluntary basis. In early October 2018 three peer worker were recruited from a group of locally treated clients at the hospital and the Mental Health Centre in Kotor. Inclusion criteria were a presence of severe mental health disorder, the willingness to participate in working activities of mental health team, and the capacity to offer peer support based on the opinion of the treating psychiatrist. All healthcare proffesionals and peer workers who accepted to become a part of the CMHT participated in this study.
Siret, Romania
In Romania CMHT members were selected out of the people working at the hospital in Siret by the local principle investigator. All team members were willing to spend some of their free time on pioneering a new service in mental health. Moreover, they were the most active during the on-site training provided by the project coordination. The peer worker was selected together by the social worker, psychiatrist and psychologist based on their professional experience. All healthcare professionals and the peer worker who accepted to become a part of the CMHT participated in this study.
Skopje, North Macedonia
All CMHT members were employed at the University Clinic of Psychiatry Skopje and were recruited by the directors of the clinic. They selected the employees based on their willingness and motivation to try something new. The directors explained the project and the nature of the study and asked if they want to participate. The two peer workers, who were recruited, were treated at the clinic due to SMI previously and were recovered during psychosocial treatment. They are not employed at the clinic but were also selected by the directors of the clinic. Everybody took part in the first training that was organized in Skopje. All healthcare professionals and the peer worker who accepted to become a part of the CMHT also decided to participated in this study. Six members went to the second training in the Netherlands and updated the others.
Sofia, Bulgaria
Healthcare providers were recruited by the director of the MHC Shipkovenski. They decided on a voluntary basis if they want to participate in this project. All CMHT members were employed at the MHC “Prof. N. Shipkovenski” a hospital for the district Sofia. The psychiatrists who work at the MHC Shipkovenski and those who collaborate with the centre suggested patients who were interested to fill in the role as peer workers. The director of the MHC Shipkovenski selected ten potential participants which participated in the training in July 2019. Three of them were invited and agreed to participate in the project. All healthcare professionals and the peer worker who accepted to become a part of the CMHT also decided to participated in this study.
Training on recovery-oriented practices
After members of the CMHTs were appointed by each mental health service in the five sites, CMHT members participated in a two-week training programme, with one week of the training carried out in their home country and one week as an intensive site visit and training week in the Netherlands, hosted by GGZ Noord-Holland-Noord. In this training, healthcare professionals had the chance to improve their understanding of community based mental health approaches, with the aim of being able to implement a cohesive team in their city or district. The training programme for the CMHT were developed by a multidisciplinary Expert Panel (including a peer worker) and reviewed closely by the implementation site coordinators for local relevance and adaptation. The training covered key components of community mental health care, working with a shared caseload, and home treatments. It focused on building the hands-on skills and competencies necessary for delivering high quality community care, as well as working with peer specialists and families. A substantial component of the training (and subsequent mentoring and hands-on coaching) focused on the most difficult change process in building a sustainable CMHT, which is changing the mindset from viewing treatment for people with SMI as custodial (protection from communities/societies, care in institutions and hospitals) and shift to perceiving services as an aid to a meaningful life in the community.
Data collection
Data was collected using a paper-based survey administered among CMHT members at the local start of the study during a staff meeting in each site by the local research team after the first week of training in their home country (Table 2). The questionnaire items had been translated prior to the start of the study at each site into the local language by members of the local research team who were fluent in English and in the local language and back translated. It took the team members approximately 15 to 20 minutes to complete the questionnaire. An informed consent was signed by all selected staff members prior to the start of the study.
Table 2: Period of data collection and dates of the two training weeks per project site
Measurements
Sociodemographic data questionnaire
The sociodemographic part of the questionnaire included questions such as year of birth, sex, profession, years of professional experience, and place of work.
Recovery Self-Assessment (RSA-Provider Version)
The Recovery Self-Assessment (RSA) is a 32-item measure developed to assess to what degree a program implements recovery-oriented practice (24). It is a self-reflective tool that is designed to identify efforts made by healthcare agencies to provided recovery-oriented care. Research has shown that the RSA has moderate to strong internal reliability (Cronbach’s α 0.63 – 0.90) (24-27). Response options include a 5-point Likert scale from 1 (strongly disagree) to 5 (strongly agree) and two additional answering options (Don’t know and Not Applicable) (28). There are four different versions of the RSA: Person in recovery, Family member/advocate, provider, and CEOs/Directors. In this study the provider version (RSA-P) was used. The measure covers the following five subscales: Life Goals, Involvement, Diversity of Treatment Options, Choice, and Individually Tailored Services. The Life Goal domain refers to the extent to which staff helps with the development and achievement of life goals based on the preferences of the patient. Involvement indicates to what degree healthcare provider and clients perceive that clients are involved in their healthcare and in decision-making. Diversity of Treatment Option Subscale refers to what extend the healthcare organisation provides different treatment options and supports clients to get involved in non-mental health activities. The Choice domain indicates to what degree healthcare provider and clients feel that choices are available to clients and if the choice is respected. The subscale Individually Tailored Services refers to the perception that healthcare services are tailored to individuals’ personal needs, culture and affectations (27). Higher scores indicate a greater degree of implementation of recovery-oriented practices (24).
Team Member Self-Assessment
The Team member Self-Assessment Tool (TMSA) developed by the Advancing Integrated Mental Health Solutions (AIMS) Centre of the University of Washington, Psychiatry & Behavioural Sciences Division of Population Health and is a part of the Team building and workflow guide (29). The tool (worksheet) consists of 26-items that allow each member of a care team to think about what collaborative care roles he/she currently practices. The worksheet includes five different care roles: Identify and Engage Patients, Track Treatment Outcome, Initiate, and Provide Treatment, Proactively Adjust Treatment if Patients are not responding, and other tasks Important for our Program. Response options for the first question “Is this your role” include yes or no. Answering option for the second question “Your level of comfort with this task” include high or medium/low.
Data analysis
All questionnaires were included in the analysis. Prior to analysis, all variables were checked for data entry errors and missing values. The two response options (‘Don’t Know’ and ‘Not Applicable’) were set as user missing value when conducting the first analysis since this is a common method for categorial variables with response options like ‘Not Applicable’ or ‘Don’t Know’. Then the standard technique to calculate scale scores for each subscale of the RSA-P was used (30). This meant that, for each subscale all associate items were summarized and aggregate measures were constructed for further analysis for the whole sample as well as per project site. This method only allows for a few missing values. Thus, items with a high number (n > 6) of ‘Not Applicable’ responses were reviewed and discussed with the local research teams of each site to understand in what way this response option was interpreted by the CMHT members who completed the questionnaire. Most of these items turned out to be relating to services/treatment options which were not offered by the hospitals, the mental health institutes or not within the scope of the RECOVER-E project. Thus, the answering option ‘Not Applicable’ was combined with ‘Strongly Disagree’ in the second and final analysis, as both indicated that those services were not provided. Bivariate linear regression analyses were applied to explore the impact of predictors on the five aggregate measures for the whole sample. Predictors with significant effects were included in multiple regression models, albeit these were considered as highly tentative given the small sample size. The internal consistency of the RSA-P and its five subscales were evaluated using Cronbach’s alpha coefficient.
Descriptive statistics were used to calculate the means and standard deviations for continuous variables and frequencies and percentages for categorical variables for the TMSA tool and demographic data. For interpretation of the findings the cut-off points for the TMSA tool for each profession was 50% indicated that half of the given profession felt like a listed role is their responsibility or that half of them felt confident to fulfil these roles. Two participants indicated ‘other’ as profession and were excluded TMSA analysis. Data was analysed with the Statistical Package for Social Science SPSS version 25 (31). Statistical significance was defined as p<0.05.