Study design
This effectiveness-implementation Hybrid Type 1 trial aims to examine both the implementation and effectiveness of OTCH. The Dynamic Adaptation Process [15] will be used to adapt and implement OTCH, as well as to examine fidelity and document implementation barriers and facilitators. A cluster-RCT will be conducted among 300 participants with FEP drawn from 21 outpatient clinics that include marginalized and poor communities to test the effectiveness of OTCH (see Fig. 1 below based on PRISMA guidelines).
Description of sites
Twenty-one outpatient clinics located in five different regions in Chile will be enlisted in this study. The regions involved in the study include the Metropolitan Region of Santiago with a population of 7,112,808; the IV region of Coquimbo with a population of 757,586; the V region of Valparaíso with a population of 1,815,902; the VI region of O’Higgins with a population of 914,555; and the VII region of Maule with a population of 1,044,950. In these regions, more than 80% of the total population uses the public health system. Thirteen of the 21 clinics are located in the Santiago Metropolitan region, which is divided in 4 health districts: Southern, Western, Southeastern, and Central Metropolitan. Further information regarding population served, poverty level, and expected enrollment for each of the twenty-one clinics is available upon request.
Adaptation Process
As noted previously, we will use the Dynamic Adaptation Process to adapt and implement OTCH, examine fidelity and document barriers and facilitators. The implementation of OTCH will include four phases: Preparation, Adaptation, Implementation, and Evaluation (Fig. 2). Ongoing experience will iteratively inform adaptation as needed. We will convene multiple stakeholders (e.g., policy makers, providers, clients) qualitative interviews and focus groups to discuss contextual conditions and identify potential modifications to the OTCH model and training curriculum. Then, once OTCH teams are established, a centralized training team will provide ongoing training using a virtual learning platform which will facilitate synchronous and asynchronous learning opportunities including virtual meetings, treatment manuals, tools, videos and other resources to support implementation. We will use mixed-methods to examine programmatic fidelity and document barriers and facilitators of OTCH implementation.
Sample size and power analysis
Power was calculated conservatively based on participation of 270 participants (rather than the full 300 to be enrolled), anticipating possible participant drop-out of ~ 10%. We estimated that, with at least 10 clinics in each condition, we will have 80% power to detect a difference in mean outcomes between study conditions with a moderate effect size of Cohen’s d = 0.35 following standard detectible effect size formula for clustered randomized trials [16, 17]. This power calculation was derived assuming an intra-class correlation of 0.05 for outcomes by clinic (15 participants nested within 21 clinics). We also assume an alpha of 0.05 and that all tests are 2-tailed. Previous research has shown interventions like OTCH to have moderate to large effects on implementation and service outcomes, such as medication adherence, service utilization, and engagement in treatment [5, 18]. We have also shown that FEP providers in Chile rely primarily upon medication and are usually not trained to promote shared decision-making, recovery orientation, or person-centered care [9]. Then it is reasonable to anticipate at least moderate (and probably large) effects of OTCH on these indicators of implementation. Finally, evidence for the expected effect on participant-level symptoms and functioning comes from the RAISE Connection Program study, which indicated improvement over 24 months for participants enrolled (e.g., PANSS improvement effect size 0.91, occupational functioning 1.2, and social functioning 0.72) [18]. Power to detect mediation effects (Aim 2b) depends on the size of the effect of intervention on implementation and service outcomes (path a), and the correlation between implementation and services outcomes and participant level symptoms, functioning, and recovery (path b). Based on tabulated recommendations for sample sizes needed to test mediation effects with a sample of size N = 270 with follow-up data, we have over 80% power to detect a significant mediation effect with path a = .26 and b = .26. Path a is expected to be > 0.35 (argued for Aim 1 above), and we expect based on our previous work that there will be at least moderate associations between implementation and clinical outcomes (r > 0.40), assuming no confounding of mediator-outcome relationship.
Randomization:
The 21 participating clinics will first be divided into two groups based on poverty level of catchment area and expected enrollment of individuals with FEP for each clinic. The largest site will be matched with two clinics of similar poverty level and clinic size as well, as noted above. The matching will be done based on visual inspection of a scatter plot of expected enrollment by income of each site. Then, blocks of size 2, and one block of size 3, will be randomly generated. Randomization will be performed using a randomization scheme generated by SAS 9.4. This procedure will entail 1) generating a random number 0 or 1 ten times, and 2) assigning 0 to the Usual FEP care clinic and 1 to the OTCH clinic in each matched pair. For the block of size 3, two sites will be assigned to either OTCH or Usual FEP care, and the remaining site to either OTCH or Usual FEP care as well. The statistician will notify the contact-PI once the randomization is done, and he will then call the CMHC Director to inform him/her of the assigned treatment status, followed by a written note to the Director with the same information. The random allocation within blocks of two will enable us to bring into the study one OTCH and one Usual FEP Care CMHC per month. We note that the randomization (and notification) will be done at the start of the project, which will enable us to select equal numbers of OTCH and Usual FEP Care CMHCs for data collection (at organizational, provider and participant level) during the preparatory phase.
Inclusion/Exclusion criteria
Study inclusion criteria for clients are: (1) verified as having FEP by a psychiatrist in a mental health clinic within two years of the onset of psychosis; (2) meet ICD-10 criteria for a diagnosis of schizophrenia, schizophreniform, schizoaffective disorder, delusional disorder, brief psychotic disorder, or psychosis not otherwise specified; (3) 15–35 years old; (4) have the capacity to provide fully informed consent (for those under 18, assent plus informed consent of parent/guardian); and (5) able to participate in research assessments in Spanish. In order to conform with government legislation, individuals will be deemed ineligible if they have a non-psychiatric medical condition that impairs functioning, a psychosis due solely to another medical condition, or a developmental disability. All subjects aged 18 years or older will be required to provide written informed consent for study participation. Subjects under 18 will provide written assent and their parent/guardian written consent. As noted above, we will also enroll providers, who will be mental health workers who have direct contact with FEP clients. Of note, these two groups (clients and providers) will be recruited at both OTCH and Usual FEP care sites to assess the implementation outcomes included in this cluster RCT (i.e., fidelity, acceptability, and uptake).
For the qualitative assessments, we will interview clients enrolled in the cluster RCT, plus policy makers and health directors. These will be individuals who are 18 years of age or older and employed full- or part-time at public agencies or participating clinics, respectively. Finally, we will also interview family members, who will be individuals who live with or are close to the client, and who are involved in providing support.
Interventions: OTCH vs Usual FEP care
OTCH
We will adapt the OTNY model to the Chilean cultural and service context to create OTCH. OTNY, as previously noted, is an intensive outpatient program for young people who have experienced their first episode of psychosis. The program is implemented by a multidisciplinary team with specialized training, who provide coordinated evidence-based treatments based on the interests, needs, and preferences of each participant. The program emphasizes assertive outreach and engagement and is offered for an average of two years.
OTNY teams offer participants a range of evidence-based treatment options, including individual therapy based on Cognitive Behavioral Therapy (CBT) for psychosis, psychiatric medications, family education and support, supported education and employment, case management and community support, and a focus on physical wellness and coordination with primary health. OTCH teams will be able to offer all of these services, some of them with low availability in the country (e.g., CBT), except peer support since clinics are not staffed with peer specialists. Services are delivered in a person-centered, recovery oriented, culturally competent manner that prioritize an individual’s needs and treatment preferences using shared decision making [14]. The overarching goals is to help participants achieve personalized goals typically related to school, work and relationships. The program strives for a balance between encouraging independence and offering support when needed [14]. An OTCH team will be comprised of five members however, adaptations will be made to how the roles will be carried out by the respective team members in response to the local context (Table 1).
Table 1
Role
|
Description
|
Team coordinator (TC)
|
A mental health professional, selected by a health director/manager, who will oversee the OTCH team and ensure that all team members adequately fulfill their roles.
|
Primary clinician (PC)
|
Preferably a psychologist, who will be the primary resource for the participant and his/her family and responsible for formulating the treatment plan in accordance with the participant’s preferences and coordinate the execution of the treatment plan with the rest of the team. This professional will also deliver psychosocial interventions to participants and family education and support. Some teams may have more than one professional fill this role.
|
Community support professional (CSP)
|
Preferably an occupational therapist, who will provide supported employment and/or education and connect the participant with peers and community resources. In the OTNY model, this role is known as the “supported employment and education specialist.” The name change for OTCH reflects an expansion of this role in the Chilean adaptation.
|
Prescriber
|
A psychiatrist, who will provide medications and symptoms management using shared decision-making and evidence-based prescribing approach.
|
Nurse
|
Either carried out by a nurse or a nursing technician, who will support the psychiatrist and monitor medication use and side effects, facilitate individual and group wellness sessions, monitor physical health and vital signs, and coordinate with primary care.
|
Table 2
Typical activities of each phase of the intervention
Phase 1: Engagement and Needs Assessment
(Months: 1–3)
|
Phase 2: Ongoing Intervention and Monitoring
(Months: 3–18)
|
Phase 3: Transition
(Months: 18–24)
|
Activities:
• Introduction to all team members
• Description of OTCH program
• Intake assessment
• Needs assessment (e.g., housing, income, etc.)
• Risk and trauma assessment
• Building rapport and getting to know the participant and their support network
• Identification of individual goals (e.g., employment, education, relationships)
• Introduction of shared decision making
• Deciding on level and type of family involvement
• Development of a treatment plan
|
Activities:
• Delivery of core psychoeducation sessions
• Delivery of psychosocial interventions to help achieve goals and build resiliency (e.g., social skills training, CBTp, substance use treatment, illness self-management, coping skills, and behavioral activation)
• Suicide prevention
• Review and revise treatment plan
• Actively pursue work and school goals
• Delivery of psychoeducation and support to family
• Relapse prevention
• Explore transition readiness
|
Activities:
• Determine gains made and goals attained
• Decrease frequency of services delivered
• Develop a transition plan
• Identify and connect to mental health services in the community
• Identify and connect to community-based supports
• Engage family in transition plan and identify their role
• If possible, test whether these new services align with participant’s preferences
|
Delivery of OTCH services will be done using a phased approach; for details on each phase see Table 1. At the beginning of treatment, a participant will be assigned a Primary Clinician (PC) by the Team Coordinator (TC) and meet with the PC, and ideally the Prescriber, for an initial orientation session. During this initial phase, team members focus on engaging with participants and families, assessing their needs and helping the individual begin identifying goals. Because no part of the model is compulsory, program participants will be encouraged to use shared decision making to identify treatment areas to prioritize. Even if a participant decides not to take medication, he/she will still be encouraged to meet with the Prescriber and Nurse even if infrequently. For instance, participants should meet with the PC and Prescriber/Nurse once a week for the first month (the prescriber and nurse will alternate weeks) and then every two weeks thereafter. Meetings with the Community Support Professional (CSP) will depend on each participant’s goals, interests, and preferences.
The second phase, constitutes the bulk of the two-year program. Once a strong relationship has been established with the participant and family and individualized goals have been defined, team members will provide interventions focused on helping individuals achieve their goals and promote their recovery as well as deliver core sessions providing psychoeducation about FEP, its course, treatment and information to support recovery and resilience to participants and family members. During this time period, meetings with the PC, Prescriber/Nurse, and other team members will continue to happen as frequently as needed depending on the interventions being delivered and participant/family preferences. As participants start reaching goals and becoming more involved in community and school/work activities it is typical that meetings with team members decrease in frequency. The final phase begins six months prior to discharge from the program and is focused on understanding the participant’s achievements, and the identification of future needs to determine connection to long-term services.
Training and supervision
The training approach for the OTCH teams will follow the established training strategies utilized by the OnTrack Central training team at the Center for Practice Innovations, for training teams in the state of New York and nationally [19]. The training will be delivered primarily through an online learning platform that will allow for virtual meetings focused on individualized and group consultation and coaching, didactic webinars, opportunities for sharing and learning from each other, and discussing data to help maximize the quality of the implementation. The OTCH trainers will also be available via email or phone for specific consultation needs that arise over the duration of the program and they will have access and regular meetings with OnTrack Central trainers in New York to help adapt the training strategy to meet the local needs.
OTCH teams, will receive an initial, remote training providing a general overview of the treatment model before starting implementation. This allows for an opportunity to discuss practical components of the implementation, understand where each team is at regarding their readiness for implementation of the treatment model, and provides an opportunity for engaging in in-vivo problem solving to address questions and barriers faced at the individual agencies hosting the programs. Training then continues following a set monthly schedule of role and team-based learning collaborative calls, so that each team receives the same amount of training and information on a regular basis (Table 3). Learning collaboratives have traditionally been used in health care and are increasingly used in behavioral health care [19]. Fundamental elements of a learning collaborative include a number of organizations working together, using quality improvement methods to close the gap between potential and actual performance, learning from experts as well as from one another, and using data to track performance [19]. This will be complemented with resources for self-paced learning that teams can access through the virtual learning platform. Trainers and trainees will also receive site-level data (both related to outcomes and fidelity) that will be utilized during virtual meetings to guide discussions focused on improving the implementation, clarifying questions related to the model, and problem-solve challenges they may be facing.
Table 3
Description of Training Approach
Training Event
|
Description
|
Initial Implementation Calls
|
Consists of individual phone calls which provide an opportunity for the agency leadership and the team leader to develop a strategy for forming the team and identifying the agency-level infrastructure that needs to be in place to ensure good team functioning.
|
Synchronous and Asynchronous Initial Training
|
Consists of a 3-month training period where individuals will be assigned materials for self-paced learning and then will join a monthly videoconference meeting. Training will consist of a general overview of the treatment model and all of its components delivered before starting implementation of OTCH.
|
Individual role-based videoconference meetings
|
Consists of monthly learning collaborative virtual meetings facilitated by a trainer focused on the implementation of specific elements of the role and gives providers an opportunity to learn from each other’s experiences
|
Care consultation videoconference meetings
|
Consists of monthly virtual meetings attended by two entire teams and at least two trainers focused on discussing a program participant in detail and getting advice and feedback
|
Special topic webinars
|
All teams are invited to receive training on a specific topic requested by trainees (e.g., prescribing long acting injectable medications, cultural competency, suicidality, etc.). These will take place as needed.
|
Usual care
The FEP policy will ensure that participants from the control sites will have access to Usual FEP Care. This is generally provided in mental health outpatient clinics, which (i) serve a population enrolled in the public health care system, (ii) provide services such as psychiatric medication, psychotherapy, and psycho-education for people with FEP and their families, and (iii) have a diverse team of professionals that include psychiatrists, psychologists, psychiatric nurses, social workers, occupational therapists, and family counselors. The services offered in these clinics, however, tend to have some limitations, three of which are relevant to this trial.
First, these clinics offer services to clients with a wide range of mental health conditions, not just those with FEP and therefore, providers lack specialized training on treating FEP (e.g., evidence-based prescribing of antipsychotic medications or evidence-based psychosocial approaches). Second, recovery-oriented services based on shared-decision making and person-centeredness are rarely offered in these clinics since a more biomedical model of care tends to be used. Third, they tend to have minimal resources to provide services in the community and often face several challenges in delivering coordinated care between different types of services (i.e., primary care, other specialized mental health services, emergency services, etc.) and likely across professionals in a team-based model.
Recruitment
Given the current COVID-19 pandemic, most research and intervention procedures will be conducted remotely, but there might be some exceptions as described below. This was decided after consulting with local stakeholders, including providers from the participating clinics, who noted that many clinics are providing mental telehealth services via telephone, email, and video conferencing. Considering that most people in Chile have smart phones, this will be the method of choice to contact, interview, and follow up potential participants via either telephone or Zoom/Webex calls.
After a site enters the study, the participating clinic will be asked to refer all individuals with FEP to our study team. A health provider in each of the participating clinics will be designated to be the local point person for the coordination of recruitment and follow-up of study participants in his/her site; this role will be referred to as “Local Coordinator.” As noted previously, an FEP registry is in place in Chile to identify individuals with an FEP to ensure they receive timely and appropriate treatment. The Local Coordinators are able to use that database to identify potential participants at their clinics. Using this tool, the Local Coordinator will assign a numeric screening ID number to each potential participant and maintain a log with this information in a secure cabinet in each site. Each site will be assigned a numeric range (1000s, 2000s, 3000s, etc.). The Local Coordinator, who is not part of the OTCH research team, will contact the potential participants by phone, videoconference, or in-person when possible. If verbal consent is obtained, the Local Coordinator proceeds with a brief introduction to study. The Local Coordinator will briefly explain that he or she has the opportunity to participate in a research study and will describe the study’s purpose. If the client accepts to participate, the Local Coordinator will notify a member of the Research Team, known as the Registration Designee, who will be in charge of assessing capacity of consent and administering the informed consent form. If someone serves as a Registration Designee for the participant, this person will not be allowed to serve as an interviewer for this participant in order to maintain the blind. This process will take place remotely unless the clinic is open for in-person appointments. Once consent is obtained, an interviewer will be notified so he/she can contact the potential participant and conduct the baseline assessment. Similar procedures will be followed for minors, and for providers, policy makers, health directors, and family members who participate in the study.
Following the recommendations by McRae et al. (18) and Giraudeau et al. (19) on administering consent forms in cluster RCTs, there will be two different consent forms: one for participants from interventions clinics and another one for participants from controls clinics. In addition, we will have other consent forms for policy makers, health directors, providers, and family members.
Interviewers who conduct both the clinical assessments and qualitative interviews will be mental health professionals with prior research training and experience conducting clinical assessments. They will be trained on research and assessment procedures by the Research Team through 3 on-line, videoconferencing sessions. The first session will entail going over the research protocol and the interviewer’s manual. The second session will address the required procedures to enter data into RedCap and report adverse events. Finally, in the third session, a thorough review of the quantitative and qualitative measures will take place, with special emphasis in the administration of the clinical-based measures such as the PANSS.
Data collection
For the trial
Table 4 provides a description of all of the measures that will be used in the trial. All instruments will be administered at baseline 12 and 24 months. Selection of measures was guided by the PhenX toolkit for clinical measures in early psychosis and key moderators that are known to influence the implementation of evidence-based practices [20]. All measures will be translated into Spanish if a Spanish language version is not available using the WHO guidelines for translating and adapting measures.
Table 4
|
Construct
|
Group
|
Measure description
|
When
|
Time
|
Implementation outcomes
|
Fidelity
|
Providers, families, and participants
|
Adapted version of the OTNY fidelity scale. The scale assesses the degree to which FEP services deliver evidence-based practices. Fidelity assssesment will be based on providers’ feedback, supervision calls, and site visits. Site visits have been established as a best practice for fidelity assessment [21].
|
6, 12, 24 months
|
NA
|
Acceptability
|
Providers
|
Providers’ attitudes to evidence-based practices will be measured by the Evidence-Based Practice Attitude Scale (EBPAS) [22]. The EBPAS consists of 15 items rated on a five-point scale from 0 (“Not at all”) to 4 (“Very great extent”). Range: 0–60 points.
|
Baseline, 12, 24 months
|
15 minutes
|
Uptake
|
Participants
|
The CollaboRATE [23] is a 3-item scale that measures Shared-Decision Making from users’ perspective. Each of the three items is rated on a scale of 0 (“no effort was made”) to 9 (“every effort was made”). Range: 0–27 points.
|
Baseline, 12, 24 months
|
5 minutes
|
Providers
|
The Shared Decision Making Questionnaire – Physician Version (SDM-Q-Doc) [24] will be employed to measure SDM at the provider level. This asks the clinician first to enter the health problem the consultation was about and which decision was made.The questionnaire continues with nine items about the steps in the SDM process, scoring at a six point scale that ranges from 0 (completely disagree) to 5 (completely agree). A total score can be calculated by summing the scores of all items. A high score indicates more SDM.
|
Baseline, 12, 24 months
|
10 minutes
|
Providers and participants
|
The Recovery Self-Assessment (RSA) [25] provider and patient versions is a 32-item, self-administered rating scale that focuses on perceptions of recovery principles and overall quality of services. It captures both providers’ and patients’ perceptions that a specific recovery-oriented intervention, in this case OTCH, is agreeable, palatable or satisfactory. Each item is rated on a 5-point scale (1 = Strongly Disagree; 5 = Strongly agree). Higher scores indicating greater quality care. Range: 32–160 points.
|
Baseline, 12, 24 months
|
15 minutes
|
Providers and partcipants
|
Ten ad-hoc questions regarding receipt of services that are central to OTCH including supported employment-education, family intervention/support, psychosocial interventions, and personal strenghts and resiliency training
|
Baseline, 12, 24 months
|
3 minutes
|
Service outcomes
tors of implementation
|
Patient-centeredness
|
Participants
|
The Youth Services Survey (YSS) [26] is a scale that assesses perceptions about accessibility, quality, and impact of mental health service over a period of time. Only the first 15 items will be used in this study. Each item is rated on a 5-point scale from 1= “Strongly disagree” to 5 = “Strongly agree” (Range: 15–75 points).
|
Baseline, 12, 24 months
|
10 minutes
|
Medication adherence
|
Participants
|
Measured by the Brief Adherence Rating Scale (BARS) [27] It consists of 4 items: 3 questions and an overall visual analog rating scale to assess the percentage of antipsychotic medication doses taken by the user in the past month (0–100%).
|
Baseline, 12, 24 months
|
5 minutes
|
Retention
|
Participants
|
Time remaining in treatment will be estimated by counting the number of days between randomization to the time of the last mental health service received as defined by the RAISE-ETP report. The Service Utilization and Resources Form for Schizophrenia (SURF) [28] will be used to record users’ utilization of psychiatric and psychosocial rehabilitation services by conducting telephone contacts on a three-monthly basis. Range: from 0 to 365 days over a 12 month period; from 0 to 730 days over a 24-month period.
|
Every 3 months
|
15 minutes
|
Clinical outcomes
|
Psychotic symptoms
|
Participants
|
Positive and Negative Syndrome Scale (PANSS) [29] scale will be used to assess positive and negative symptoms (only the first 14 items). This scale includes seven rating points that represent increasing levels of psychopathology, from 1 (“absent”) to 7 (“extreme”). Range 7–49 points.
|
Baseline, 12, 24 months
|
20 minutes
|
Functioning
|
Participants
|
The Social and Role Functioning in Psychosis and Schizophrenia (SRFP) [30] scale asks for social, behavioral, and occupational difficulties associated with mental illness, including psychosis and schizophrenia. The scale rates functioning on a scale of 0 (“extreme role dysfunction”) to 10 (“superior social/interpersonal functioning”). Range: 7–70 points.
|
Baseline, 12, 24 months
|
20 minutes
|
Recovery orientation
|
Participants
|
The Questionnaire about the Process of Recovery (QPR) [31] assesses users’ perceptions about recovery from psychosis. It has 15 items each scored on a 4-point scale ranging from 0 (“strongly disagree”) to 4 (“strongly agree”). Higher scores on the measure are indicative of recovery (Range: 0–60 points).
|
Baseline, 12, 24 months
|
15 minutes
|
Moderators
|
Poverty
|
Community-level
|
Administrative data from the Chilean System of Social Protection [32] will be utilized to estimate percentage of the population living below the national poverty lines (0–100%).
|
As available across study period
|
N/A
|
Providers’ Attitudes to EBP
|
Providers
|
This will be measured by the Evidence-Based Practice Attitude Scale (EBPAS) [22] as described above under implementation outcomes.
|
Baseline
|
10 minutes
|
Providers’ Recovery orientation
|
Providers
|
The Recovery Self-Assessment (RSA) [25] as described above under implementation outcomes
|
Baseline
|
15 minutes
|
Symptoms
|
Participants
|
Measured by PANSS [29] as described above under clinical outcomes
|
Baseline
|
20 minutes
|
DUP
|
Participants
|
The time between the onset of psychotic symptoms and initiation of treatment at a mental health clinic based on the FEP registry.
|
Baseline
|
NA
|
Functioning
|
Participants
|
Measured by SRFP [30] as described above under clinical outcomes
|
Baseline
|
20 minutes
|
Socio-demographics
|
Participants
|
Age (15–19, 20–24, 25–29, 30–34, 35–40 years), gender (male, female), ethnicity, education, employment, and marital status
|
Baseline
|
10 minutes
|
For estimating total health care costs, we will rely on both administrative data on expenditures for implementing and delivering OTCH, and interview data, collected every three months from participants on their use of services other than OTCH. Research assistants will document service use using the Service Use and Resource Form (Table 4) every three months, during the first year, to assure adequate recall [28].
For the qualitative assessments
We will use qualitative methods to identify and characterize potential barriers/facilitators and processes that may inform the implementation of OTCH and its continuous adaptation process. In particular, we will explore factors at multiple levels (system-, organizational-, provider- and client/family member-level) by conducting semi-structured, on-line qualitative interviews with policy makers, director/health managers and providers and focus groups or qualitative interviews with users and family members.
Qualitative interviews and focus groups will be conducted when OTCH teams begin enrollment activities and at 6- and 18-month post-randomization at all OTCH sites and at five sites randomized to usual care. We will conduct these interviews and focus groups via telephone/teleconferences. All interviews and focus groups will be audio recorded with participant’s informed consent. We will employ a purposive sampling approach to identify and invite key informants representing each group of participants (17–19).
Data analysis
Figure 3 presents the conceptual model that guides our analysis. The model posits, first and foremost, that by 12 months’ implementation and service outcomes will reflect what should be seen in participants if implementation is properly done, based on previous studies [11, 20, 33, 34]. It also indicates that factors at community- (e.g., % poverty), provider- (e.g., attitudes to evidence-based practices), and participant (e.g., socio-demographics) levels could moderate the effect of OTCH on implementation and service outcomes. The model also posits that the effect of OTCH vs FEP Usual Care on clinical outcomes (symptoms, functioning, and recovery orientation) will be evident at 24 months. It further proposes that 12-month implementation and service outcomes will mediate 24 month clinical outcomes.
We will carry out all analyses on an Intention-to-treat basis. All tests will be two-sided with critical value α = 0.05. Prior to analysis, we will examine all variables for outliers and inconsistencies and examine their distributions. The distributions of all continuous variables will be checked for normality, and transformations will be employed to normalize distributions, if necessary, before applying specific parametric techniques. Missing data will be imputed, using multiple imputation procedures in standard statistical software (e.g. Stata or SAS).
Aim 1a and Aim 1b analyses
We have selected 3 measures of implementation (fidelity, acceptability, and uptake) and 3 measures of service outcomes (patient-centeredness, adherence, and retention) that represent the active ingredients of CSC for people with FEP. These six measures reflect care processes that will be fundamental to the implementation of the OTCH program [14]. Tests for treatment differences between the intervention arms on each of the six outcomes will be performed using a hierarchical linear model (HLM) for each outcome including clinic-level random effects to account for clustering of participants within clinics and participant-level random effects to account for repeated measures on the same user over time. The primary fixed effects will be an indicator for intervention status (OTCH, FEP Usual Care), time (12 or 24 months), and intervention*time. The magnitude and statistical significance of the beta coefficients from the HLMs will be used to test whether implementation and service outcomes differ between intervention arms at 12 and 24 months. In the event that descriptive analyses identify any baseline differences between randomized groups of participants at baseline, we will consider adding the respective covariate to the model to adjust for this.
We will examine multilevel factors that may moderate the implementation of OTCH (see Table 1). 1) Community level: there is evidence that access to and engagement with mental health services is low among those who reside in poor communities [35]. 2) Provider level: we hypothesize that more positive attitudes toward evidence-based practices and recovery orientation will be associated with more successful implementation of programs which have these characteristics like OTCH [36]. 3) Participant-level: we also hypothesize that socio-demographics and baseline clinical state will be predictors of implementation and service outcomes based on prior studies with clinical outcomes [18]. We will test this aim by including all three types of moderators into the HLM utilized for Aim 1.a. Specifically, because we aim to test whether these factors differentially affect the implementation of the OTCH compared to FEP usual care (i.e. an interaction with treatment), we will incorporate a random intervention effect into the HLM and allow the factors to predict the random intervention effect at both the clinic and participant level. The magnitude and statistical significance of the beta coefficients for these factors will provide the test of them as treatment effect moderators.
Aim 2a and 2b analyses
We hypothesize that the OTCH program will be effective and that, over time, user level psychotic symptoms, functioning, and recovery orientation will improve. Tests for treatment differences between the intervention arms on each outcome (symptoms, functioning, and recovery orientation) will be performed using an HLM including an indicator for intervention status, time (baseline, 12, 24 months), and intervention*time. The magnitude and statistical significance of the beta coefficient for the interaction will provide an assessment of the extent to which the change in psychotic symptoms, functioning, and recovery orientation since baseline over time differs by intervention arm.
Based on prior research [37, 38, 39], we have hypothesized that service and implementation outcomes at 12 months will mediate the impact of OTCH on clinical outcomes at 24 months. To establish service and implementation outcomes at participant- and provider-level at 12 months as mediators of the OCTH effect on user level clinical symptoms, functioning, and recovery orientation at 24 months, in addition to finding intervention effects (OCTH vs FEP usual care) as in Aim 1a and Aim 2a, we must also establish that service and implementation outcomes at 12 months are associated with better participant symptoms, functioning, and recovery orientation at 24 months. The mediation effect [40] is estimated by taking the expected value of the total effect of OTCH vs FEP Usual Care on symptoms, functioning, and recovery, and subtracting the direct effect when service and implementation outcomes are taken into account. This indirect (or mediation) effect of the implementation factors will be estimated and tested by fitting a Structural Equation Model using MPlus version 7.4 which our lead statistician has extensive expertise. We will estimate mediation models that consider one implementation or service mediator at a time, as well as including all indicators of implementation or service simultaneously to identify independent effects. The bootstrapping method (available in Mplus) will be used to obtain standard errors, confidence intervals, and the test for statistical significance [41]. Even in a RCT, the association between the mediator and the outcome can be confounded by other time-varying confounders and recent developments in mediation analysis suggest the need for sensitivity analyses to explore potential impact of unmeasured confounders. We will perform these sensitivity analyses which provide a range of plausible results that may have been obtained under different assumptions about unmeasured confounders [42].
Study Management
This study will have several teams in charge of the varying components. There is an overarching study steering committee that oversees all decisions regarding study procedures. An adaptation team will be responsible for performing initial and ongoing qualitative interviews with stakeholders at various levels to inform adaptations to the OTCH model. A clinical team will be responsible for determining clinical adaptations to be adopted, training all of the providers OTCH and examining trend in the data collected to provide feedback to OTCH teams about their implementation. A research team, directed by a Data Center Manager, will be responsible for all quantitative and qualitative data collection.