Study design
This study will use a randomized cluster factorial design(Figure 2), partially following the MOST framework for intervention optimization[15]. Both the pilot and full trial will feature a factorial design with three factors, each having two levels. This results in eight possible configurations (2×2×2=8) of SMA interventions (Table 2). If a specific component, such as the group consultation (patients coming in groups), is deemed essential based on the pilot trial results. It may be applied as a constant component across all SMA configurations in the full trial. The factorial design was chosen for its efficiency in testing both main effects and interaction effects among the SMA components. Both effects are essential to address our core question of identifying the optimal configuration of SMA. The main effect will evaluate the average effect of each particular component of SMA, while the interaction effect will assess the extent to which components affect each other's performance.
The protocol report adheres to the Standards for Reporting Implementation Studies (StaRI) [16], the Criteria for Reporting the Development and Evaluation of Complex Interventions in Healthcare (revised guideline CReDECI 2)[17], and the Standard Protocol Items for Clinical Trials (SPIRIT) [18], Reporting of Factorial Randomized Trials: Extension of the CONSORT 2010 Statement[19],where applicable.
Figure 2 inserts here.
Settings
Guizhou is a less developed province with resources more limited than most other parts of the country [20]. Compared to the national average, Guizhou has a patient population with less knowledge about diabetes [21] and poorer management of glucose levels [22]. The study will be conducted at primary care institutions in two purposefully selected districts of Guizhou: Bozhou (population 0.76 million; GDP per capita CNY¥ 48,390, equal to US $7,584)[23] and Bijiang (population 0.45 million; GDP per capita CNY ¥57,399, equal to US $8,416)[24]. Their GDP per capita is comparable to that of countries such as Bosnia and Herzegovina(US$7,585) and Dominica(US $8,414) in 2022 [25]. We selected those districts as they present an opportunity to examine the implementation of SMA in the resource-constrained setting and thus enhance the study's relevance to other low- and middle-income countries.
Participants
The study will include institutional and individual participants, with selection criteria as follows:
Institutions: The trial targets primary health care institutions in Bozhou and Bijiang districts. These are referred to as community health centers in the urban areas and township health centers in the rural areas. Those centers, comprising both public and private entities, serve as the primary providers of the government's diabetes management program, typically through government service contracts.
Providers: These centers typically employ public health doctors and clinical doctors responsible for executing the government's diabetes program. Public health doctors are usually trained in public health and perform disease management tasks. Their responsibilities typically include routine home-based follow-ups to test glucose levels and blood pressure, check symptoms (and make referrals when necessary), provide health education, and monitor patients' medication status. On the other hand, clinical doctors are clinically trained and focus on curative care and brief verbal health education, usually in outpatient settings. Our trial intends to recruit all doctors involved in the government diabetic program.
Patients: Our trial will include individuals who (1) are currently enrolled in the government's diabetes management program at the sampled institutions (people meeting the national diagnostic criteria for type 2 diabetes [26, 27] and 35 years or older are eligible for the program), and (2) reside in the community and have no plans for relocation in the coming six months. People with cognitive and physical disabilities that prevent participation in the group clinical consultation will be excluded. The cognitive status will be assessed using the Chinese Mini-Mental State Examination (C-MMSE)[28], with patients scoring 23 or lower being excluded. Participants will be asked about any physical limitations that prevent them from coming to the clinic individually, and those who report such constraints will be excluded.
Other stakeholders: In addition to clinicians and patients, the following stakeholders will be invited to participate in the qualitative portion of the trial: government policy-makers, health institution managers, other clinicians who play a supportive role in team-based SMA care delivery, and relatives or friends of the patients.
Sampling method and sample size
Institutions: There are 40 primary care centers or township health centers in Bozhou and Bijiang. Of these, eight from Bozhou and four from Bijiang were selected to develop and preliminarily test the SMA components in the pilot trial (Phase 1). We have purposively picked six facilities from resource-limited settings and six from relatively well-resourced settings (Figure 3). The later full optimization phase (Phase 4) will involve the remaining 28 facilities in Bozhou and Bijiang. Details on the statistical power for 28 institutions (i.e., clusters) and estimated sample size based on the number of SMA components to be tested are provided in Table 1. The codes used to calculate the sample size are listed in Appendix 2.
Figure 3 inserts here.
Table 1 inserts here.
Patients: In the pilot trial, the number of patients enrolled in each institution will be six, in line with the recommendation by Edelman et al.'s systematic review [9]. However, during the optimization phase, the group size may be adjusted based on stakeholder feedback.
Stakeholders: We will conduct several in-depth interviews with stakeholders across the 12 selected facilities throughout the study phases. The level of information saturation reached will determine the sample size for these interviews.
Phases of the study
Phase 1: Developing prototype SMA
Based on our current understanding and literature reviews, we have developed a preliminary conceptual framework(Figure 1) for SMA to illustrate the potential components, their levels, and the mechanism of their effectiveness on implementation and health outcomes. The framework will be discussed among researchers and stakeholders in iterative consensus meetings to determine three candidate components and their levels. We will limit the component number to three as more numbers will lead to cumbersome trial execution. At the consensus meeting, the research team members will present the rationale for including or excluding a component and its levels for the group to provide feedback. Consensus meetings[29, 30] will be held iteratively until the team determines the three most important SMA components.
The existing systematic review highlights the patient-group-based consultation model as a defining feature of SMA, where healthcare providers deliver healthcare services to a group of patients with similar conditions[9, 14]. Health education provided through social media groups[31] is also crucial in other studies[9, 14]. Therefore, the pilot trial may concentrate on these three essential components (subject to consensus meeting discussion)(Table 2):
- Group of clinicians vs. single clinician providing consultation
- Group of patients coming for the clinical consultation vs. single patient
- Health education sessions provided in a virtual social media group vs. in-person education provision.
All service providers will receive training following the SMA service protocol (Appendix 3). The SMA services protocol may be modified according to pilot trial results.
Table 2 inserts here.
Phase 2: Identifying implementation barriers
This phase (i.e., the preparation phase in Figure 2) involves conducting a small-scale, cluster-randomized factorial trial to examine the effectiveness of the prototype SMA components preliminarily. The trial outcomes will guide power calculations for a subsequent full-scale study. Concurrently, we will use the Rapid Qualitative Research method to uncover potential implementation barriers. The Consolidated Framework for Implementation Research (CFIR) [32]will direct our interview structure, data collection, and analysis. To streamline the process, we will analyze audio recordings directly using structured interview notes, bypassing traditional transcription coding. These interviews will shape a survey centered on implementation determinants to be completed by participating patients, clinicians, and administrators. The survey responses will amass a comprehensive list of potential barriers to the implementation process.
Phase 3: Prioritizing barriers for the optimization process
Our aim is to prioritize barriers that can be addressed strategically through SMA optimization. We will convene a panel of stakeholders and researchers for a consensus conference[29, 30] to facilitate this task following the principles below:
- Barriers' impact on implementation: Priority will be given to barriers whose removal could lead to significant improvements in SMA implementability.
- Difficulty in Addressing the Barriers through Implementation Strategies: If a barrier is challenging to address with an implementation strategy, it will be recognized as a constraint and converted into an optimization criterion (e.g., the total expense of SMA must be under 100 USD, or the frequency should be no more than once per month, or the time duration should be no more than 1 hour per time) .
- Feasibility of adjusting SMA to tackle the barriers: We will evaluate to what extent a barrier can be addressed by modifying the SMA configurations. For instance, if a barrier is a lack of evidence for the moderate frequency of SMA sessions, we can vary the SMA session frequency in the trial to provide this evidence. Then, this barrier can be prioritized.
Phase 4: Full optimization cluster factorial trial
Details of this phase are described throughout the method section.
Phase 5: Apply optimization criteria to determine optimal SMA configuration
While Phase 5 will demonstrate each SMA component's main effects and interaction effects, the key next step is deciding which components should be retained in the final optimized SMA package. To make this determination, we will follow the procedures recommended in MOST, specifically applying the principles of effect hierarchy, sparsity, and heredity to the factorial results (Figure 4):
- Identify Important Effects: Based on the results from Phase 4 for the designated outcome, identify important main effects and interactions. For main effects, the "important effect" will be determined if the effect size is larger than the stakeholder-specified minimum important effect size. The interactions will be considered important if the lower-order interaction effect size, which must contain at least one active parent main effect, exceeds stakeholder-specified minimums.
- Screen Components: Screen components into the screened-in set if they demonstrate an important, positive main effect or are involved in an important, positive interaction. Other components are screened out provisionally.
- Reconsider Screened-Out Components: Components with an important negative main effect are reconsidered for inclusion in the screened-in set at the lower level.
- Apply Optimization Criteria: Determine the combination of screened-in components that optimizes the outcome within the constraints. For example, If constraints on human resources, cost, and time are set, determine the combination of screened-in component levels that are expected to produce the best outcome while satisfying the constraints.
- Satisfy Constraints: If needed, omit screened-in components to satisfy the constraints in this order: first, delete those involved only in higher-order interactions, and then delete those with the smallest main effects.
Figure 4 inserts here.
Group assignment and Blinding
A statistician will complete the independent intervention assignment for both the pilot and full factorial trial. The SMA intervention assigned to each institution (i.e., each cluster) will follow the random number table generated by the "Random Assignment Generator()" in the MOST package (Version 0.1.2) of the R software (see Appendix 2). The participating primary care institutions will be randomly assigned to one of those eight interventions. In the factorial design, there will be no single control group as in the conventional two-arm trials. It was impossible to blind the program services providers and patient participants regarding their group assignment. Once the institutions in the optimization trial are determined, we will conduct the same randomized assignment method.
Outcomes, variables, and measurement
We have combined Proctor's Implementation Outcome Framework (IOF) and the RE-AIM framework [33]to form the SMA trial outcome set using the Delphi method (the development and validation process will be published elsewhere). Except for the domains of "acceptability," "appropriateness", and "feasibility", the domains of IOF overlap with those of the RE-AIM[33]. These three domains are essential and will be included in our optimization study. Table 3 summarizes the domains, indicators, definitions of the indicators, formulas for calculating the indicators, measurement tools, measurement time points, and the level of analysis. We have provisionally selected patient participants' HbA1c level (in the effectiveness domain of RE-AIM) as the primary outcome and used it for statistical power calculation. We select this clinical outcome as the primary outcome to ascertain the clinical effectiveness of the optimized SMA intervention in our study. However, during the pilot phase, we intend to test all outcomes listed in Table 3 to evaluate their sensitivity to intervention, affordability, and feasibility of data collection. The pilot trial results may necessitate adjustments to the outcome sets, including the primary outcome.
In addition to measuring the outcomes, we will also gather data on various basic and process covariates and indicators, as summarized in Table 3. Throughout the program, we will use the Research Electronic Data Capture (REDCap) platform[34], a reliable data collection, storage, and management tool. A group of graduate students will monitor the data collection process and conduct quality control after the collection. The principal investigator will supervise the group members.
Table 3 inserts here.
Data analysis
Describing baseline characteristics and outcomes
We will conduct a descriptive analysis to examine the balance of baseline characteristics and compare raw outcomes across different groups. This will involve comparing demographic characteristics, cognitive and physical function, diabetes behavior, blood glucose level, and other baseline conditions of patient participants and the quality of care and health management checklist. The statistical method will be determined based on the data type and distribution.
Assessing the SMA effectiveness
For both the pilot and full optimization trials, our data will exhibit a hierarchical or multilevel structure: patients (level 1) within doctors (level 2) within institutions (level 3). This 3-level structure will present a potential correlation within clusters (i.e., different patients will visit the same doctor in the same institution). If the variability between doctors and institutions (i.e., random effects) proves statistically significant, we will use multilevel linear mixed modeling (MLM) [35].to conduct the effectiveness analysis. MLM is well-suited to handle this hierarchical data structure and can provide insight into the main effects of all SMA components and their potential interactions on the study's primary outcomes. For the coding of factor levels within the MLM, we will employ effect coding [36, 37], where factors exhibiting a higher level will be coded as "+1", and those at a lower level will be coded as "-1". The MLM will include cross-level interactions in sensitivity analyses, which refer to the interaction between a level-1 and a level-2 variable. If these interactions are significant, the relationship between the level-1 variable and the outcome depends on the level-2 variable. Should the random effects not prove statistically significant, indicating no substantial doctor or institution-level variability, we will use a simpler linear regression model instead of MLM. The treatment effect will be further estimated through two subgroup analyses, stratified by gender and patients' primary baseline outcome levels. All analyses will adjust for strong predictors of the outcomes, such as the baseline socio-demographic covariates and functional status [38]. We will conduct all analyses on an intention-to-treat basis, ensuring that all participants are analyzed in the groups to which they were randomized. Missing data will be handled through multiple imputation[39-41].
Analyzing contexts for the SMA implementation.
The purpose of this analysis is to determine the contextual conditions that are necessary for the successful implementation of SMA. We will gather data through surveys and qualitative studies (presented in the "Determinants" part of Table 3) to identify contextual factors, while implementation success will be measured by the extent to which SMA is implemented as intended or its fidelity. The analyses will use fuzzy-set Qualitative Comparative Analysis (QCA) (fsQCA)[42] and System Dynamic Modeling (SDM)[43]. QCA will use Boolean algebra to simplify and analyze the relationships between the contextual factors and the successful implementation of SMA both on implementation outcomes and clinical outcomes. The approach will allow us to identify the minimum set of contextual conditions required for successful SMA implementation. Similarly, the SDM method will be used to determine the essential factors for implementing SMA. However, the SDM is a way to study complex systems and qualitatively understand how different factors interact over time. The method involves creating a system simulation model using software R (deSove package) and then running the model to see how different factors affect the system's behavior. In our study, the SDM method will identify the key factors essential for implementing SMA: understanding their relationships and how they interact over time. By simulating the model, we can see how different factors affect the outcome of SMA implementation fidelity (See Appendix 4 for QCA and SDM methods details).
Economic Evaluation
We aim to compare the cost-effectiveness of various SMA configurations using a cost-effectiveness analysis (CEA) [44]. The analysis will consider two outcome variables: (1) the degree of improvement in patients' glycemic levels with different component combinations and (2) the cost per percentage of improvement in patients' glycemic levels. Appendix 4 provides a brief introduction to our approach to CEA.