Study design and setting
We will test the effectiveness of the intervention using a parallel two-arm, cluster-randomised, controlled, superiority trial. We will run the trial in township hospitals of two rural counties in Shaoguan city, Guangdong province: Lechang and Nanxiong county. Shaoguan is in the north of Guangdong and remains one of the poorest areas in the province. In rural China primary care is provided by public township hospitals and private village clinics. Each township hospital covers 50,000 to 100,000 people and typically has less than 100 beds and 20-40 family physicians. Family physicians are medical doctors who receive formal medical training for either 3 or 5 years to practice acute care, and employed by the township hospital. Village clinics are run by one or two paramedics (called village doctors), who receive limited medical training equivalent to a high school level, and are guided by the township hospital. Village doctors are self-employed, primarily relying on three sources of income: the consultation fees paid by the rural health insurance, the Government’s public health package for preventative care, and income from private practice which are not properly documented.
Since 2013 the Chinese government has implemented a “zero mark-up policy”, which prevents township hospitals and village clinics from earning profits by prescribing or dispensing medicines on the Essential Medicine List. Only medicines on the Essential Medicines List, issued by the national and provincial governments, can be prescribed in township hospitals and village clinics. These medicines are purchased through an open bidding platform run by the provincial government. Village clinics have access to a much lower variety of medicines compared to township hospitals. Formally, village doctors are not allowed to prescribe antibiotics based on their practice regulations. However, in our previous work in Guangxi we found that some village doctors did actually prescribe antibiotics for common colds, and that villagers often visited village clinics for convenience but frequently got the wrong advice, which is likely to drive demand for antibiotics when they seek care for themselves or their children in township hospitals . For these reasons, village doctors need to be targeted by the invention as well.
We define eligible clusters as all willing township hospitals, and family physicians in those township hospitals, that have functional and extractable EMR, along with their associated village clinics, and village doctors in those village clinics, from within the two selected counties of Shaoguan. However, the two township hospitals and their associated village clinics selected for the pilot study (see below) will not be eligible for the trial. Preliminary work suggests all township hospitals and their associated village clinics within the two counties are eligible.
Our trial targets township hospitals, which provide the majority of primary care in rural China. Thus, we will collect all outcome data from eligible patients’ prescriptions issued within participating township hospitals. Similar to our previous trial setting, all outpatient consultations in China result in a prescription due to the request of health insurance scheme for documenting the consultation and the routine practice to give patients medications/ herbs after the consultation. We will not collect prescriptions from village doctors because there is not sufficient documentation (such as prescriptions) available in private village clinics. We define eligible patients as outpatients aged between 0 and 75 years-old who receive a primary diagnosis of an ARI and receive a prescription following their consultation with a family physician in a township hospital. This includes a diagnosis of any URTIs according to the International Classification of Disease 10th Revision (ICD-10), and acute bronchitis as an uncomplicated acute lower respiratory tract infections (LRTIs) (Table 1). However, patients and their prescriptions will not be eligible for inclusion in the trial if they are high risk patients diagnosed with either: 1) non-ARIs, 2) pneumonia, as it is severe and clinically challenging to group with other uncomplicated ARIs (but we will extend the invention in the future to include pneumonia), 3) chronic conditions including asthma, chronic obstructive pulmonary disease, non-infective or non-acute disorders (e.g., cystic fibrosis, pulmonary embolus, heart failure, oesophageal reflux and/or allergies), non-respiratory infections (e.g., cutaneous infections, urinary tract infections, trauma related infections, bacterial enteritis and/or cellulitis/abscess), immunological deficiencies, tuberculosis or any form of cancer.
The trial has obtained ethical approval from the Ethics Committee of the First Affiliated Hospital of Guangzhou Medical University, China (2019-53) and the University of Toronto Office of Research Ethics, Canada (38265).
In control arm clusters we will not make any changes or provide any inputs of any sort. Township hospital and village clinic providers will be allowed to continue treating patients with ARIs according to their existing usual treatment practices and guidelines, and as all data is collected from EMR there will also be no observable impacts for facility staff due to data collection.
Based on our previous intervention we have developed a comprehensive antibiotic stewardship programme to reduce inappropriate prescribing of antibiotics for ARIs in the general population within rural primary care settings . The intervention targets family physicians and patients in township hospitals, as well as the village doctors associated with township hospitals. The intervention activities are designed to fit within the existing policy requirements around antibiotic prescribing, which are detailed in Table 2. In all our training materials we define appropriate use of antibiotics as: 1) for all URTIs other than bacterial pharyngitis and sinusitis any use of antibiotics is considered inappropriate, and 2) for bacterial pharyngitis, sinusitis, otitis media and bronchitis, amoxicillin or penicillin are the first-line recommended antibiotics and are considered appropriate.
Before the trial we will test the feasibility of the intervention strategies and research design using qualitative and quantitative methods. Specifically, we will explore the feasibility and acceptability of the intervention strategies from the perspective of the hospital management staff and family physicians, village doctors and patients/caregivers. We will also explore the feasibility of the study design process, including recruiting family physicians and recording the number of eligible outpatient visits.
We will purposively select two township hospitals, with one township hospital randomly allocated to the intervention treatment and the other allocated to the control arm. We will also invite private village doctors associated with the intervention site for the initial training. We will test all interventions and study processes in one township hospital, while observing what the usual-care is in another. We follow the same informed consent process as outlined in the trial.
The pilot study will run for three months. For the pilot evaluation we will employ a mixed methods approach and collect data using in-depth interviews with family physicians in township hospitals and patients, as well as a questionnaire administered to all family physicians after training. The questionnaire methodology is very similar to that used in our process evaluation (see Process Evaluation section). The pilot evaluation is guided by the MRC framework for process evaluations . A planning matrix, adapted from our previous trial in Guangxi, can be found in Additional file 1 that illustrates our objectives, intended topics, information required, sources of information and methods for data collection. In the intervention cluster we will conduct in-depth interviews with two family physicians, two village doctors and one township hospital director, and we will conduct one focus group discussion with patients or their caregivers. We plan to do 5 in-depth interviews with family physicians and 1 focus group discussion with 2-3 patients or their caregivers during the pilot study. We will administer the questionnaire to family physicians before and near the end of the pilot study period. We will also perform structured observations to record training and implementation (e.g. consultations) processes during the pilot period.
Our qualitative methods adopt an interpretive description approach and we will employ thematic analysis as described by Braun and Clarke [32, 33]. We will analyse qualitative data as soon as possible after it has been collected. This approach allows for reflexive identification of themes from interviews and observations which can feed into subsequent interviews. For example, if interesting issues emerge they can be followed up in further interviews. As this is a mixed-methods pilot evaluation, our qualitative results will be analysed with our quantitative questionnaire results to provide feasibility and acceptability on both the intervention and control clusters, as well as on feasibility of meeting the sample size (of the prescriptions).
At the end of the pilot study period we will decide whether to continue with the full trial based on the analysis of the planning matrix. The key criteria would be based on: 1) having sufficient levels of recruitment to likely meet the required number of prescriptions for the trial, and 2) the intervention implementation is judged to be feasible: specifically it must appear feasible to train at least 60% of family physicians; 80% of family physicians who are trained employ the intervention guidelines at the end of the pilot evaluation period.
We will collect all outcomes and variables from EMR data on all prescriptions issued to eligible outpatients attending trial township hospitals during the 12-months before randomisation (the “baseline” period) and the 12-month trial period. We define all outcomes at the cluster-level and will calculate them from the individual-level prescription data for analysis as single summary outcome values (proportions/means) per cluster.
Our primary outcome is the proportion of prescriptions for eligible patients (aged 0-75 with a diagnosed ARI, excluding pneumonia and other complications – see eligibility criteria) that contain one or more antibiotics, which (as per related literature) we refer to as the antibiotic prescription rate (APR). Like previous trials [17, 34] we will use the APR to measure the proportion of unnecessary antibiotic prescribing/use, given that most patients with URTIs and acute bronchitis do not benefit from antibiotics.
We will also collect and create the following set of secondary outcomes, which will allow us to evaluate whether the intervention affects the proportion of antibiotic-containing prescriptions where the antibiotic(s) are of a specific class or method of delivery. The following set of indicators are based only on those eligible prescriptions containing at least one antibiotic. 1) The broad-spectrum antibiotic prescription rate: the proportion that contain one or more broad-spectrum antibiotics (we define broad-spectrum antibiotics as those antibiotics that act on the two major bacterial groups gram-positive and gram-negative, or any antibiotic that acts against a wide range of disease-causing bacteria). 2) The fluoroquinolones prescription rate: the proportion that contain one or more fluoroquinolone antibiotics. 3) The multiple antibiotic prescription rate: the proportion that contain two or more antibiotics of any kind. 4) The intravenously-injected antibiotic prescription rate: the proportion that contain any antibiotics delivered by intravenous injection. As the intervention will address appropriate use of antibiotics for bacterial ARIs, we will also measure, taking the total number of antibiotics prescribed per township hospital as the denominator, 5) the proportion that contain any antibiotics in the Access group of the WHO’s 2019 Essential Medicine List classification . As all these outcomes have denominators that only include eligible prescriptions containing one or more antibiotics, when analysed they will effectively be creating “outcome-based subgroups” that are defined post-randomisation . This is a common but often unrecognised issue that can introduce bias into the treatment effect estimates for such outcomes. However, there are no clearly applicable/feasible solutions for our situation , and so we will treat the results for these specific outcomes as exploratory.
We will also collect and create the following secondary outcomes based on all eligible prescriptions. In our previous trial, we observed increased use of Traditional Chinese Medicines, likely as alternatives to antibiotics, and a wide spread misuse of glucocorticoids . Thus, we will also measure: 6) the proportion containing any Traditional Chinese Medicines, and 7) the proportion containing any glucocorticoids. Lastly, to evaluate if and how the intervention affects the mean cost of prescriptions issued to eligible patients we will also measure: 8) the average cost of a prescription, based on the cost of any medicines, and 9) the average cost of a consultation (1 per prescription), based on all costs including medicines, tests and the consultation.
Patient safety indicator
We will evaluate whether the intervention appears to increase adverse events, for example because of antibiotics being withheld for appropriate conditions more frequently due to the intervention, by creating and evaluating an indicator of unintended harms. To do this we will use the EMR to track whether any patients subsequently become hospitalised in any hospital in the Shaoguan Prefecture, including its county and prefectural level teaching hospitals, due to respiratory infections or sepsis, within 30 days of their index visit to a participating township hospital during the trial period. This will allow us to calculate a cluster-level hospitalisation rate for trial participants as the number of hospitalisations for respiratory infections or sepsis per 100 outpatient consultations. We will compare both patient safety indicators between treatment arms.
Our sample size is calculated based on our primary APR outcome. Previous exploratory work suggested the existing township hospital APR level to be around 80%. We assume that the treatment will reduce the intervention arm APR by more than 15 percentage points during the trial period, based on how effective of previous related trial was ., which is viewed as the minimum clinically important effect in the context of the extremely high existing APR levels. Based on our previous trial  we assume the intra-cluster correlation coefficient will be 0.14 in the intervention arm and 0.09 in the control arm here, and based on our exploratory work we assume that we will be able to collect at least 500 eligible prescriptions per township hospital during the trial period, which we assume as our fixed cluster size. Based on these assumptions we estimated that we will require 17 clusters per arm to detect a 15 percentage point or greater absolute reduction in the APR with 80% power using a two-sided hypothesis test with an alpha of 0.05 . Because our previous trial data indicated potentially important county-level differences in baseline and during trial period APR levels and in the magnitude of the treatment effect we also plan to stratify our randomisation by county. Therefore, this sample size also accounted for an unequal allocation ratio within strata, and specifically given the number of clusters available in each county it accounted for a within-county intervention-to-control allocation ratio of 8:9, which we chose in favour of the control arm for logistical purposes.
Recruitment, randomisation and blinding
We will select and seek to recruit all eligible township hospitals and their associated village clinics (i.e. clusters) in the two study counties. There are 17 eligible township hospitals in Lechang county and 17 in Nanxiong county, but we will exclude the two township hospitals from Nanxiong county that will be part of the pilot (see below). We will seek written informed consent from the director of each township hospital on behalf of family physicians in township hospitals and the use of their EMR data. We will also seek written informed consent from all village doctors when participating in the initial training session. We will collect de-identified patient prescriptions from routine EMR system, so individual patient consent is not necessary. Following recruitment the study statistician (JPH) will randomise all recruited township hospitals at the same time, stratified by county, to the intervention or control arm in an overall 1:1 ratio, but an 8:9 intervention-to-control arm ratio within each county, using a simple custom-written computer program in R . Township hospital family physicians and village doctors in intervention group will then be invited to attend the training.
As the intervention will be applied at the cluster level all patients visiting township hospitals during the trial period will receive the treatment allocated to their cluster. Due to the nature of the intervention it will not be possible to blind providers or patients/caregivers to treatment allocation. However, we will blind the assessors of the adverse events outcome (see below), while all other data will be collected from routine databases, and we will also blind the data analysts.
Data collection and management
Clinical consultations and prescriptions
In township hospitals a prescription is required for each consultation to record the clinical visit to enable patients to receive reimbursements from the rural health insurance scheme. Therefore, although in theory patients may visit a family physician but not obtain a prescription this is rare in practice because patients do not feel they are “being taken care of” without receiving medication(s) . Also, even when outpatient visit costs are not covered by the health insurance scheme patients still prefer to register their visits because the scheme covers much of their consultation costs .
Primary and secondary outcomes
Following the baseline and the trial periods we will extract all eligible patient prescription data from the EMR for all township hospitals. The Prefecture Health Information Centre manages all EMR, and will provide encrypted and de-identified electronic prescription data (e.g. names, addresses and health insurance numbers will not be collected). We will then clean the data and enter it into a standard format database to store all outcome and covariate data (including diagnoses, medicines prescribed, medicine and consultation costs, plus patients’ age, sex, insurance status, date of visit, any related treatments, payment details, and any re-visit or hospitalisation data). We will also collect relevant covariate data from all township hospitals, including family physicians’ age, sex, experience and qualification level, which we can be link to each prescription via unique prescriber IDs.
To allow us to exclude patient prescriptions from outcomes where the patients have excluding comorbidities, we will also record any comorbidities as per secondary diagnoses shown on prescriptions. We will also check medications listed on prescriptions to link them to possible diagnoses. For example, steroid inhalers with asthma or chronic obstructive pulmonary disease, anti-diabetic medications with diabetes, and anti-hypertensive medications with hypertension. The Prefecture Health Information Centre will use the encrypted personal health insurance numbers in each prescription to identify any long-term comorbidities recorded such as diabetes, hypertension, cardiovascular disease. We will also develop a list of keywords used locally in prescription diagnoses for respiratory infections (which are often symptom based), and will manually screen and code all prescriptions not containing an ICD-10 code of diagnosis.
Patient safety indicators
We will collect all inpatient charts from the EMR managed under the Prefectural Health Information Centre for any patients who are hospitalised (in any hospital within the Shaoguan Prefecture, including its county and prefectural level teaching hospitals) within 30 days of their index visit to a trial township hospital for an ARI or for sepsis. The Prefectural Health Information Centre will provide de-identified patient charts for review after identifying patients as having visited a trial township hospital within 30 days of becoming hospitals, using patients’ encrypted insurance numbers. We will then have these inpatient charts reviewed by a group of 3-5 physicians (blinded to the township hospital of all patients) who will decide if the hospitalisation is due to one of the following reasons: 1) not providing antibiotics during the index visit, 2) other inappropriate usage of antibiotics, such as prescribing the wrong antibiotics, 3) side-effects from antibiotics or other medications, and 4) undetermined. Any patients who are determined to have been hospitalised due to either the first or second of these reasons will be included in the calculation of the adverse event outcome.
We will report all results according to the “Consolidated Standards of Reporting Trials: Extension for Cluster Trials” (CONSORT) guidelines . Prior to the end of the trial and before the trial dataset is created we will produce a full statistical analysis plan pre-specifying and detailing all planned analyses. In summary though, in our main results paper we will present appropriate descriptive statistics for all relevant patient, family physician, township hospital and village clinic characteristics, along with appropriate summary statistics and their associated 95% confidence intervals for all outcomes by treatment arm at baseline and during the trial. Then for all outcomes we will produce a main set of estimates of treatment effectiveness using cluster-level methods of analysis suitable for cluster trials. This set of estimates will be used as the primary evidence for determining how effective the treatment appears to be for the primary and all secondary outcomes. There will no interim analyses.
For all outcomes these analyses will follow a two-stage process . For the primary outcome and all secondary proportion-based outcomes we will first fit a multiple logistic regression model to the relevant individual-level binary variable from the prescription data, adjusting for (likely) influential individual- and cluster-level covariates, including the cluster-level value of the outcome during the baseline period (we will fully define all adjustment covariates in the analysis plan). We will then use the model’s residuals to create the cluster-level outcome as covariate-adjusted cluster-specific proportions. We will then estimate the treatment effect based on the covariate-adjusted cluster-specific proportions using a stratified (by county) independent t-test to compare the cluster-level outcome values in each arm. We will repeat this process for the continuous outcome but using a multiple linear regression model to do the initial covariate adjustment. By estimating covariate-adjusted treatment effect estimates we will reduce the risk of bias in our results due to imbalances in the cluster-randomisation, and we will increase the precision of the estimates. By ultimately estimating treatment effectiveness using a t-test, for our cluster-level proportion outcomes we will estimate covariate-adjusted risk differences (i.e. absolute differences in cluster-level outcome proportions), as recommended by CONSORT, and for our cluster-level continuous outcome we will estimate treatment effectiveness as a covariate-adjusted mean difference. We will base our inferences about the effectiveness of the treatment on the outcomes by interpreting the 95% confidence intervals around our outcomes’ treatment effect estimates along with the associated two-sided t-statistic based p-values. We will adjust all confidence intervals and p-values for our secondary outcomes for “multiple comparisons” using the Holm method [43, 44].
For our main analyses we will including all clusters originally recruited into the trial as per their original treatment allocations, and all patients who are originally established as being eligible and having received treatment in a trial township hospital during the trial period. We will ensure that all eligible patients who received treatment at a trial township hospital during the trial period are included in our main analyses by dealing with any missing outcome and/or covariate data (used in the two-stage process of adjusting for covariates) using multilevel multiple imputation methods combined with our cluster-level methods of analysis, if needed .
To explore the robustness of our main analysis results we will also do a range of sensitivity analyses. Specifically, we will also produce a set of treatment effect estimates for all covariate-adjusted cluster-level outcomes but without any multiple imputation of outcome or covariate data (used in the two-stage adjustment process), so that only patient prescriptions that include all outcome and covariate data are included in the analyses (these analyses will also include all clusters, as originally recruited and as per their original treatment allocations, and all patients treated during the trial period with any missing outcome or covariate data imputed). Then we will also produce a final set of treatment effect estimates for all cluster-level outcomes but without any initial covariate adjustment, but with imputation of any missing outcome data (and again including all clusters, as originally recruited and as per their original treatment allocations, and all patients treated during the trial period with any missing outcome or covariate data imputed). We will adjust the confidence intervals and p-values from each set of sensitivity analysis results for multiple comparisons as per our main analyses to allow comparison.
Lastly, conditional on obtaining “statistically significant” results for our primary outcome, we will also do a small number of pre-planned subgroup analyses of the primary outcome, which will be fully detailed in the analysis plan. These subgroup analyses will be adjusted for the same range of covariates as the main analyses, and will also include all clusters, as originally recruited and as per their original treatment allocations, and all patients treated during the trial period with any missing outcome or covariate data imputed.
We will conduct a mixed-methods, theory-driven process evaluation (PE), guided by the MRC’s 2008 framework  and Grant’s framework for process evaluations of cluster randomised trials of complex interventions . Our PE will be based on the underlying programme theory, namely the social cognitive theory, which views behaviour change and maintenance as a dynamic and reciprocal relationship determined by the person, their environment (their external social context) and their behaviour (their response to stimuli to achieve goals) . Our PE aims to offer an exploration of ‘what worked, for whom and why’ in the implementation of the intervention . Our specific objectives for the process evaluation are: 1) to describe the health system and service delivery context in which the intervention was delivered, 2) to examine recruitment processes, both at the cluster level (township hospital) and the individual level (patient consultations and prescriptions), 3) to report on intervention fidelity, both at the cluster level (training) and the individual level (provider delivery), and 4) to explore the responses to the intervention both at the cluster level (managers and providers) and the individual level (patients). Methods will include document review (e.g. recruitment records, meeting minutes), structured observation of trainings and consultations in the township hospital, questionnaires with township hospital family physicians, and interviews of more than 50 participants including 6 township hospital directors, 18 township hospital family physicians, 12 village doctors and 18 patients/their caregivers, all distributed between the intervention and control arms at the ratio of 2:1. We will also observe 4 training sessions in the intervention arm. In addition, we will conduct 12 observations of clinical consultations in the intervention arm and 6 in the control arm. The sample size of qualitative study is purposively set according to our previous trial in Guangxi and will be adjusted during the study. Our qualitative methods will be guided by an interpretive description approach, which focuses on developing knowledge to inform clinical practice . We will develop a sampling frame, purposively select participants for inclusion and collect data at the 3, 6 and 12 months after the start of the intervention in four clusters in the intervention arm and two clusters in the control arm.
Analysis of process evaluation
We will analyse qualitative data from the PE using a framework analysis as described by Gale et al. to identify themes related to our study objectives . This approach allows for inductive discovery of new themes outside of our framework during analysis. We will transcribe and translate into English for all qualitative data, and do our analyses using NVivo 10 software. We will report our qualitative work following the Consolidated Criteria for Reporting Qualitative Research (COREQ) guidelines .
We will conduct an incremental cost-effectiveness analysis along with the trial. The primary outcome in this costing study is the cost per percentage point decrease in the APR (as defined in the Outcomes section) in the intervention arm compared to the control arm. We will compare direct costs and outcomes of patients randomised to the intervention arm compared to the usual care arm over the 12-month time horizon of the trial. We will not discount of costs and benefits due to the short period of the trial . The perspective adopted for the analysis will be that of the healthcare provider.
We will develop a questionnaire to collect data on the resources used to deliver the intervention, which we will aim to administer to the directors of all 34 township hospitals between the 9th and 12th month of the trial. We will collect information on: 1) average salaries, in RMB, for each level of staff in their hospitals, 2) the duration of consultations, 3) the amount of time spent reviewing prescriptions in preparation for the prescribing peer-review meetings, and 4) the frequency and duration of peer-review meetings and the staff involved in each process. We will ensure data quality by using double entry and by checking a random subset of the data.
Estimation of resource use and costs
Healthcare resource use includes patient visits to the health facility. We will include the cost of consultations, medications, medication reviews and subsidies. The total cost per patient will be calculated as the sum of the three elements: consultation, medication and medication reviews. We will calculate the total costs to the healthcare provider (township hospital) for the main analysis population (see Statistical Analyses section) accounting for clustering and stratification .
Estimation of implementation costs
Implementation costs represent upfront costs and are estimated and reported separately, and will not be included in the cost-effectiveness analysis. However, policy makers would need to consider these costs when deciding whether to implement the intervention at scale. We will calculate implementation costs for the software development, including the development of the WeChat app function and the EMR improvement. Then for each cluster we will calculate them as the sum of: the cost of a trainer to deliver training on the appropriate use of antibiotics when treating acute RTIs, the cost of staff time to attend training, plus the costs of producing one handbook and one set of guidelines (used as information aids in consultations) per training attendee, and the educational videos (displayed in waiting areas) and posters (displayed around the hospital).
As with our previous trial  we will continue to follow-up both intervention and control arm clusters for another 24 months after the trial ends to monitor the possible longer-term impact of the intervention. It will be at the discretion of individual township hospitals whether to continue the intervention activities after the trial has ended. We will investigate to what extent the township hospitals are willing to continue the intervention activities and the long-term influence on family physicians’ behaviors without any interventions efforts provided by the research team. All analyses will be fully pre-specified in the statistical analysis plan.
Prof Xiaolin Wei from the University of Toronto and Dr. Chao Zhuo from the Guangzhou Medical University will be the co-guarantees of the trial who have full access to the trial dataset. We will establish a data management committee (DMC) led by external/independent members to safeguard the safety and privacy of patients involved, and to ensure that all data are collected according to agreed ethical guidelines, properly stored and only used for research purposes. We will also form a trial steering committee (TSC) led by external members. We will organise teleconference meetings for both the DMC and TSC at the beginning of the study, and then every six months until the study completes. The committees may also meet on an ad hoc basis should the need arise. During meetings we will discuss any protocol modifications. We will also establish a trial management unit (TMU) in Shaoguan, to manage the day-to-day activities of the trial, consisting of three research associates employed locally and two research associates from the University of Toronto.