The MCH CCT program
The MCH CCT project office, supported by the government of China, launched the MCH CCT pilot program in the late spring of 2013 in 40 townships (CCT townships) in three provinces: 11 in Gansu, 17 in Sichuan, and 14 in Yunnan. All villages within each CCT township were offered CCT services. However, it is important to note that not all townships in these three provinces were CCT townships. The purpose of the CCT program was to encourage eligible pregnant women and mothers to use maternal and children health services, increase the benefit of utilizing MCH services. This was expected to lead to better knowledge about maternal and child health and better child health outcomes. All the households from targeted town with pregnant women or neonates during the implementation of the CCT program were eligible.
The premise of the CCT intervention is simple. In a series of group meetings between eligible participants in a selected number of project villages in designated project townships, pregnant women were advised that, if they undertook any one of a set of seven MCH services—of which, five are free—then they would not only receive the benefits of the service that was provided, they would also receive a cash payment of a certain amount. The program was designed to make payments to the participants shortly after completion of each visit to the MCH service provider.
The list of services and the payment schedule were displayed prominently in all treatment townships. Eligible women were told that they would receive a separate payment each time they (a) underwent a prenatal examination; (b) delivered their baby in a hospital; (c) underwent a maternal postpartum examination; (d) engaged in early breastfeeding, that is, began breastfeeding after delivery within one hour; (e) breastfed exclusively for six months; (f) gave their child all required vaccinations; or (g) took their child to a child health examination. If a mother in a CCT project township completed all CCT activities, she would receive about 1,000 Chinese yuan (equivalent to 154 US dollars). This is a relatively large sum of money in the study area, given that the average annual income in 2013 was approximately 1,500 yuan per capita [7].
Study Design
Our study is not, strictly speaking, a cluster randomized control design, but instead uses a quasi-experimental design to make a comparison between those that received the treatment and those that did not receive the treatment in two dimensions. On dimension of the comparison is a “between-village” comparison; the other dimension in a “within-village” comparison.
For the between-village comparison, we received access to 25 CCT townships, including 9 townships from four counties in Gansu and 16 townships from five counties in Sichuan, an average of about three townships per county. One treatment village was randomly selected in each treatment township to generate a total set of 25 treatment villages. The corresponding 50 comparison villages were selected from 50 comparison townships respectively, which had similar ethnic, social, economic, and infrastructural characteristics to the treatment towns. A series of control variables were used to select comparison villages, including ethnicity (as measured by the share of Han ethnicity in villages); total village population; nature of the local township road (as measured by the presence of a paved road from the township to the village); share of families who were receiving income support or welfare; and average travel time from the village office to the township health center. In the study, then, there were 2 comparison villages (and sets of respondents) for every treatment village.
For the within-village comparison, we adopted three groups to present the status of the eligibility of CCT programs: a fully eligible group (women who were pregnant during the CCT implementation), a partially eligible group (women with newborns during the CCT implementation), and a non-eligible group (women who had delivered their babies before the CCT implementation). We therefore can compare the differences of these three groups within a village.
The sample villages were selected (and overall sampling protocol was implemented) in the Fall of 2014, 18 months after the launch of the CCT program (as indicated above in the Spring of 2013). Figure 1 depicts the location of the sample townships, counties, and provinces.
Study sample
Our sampling frame worked as follows. First, we went to the nine CCT project counties from the Sichuan and Gansu provinces. From the five counties in Sichuan, we went to 16 townships that were offering the CCT program (treatment townships), and 32 that were not (comparison townships). From the four counties in Gansu, we went to nine CCT towns and 18 non-CCT towns.
To choose the sample villages and households, we followed a pre-specified protocol that consisted of four steps. The first step was to take a between-village matching strategy. To do so, we randomly chose a set of treatment villages from among the 25 treatment townships that would contain households eligible to participate in the CCT program. After the treatment villages were randomly chosen, the second step involved choosing a set of comparison villages from among the 50 comparison townships. To improve the probability of having a good match, we chose two comparison villages for each treatment village. The assumption of our sampling strategy was that the two comparison villages, by nature of their proximity to the treatment villages, were likely to be close matches.
To select the comparison villages, we used secondary township-level and village-level statistics. Utilizing all available variables noted above for each village in each township, we identified one village from within each of the two comparison townships that were similar to our treatment villages. No significant differences were found between the comparison and the treatment villages (details can be found in Table 1). This means that each of the two comparison villages were statistically similar to the treatment villages in terms of the relevant township-level and village-level characteristics.
The sample included 25 treatment villages and 50 comparison villages, for a total of 75 study villages. The sample villages were selected, and the overall sampling protocol was implemented in the fall of 2014. The timing of the sample selection was carried out 18 months after the launch of the CCT program, which was spring 2013.
For the third step of the sample selection protocol, we chose study households in the treatment and comparison villages. The goal was to choose three different types of households. The first type was termed Fully Eligible households (FE households), meaning that the mother became pregnant after the implementation of the CCT program in the treatment villages. This means that she would have been able to take advantage of all services offered by the program. The second type was termed Partially Eligible households (PE households), wherein the mother became pregnant prior to the launch of the CCT program but did not deliver her child until after its launch in the treatment villages. This means she would have been able to access some, but not all, program services. The third type was termed Ineligible households (IE households), meaning that the mother both became pregnant and delivered her child prior to the launch of the CCT program. This would have barred her from being able to access its services even though she was in a CCT treatment village.
To select these households, we went to each village and consulted the roster of all babies from the village doctor or the women’s representative. We grouped babies into three types based on their dates of birth. Within each type of household, we randomly selected seven babies and their mothers to become our sample households. As such, we selected 21 households per village. At the time of the final evaluation survey, FE households had babies who were between 3 and 12 months old; PE households had babies between 13 and 18 months old; and IE households had babies aged 19 to 24 months old. Thus, at the time of the launch of the CCT program, IE households had babies who were between 1 and 6 months of age, while PE and FE households had mothers who were either pregnant or not pregnant yet, respectively. For clarity of exposition, we further classify households from treatment villages as FE treatment households, PE treatment households, and IE treatment households. Likewise, households from comparison villages are classified as FE comparison (or control) households, PE comparison households, and IE comparison households, matched by the same age period, in the fourth step.
In summary, the sample included 21 households (7 FE, 7 PE, and 7 IE treatment households) in each of the 25 treatment villages and 21 households (7 FE, 7 PE, and 7 IE comparison households) in each of the 50 comparison villages.
In total, the sample included 25 treatment villages and 50 comparison villages, for a total of 75 study villages. In addition, 21 households in each target village were selected based on their eligibility status for the CCT program. The assessment profile of this CCT program is depicted in Fig. 2.
Data collection
The research team conducted the survey in October 2014 in Sichuan Province and November 2014 in Gansu Province. The survey comprised five modules that were designed to meet our objectives of measuring the impact of the CCT program on the outcomes of interest: uptake of MCH services; knowledge of the mother about MCH issues; health outcomes of the child; participation in and receipt of the CCT payments in the CCT eligible households in the treatment villages; and information on individual and family characteristics that we used as control variables in the analysis.
The first module involved the collection of information needed to assess the uptake of MCH services. This included asking whether the caregiver of the child had participated in antenatal check-ups, in-hospital delivery, post-partum checkups, child physical checkups, or child vaccinations. Using our data, we constructed seven measures of MCH service uptake: made any antenatal care visit (1 = yes, 0 = no), baby delivered in hospital (1 = yes, 0 = no), made any postpartum care visit (1 = yes, 0 = no), early breastfeeding is defined as a practice of a woman that begins to breastfeed within one hour after delivery (1 = yes, 0 = no); exclusive breastfeeding is defined as a characteristic of a woman that fed her child for six months with only breast milk, without supplementing any complementary foods or infant formula (1 = yes, 0 = no), compliance rate of child physical examinations (%), and compliance rate of child vaccinations (%). The compliance rates of child physical examinations and vaccinations were calculated based on the requirements of the national standards of basic public health services, which has a required schedule of physical examinations and vaccinations a child should receive from birth to 6 years old. Therefore, the compliance rate is the actual frequency of physical examination and vaccination divided by the frequency of the national standards.
The second module concerned the assessment of the mother’s knowledge of the MCH services that she was asked to take her child to, as well as knowledge of infant nutrition. In this knowledge scale, there was a total of 22 items, scored by giving the respondent one point for each correct answer; a mother with complete knowledge would score 22 points. Appendix A1 provides an English translation of the knowledge test [see Additional file]. Using our data, we constructed five measures of mother’s knowledge about MCH services: total score on the 22-item knowledge test (full = 22 points), at least 60 percent of the 22-item knowledge test correct (1 = yes, 0 = no), score on items related to maternal care (full = 8 points), score on items related to child nutrition (full = 6 points), and thinking child physical exam were necessary (1 = yes, 0 = no).
In the third module, each sample child in both the treatment and comparison groups received physical examinations. Trained nurses, as part of the survey team, collected data on three indicators of health outcomes for each child. The three measures included hemoglobin concentrations, height, and weight. Hemoglobin levels were measured using HemoCue Hb 201+ systems (HemoCue Inc., Angelholm, Sweden). Following international standards for our sample age group, we defined anemia as a hemoglobin count of less than 110 grams per liter [27]. Height and weight measurements were obtained following the World Health Organization (WHO) standard protocol. The children were measured in light clothing without shoes, hats, or accessories. Height was measured using a standard tape measure. Weight was measured with a calibrated electronic scale (Tanita, HD-388, Japan). The nursing team was trained to set up the weighing station on level ground to ensure accuracy of the equipment. The anthropometric data were used to develop standard indicators of child development, such as length-for-age Z-scores (LAZ) and weight-for-length Z-scores (WLZ), based on international standards [28]. Using our data, we constructed four measures of child health outcomes, following WHO guidelines: low birth weight (1 = less than 2,500 g, 0 = 2,500 g or more), anemia (1 = hemoglobin less than 11.0 g/dl, 0 = 11.0 g/dl or more), stunted growth (1 = LAZ less than -2 standard deviations, 0 = -2 or more), and wasting (1 = WLZ less than -2 standard deviations, 0 = -2 or more).
Although all other modules were administered to all sample households regardless of their eligibility status, the fourth was not. For households in the treatment groups, namely, FE and PE treatment households, we had one extra module in which enumerators asked detailed information about their participation in the CCT program. Specifically, enumerators asked whether the mother registered for the program. Mothers were also asked to report the amount of cash that they had received for participating in the CCT MCH activities.
The fifth module of the survey was designed for the collection of information on various factors, statistically, controls, that might directly or indirectly affect the uptake of MCH services or health outcomes. The survey contained items for mothers about their child’s age, gender, ethnicity, gestational age, and pregnancy order. Enumerators also quizzed mothers about their own characteristics, including age, education, ethnicity, and occupation. A final set of items concerned overall household characteristics, including distance from home to the township health center, in terms of kilometers and travel time, and the nature of each household’s durable assets.
Statistical Analysis
A power analysis was conducted based on one of the main outcomes: the rates of hospital delivery. With the power of 0.8 to detect a difference in hospital delivery rates between the treatment and control groups in a cluster controlled trial, a suitable sample size depends on number of children per village, number of villages, probability of hospital delivery in treatment villages and controlled villages at a 95% plausible interval. According to our study design with 1 CCT village and 2 control villages, the total number of villages was 75. Based on previous studies [29], we assumed that hospital delivery rates were 50% in control villages and 60% in treatment villages. We then assumed a 95% plausible interval of 0.4 to 0.75. On the basis of these parameters, we calculated that we required 18 women per village. Considering the possible sample loss and assumed impacts, we added 3 women to each village to overpower the study when the budget allowed.
All statistical analyses were performed using STATA 12.0 (StataCorp, College Station, Texas, USA); p-values below 0.05 were considered statistically significant. We reported coefficients and 95% confidence intervals (CIs) for all main variables of interest. Comparisons between the treatment and comparison groups for all outcomes by subgroup populations were assessed using t-tests or chi-square tests.
To examine the effect of the CCT program on the uptake and knowledge of MCH services as well as on child health status, the evaluation used two dimensions of variation, i.e., between-village analysis (Evaluation Strategy 1) and within-village analysis (Evaluation Strategy 2). The first is cross-sectional and comes from comparing households with the same eligibility but from villages with different treatment status, namely, CCT treatment villages versus non-CCT comparison villages, utilizing Evaluation Strategy 1. Under Strategy 1, we estimated the impact of the CCT program, using the following least squares regressions model: (see Equation 1 in the Supplementary Files)
Yi is the outcome of interest for household i, including uptake of MCH services, knowledge of MCH services, and health status of children; CCTi is a dummy variable that indicates whether a household comes from a CCT village, which makes β the parameter of interest; and Xi is a vector of covariates that are included to capture the characteristics of children, mothers, and households. In all cases, we adjusted standard errors for clustering at the township level, using a cluster-corrected estimator.
The second dimension of variation is temporal and comes from comparing households that are fully or partially eligible (FE/PE households) against those households that are ineligible (IE households) for this CCT program under Evaluation Strategy 2. In this evaluation strategy, we estimated the impact of the CCT program, using the following least squares regressions model: (see Equation 2 in the Supplementary Files)
Note that the only difference between equation (2) and equation (1) is that we replaced CCTi with the dummy variable Eligibilityi, indicating whether a household is fully or partially eligible (FE/PE households). The rest of the variables are the same as described in equation (1). Together, Strategies 1 and 2 consist of comparing households whose children were born at different times—before, during, or after the launch of the CCT program—and by CCT status. The CCT program can be considered the treatment, and our sample households were divided into two treatment groups and a comparison/control group. The treatment groups include (a) the FE households in the CCT villages and (b) the PE households in the CCT villages. The comparison group includes all of the households in the non-CCT comparison villages (FE, PE, and IE households) as well as the IE households in the CCT villages.
A within-village difference for the first stage difference and between village but within-township difference for the second stage allows us to apply the difference-in-difference strategy to evaluate the effect of CCT on utilization of maternal health services and health outcomes. The baseline values for the difference-in-difference analysis were generated with the data of households that were ineligible for the CCT program.
We supplemented our intention-to-treat (ITT) multivariable analysis described above by examining the average-treatment-effects-on-the-treated (ATT analysis) to measure the impact on outcomes among the subpopulation of households who had heard about the CCT program. This allowed us to control for any confounding due to non-compliance, which we define as usage of MCH services without receiving a monetary transfer. For the ATT analysis, we utilized an instrumental variable (IV) approach [30], in which the treatment assignment (receiving CCT information or not) was used to account for observed compliance, or receiving a monetary transfer for using MCH services. This analysis is based on the assumption that the only reason for a woman in a CCT village to not receive a monetary transfer for using an MCH service is because she was unaware of the CCT program. The IV approach allows us to measure the average effect of treatment on the use of MCH services, mother’s knowledge, and child health outcomes among the subpopulation of households that knew about the program and, thus, control for confounding due to non-compliance. The ATT analyses for the continuous outcome measures were performed using STATA’s ivreg model. The ATT analyses for the binary outcome measures were performed using STATA’s ivprobit model. In estimating both models, we clustered the standard errors at the village level. STROBE [31] and BMC guidelines were used to organize our paper.