Implementation Research for the Evaluation of the Child Health Education and Surveillance Tool Application: A Randomized-Controlled Trial

When community health workers (CHWs) are effective, they can help teach essential knowledge of healthy child rearing practices in their communities. An effective mHealth tool can potentially improve the performance of CHWs and create greater child health and development outcomes. The Child Health Education and Surveillance Tool (CHEST App) was developed and implemented in a CHW program in the Amazon of Peru to improve CHW performance and early childhood development. The intervention was developed and implemented with the guidance of several implementation research tools. This study presents the results of the intervention METHODS The CHEST App intervention was evaluated using an effectiveness-implementation hybrid type 2 study design. A group of communities (8) in Loreto, Perú were matched by propensity scores and then randomly assigned to an arm of the study. The implementation outcomes were evaluated with a mixed-methods approach to how well the intervention was installed into the local CHW program. The effectiveness of the intervention was determined by assessing knowledge scores, hemoglobin gm/dL, early childhood development motor scores and cognitive scores. An independent samples t-test and one way ANOVA analysis were conducted with each intervention outcome to assess differences between the intervention and control groups. The results of the study are presented within the framework of the Implementation Research Logic Model. intervention associated with an increasing or decreasing risk of each outcome. A one-way analysis of variance (ANOVA) was conducted to determine if the mean of the outcome variables are different in the three groups (0, 1-2, or 3+ home visits). If the one-way ANOVA showed a statistically signicant difference between groups than a Tukey post hoc test was conducted to determine where any differences lie. The following null hypothesis was tested by the one-way ANOVA for each of the intervention outcome variables (H 0 2.1: knowledge H 0 2.2: hemoglobin H ECD motor H 0 2.4: ECD

and delity. However, adoption and delity of the surveillance function of the CHEST App by program coordinators was not achieved.

CONCLUSION
The CHEST App intervention is a promising tool to improve the performance of CHWs during their home visits and their ability to teach caregivers healthy child rearing practices.

Contributions To The Literature
The current study is exemplary for how implementation research methods and tools can be used to design, evaluate, and report a pilot study, to improve learning and replicability.
The research presented here provides important generalizable knowledge about the feasibility and value of providing community health workers a digital tool to improve their capacity to teach caregivers healthy childrearing practices.
Animated videos, automated message selection, and the novelty of an electronic device are associated with increase knowledge acquisition by caregivers in rural Amazonian communities.

Background
Children around the world continue to suffer from ailments such as malnutrition and developmental delay due to unhealthy practices in the household. [1][2][3][4] A large portion of the ailments could be alleviated by the adoption of better sanitation practices, diet, and disease prevention, however, caregivers' knowledge of healthy practices can be limited due to poor education and health promotion efforts in impoverished settings. [5][6][7][8][9] Effective transmission of knowledge of healthy practices has been di cult to achieve at scale in rural communities, but when accomplished it can substantially improve child health and development. 10 The utilization of community health workers (CHWs) to conduct health promotion and education in their communities has been shown to be effective, though outcomes vary greatly. [11][12][13][14] Effective, consistent, and scalable implementation of CHWs programs have been elusive. 14,15 The utilization of information and communication technology (ICT) and mHealth technology have also been shown to be effective at improving health behavior in the household. [16][17][18][19][20][21][22][23][24] However, effective implementation of ICT tools in rural settings has also been varied, and most populations in impoverished settings have not received the bene ts of knowledge transmission through modern technology. 25 Ineffective implementation of evidence-based interventions, such as CHW programs and ICT tools for behavior change, is partly due to an imprecise understanding of what has been done, what has worked, and what has failed. The implementation of innovations needs to be mapped, evaluated, and reported with su cient detail to support continuous learning and improvement. [26][27][28] The current study reports on the implementation and impact of an mHealth innovation to improve CHW performance in the Amazon of Peru. The implementation research was conducted to generate data on the success and failures of the implementation strategies to install the innovation and the feasibility of the intervention to function effectively in the rural setting.
The complete study protocol is described in the article, Westgard, et. al., 2018. 29 The speci c stages of implementation (based on the Active Implementation Frameworks 30,31 ) and implementation strategies that were used to install the intervention are described in the article, Westgard, et. al., 2020. 32 The intervention was implemented in the northern Amazon region of Peru, in the department of Loreto. In Loreto, 57% of children under 3 have anemia, 20% under 5 have chronic malnutrition, and 26.7% of children are developmentally delayed. 33,34 Caregivers' knowledge of healthy childrearing practices are limited as they transition from traditional practices to modern medicine. [5][6][7]9 In many rural communities in Peru, CHWs conduct home visits with new mothers to teach them health topics such as sanitation, diet, disease prevention, and early childhood development. However, the impact of the CHW programs has been hindered by poor performance of CHWs and a lack of effective educational material. The CHWs are often unequipped and under-trained to successfully transmit the information to the caregivers and convince them the importance of the behavior change. 15,35,36 To address the underperformance, the research team at the Peruvian research organization, Elementos, developed a tablet-based application and animated videos to support the CHWs in their task to teach caregivers key health messages and collect child health indicators during home visits. The innovative tool is titled, The Child Health Education and Surveillance Tool Application (The CHEST App). A video of the CHEST App and the animated videos that accompany the App can be viewed online. 37 The primary objectives of the CHEST App intervention are to improve early childhood development (ECD) scores, reduce anemia rates, and reduce chronic malnutrition rates. The theory of change of the intervention is displayed in Appendix 1 and the Conceptual Model of the intervention is displayed in Appendix 2.
The pilot study was interrupted by the COVID-19 pandemic, causing CHW program shutdowns and thus a signi cant loss of program participants and sample size. After 6 months of shutdowns, one community in the intervention group and one community in the control group reactivated their CHW programs. The two communities were matched with propensity scores before assignment to the intervention and control study group.

Study Design
The implementation process and CHEST App intervention are evaluated using a Hybrid Type II evaluation study design. 38 In a Hybrid Type 2 study, the implementation process is evaluated alongside the intervention (the CHEST App) to determine what mechanisms are responsible for the changes associated with the intervention. 38 The study took place in the Amazon of Peru, department of Loreto. The study initially included 8 communities that were matched 1-1 with propensity score matching and then randomly assigned to the intervention or control group. Both communities are similar in size (8-10,000), access to health facility (Level I-2 in community), distance to province capital (1.5 hours in fast boat), rates of anemia (25-32%), and structure of CHW program. The study was reduced to one intervention group (Indiana) that continued to use the CHEST App after COVID and one control group (Tamshiyacu) pair that continued their CHA activities. The intervention group's CHWs received the CHEST App and the control group's CHWs did not. The CHW programs had to randomly select children in their communities to receive home visits when they were not able to visit all children aged 0-3 years. The intervention group included the children and caregivers that received a home visit with the CHEST App. The control group included all children in both communities that did not receive the CHEST App. Most of the participants at endline were not the same as the participants at baseline, although they lived in the same community. Many of the children that participated in the baseline assessment aged out of the program by the endline assessment. Therefore, a ANCOVA analysis could not be used to assess impact.
A baseline survey was conducted prior to implementation of the intervention and an endline survey was conducted 16 months following implementation, in the intervention and control communities. The research team conducted a consensus in each community to survey all children ages 6-35 months, regardless of if they received home visits by the CHWs. A detailed description of the selection process and study population is included in the published study protocol. 29

Implementation Outcomes
The evaluation of the implementation of the intervention was conducted with a mixed methods approach to identify the extent to which the intervention was successfully adopted and incorporated into the local CHW program. The implementation process, including the barriers and adaptations, is described in the publication by Westgard, et. al., 2020. 32 Reporting the implementation process was done to allow others (researchers and program implementers) to understand what activities took place during the implementation of the intervention that may have in uenced the success or failure of the intervention. 39 The implementation research reported here follows the C.A.R.E. guidelines (Consolidated Advice on Reporting ECD Implementation Research) to ensure the necessary information is included in the evaluation. 40 An Implementation Research Logic Model (IR Logic Model) was used to design the evaluation process and specify the relationship between the determinants of implementation, implementation strategies, and the implementation, service, and clinical outcomes (Appendix 3). 26 The implementation outcomes were selected based on the conceptual framework presented by Proctor, et. al., 2011. 41 The implementation outcomes serve as a precondition for attaining the desired changes in service outcomes and clinical outcomes. The implementation outcomes relevant to this study include, acceptability, adoption, delity, and cost. Other implementation outcomes were not measured because the intervention had not yet reached the point in its maturation to measure them with con dence, including, feasibility, penetration, and sustainability. An additional implementation outcome, dosage, was also measured. Acceptability was measured to identify the extent to which the implementation stakeholders (caregivers and providers) perceive the intervention to be satisfactory. 41 Both the CHWs and caregivers in the intervention group received a semi-structured interview to determine their level of acceptance of the CHEST App. The quantitative portion of the survey was analyzed and reported using descriptive statistics. The qualitative responses were analyzed by identifying a set of sub-themes from the responses then assessed for similarities and differences in perspectives of the participants. Key quotes that best re ect the position of each actor group are reported.
Adoption was measured to determine the uptake of the intervention and continued use by the providers throughout the study period. Adoption indicates that when the CHW realized a home visit, they used the CHEST App to do it. It was measured by analyzing data that was collected with the CHEST App and then uploaded to the server. The number of home visits conducted with the CHEST App was compared to the number of home visits the CHWs were reported to complete each month, to determine a percentage of adoption. Also, the CHWs received a semi-structured interview to provide them the opportunity to describe the extent to which they use the CHEST App during their home visits, and the components of the CHEST App they use with consistency. The caregivers in the intervention group also received a semi-structured interview to determine the number of home visits with an electronic tablet they have received. This indicator was used as the independent variable of the study. The responses from the CHWs and caregivers are reported with descriptive statistics. The program coordinator of the CHW program received a semi-structured survey pertaining to the adoption of the CHEST App and the use of each of its components. The response to the survey is summarized.
Dosage is a measure of the total number of home visits realized with the CHEST App during the study period. The program was expected to conduct home visits with at least 100 children and provide an average of 10 home visits with each child. Dosage is be measured to determine to what extent the intervention reached the recipients. It will be calculated by dividing the number of home visits conducted during the program by the expected number of home visits (1000) to be conducted.
Fidelity was measured to determine the degree to which the intervention was delivered as prescribed. The research team conducted observations of home visits with the CHEST App to determine if each component of the CHEST App was being delivered with quality and as intended in the intervention protocol. The observers had an observation checklist to mark the completion, or incompletion, of each component. The results of the checklist are reported as the measure of delity. The delity measure instrument (in Spanish) can be seen in Appendix 4.
Implementation cost is reported to provide the nancial impact of the implementation process. The costs include equipment, training, and supervisory services. The costs are broken down to provide a per CHW cost and the cost to implement the intervention with 20 CHWs. The costs to development of the App and educational material are not included.

Intervention Outcomes
The effectiveness of the intervention was determined by measuring differences in the service outcome (intermediary) and clinical outcomes between the intervention and control group. The service outcome was the knowledge scores by caregivers. The clinical outcomes were the hemoglobin gm/dL levels and early childhood development (ECD) scores (motor and cognitive). To determine if the intervention and control groups were similar at baseline, an independent t-test analysis was conducted on each intervention outcome. To assess the impact of the CHEST App, two analyses were conducted on the outcome variables. First, an intention-to-treat analysis was conducted with the treatment variable coded as 1 or 0 (1= 1 or more home visits with the CHEST App, 0= no home visits with the CHEST App). An independent t-test was used to determine if the mean of the outcome variables (knowledge, hemoglobin, and ECD scores) are signi cantly different in the two groups (received CHEST App vs. did not receive CHEST App). The following null hypothesis was tested with the independent t-tests for each of the intervention outcome variables (H 0 1.1: knowledge scores, H 0 1. Secondly, a dose-response relationship analysis was conducted with the treatment variable coded as three categories: 0 home visits with the CHEST App, 1-2 home visits with the CHEST App, 3+ home visits with the CHEST App. The relationship was tested to identify if an increase level of exposure to the intervention is associated with an increasing or decreasing risk of each outcome. 42 A one-way analysis of variance (ANOVA) was conducted to determine if the mean of the outcome variables are different in the three groups (0, 1-2, or 3+ home visits). If the one-way ANOVA showed a statistically signi cant difference between groups than a Tukey post hoc test was conducted to determine where any differences lie. The following null hypothesis was tested by the one-way ANOVA for each of the intervention outcome variables (H 0 2. H 0 2: The outcome variable for those in each of the three groups (0 home visits, 1-2 home visits, 3+ home visits) will be signi cantly different at endline.
The knowledge score represents the service outcome of the study, as shown in IR Logic Model (Appendix 3). The service outcome of the study re ects the effectiveness of the intervention to improve CHW performance. 41 If CHWs are effective at delivering their intended service, the caregivers will have greater knowledge of healthy childrearing practices. Improved knowledge is expected to lead to improvements in the clinical outcomes (anemia and ECD scores). 43 Knowledge scores were measured by an opened-end questionnaire with caregivers, at baseline and endline. The questionnaire was designed to give the participant ample opportunity to describe what they know about each question. The surveyors were trained to ask the participant the survey question, then probe the participant to provide further information. For example, the participant was asked, what are the bene ts of breast milk for a child, then follow-up with statements such as, "what other bene ts" and "have you heard of any other bene ts". Probes continued until the participate indicated that they do not know any further information. For each correct answer, the participant received a point. The points were totaled to provide the knowledge score for the participant. The questionnaire included 15 questions with a total possible score of 91. The topics included, nutrition, sanitation and hygiene, disease prevention, and early childhood development. The knowledge questionnaire (in Spanish) can be seen in Appendix 5. ECD was evaluated with the Caregiver-Reported Early Development Instrument (CREDI), an instrument that utilizes caregiver-reported data to assess the ability of the child to perform age-related developmental activities. [44][45][46][47][48][49] The CREDI has been validated as an effective instrument to evaluate ECD in 17 low-and middle-income countries. 49 The CREDI is conducted by asking the caregiver if the child is able to complete a list of age-appropriate activities that become progressively more di cult. The instrument was used to assess motor development and cognitive development. As recommended by the CREDI manual, a raw score was calculated for each development measure using the statistical software, R 50 , and the script package provided by the CREDI instrument. The raw scores were used as the ECD motor and ECD cognitive scores for the analysis.
A SPIRIT Checklist was completed to ensure the manuscript includes all research reporting components. The SPIRIT Checklist is included in Appendix 6.

Implementation Outcomes
The results of the implementation research can be seen in the IR Logic Model in Appendix 3. The evaluation of the implementation outcomes included 41 caregivers and 6 CHWs that received the intervention.

Acceptability by Caregivers
Most caregivers (92%) that received home visits with the CHEST App expressed that they prefer the App than use of traditional methods (pen, paper, and ipcharts). 5% expressed that they had no preference and 3% expressed that they prefer traditional methods of home visits.
When caregivers were asked if they learned the health messages delivered during the home visits better with the CHEST App or without the CHEST App, 84% expressed that they learned better with the CHEST App, 13% expressed that they learned the same with or without it, and 3% (n=1) expressed that they learned better with traditional methods. When asked what they liked most about receiving home visits with the CHEST App, all caregivers answered that they most enjoyed the educational component of the App. Many (43%) speci cally mentioned the animated videos as a reason why they prefer the CHEST App to traditional methods. For example, a mother in the community of Indiana said, "I like that they show use the videos and how to feed the children. You learn better because you can see how to do it through the animations".
(Me gusta que nos hacen mirar los videos y la alimentación de los niños. Se aprende mejor porque se ven como se hace para aprender mediante dibujos) Acceptability by CHWs All CHWs expressed that they prefer to conduct the home visits with the CHEST App than their traditional methods (pen, paper, and ipcharts). When asked what they liked about using the App, the CHWs reported that they most liked learning from the educational material, the animated videos, and the child health indicators displayed in red or blue. The animated videos were the most cited reason that they liked the CHEST App and how it helped them better conduct their home visits. A CHW in the community of Indiana said, Yes, it (the App) has everything summarized and is faster. They like the videos. They laugh and understand more quickly. The child points. They like it a lot.
(Si. Tiene todo resumido y mas rápido. Les gustan los videos. Se ríen. Aprenden más rápido y el niño apunta. Les gusta mucho) The CHWs expressed that the most di cult part about using the CHEST App was sending the data and registering the information in the tablet.

Adoption by CHWs
The CHWs were registering health indicators of 140 children per month with the CHEST App, the same number of children they were assigned to visit.
Thus, adoption was con rmed to be 100%., meaning the App was used by the CHWs during every scheduled home visit. After 18 months of use, the CHWs expressed that they continue to use the CHEST App during all their home visits. The caregivers veri ed adoption of the CHEST App by CHWs by reporting the number of home visits they have received, and number of home visits received with an electronic tablet. The results were similar (8.6 vs. 7.8 visits with tablet), indicating that the CHWs use the CHEST App during their home visits. All the CHWs reported that they use all the App's functions when conducting a home visit (share health indicators with caregiver, register health indicators, scheduling, educational images, and animated videos).
Adoption of the CHEST App was hindered due to the cancelation of the CHW programs in several communities, both before COVID and during COVID. The instability of the local CHW programs makes adoption and sustainability of the CHEST App di cult to measure because without the infrastructure of the CHW program the CHEST App cannot be utilized.

Adoption by CHW Program Coordinators
CHW Program Coordinators were tasked to upload and utilize the data collected with the CHEST App. Adoption of this practice was not achieved. The CHW Program Coordinators were not interested in uploading the data to the server. The task was completed during the pilot study, but only to satisfy the external research team at Elementos. Therefore, the research team is not able to monitor the data from the tablet unless they visit the community and upload the data from the tablets to the server. The Program Coordinator and local municipality continue to desire a paper-based list of results of the surveillance data and did not have a printer readily available to print the data from the tablets. The CHW Program Coordinator wrote the data displayed in the tablet on paper and submitted the paper report to the municipality. Thus, re ecting low adoption and poor delity of the surveillance function of the CHEST App.

Fidelity
Fidelity of the CHEST App by the CHWs, as observed during home visits, remained high throughout the pilot. At 18 months, each intended activity associated the CHEST APP showed delity scores between 80-100%, as shown in Table 1. The score indicated a high degree of delity and high quality of home visits with the CHEST App.   Table 2. The average knowledge scores by the caregivers, at baseline and endline, are shown in Table 3. The intervention and control group showed no signi cant difference in knowledge scores at baseline (p=0.9216). The independent samples t-test (intention to treat) found a signi cant increase in knowledge scores (M=5.53, SD=1.26) by those in the intervention group compared to those in the control group (t(184)=-4.39, p<0.001). The results of the independent t-test analysis can be seen in Table 4. A one way ANOVA showed that the effect of the CHEST App on knowledge scores yielded signi cant variation among groups, F(2,185)=12.9, p<0.000. The results of the one-way ANOVA analysis can be seen in Table 5. A post hoc Tukey test indicated that the average knowledge score for those that received 1-2 home visits (M=36.05, SD=6.11) was signi cantly higher than those that received no home visits (M=27.66, SD=7.38), p<0.000. The average knowledge score of those that received 3+ home visits (M=30.77, SD=8.57) was signi cantly different than those that received 1-2 home visits, p=0.040. The comparison between those that received no home visits and those that received 3+ home visits was not signi cant, p=0.125. The results of the post hoc Tukey test can be seen in Table 6.

Clinical Outcomes
The intervention and control group communities showed no signi cant difference in any of the clinical outcomes at baseline.

Hemoglobin
The average hemoglobin gm/dL of the children in intervention group and control group, at baseline and endline, are shown in Table 3. The independent samples t-test found no signi cant difference in hemoglobin levels in the intervention and control group at endline (t(140)=-0.37, p<0.711). The results of the t-test analysis can be seen in Table 4. A one way ANOVA showed that there was no signi cant difference between the three analysis groups, No home visits, 1-2 home visits, and 3+ home visits, in terms of hemoglobin levels at endline, F(2,139)=1.86, p=0.160. The results of the one-way ANOVA analysis can be seen in Table 5.

Early Childhood Development, Motor
The average ECD Motor scores for the children in the intervention and control group, at baseline and endline, can be seen in Table 3. Although the mean ECD motor score was 0.38 standard deviations higher in the intervention group at endline, the independent samples t-test found no signi cant difference (t(133)=-1.60, p<0.111). The one way ANOVA analysis found no signi cant difference between the three analysis groups, F(2,141)=1.28, p=0.282.

Early Childhood Development, Cognitive
The average ECD cognitive scores for the children in the intervention and control group, at baseline and endline, can be seen in Table 3. The independent samples t-test showed that there was no signi cant difference in ECD cognitive scores at endline (t(135)=-1.20, p<0.232). The one way ANOVA analysis also found no signi cant difference between the three analysis groups, F(2,141)=0.79, p=0.457 .

Discussion
The results suggest that the CHEST App improved the performance of CHWs to teach caregivers important knowledge pertaining to healthy child rearing. The caregivers that received visits from CHWs that utilized the CHEST App produced greater knowledge scores (1.26 standard deviation increase) than caregivers that received visits from CHWs with traditional material ( ip charts and brochures). Improvement in knowledge scores by the caregivers re ects the immediate impact of the CHEST App. As displayed in other studies, the mHealth tool improved the ability of the CHWs to conduct health promotion and education in the household. 21 The CHEST App intervention failed to create statistically signi cant change in the clinical outcomes of the participants. It is believed to be due to the low dosage outcome that was ultimately achieved. The dosage implementation outcome revealed that the intervention reached only 5.7% of its objective, the number of home visits needed to make the hypothesized change to the clinical outcomes. Failure to meet the dosage objective created a breakdown in the theory of change. Full implementation of the intervention was not achieved during the study time frame. The reduced sample size and disruption in CHW services was due to the COVID-19 pandemic. The improved knowledge by caregivers is expected to contribute to improved clinical outcomes, after more time passes and more home visits can be conducted. The authors will review the health indicators in the intervention community in one year to again test the null hypotheses.
Prior to the implementation of the intervention, applied implementation research was conducted to identify effective strategies to install the intervention in the local context. The research team rst assessed the key implementation determinants in the local context to identify potential barriers and facilitators to success. The determinants were based on the categorization from the Consolidated Framework for Implementation Research/CFIR; Damschroder et al., 2009. 51 To help determine which implementation strategies should be included, the research team used the CFIR ERIC Matching Tool. 52 The strategies that were used during the implementation process are listed in the IR Logic Model. Not all strategies suggested by the Matching Tool were utilized. For instance, funding strategies were not used due to a lack of receptibility by the local government to discuss changes to funding structures. For each implementation strategy, the research team implemented a set of discrete activities to operationalize the strategies. 53,54 Methods from Intervention Mapping (and Implementation Mapping) were used to develop the implementation strategies and evaluate the implementation outcomes. 55 The speci c implementation activities are described in Westgard, et al., 2020. 32 Finally, the implementation outcomes, service outcomes, and clinical outcomes were de ned, monitored, evaluated, and reported, as displayed in the IR Logic Model (Appendix 3).
The implementation outcomes that were evaluated suggest that the implementation strategies were affective at installing the intervention in the established CHW program, with high degrees of acceptability, adoption, and delity. Acceptability and adoption by the CHWs were greatly in uenced by the novelty of the CHEST App technology. The CHWs felt empowered from using the modern technology and more prepared to teach the caregivers due to the extensive educational material they had at hand. Fidelity rates of the CHEST App by the CHWs were consistently high, which is especially promising given the dynamic nature of home visits. Registering the child health indicators with the CHEST app was the most likely to be omitted during the home visit. For instance, if a home visit occurred and the caregiver did not have new information written on their child's growth monitoring card, the CHWs would not register child health indicators during that visit. This led to missed opportunity to record cases of infections that may have occurred since the last home visit.
Full adoption and delity by CHW program coordinators were not achieved because there was little desire to digitize the surveillance data. The coordinators and local municipality did not have the desire to rely on technology to track the health of children in their communities. Because the programs are not integrated into a larger, regional program, the number of children managed by the program remains relatively small. The CHW program and municipality value the educational component of the CHEST App but do not value the digital surveillance component. The digital surveillance component will be more important for CHW programs that are integrated into a regional or national level program, which demands greater management of data.
Outcomes related to sustainability and penetration need to be assessed after the communities have had more time with the CHEST App and following the end of support from the external research team from Elementos. Sustainability of the CHEST App intervention is greatly in uenced by the sustainability of the CHW programs. The local municipalities have been inconsistent in their support of the CHW programs. The three municipalities in the CHEST App pilot study each changed the CHW program during the 18-month trial. Two of the municipalities temporarily canceled the CHW program and one of the municipalities cut back the number of CHWs from 10 to 5, due to a lack of available funds. The National CHW program (Cuna Mas) 56 that also conducts home visits throughout the poorest regions of Peru is more stable, and thus a better alternative to incorporate the CHEST App. The implementation outcomes, service outcomes, and clinical outcomes from this pilot study are expected to be generalizable to the context of the National CHW program.

Limitations
The reduced sample size of the study due to program shutdowns caused by COVID-19 diminished the ability of the study to detect changes in the clinical outcomes. The restriction makes it di cult to conclude that the intervention is, or is not, effective at improving ECD or anemia.
The study design may have also been compromised by adaptations to the randomization process (to intervention and control group). Although assignment of the communities to the intervention and control group was randomized by the research team, the CHW program than led the randomization of individuals within the intervention community to receive the intervention. It was expected that the CHWs would be able to visit all children in the intervention community, however, reduction in the number of CHWs in the program hindered that possibility. The CHWs may had choose families to be involved in the program for reasons other than randomization. By checking for balance between the two groups at baseline in the key outcomes, and nding no statistically signi cant difference, the authors believe the objective of randomization was maintained.
An additional limitation was caused by the method to select who received the hemoglobin test. Hemoglobin levels were measured by the local health center and were not random nor a census of all children in the community. There may be selection bias of those that received the hemoglobin test and reported hemoglobin levels to the study. The anemia rates reported in the study suggests that there was a selection bias present in the sampling because the anemia rates are signi cantly lower than the regional average. To accurately detect the effect of the CHEST App on anemia rates a more rigorous method of hemoglobin assessment will be needed.

Conclusions
The effectiveness-implementation hybrid type 2 study provided the following results: 1. The CHEST App intervention can be effectively incorporated into a CHW program with high degrees of acceptability, adoption, and delity. 2. Adoption and delity of the surveillance function of the CHEST App by program coordinators was not achieved. 3. The CHEST App intervention is associated with improvements in knowledge of healthy child rearing practices by caregivers.
4. Program interruptions caused by COVID-19 created a decrease in nal dosage of the intervention, interrupting or delaying the theory of change.
The improved knowledge scores by the caregivers are theorized to contribute to a reduction in anemia rates and improve ECD scores, with more time passed with the intervention. The CHEST App is a promising tool to improve the performance of CHWs during their home visits, to accomplish their objective of teaching caregivers healthy childrearing practices and improving childhood health and development in their communities.  : 19-3097). The participants provided their written informed consent to participate in this study.

Consent for Publication
Not Applicable

Availability of Data and Material
The dataset supporting the conclusions of this article is available in the gshare repository, 10.6084/m9. gshare.16725271.

Competing Interests
The author declares that they have no competing interests

Authors Contributions
CW designed, implemented, analyzed, and reported the study. LOF designed, analyzed, and reported the study.