Cash Transfers for Improved Maternal and Child Health Data: A Pilot Study Supporting Health Management Information System in Malawi

The health management information system (HMIS) is an integral component of a strong health care system. Despite its importance for decision-making, the quality of HMIS data remains of concern in low-and middle-income countries. To address challenges with the quality of maternal and child health (MCH) data gathered within Malawi's HMIS, we designed a pilot study consisting of performing regular cash transfers to district-level HMIS oces. We hypothesized that providing regular cash transfers to HMIS oces would empower staff to establish strategies and priorities based on local context, consequently obtaining and maintaining accurate, timely, and complete MCH data. This pilot a non-randomized control group pretest/post-test It a baseline (pre-intervention) data quality assessment, a ten-month implementation phase, and an end line (post-intervention) data quality assessment and qualitative evaluation.


Page 3/19
Background Health management information systems (HMIS) are deployed to assist health o ces in recording, storing, and processing health data [1]. These data guide public health programs and policies and the effective allocation and use of health resources [1,2]. The information gathered within the HMIS also plays a critical role when planning health targets and assessing progress towards reaching national and international health targets. Researchers also use routine HMIS data to answer pertinent health system research questions [3].
Despite its critical importance, HMIS in most low-and middle-income countries (LMICs) remain inadequately used by policymakers, researchers, and non-governmental organizations (NGOs) [3,4]. This has been partially attributed to concerns over the completeness, timeliness, and accuracy of data collected [5][6][7][8] In Malawi, according to Malawi's National Health Information System Policy, the HMIS remains "fragmented and unable to generate quality information at the time they are needed" [9]. Commonly cited issues related to HMIS data quality include data duplication and loss, untimely submission of reports, data fabrication, incomplete data, and poor agreement between registers, monthly summary reports, and electronic reports [10][11][12].
Increased efforts to support and strengthen HMIS have been noted in LMICs, including Malawi. Common data quality improvement initiatives include training health workers and managers, identifying and training focal HMIS persons, organizing frequent HMIS data quality audits and review meetings, and increasing supportive supervision in low-performing health facilities [13][14][15][16]. Other major initiatives to facilitate the collection of high-quality HMIS data have focused on introducing electronic HMIS in health facilities and simplifying data reporting tools [16][17][18]. However, barriers to the success of past efforts to improve HMIS data quality have been observed [16]. Poor data quality has been shown to persist due to the high turnover of health staff, interruption of funding, and lack of resources [16]. In preparation for this study, we recognized, through visits to district-level HMIS o ces and health facilities in four districts of Malawi, that the lack of material and nancial resources to HMIS o ces for training and supervision was the main limiter of successful execution of HMIS tasks. This challenge was apparent given the irregular and short-term involvement of the Ministry of Health (MoH) and NGOs supporting HMIS o ces.
In response to the barriers to HMIS data quality noted in Malawi, we designed an intervention consisting of performing regular cash transfers to HMIS o ces. This intervention was nested within a multi-country development project focused on improving maternal and child health (MCH) outcomes in Sub-Saharan Africa. It aimed to assess the ability of a cash transfer strategy to improve MCH data in Malawi. It represents a novel use of cash transfers to inform MoHs and NGOs in their initiatives to strengthen HMIS data quality. We hypothesized that this strategy would give HMIS o ces more autonomy and capacity to establish their strategies and set their priorities to obtain and maintain accurate, timely, and complete MCH reports from health facilities.

Aim of the Study
We aimed to assess whether providing direct nancial assistance to facilitate the operations of districtlevel HMIS o ces in Malawi would lead to improvement in the quality of MCH data gathered within the HMIS.

Study Design
This pilot study used a non-randomized control group pretest/post-test design. It had three phases: a baseline (pre-intervention) data quality assessment, a ten-month implementation phase, and an end line (post-intervention) data quality assessment and qualitative evaluation.

Study Setting and Participants
This study was nested within the Canada-Africa Initiative to Address Maternal, Newborn and Child Mortality (CAIA-MNCM), a four-year project aiming at improving maternal, newborn, and child health in Ethiopia, Kenya, Malawi, and Tanzania. This project was carried out by a consortium of organizations including Amref Health Africa, WaterAid, and Children Believe, with research, evaluation and capacity building support from the SickKids Centre for Global Child Health. In Malawi, the CAIA-MNCM project was implemented by Amref Health Africa in four districts: Ntchisi, in the Central Region; and Neno, Mwanza, and Chikwawa, in the Southern Region. Malawi was chosen to pilot test this study given the number of project-supported districts and the project implementers' interest in strengthening HMIS performance in a novel way.
Malawi uses the District Health Information System version 2 (DHIS2) software to manage its HMIS [19].Reporting to Malawi's DHIS2 is mostly paper-based in communities and primary health facilities. In communities, health surveillance assistants (HSAs) are responsible for recording all services provided into paper registers. In health facilities, nurses, clinical o cers, and data clerks maintain separate registers for various MCH-related various services (e.g., child health consultations, antenatal care, deliveries, etc.). On a monthly basis, data clerks combine data from their registers and the HSAs' registers into summary reports and share those reports with their respective district-level HMIS o ces. At that level, the reports are reviewed and entered into the DHIS2 by the district statistical clerks and HMIS o cers. Once entered into the DHIS2, the data are made available to district-and national-level decision-makers to inform health planning. The overall process is supervised by the HMIS o cers and the district environmental health o cer (DEHO).
The study focused on the HMIS o ces and the health facilities (i.e., health centers and district hospitals) that report data to these o ces in the four districts of interest. Each district is served by at least one hospital and several smaller health centers. Neno has nine health centers, while 10, three, and 15 health centers are found in Ntchisi, Mwanza, and Chikwawa, respectively. Given the relatively large number of health facilities in Chikwawa, we randomly selected nine to participate in the study.

Processes, Intervention, and Comparisons
The cash transfer strategy was implemented in Mwanza, while Chikwawa, Neno, and Ntchisi served as control sites. Mwanza was selected to test the cash transfer strategy because the HMIS o ce was not receiving external support from NGOs other than Amref Health Africa in Malawi. The HMIS o ce in Mwanza consented to not receiving support from other NGOs during the study period to participate in the study. The support the HMIS o ce received from the Mwanza district health o ce (DHO) was minimal.
The HMIS o ces in Neno and Chikwawa, on the other hand, were supported by different NGOs. Ntchisi served as a 'pure' control, given the absence of de ned assistance from external partners to the HMIS o ce. However, Ntchisi did receive sporadic support from its District Health Management Team (DHMT).
In contrast, Neno and Chikwawa districts reported no support from the district-level MoH during the study period. Neno, Chikwawa, and Ntchisi represented suitable comparisons to assess the bene ts and limitations of our intervention compared to more standard NGO strategies (providing less autonomy to HMIS o cers to implement activities) and to the absence of clear and consistent support.

Baseline Data Quality Assessment
Before starting the intervention, we assessed the various MCH data in health centers, district hospitals, HMIS o ces, and the DHIS2 platform. We reviewed registers and monthly paper and electronic reports to assess the quality of the information reported on antenatal care (ANC), maternity and postnatal care (PNC) services. We also reviewed HMIS15 reports, which capture a subset of indicators from the various summary reports.
The quality of key MCH data within the HMIS was assessed considering four dimensions of data quality, as highlighted in the World Health Organization's data quality review toolkit [20][21][22] At the beginning of each month, HMIS o cers developed and shared activity plans and budgets with the study investigators for review. The investigators veri ed that the activities were speci c to HMIS strengthening and that the budget for the proposed activities did not exceed their monthly amount of 250,000 Malawian Kwacha. Once approved, the monthly allocation was transferred to a bank account opened speci cally for cash withdrawals by the HMIS o cers. At the end of each month, HMIS o cers provided the study team with details on the activities, including a description of the location, duration, and targeted individuals or institutions. For further accountability, detailed costs, along with itemized receipts, were provided to the study team. The study team also conducted onsite monitoring visits to ensure activities were conducted. The HMIS o ce staff were blinded to the indicators being assessed in the study and could use the funds to implement activities to reach overall greater HMIS data quality.
In parallel to the data quality assessment, we conducted interviews on a quarterly basis with HMIS o cers and representatives from implementing NGOs in the control districts. The discussions helped establish the activities in the comparison districts and highlight data quality strengths, challenges, and opportunities for improvement.

End line Evaluation
Following a 10-month intervention, trained research assistants visited health facilities and HMIS o ces to repeat the data quality assessment. The post-intervention data quality assessment focused on data collected between August 2019 and January 2020. Additionally, we conducted in-depth interviews in Mwanza with relevant informants to assess their opinion on the district's current performance in MCH data quality and their general perceptions of the intervention, speci cally on the acceptability and appropriateness of the intervention [23]. Participants for the interviews were selected using purposive sampling, targeting HMIS o ce staff and primary data handlers from the district hospital and the three health centers.

Data Analysis
Given the limited number of data points (i.e., health facilities and hospitals) in each district, we estimated district-level median scores in availability, timeliness, completeness, and consistency of selected MCH indicators across different levels of data aggregation. Our pilot study did not attempt to assess the statistical signi cance of our intervention on MCH data quality. Instead, we emphasize describing the change observed between the baseline and endline assessments, keeping in mind differences that would be considered relevant.
Qualitative interviews were conducted in Mwanza either in English or Chichewa, depending on the participants' preference. Participants were sampled purposively given their participation in the intervention and/or their role in HMIS data collection in the health facilities. Key respondents included the Mwanza's HMIS o cer (N = 1), the two district statistical clerks (N = 2), and the facility data clerks (N = 3). Qualitative data were collected using audio recording devices. Interviews conducted in Chichewa were translated into English at the point of transcription. Transcripts were coded using the NVivo software program to identify the main themes about the usefulness, perceived effectiveness, challenges, and sustainability of the intervention.

Summary of Activities
Over 10 months, Mwanza's HMIS o ce carried out eight activities to enhance the availability, timeliness, completeness, and consistency of the data (Table 1). No implementation occurred in September and December 2019 due to delays in transferring funds from the study team o ce to the HMIS o ce's bank account. The intervention was therefore implemented on eight of the 10 months scheduled for the intervention.
One activity (purchasing airtime and internet dongles) speci cally targeted the HMIS o ce to improve the timeliness of data entry into DHIS2, while the other activities aimed at reaching health facility-level staff. Through their various actions, the HMIS o cers oriented the data clerks on data collection tools, assessed how they were entering the data into the registers and summary forms, and provided advice to improve reporting. Using the nancial resources, the HMIS o ce supported MCH data, as well as data on outpatient services, infectious diseases, immunizations, palliative care, and youth-friendly services.
There were no major differences in the type of activities implemented in the intervention and control sites. While Ntchisi conducted their HMIS-related activities using sporadic support from the MoH, Chikwawa and Neno were supported by different international NGOs. In all districts except Ntchisi, there was a focus on improving the timeliness of data entry by securing an internet connection, and on conducting supervision visits to lower-level health facilities. Contrary to Mwanza, HMIS-related activities in the comparison districts did not include data quality assessment or reviews. While no important distinctions were noted in the type of activities conducted in the intervention and control districts supported by NGOs (Chikwawa and Neno), differences were observed in the frequency of activities. Using the cash transfers, Mwanza had to prioritize speci c interventions, while NGOs in Neno and Chikwawa had set activities and nancial allocations for the HMIS o ce. For example, 300,000 Malawi Kwacha per month (equivalent to approximately 406 US Dollars as of December 1st, 2019) was allocated to Chikwawa's HMIS o ce for allowances and transportation for supervision visits and data veri cation exercises. Our ndings highlighted changes in data quality following the 10-month intervention period in the intervention and control sites (Fig. 1). In the intervention district, improvements in the availability and completeness of MCH registers in health facilities increased by 22% (67% at baseline to 89% at endline) and 18% (46% at baseline to 64% at endline), respectively. The completeness of MCH reports sent to the HMIS o ce also improved from 84-100%. The availability of DHIS2 reports also increased from 75-83%. We noted no changes in the consistency of recounted and reported MCH data. Similar improvements in data quality were noted in the control districts. In Mwanza, we observed some progress in the consistency between monthly reports and DHIS2 entries (from 73% at baseline to 94% at endline). Neno district also reported important changes in data quality, with a 33% increase in the availability of MCH registers and a 50% rise in the availability of MCH reports at the HMIS o ce. In Chikwawa and Neno districts, progress was only noted in the consistency between monthly report and DHIS2 data. Minimal changes and, in some instances, reductions were noted for most data quality dimensions.

Qualitative Assessment of the Cash Transfer Strategy
Acceptability, appropriateness, and perceived effects on data quality The respondents who participated in the intervention discerned the usefulness and potential of the cash transfer strategy to support HMIS data quality. As the funds dedicated to the HMIS o ce by the district health o ce were limited and no external partners provided additional support to the HMIS o ce, the study respondents appreciated the objective and novelty of the study. In addition, the HMIS o ce staff reported that the cash transfer approach enabled them to do their job and gave them the exibility to implement important HMIS-related activities.
(The intervention) showed that we had a purpose, this has imparted us to do review meetings and (carry out activities) other than supervision. (HMIS o cer) Through their activities, HMIS o cers reported being better positioned to support statistical clerks in hospitals and health centers. Regular supervision visits and data quality assessments enabled them to assess whether data clerks were adequately entering the data into the registers and summary forms and provide feedback when needed. This is con rmed by the observed improvement in the completeness of MCH registers (Fig. 1) but contradicted by the ndings related to the consistency of reporting from the registers to the monthly summary forms.
The HMIS o ce staff also highlighted that the cash transfer approach enabled them to address gaps they had been observing in the district's health facilities. For instance, in the past, data collection tools were supplied to the facilities without providing proper orientation to staff. The HMIS o ce staff reported now being able to spend time and resources to properly orient health workers, data clerks, and program coordinators on new and revised reporting tools. Given more frequent interactions with data clerks, the HMIS o ce staff highlighted being in a better position to recognize and address emerging data quality weaknesses.
Orientation on data collection tools helped to give the people feedback on the gaps we had found and how they could ll them up. The review helped us to analyze the report that we had written after orientating them, and the data handlers could give their feedback on how we could best improve the report.(District Statistical Clerk) HMIS o ce staff believe that the intervention ultimately helped them improve their data quality through ensuring greater and more consistently available data collection tools and more frequent collaborations with facility data clerks. This is also re ected in the results of the DQAs, where we observed increased availability of reporting forms (registers and monthly summary forms) in health facilities. The support received by the HMIS o ce was also indicated during the interviews with the facility data clerks. However, although HMIS o ce personnel were reported to visit the health facilities, no improvements in the consistency between the data recounted from the registers and the data reported in the summary forms were noted. Because of this intervention, we had time to write the report after each activity and submit to the DHMT. Every month and quarter, we (could) come up with reports and submit to different people so that they know how they are doing in their program. (District Statistical clerk) Compared to previous support received to strengthen HMIS functions, the interviewees noted that our approach had the advantages of closely involving the individuals who are responsible for the data. They also highlighted the fact that other NGOs and the DHO could learn from this intervention, and the HMIS o ce's ability to run activities with minimal funding. Challenges and unintended consequences While some respondents perceived progress in HMIS data quality as a result of the cash transfer strategy, some challenges were also reported. There were unforseen delays with the transfer of the monthly funds from the study team to the HMIS o ce, which resulted in delayed implementation. As a consequence of delayed transfer of funds from the study team end, no activities were conducted in September and December 2019.
We could plan to do the activity in March, but we could be given the funds in April. [...] Like now, we have made a plan for February, but the money is not yet in. (HMIS o cer) The amount of the monthly cash transfer was also determined as insu cient to carry out all activities needed to reach the desired level of data quality. Fluctuations in bank charges and fuel prices also signi cantly reduced the amount that the HMIS o ce was receiving every month. These unexpected charges prevented the HMIS o ce from realizing planned activities.
We could write a proposal of about 250,000 MWK, and then we would add a bank charge of 30 000 MWK, you would nd that the bank charge was more than what we thought, and we would get less than the amount we had planned for. As a result, we would cut some activities. (HMIS o cer) These challenges ultimately prevented the HMIS o ce staff from proceeding with some activities and required them to be selective of which individuals, departments, facilities, and data collection tools to target.
There are some programs that needed formal training and not just orientations. There are some data tools that were introduced, but people were just oriented and not trained. They just got a glimpse of the job, and that still compromises data documentation. Maybe if there can be a chance for them to be trained in different tools in data documentation. (HMIS o cer) [The study team] was not doing everything because they only provided a portion of the funding. Maybe when training coordinators, we were not training all of them, we were just taking the front-line workers who were the core coordinators. (HMIS o cer) Echoing the concerns raised by the HMIS o ce staff, health facility-level data clerks also reported that some of the activities conducted by the HMIS o ce only reached a small portion of needed participants.
The DQAs require everyone involved in the program to be present, but sometimes they just say we want the in-charge and other selected people. Everyone should be present, so they understand their program. Then maybe the programs can run smoothly. (Facility data clerk) The launch of the intervention had the unintended consequence of sparking tensions within the district health o ce and discordance on who would manage the funds. These concerns ultimately delayed the start of the intervention but were overcome following discussions between the investigators and the DHMT on the study and its intended bene ts.
When the (intervention) came, (some members of the DHMT) thought I would be very rich since they thought (the study team) would be providing a lot of money and even (some people were) not in agreement with me (participating in the study At the end of the 10-month intervention period, we noted progress in data quality in Mwanza, while in Chikwawa, no changes were observed. The absence of major improvements in data quality in Chikwawa could be explained by the fact that the HMIS o ce was limited in the number of facilities they could support using assistance from the NGOs. In fact, some hard-to-reach health facilities were excluded from the HMIS support. An approach like the one piloted in this study could overcome such challenges by enabling HMIS o ce staff to target health facilities in critical need of support.
Although Ntchisi did not have any intervening partners, the overall quality of the data was high, which could be explained by the HMIS o ce staff's dedication and the district's proximity to Lilongwe, which may have facilitated access to reporting tools and training from the MoH. Yet, some non-negligible reductions were noted on the completeness of registers and the timeliness of report entry into DHIS2. As identi ed by Ntchisi's HMIS o cer during the quarterly interviews, the absence of external partners and the limited assistance provided by the DHO hindered the HMIS o ce's access to the Internet and their ability to secure transport and allowances for supervision visits. This indicates that regular support to HMIS o ces is required to prevent data quality loss.
Interestingly, in the intervention and control sites, we did not observe changes in the consistency between the MCH data recounted from the registers and reported in the monthly summary forms, highlighting the need for more targeted support in health centers. Our qualitative interviews in both the intervention and control districts provided indications that extracting some data from the registers, particularly the ANC and PNC registers, can be di cult for data clerks and medical professionals, given these registers follow women in cohorts (6 months for ANC and 3 months for PNC registers). The data handlers are required to determine in which cohort the women belong, based on their rst ANC visit date. New or untrained professionals are more likely to assign ANC clients to the wrong cohort when aggregating the data. We also identi ed that, even when available, health facility personnel would sometimes refrain from using the PNC registers. The comparison districts also recorded low agreement between ANC registers and monthly reports, reinforcing the idea that facility-level health professionals and data clerks may face challenges when reporting ANC and PNC data on monthly summary forms and that more training and orientation opportunities should be provided.
Our quantitative results in Mwanza were also supported by qualitative interviews, through which the HMIS o ce personnel identi ed some advantages to our approach. Challenges were also noted, including the fact that the funds were deemed insu cient to run activities such as intensive training sessions in facilities, which could have helped improve the consistency of data from the registers to the summary reports. It is also possible that a longer intervention period, beyond 10 months could have led to more consistency of data reporting. In future studies considering this type of cash transfer strategy, efforts should be made to support the HMIS o ce for more than 10 months.
Although some improvements in MCH data quality were observed at the end of the intervention period, our pilot intervention had some limitations. First, we implemented the intervention in a district with a small number of facilities, which might have eased the implementation of the strategy. The effects of our strategy could have been very different in a district with a large number of facilities with competing priorities for HMIS data quality. Second, the qualitative interviews were of variable quality. Some of the responses were unclear, hence making transcription di cult. Although participants had the option of doing their interviews in Chichewa, some interviews were still done in English. This challenged some informants' ability to answer the questions comprehensively. Lastly, as the endline data collection coincided with the rst reported cases of COVID-19 in Mwanza, we missed the perspectives of additional key informants, including the DHO and the DEHO.

Conclusions
This pilot intervention provides preliminary evidence that our approach -providing targeted funding directly to the HMIS o ce -can produce data quality changes in terms of the availability and completeness of data. A comparison of data quality changes in the intervention and some control sites suggests that our intervention could lead to greater data quality improvement than more standard approaches considering by NGOs to HMIS strengthening. Meaningful improvement in the consistency of reporting may require more time to allow HMIS o cers to implement frequent and intensive activities such as training, supervision visits, and data quality assessments. Our approach empowered HMIS o ce personnel to set their own priorities to improve data quality. The cash transfer strategy was even preferred to more standard ways of supporting HMIS that provide less autonomy to HMIS o cers to make decisions. Given the MoH's focus on scaling up the performance of its DHIS2, our intervention provides a strategy that NGOs and DHOs could implement with minimal oversight.