Cash Transfers for Improved Maternal and Child Health Data: A Pilot Study Supporting Health Management Information System in Malawi

doi:10.21203/rs.3.rs-728453/v1

Download PDF

Research Article

Cash Transfers for Improved Maternal and Child Health Data: A Pilot Study Supporting Health Management Information System in Malawi

https://doi.org/10.21203/rs.3.rs-728453/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

Background

The health management information system (HMIS) is an integral component of a strong health care system. Despite its importance for decision-making, the quality of HMIS data remains of concern in low- and middle-income countries. To address challenges with the quality of maternal and child health (MCH) data gathered within Malawi's HMIS, we designed a pilot study consisting of performing regular cash transfers to district-level HMIS offices. We hypothesized that providing regular cash transfers to HMIS offices would empower staff to establish strategies and priorities based on local context, consequently obtaining and maintaining accurate, timely, and complete MCH data.

Methods

The pilot intervention was implemented in Mwanza district, while Chikwawa, Neno, and Ntchisi districts served as control sites. The intervention consisted of providing cash transfers to Mwanza's HMIS office following the submission of detailed budgets and lists of planned activities with their respective targets and outputs. In the control districts, we performed regular interviews with the HMIS officers to track the HMIS-related activities. We evaluated the intervention by comparing data quality between the post-intervention and pre-intervention periods in the intervention and control districts. Additionally, we conducted interviews with Mwanza's HMIS office staff to determine the acceptability and appropriateness of the intervention.

Results

Following the 10-month intervention period, we observed improvements in MCH data quality in the intervention district (Mwanza). The availability and completeness of MCH data collected in the registers increased by 22% and 18%, respectively. The consistency of MCH data between summary reports and electronic HMIS improved from 73–94%. The qualitative interviews confirmed that, despite some challenges, the intervention was well received by the participating HMIS office. Participants preferred our strategy to other conventional ways of supporting HMIS that fail to give HMIS offices the independence to make decisions.

Conclusions

This pilot intervention demonstrated an alternative approach to support HMIS offices in their daily efforts to improve data quality. Given the Ministry of Health (MoH)'s interest in strengthening its HMIS, our intervention provides a strategy that the MoH and local and international partners could consider to rapidly improve HMIS data with minimal oversight.

Health Policy

Health Economics & Outcomes Research

Health Management Information System

Maternal and Child Health

Pilot Intervention

Malawi

Health management information systems (HMIS) are deployed to assist health offices in recording, storing, and processing health data [1]. These data guide public health programs and policies and the effective allocation and use of health resources [1, 2]. The information gathered within the HMIS also plays a critical role when planning health targets and assessing progress towards reaching national and international health targets. Researchers also use routine HMIS data to answer pertinent health system research questions [3].

Despite its critical importance, HMIS in most low- and middle-income countries (LMICs) remain inadequately used by policymakers, researchers, and non-governmental organizations (NGOs) [3, 4]. This has been partially attributed to concerns over the completeness, timeliness, and accuracy of data collected[5–8] In Malawi, according to Malawi's National Health Information System Policy, the HMIS remains "fragmented and unable to generate quality information at the time they are needed"[9]. Commonly cited issues related to HMIS data quality include data duplication and loss, untimely submission of reports, data fabrication, incomplete data, and poor agreement between registers, monthly summary reports, and electronic reports [10–12].

Increased efforts to support and strengthen HMIS have been noted in LMICs, including Malawi. Common data quality improvement initiatives include training health workers and managers, identifying and training focal HMIS persons, organizing frequent HMIS data quality audits and review meetings, and increasing supportive supervision in low-performing health facilities [13–16]. Other major initiatives to facilitate the collection of high-quality HMIS data have focused on introducing electronic HMIS in health facilities and simplifying data reporting tools [16–18]. However, barriers to the success of past efforts to improve HMIS data quality have been observed [16]. Poor data quality has been shown to persist due to the high turnover of health staff, interruption of funding, and lack of resources [16]. In preparation for this study, we recognized, through visits to district-level HMIS offices and health facilities in four districts of Malawi, that the lack of material and financial resources to HMIS offices for training and supervision was the main limiter of successful execution of HMIS tasks. This challenge was apparent given the irregular and short-term involvement of the Ministry of Health (MoH) and NGOs supporting HMIS offices.

In response to the barriers to HMIS data quality noted in Malawi, we designed an intervention consisting of performing regular cash transfers to HMIS offices. This intervention was nested within a multi-country development project focused on improving maternal and child health (MCH) outcomes in Sub-Saharan Africa. It aimed to assess the ability of a cash transfer strategy to improve MCH data in Malawi. It represents a novel use of cash transfers to inform MoHs and NGOs in their initiatives to strengthen HMIS data quality. We hypothesized that this strategy would give HMIS offices more autonomy and capacity to establish their strategies and set their priorities to obtain and maintain accurate, timely, and complete MCH reports from health facilities.

Aim of the Study

We aimed to assess whether providing direct financial assistance to facilitate the operations of district-level HMIS offices in Malawi would lead to improvement in the quality of MCH data gathered within the HMIS.

Study Design

This pilot study used a non-randomized control group pretest/post-test design. It had three phases: a baseline (pre-intervention) data quality assessment, a ten-month implementation phase, and an end line (post-intervention) data quality assessment and qualitative evaluation.

Study Setting and Participants

This study was nested within the Canada-Africa Initiative to Address Maternal, Newborn and Child Mortality (CAIA-MNCM), a four-year project aiming at improving maternal, newborn, and child health in Ethiopia, Kenya, Malawi, and Tanzania. This project was carried out by a consortium of organizations including Amref Health Africa, WaterAid, and Children Believe, with research, evaluation and capacity building support from the SickKids Centre for Global Child Health. In Malawi, the CAIA-MNCM project was implemented by Amref Health Africa in four districts: Ntchisi, in the Central Region; and Neno, Mwanza, and Chikwawa, in the Southern Region. Malawi was chosen to pilot test this study given the number of project-supported districts and the project implementers' interest in strengthening HMIS performance in a novel way.

Malawi uses the District Health Information System version 2 (DHIS2) software to manage its HMIS [19].Reporting to Malawi's DHIS2 is mostly paper-based in communities and primary health facilities. In communities, health surveillance assistants (HSAs) are responsible for recording all services provided into paper registers. In health facilities, nurses, clinical officers, and data clerks maintain separate registers for various MCH-related various services (e.g., child health consultations, antenatal care, deliveries, etc.). On a monthly basis, data clerks combine data from their registers and the HSAs' registers into summary reports and share those reports with their respective district-level HMIS offices. At that level, the reports are reviewed and entered into the DHIS2 by the district statistical clerks and HMIS officers. Once entered into the DHIS2, the data are made available to district-and national-level decision-makers to inform health planning. The overall process is supervised by the HMIS officers and the district environmental health officer (DEHO).

The study focused on the HMIS offices and the health facilities (i.e., health centers and district hospitals) that report data to these offices in the four districts of interest. Each district is served by at least one hospital and several smaller health centers. Neno has nine health centers, while 10, three, and 15 health centers are found in Ntchisi, Mwanza, and Chikwawa, respectively. Given the relatively large number of health facilities in Chikwawa, we randomly selected nine to participate in the study.

Processes, Intervention, and Comparisons

The cash transfer strategy was implemented in Mwanza, while Chikwawa, Neno, and Ntchisi served as control sites. Mwanza was selected to test the cash transfer strategy because the HMIS office was not receiving external support from NGOs other than Amref Health Africa in Malawi. The HMIS office in Mwanza consented to not receiving support from other NGOs during the study period to participate in the study. The support the HMIS office received from the Mwanza district health office (DHO) was minimal. The HMIS offices in Neno and Chikwawa, on the other hand, were supported by different NGOs. Ntchisi served as a ‘pure’ control, given the absence of defined assistance from external partners to the HMIS office. However, Ntchisi did receive sporadic support from its District Health Management Team (DHMT). In contrast, Neno and Chikwawa districts reported no support from the district-level MoH during the study period. Neno, Chikwawa, and Ntchisi represented suitable comparisons to assess the benefits and limitations of our intervention compared to more standard NGO strategies (providing less autonomy to HMIS officers to implement activities) and to the absence of clear and consistent support.

Baseline Data Quality Assessment

Before starting the intervention, we assessed the various MCH data in health centers, district hospitals, HMIS offices, and the DHIS2 platform. We reviewed registers and monthly paper and electronic reports to assess the quality of the information reported on antenatal care (ANC), maternity and postnatal care (PNC) services. We also reviewed HMIS15 reports, which capture a subset of indicators from the various summary reports.

We extracted the following MCH indicators from the different records: (1) first ANC visits, (2) facility-based deliveries, (3) breastfeeding within 60 minutes of birth, (4) postnatal care within 48 hours (PNC48) offered to mothers, and (5) PNC48 for babies. We reviewed data collected between May and October 2018.

The quality of key MCH data within the HMIS was assessed considering four dimensions of data quality, as highlighted in the World Health Organization’s data quality review toolkit[20–22]:

Availability of relevant MCH registers and monthly reports: Number of registers/monthly reports available divided by the total number of registers/monthly reports expected to be available.

Timeliness of submission into DHIS2: Number of monthly reports entered into the DHIS2 on time (by the deadline for reporting) divided by the total number of monthly reports expected to be in the DHIS2.

Completeness of MCH registers and monthly reports: Number of MCH registers/monthly reports with non-missing values for selected MCH data elements divided by the total number of registers/monthly reports available.

MCH data consistency from health facilities to HMIS offices to the DHIS2 platform: a) percentage of health facilities for which the counts for selected MCH indicators in registers and the values of MCH indicators reported on the monthly reports found in the HMIS offices matched; b) percentage of health facilities for which the values of selected MCH indicators reported on monthly reports found in the HMIS offices and those found in the DHIS2 matched.

Data Quality Improvement Strategy

The intervention consisted of providing cash transfers to Mwanza's HMIS office following the submission of detailed monthly budgets and summaries of planned activities by the HMIS officers. Considering the amount that Mwanza's HMIS office requests from its DHO management to maintain HMIS activities and what would be sustainable to the DHO, we established a fixed amount to allocate to the HMIS office for the duration of the study. The amount of 250,000 Malawian Kwacha per month (equivalent to approximately 338 US Dollars as of December 1st, 2019) was provided for 10 months, between June 2019 and March 2020.

At the beginning of each month, HMIS officers developed and shared activity plans and budgets with the study investigators for review. The investigators verified that the activities were specific to HMIS strengthening and that the budget for the proposed activities did not exceed their monthly amount of 250,000 Malawian Kwacha. Once approved, the monthly allocation was transferred to a bank account opened specifically for cash withdrawals by the HMIS officers. At the end of each month, HMIS officers provided the study team with details on the activities, including a description of the location, duration, and targeted individuals or institutions. For further accountability, detailed costs, along with itemized receipts, were provided to the study team. The study team also conducted onsite monitoring visits to ensure activities were conducted. The HMIS office staff were blinded to the indicators being assessed in the study and could use the funds to implement activities to reach overall greater HMIS data quality.

In parallel to the data quality assessment, we conducted interviews on a quarterly basis with HMIS officers and representatives from implementing NGOs in the control districts. The discussions helped establish the activities in the comparison districts and highlight data quality strengths, challenges, and opportunities for improvement.

End line Evaluation

Following a 10-month intervention, trained research assistants visited health facilities and HMIS offices to repeat the data quality assessment. The post-intervention data quality assessment focused on data collected between August 2019 and January 2020. Additionally, we conducted in-depth interviews in Mwanza with relevant informants to assess their opinion on the district's current performance in MCH data quality and their general perceptions of the intervention, specifically on the acceptability and appropriateness of the intervention [23]. Participants for the interviews were selected using purposive sampling, targeting HMIS office staff and primary data handlers from the district hospital and the three health centers.

Data Analysis

Given the limited number of data points (i.e., health facilities and hospitals) in each district, we estimated district-level median scores in availability, timeliness, completeness, and consistency of selected MCH indicators across different levels of data aggregation. Our pilot study did not attempt to assess the statistical significance of our intervention on MCH data quality. Instead, we emphasize describing the change observed between the baseline and endline assessments, keeping in mind differences that would be considered relevant.

Qualitative interviews were conducted in Mwanza either in English or Chichewa, depending on the participants' preference. Participants were sampled purposively given their participation in the intervention and/or their role in HMIS data collection in the health facilities. Key respondents included the Mwanza's HMIS officer (N = 1), the two district statistical clerks (N = 2), and the facility data clerks (N = 3). Qualitative data were collected using audio recording devices. Interviews conducted in Chichewa were translated into English at the point of transcription. Transcripts were coded using the NVivo software program to identify the main themes about the usefulness, perceived effectiveness, challenges, and sustainability of the intervention.

Summary of Activities

Over 10 months, Mwanza's HMIS office carried out eight activities to enhance the availability, timeliness, completeness, and consistency of the data (Table 1). No implementation occurred in September and December 2019 due to delays in transferring funds from the study team office to the HMIS office’s bank account. The intervention was therefore implemented on eight of the 10 months scheduled for the intervention.

One activity (purchasing airtime and internet dongles) specifically targeted the HMIS office to improve the timeliness of data entry into DHIS2, while the other activities aimed at reaching health facility-level staff. Through their various actions, the HMIS officers oriented the data clerks on data collection tools, assessed how they were entering the data into the registers and summary forms, and provided advice to improve reporting. Using the financial resources, the HMIS office supported MCH data, as well as data on outpatient services, infectious diseases, immunizations, palliative care, and youth-friendly services.

There were no major differences in the type of activities implemented in the intervention and control sites. While Ntchisi conducted their HMIS-related activities using sporadic support from the MoH, Chikwawa and Neno were supported by different international NGOs. In all districts except Ntchisi, there was a focus on improving the timeliness of data entry by securing an internet connection, and on conducting supervision visits to lower-level health facilities. Contrary to Mwanza, HMIS-related activities in the comparison districts did not include data quality assessment or reviews. While no important distinctions were noted in the type of activities conducted in the intervention and control districts supported by NGOs (Chikwawa and Neno), differences were observed in the frequency of activities. Using the cash transfers, Mwanza had to prioritize specific interventions, while NGOs in Neno and Chikwawa had set activities and financial allocations for the HMIS office. For example, 300,000 Malawi Kwacha per month (equivalent to approximately 406 US Dollars as of December 1st, 2019) was allocated to Chikwawa’s HMIS office for allowances and transportation for supervision visits and data verification exercises.

Table 1

Summary of activities conducted between June 2019 and March 2020 in the intervention (Mwanza district) and control sites (Chikwawa, Neno, Ntchisi districts) in Malawi
District	Activities
Mwanza (Intervention site)	• Purchasing of airtime and internet dongles to facilitate entry into the DHIS2 at the HMIS office • Printing and provision of data collection and reporting tools to health facilities • Data quality assessment of ANC, maternity, sexually transmitted infection (STI) and outpatient registers in health facilities • District- and facility-level orientations on new and revised data collection tools. • Supportive supervision in health facilities to review ANC, maternity, family planning, and HMIS15 data • District- and facility-level data reviews
Chikwawa (Control site)	• Allowances and transportation for HMIS office staff to perform supervision visits and data verification exercises in health facilities • Monthly airtime subscription of 10GB provided to the HMIS office to facilitate the submission of reports to DHIS2 • HMIS office staff provide access to the NGO office to print registers, summary sheets, and supervision checklists • Transportation of new logs from the National Statistical Office to Chikwawa • Deployment of data clerks in targeted health facilities
Neno (Control site)	• Monthly financial allocation for HMIS office to buy stationery • Creation of a booklet for tracking the submission of monthly reports by health facilities • Quarterly financial assignment to HMIS office to cover daily allowances and fuel costs for in-facility supervision visits • Internet bundles to facilitate timely data entry into DHIS2 • Deployment of data clerks in targeted health facilities
Ntchisi (Control site)	• Deployment of data clerks in targeted health facilities • Stationary (tonner, papers, writing materials) and airtime for data entry • Transport of monthly reports from facilities to HMIS office

Data Quality Assessment

Our findings highlighted changes in data quality following the 10-month intervention period in the intervention and control sites (Fig. 1). In the intervention district, improvements in the availability and completeness of MCH registers in health facilities increased by 22% (67% at baseline to 89% at endline) and 18% (46% at baseline to 64% at endline), respectively. The completeness of MCH reports sent to the HMIS office also improved from 84–100%. The availability of DHIS2 reports also increased from 75–83%. We noted no changes in the consistency of recounted and reported MCH data. Similar improvements in data quality were noted in the control districts. In Mwanza, we observed some progress in the consistency between monthly reports and DHIS2 entries (from 73% at baseline to 94% at endline). Neno district also reported important changes in data quality, with a 33% increase in the availability of MCH registers and a 50% rise in the availability of MCH reports at the HMIS office. In Chikwawa and Neno districts, progress was only noted in the consistency between monthly report and DHIS2 data. Minimal changes and, in some instances, reductions were noted for most data quality dimensions.

Qualitative Assessment of the Cash Transfer Strategy

Acceptability, appropriateness, and perceived effects on data quality

The respondents who participated in the intervention discerned the usefulness and potential of the cash transfer strategy to support HMIS data quality. As the funds dedicated to the HMIS office by the district health office were limited and no external partners provided additional support to the HMIS office, the study respondents appreciated the objective and novelty of the study. In addition, the HMIS office staff reported that the cash transfer approach enabled them to do their job and gave them the flexibility to implement important HMIS-related activities.

(The intervention) showed that we had a purpose, this has imparted us to do review meetings and (carry out activities) other than supervision. (HMIS officer)

Through their activities, HMIS officers reported being better positioned to support statistical clerks in hospitals and health centers. Regular supervision visits and data quality assessments enabled them to assess whether data clerks were adequately entering the data into the registers and summary forms and provide feedback when needed. This is confirmed by the observed improvement in the completeness of MCH registers (Fig. 1) but contradicted by the findings related to the consistency of reporting from the registers to the monthly summary forms.

The HMIS office staff also highlighted that the cash transfer approach enabled them to address gaps they had been observing in the district’s health facilities. For instance, in the past, data collection tools were supplied to the facilities without providing proper orientation to staff. The HMIS office staff reported now being able to spend time and resources to properly orient health workers, data clerks, and program coordinators on new and revised reporting tools. Given more frequent interactions with data clerks, the HMIS office staff highlighted being in a better position to recognize and address emerging data quality weaknesses.

Orientation on data collection tools helped to give the people feedback on the gaps we had found and how they could fill them up. The review helped us to analyze the report that we had written after orientating them, and the data handlers could give their feedback on how we could best improve the report.(District Statistical Clerk)

HMIS office staff believe that the intervention ultimately helped them improve their data quality through ensuring greater and more consistently available data collection tools and more frequent collaborations with facility data clerks. This is also reflected in the results of the DQAs, where we observed increased availability of reporting forms (registers and monthly summary forms) in health facilities. The support received by the HMIS office was also indicated during the interviews with the facility data clerks. However, although HMIS office personnel were reported to visit the health facilities, no improvements in the consistency between the data recounted from the registers and the data reported in the summary forms were noted.

(The HMIS office people) come to supervise. They come to verify what we have sent them as original forms and if it is (agreeing) with the registers. (Facility data clerk)

The approach used in this pilot study also revived HMIS office staff’s interest in their data and promoted better engagement and cooperation between the HMIS office and the various departments in hospitals and health centers. The HMIS office collaborated and communicated with the District Health Management Team (DHMT) through frequent data quality reports.

Because of this intervention, we had time to write the report after each activity and submit to the DHMT. Every month and quarter, we (could) come up with reports and submit to different people so that they know how they are doing in their program. (District Statistical clerk)

Compared to previous support received to strengthen HMIS functions, the interviewees noted that our approach had the advantages of closely involving the individuals who are responsible for the data. They also highlighted the fact that other NGOs and the DHO could learn from this intervention, and the HMIS office’s ability to run activities with minimal funding.

(The) intervention started from the grassroots, thus the data handlers, what they were facing as challenges […]. I can say (this) intervention is better than the previous ones that just wanted to see the data but not concerned with the one collecting the data. (HMIS officer)

Not only the NGOs, but maybe even the DHO can borrow a leaf from what (this study) was doing to help us. That with the little we were getting from (this study), we were able to do some activities. […] It’s the simple things; you can do a lot with 100,000 or 50,000 Malawi Kwacha. (HMIS officer)

Challenges and unintended consequences

While some respondents perceived progress in HMIS data quality as a result of the cash transfer strategy, some challenges were also reported. There were unforseen delays with the transfer of the monthly funds from the study team to the HMIS office, which resulted in delayed implementation. As a consequence of delayed transfer of funds from the study team end, no activities were conducted in September and December 2019.

We could plan to do the activity in March, but we could be given the funds in April. [...] Like now, we have made a plan for February, but the money is not yet in. (HMIS officer)

The amount of the monthly cash transfer was also determined as insufficient to carry out all activities needed to reach the desired level of data quality. Fluctuations in bank charges and fuel prices also significantly reduced the amount that the HMIS office was receiving every month. These unexpected charges prevented the HMIS office from realizing planned activities.

We could write a proposal of about 250,000 MWK, and then we would add a bank charge of 30 000 MWK, you would find that the bank charge was more than what we thought, and we would get less than the amount we had planned for. As a result, we would cut some activities. (HMIS officer)

These challenges ultimately prevented the HMIS office staff from proceeding with some activities and required them to be selective of which individuals, departments, facilities, and data collection tools to target.

There are some programs that needed formal training and not just orientations. There are some data tools that were introduced, but people were just oriented and not trained. They just got a glimpse of the job, and that still compromises data documentation. Maybe if there can be a chance for them to be trained in different tools in data documentation. (HMIS officer)

[The study team] was not doing everything because they only provided a portion of the funding. Maybe when training coordinators, we were not training all of them, we were just taking the front-line workers who were the core coordinators. (HMIS officer)

Echoing the concerns raised by the HMIS office staff, health facility-level data clerks also reported that some of the activities conducted by the HMIS office only reached a small portion of needed participants.

The DQAs require everyone involved in the program to be present, but sometimes they just say we want the in- charge and other selected people. Everyone should be present, so they understand their program. Then maybe the programs can run smoothly. (Facility data clerk)

The launch of the intervention had the unintended consequence of sparking tensions within the district health office and discordance on who would manage the funds. These concerns ultimately delayed the start of the intervention but were overcome following discussions between the investigators and the DHMT on the study and its intended benefits.

When the (intervention) came, (some members of the DHMT) thought I would be very rich since they thought (the study team) would be providing a lot of money and even (some people were) not in agreement with me (participating in the study). (HMIS officer)

Areas for improvement and sustainability

When asked about ways to improve our intervention, the respondents highlighted that a monthly allocation of 700,000 Malawian Kwacha instead of 250,000 would have been adequate (approximately 947 US Dollar instead of 338 (based on rate from December 1st, 2019)). Other recommendations included circumventing delays associated with the transfer of funds to the HMIS office’s bank account and settling the bank charges in advance of the study.

Sustainability by the district-level MoH is important to consider, particularly when support from NGOs is limited and unsteady. We therefore asked the interviewees if they thought this approach could be considered and adopted by the DHO. Money was identified as the main barrier that could prevent the DHO from considering this intervention in the long-term. Additional reasons challenging the sustainability of the cash transfer approach were the perceptions that the DHO may have on the value of the HMIS office in supporting and informing the health system.

The limiting factor [...] could be money. We receive little money from the DHO. Sometimes they don’t (think about) HMIS because (they see it as) a preventive department. (The DHO) can’t give much money to HMIS, so they put it into other things. They are looking at assisting the sick people and not follow up on the data trends. (HMIS officer)

In the past, when HMIS came up with activities, they were removed by the DHO. They thought it was just a waste of resources forgetting that it is a good department where data can be accessed to help the hospital move forward. (HMIS officer)

Improving and maintaining the quality of HMIS data is critical. Facilities' failure to submit complete and accurate MCH reports to their district may result in a partial and incomplete representation of MCH service provision. This could have important implications for the health of pregnant women, newborns, and children living in these districts, as the DHOs use information reported by health facilities to guide health planning and resource allocations. In this study, we explored an alternative strategy to support HMIS offices in their efforts to improve data quality. To our knowledge, no study has attempted to evaluate improvements in the quality of HMIS data in response to direct financial assistance provided to HMIS offices. Most of the initiatives conducted to improve HMIS data quality had defined activities such as introducing technology and capacity-building activities of HMIS personnel [16]. The interventions reported in two of the control districts (Neno and Chikawa) fell into that category, where NGOs had a list of areas to support HMIS offices. On the other hand, our pilot intervention aimed to give HMIS office personnel the flexibility to choose and implement activities to address HMIS data quality gaps in their health facilities.

At the end of the 10-month intervention period, we noted progress in data quality in Mwanza, while in Chikwawa, no changes were observed. The absence of major improvements in data quality in Chikwawa could be explained by the fact that the HMIS office was limited in the number of facilities they could support using assistance from the NGOs. In fact, some hard-to-reach health facilities were excluded from the HMIS support. An approach like the one piloted in this study could overcome such challenges by enabling HMIS office staff to target health facilities in critical need of support.

Although Ntchisi did not have any intervening partners, the overall quality of the data was high, which could be explained by the HMIS office staff’s dedication and the district’s proximity to Lilongwe, which may have facilitated access to reporting tools and training from the MoH. Yet, some non-negligible reductions were noted on the completeness of registers and the timeliness of report entry into DHIS2. As identified by Ntchisi’s HMIS officer during the quarterly interviews, the absence of external partners and the limited assistance provided by the DHO hindered the HMIS office’s access to the Internet and their ability to secure transport and allowances for supervision visits. This indicates that regular support to HMIS offices is required to prevent data quality loss.

Interestingly, in the intervention and control sites, we did not observe changes in the consistency between the MCH data recounted from the registers and reported in the monthly summary forms, highlighting the need for more targeted support in health centers. Our qualitative interviews in both the intervention and control districts provided indications that extracting some data from the registers, particularly the ANC and PNC registers, can be difficult for data clerks and medical professionals, given these registers follow women in cohorts (6 months for ANC and 3 months for PNC registers). The data handlers are required to determine in which cohort the women belong, based on their first ANC visit date. New or untrained professionals are more likely to assign ANC clients to the wrong cohort when aggregating the data. We also identified that, even when available, health facility personnel would sometimes refrain from using the PNC registers. The comparison districts also recorded low agreement between ANC registers and monthly reports, reinforcing the idea that facility-level health professionals and data clerks may face challenges when reporting ANC and PNC data on monthly summary forms and that more training and orientation opportunities should be provided.

Our quantitative results in Mwanza were also supported by qualitative interviews, through which the HMIS office personnel identified some advantages to our approach. Challenges were also noted, including the fact that the funds were deemed insufficient to run activities such as intensive training sessions in facilities, which could have helped improve the consistency of data from the registers to the summary reports. It is also possible that a longer intervention period, beyond 10 months could have led to more consistency of data reporting. In future studies considering this type of cash transfer strategy, efforts should be made to support the HMIS office for more than 10 months.

Although some improvements in MCH data quality were observed at the end of the intervention period, our pilot intervention had some limitations. First, we implemented the intervention in a district with a small number of facilities, which might have eased the implementation of the strategy. The effects of our strategy could have been very different in a district with a large number of facilities with competing priorities for HMIS data quality. Second, the qualitative interviews were of variable quality. Some of the responses were unclear, hence making transcription difficult. Although participants had the option of doing their interviews in Chichewa, some interviews were still done in English. This challenged some informants’ ability to answer the questions comprehensively. Lastly, as the endline data collection coincided with the first reported cases of COVID-19 in Mwanza, we missed the perspectives of additional key informants, including the DHO and the DEHO.

This pilot intervention provides preliminary evidence that our approach – providing targeted funding directly to the HMIS office – can produce data quality changes in terms of the availability and completeness of data. A comparison of data quality changes in the intervention and some control sites suggests that our intervention could lead to greater data quality improvement than more standard approaches considering by NGOs to HMIS strengthening. Meaningful improvement in the consistency of reporting may require more time to allow HMIS officers to implement frequent and intensive activities such as training, supervision visits, and data quality assessments. Our approach empowered HMIS office personnel to set their own priorities to improve data quality. The cash transfer strategy was even preferred to more standard ways of supporting HMIS that provide less autonomy to HMIS officers to make decisions. Given the MoH's focus on scaling up the performance of its DHIS2, our intervention provides a strategy that NGOs and DHOs could implement with minimal oversight.

ANC: Antenatal Care

CAIA-MNCM: Canada-Africa Initiative to Address Maternal, Newborn and Child Mortality

DEHO: District Environmental Health Officer

DHIS2: District Health Information Software 2

DHMT: District Health Management Team

DHO: District Health Officer

HMIS: Health Management Information System

HSA: Health Surveillance Assistant

LMIC: Low- and Middle-Income Country

MCH: Maternal and Child Health

MoH: Ministry of Health

NGO: Non-governmental organization

PNC: Postnatal Care

PNC48: Postnatal Care within 48 hours

Ethics approval and consent to participate

Approval for this protocol was obtained from Malawi’s National Health Science Research Committee (NHSRC) (Approval number: 2101) and The Hospital for Sick Children Research Ethics Board (REB) (REB number: 1000061129). The authors confirm that all methods were performed in accordance with relevant guidelines and regulations. Before seeking REB clearance, we obtained approval for this study from the DHOs of the four districts. We also obtained informed consent from the HMIS officers and health facility in-charges to conduct the data quality assessments. The HMIS office staff also had to provide informed consent to participate in the intervention. We also sought informed written consent from all the participants in the qualitative interviews.

Consent for publication

Not applicable

Availability of data and materials

The HMIS datasets used and/or analysed during the current study belong to Malawi’s Ministry of Health and therefore cannot be shared due to legal and privacy restrictions. Qualitative data cannot be shared owing to the consents given by the participants, which limits data to the study team only.

Competing interests

The authors declare that they have no competing interests.

Funding

This research study, nested within the Canada-Africa Initiative to Address Maternal, Newborn and Child Mortality, was supported by Global Affairs Canada, under the Partnerships for Strengthening Maternal, Newborn, and Child Health initiative.

Authors' contributions

MO, MT, and BM conceived the project. KM and DG provided valuable feedback during the design phase of the pilot study. JM performed the monitoring of the pilot study in Mwanza. MO, MT, and JM supported data collection efforts in the four districts. MO performed all data preparation and analyses. MO, MT, JM, KM, DG, BM participated in the interpretation of the findings and the writing and completion of the manuscript. All authors read and approved the final version of the manuscript.

Acknowledgements

The study team acknowledges the HMIS office and NGO staff for their time and participation during this study. We also want to acknowledge Vincent Magombo, Bester Gondwe, Dumisani Mfipa, and Virginia Kayoyo from Amref Health Africa in Malawi for their contribution to this pilot study.

AbouZahr C, Boerma T. Health information systems: The foundations of public health. Bulletin of the World Health Organization. 2005;83:578–83. doi:/S0042-96862005000800010.
Nyamtema AS. Bridging the gaps in the health management information system in the context of a changing health sector. BMC Medical Informatics and Decision Making. 2010;10.
Hung YW, Hoxha K, Irwin BR, Law MR, Grépin KA. Using routine health information data for research in low- And middle-income countries: A systematic review. BMC Health Services Research. 2020;20:1–15.
Wagenaar BH, Sherr K, Fernandes Q, Wagenaar AC. Using routine health information systems for well-designed health evaluations in low- and middle-income countries. Health Policy and Planning. 2016;31:129–35.
Ouedraogo M, Kurji J, Abebe L, Labonté R, Morankar S, Bedru KH, et al. A quality assessment of Health Management Information System (HMIS) data for maternal and child health in Jimma Zone, Ethiopia. PLoS ONE. 2019;14.
Kebede M, Adeba E, Adeba E, Chego M. Evaluation of quality and use of health management information system in primary health care units of east Wollega zone, Oromia regional state, Ethiopia. BMC Medical Informatics and Decision Making. 2020;20:1–12. doi:10.1186/s12911-020-01148-4.
Amoakoh-Coleman M, Kayode GA, Brown-Davies C, Agyepong IA, Grobbee DE, Klipstein-Grobusch K, et al. Completeness and accuracy of data transfer of routine maternal health services data in the greater Accra region. BMC Research Notes. 2015;8:1–9. doi:10.1186/s13104-015-1058-3.
Rumisha SF, Lyimo EP, Mremi IR, Tungu PK, Mwingira VS, Mbata D, et al. Data quality of the routine health management information system at the primary healthcare facility and district levels in Tanzania. BMC Medical Informatics and Decision Making. 2020;20:1–22. doi:10.1186/s12911-020-01366-w.
Malawi National Health Information System Policy.
Admon AJ, Bazile J, Makungwa H, Chingoli MA, Hirschhorn LR, Peckarsky M, et al. Assessing and improving data quality from community health workers: a successful intervention in Neno, Malawi. Public Health Action. 2013;3:56–9.
O’Hagan R, Marx MA, Finnegan KE, Naphini P, Ng’ambi K, Laija K, et al. National assessment of data quality and associated systems-level factors in Malawi. Global Health Science and Practice. 2017;5:367–81.
Hjemås G, Bråthen R, Terje Vikan S, Åge Haugen J. Improving quality on health data, recommendations and guidelines Based on the case of the Health Management Information System in Malawi and DHIS2. Statistisk sentralbyrå; 2017.
Mutale W, Chintu N, Amoroso C, Awoonor-Williams K, Phillips J, Baynes C, et al. Improving health information systems for decision making across five sub-Saharan African countries: Implementation strategies from the African Health Initiative. BMC Health Services Research. 2013;13 SUPPL.2.
Doherty T, Chopra M, Nsibande D, Mngoma D. Improving the coverage of the PMTCT programme through a participatory quality improvement intervention in South Africa. BMC Public Health. 2009;9:1–9.
Mphatswe W, Mate KS, Bennett B, Ngidi H, Reddy J, Barker PM, et al. Improving public health information: a data quality intervention in KwaZulu-Natal, South Africa. Bull World Health Organ. 2012;90:176–82.
Lemma S, Janson A, Persson LÅ, Wickremasinhe D, Källestål C. Improving quality and use of routine health information system data in low- And middle-income countries: A scoping review. PLoS ONE. 2020;15 10 October:e0239683. doi:10.1371/journal.pone.0239683.
Kiberu VM, Matovu JK, Makumbi F, Kyozira C, Mukooyo E, Wanyenze RK. Strengthening district-based health reporting through the district health management information software system: The Ugandan experience. BMC Medical Informatics and Decision Making. 2014;14.
Krishnan A, Nongkynrih B, Yadav K, Singh S, Gupta V. Evaluation of computerized health management information system for primary health care in rural India. BMC Health Services Research. 2010;10:1–13.
Dehnavieh R, Haghdoost AA, Khosravi A, Hoseinabadi F, Rahimi H, Poursheikhali A, et al. The District Health Information System (DHIS2): A literature review and meta-synthesis of its strengths and operational challenges based on the experiences of 11 countries. Health Information Management Journal. 2019;48:62–75. doi:10.1177/1833358318777713.
World Health Organization. Data quality review: module 1: Framework and metrics . 2017. https://apps.who.int/iris/handle/10665/259224. Accessed 21 Jun 2021.
World Health Organization. Data quality review: module 2: Desk review of data quality. 2017. https://apps.who.int/iris/handle/10665/259225. Accessed 19 Jun 2021.
World Health Organization. Data quality review: module 3: Data verification and system assessment. 2017. https://apps.who.int/iris/handle/10665/259226. Accessed 19 Jun 2021.
Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for Implementation Research: Conceptual Distinctions, Measurement Challenges, and Research Agenda. 2010.

No competing interests reported.

Download PDF

Editorial decision: Major revision
05 Oct, 2021
Reviews received at journal
31 Aug, 2021
Reviewers agreed at journal
29 Aug, 2021
Reviewers agreed at journal
17 Aug, 2021
Reviewers agreed at journal
23 Jul, 2021
Reviewers invited by journal
21 Jul, 2021
Editor assigned by journal
21 Jul, 2021
Editor invited by journal
21 Jul, 2021
Submission checks completed at journal
21 Jul, 2021
First submitted to journal
17 Jul, 2021

You are reading this latest preprint version

Cash Transfers for Improved Maternal and Child Health Data: A Pilot Study Supporting Health Management Information System in Malawi

Status:

Version 1

Abstract

Background

Methods

Results

Conclusions

Figures

Background

Methods

Aim of the Study

Study Design

Study Setting and Participants

Processes, Intervention, and Comparisons

Baseline Data Quality Assessment

Data Quality Improvement Strategy

End line Evaluation

Data Analysis

Results

Summary of Activities

Data Quality Assessment

Qualitative Assessment of the Cash Transfer Strategy

Acceptability, appropriateness, and perceived effects on data quality

Challenges and unintended consequences

Areas for improvement and sustainability

Discussion

Conclusions

Abbreviations

Declarations

Ethics approval and consent to participate

Consent for publication

Availability of data and materials

Competing interests

Funding

Authors' contributions

Acknowledgements

References

Additional Declarations

Status:

Version 1