A Strategy That Improves the Culture of Information Use at Primary Health Care Units in Ethiopia

Background: Health systems require high-quality data production for health service delivery and program improvement. Ethiopia’s health sector was challenged with lack of quality data from its routine health information system which is an essential element of evidence-based decision making. To overcome this, the Ethiopian Ministry of Health introduced a health information system (HIS) performance monitoring tool named the connected woreda strategy (CWS). This study aims to assess the effectiveness of the CWS to improve culture of information use at primary health care entities. Methods: The study employed a repeated cross-sectional study design through pre and post testing of district health oces using connected woreda assessment tools. A total of 78 districts were enrolled in this study to assess their performance on the implementation of the CWS. The CWS assessment checklist is comprised of 54 questions, all of which were assigned numeric number response types and categorized into HIS capacity (30%), data quality (30%), and administrative data use (40%) and further categorized as districts that meet the highest standards, (>90% of common set of criteria) i.e. ‘model woredas’, districts that meet medium standards, (between 65% to 90% score of common set criteria) i.e. ‘candidate woredas’ and districts that meet lowest standards (<65 of common set of criteria) i.e. ‘emerging woredas’. Finally, the data were analyzed using the SPSS-version 25 software. Results: A total of 78 districts were employed and described and a paired sample t-test was used to measure the performance of districts on CWS implementation. Due to CWS implementation, 10% of districts were ‘models’ in HIS performance; ‘candidate’ districts increased from 40% to 73%; and ‘emerging’ districts decreased from 60% to 17%. Finally, overall CWS scores showed signicant improvements after the intervention with 73.4 ± 10.48 SD compared to before intervention 60.4±13.69 SD, with t (77) =-7.18 and p=0.001. Conclusions: This study has revealed that the CWS implementation has a positive effect towards cultural transformation of use of data for decision making at primary health care entities. Thus, scaling up the connected woreda implementation is recommended at the national level to improve the performance of primary health


Background
Strong health systems are highly dependent on good performance of health information systems at primary health care entities (1). Data quality is characterized as timely, credible, accurate and complete data that can be effectively used to track program performance, and thus positively impact strategic planning and decision-making processes (4,5). Low-and middle-income counties have been working to strengthen their routine health information system in terms of building sustainable use of quality data for evidence-based practices and patient care at primary health care units to ensure universal health care (1).
Ethiopia has made signi cant investments on improving the performance of HIS at all levels of the health system (2). Moreover, the performance of HIS contributes towards the country's HIS capacity, data quality and information use (2,3). Despite the several efforts that have been exerted to strengthen the performance of HIS, there are persisting gaps in governing the HIS to produce quality data and information use for decision making at all levels of the health system. Among the gaps are the absence of a HIS governance framework, weak enforcement on standards, lack of standards for some digitalized health systems (e.g. EMR), absence of interoperable systems, and poor data demand and information use culture.
In addition, Ethiopia grapples with poor quality data production and limited information use at primary health care entities due to poor documentation, incomplete and untimely reporting, and low HIS capacity at the lower levels of the health system (6). This can lead to limited information use and wrong decision making in health service delivery and health programs (6). Ethiopia has introduced the connected woreda strategy as an integral part of its national HIS strategy termed the information revolution agenda (2) which is one element of the country's transformation agenda stated in its ve-year Health Sector Transformation Plan (HSTP) (2,6).
The connected woreda strategy (CWS) is de ned as a tool that strengthens the performance of the health information system (HIS) at primary health care entities (7). The CWS is an indigenous, district level HIS performance monitoring tool that improves HIS capacity, data quality and information use culture at the point of data production (2,6,9).
The CWS applies assessment tools that check the status of primary health care units on a quarterly basis through continuous mentorship and coaching activities (7). The assessment tool has 18 major and 36 composite criteria categorized as: HIS capacity (30%), data quality (30%), and information use (40%). The connected woreda strategy is being implemented all over the country at the primary health care level as part of Ethiopia's transformation agenda (7).
The CWS has begins with an accreditation process where woredas/districts are evaluated and scored against a common set of criteria related to HIS resource, data quality, and administrative processes.
Woredas/districts that meet the highest standards, ( > = 90% of common set of criteria), are recognized as 'model woredas', woredas/districts that meet medium standards, (between 65-90% score of common set criteria) are recognized as 'candidate woredas' and woredas/districts that meet lowest standards (< 65 of common set of criteria) are recognized as 'emerging woredas'. Woredas that enable online data access and transmission in addition to being 'model woredas' are recognized as 'connected woredas' (6). The contribution of the CWS towards the improved performance of HIS has yet to be documented.
The USAID Transform: Primary Health Care Activity has been providing technical and nancial support to the government's strategic initiatives in Amhara, Oromia, SNNPR and Tigray regions. A total of 400 districts, 1,856 health centers and 9,291 health posts, and 122 primary hospitals are covered by the Activity through the provision of phased and adaptive technical assistance. The Activity has integrated the CWS practices as part of the key components of its interventions to improve the performance of HIS and create a culture of information use for evidence-based practices at primary health care entities. This study will assess the connected woreda strategy in improving the culture of information use through pre and post analysis of primary health care entities in Ethiopia.
Operational De nitions I. The connected woreda strategy (CWS) is a tool to create model/high performer woredas/districts in use of data for planning, budgeting and evidence-based practice to improve service delivery and patient care. The initiative operationalizes data-use innovations through instituting a tiered pathway for woredas/districts to achieve the highest standards in data quality and use. The pathway begins with an accreditation process where woredas/districts are evaluated and scored against a common set of criteria related to HIS resource, data quality, and administrative processes. Woredas/districts that meet the highest standards, ( > = 90% of common set of criteria), are recognized as 'model woredas', woredas/districts that meet medium standards, (between 65-90% score of common set criteria) are recognized as 'candidate woredas' and woredas/districts that meet lowest standards (< 65 of common set of criteria) are recognized as 'emerging woredas'. Woredas that enable online data access and transmission in addition to being 'model woredas' are recognized as 'connected woredas'. (7). See the II. Health information technicians are data managers responsible for the implementation of the routine health information system and that mainly perform activities such as data collection, data cleaning, data analysis, data production as well as data dissemination and use at the district level.
III. Health management information system (RHMIS): In Ethiopia, the HMIS is managed by the Federal Ministry of Health and is a primary source of information for the continuous monitoring of health services in the whole In response to socio-economic changes, disease patterns and the introduction of international strategies, the current HMIS incorporates standardized new initiatives.
Intervention areas USAID Transform: Primary Health Care is working on the implementation the (CWS) to improve the performance of HIS at primary health care levels. CWS applies tools that improve HIS system and capacity, data quality and administrative data use which are major factors affecting the performance of HIS at the primary health care level. As part of this effort, the Activity's major intervention areas were categorized as: 1. training, 2. mentorship and coaching, and 3. best practice sharing. Each of these categories are described below.
1. Training: USIAD Transform: Primary Health Care developed a ve-day training material on use of data for decision making (UDDM) for primary health care entities. The content of the training includes CWS as part of HIS reform agenda, health management information system (HMIS), performing lot quality assurance sampling (LQAS), district health information system 2 (DHIS2), electronic community health information system (eCHIS), etc. (Additional le 2). The Activity, in collaboration with Regional Health Bureaus (RHBs), provided training of trainers (TOT) on UDDM for 106 regional and zonal health information technicians within Amhara, Oromia, SNNP and Tigray regions. UDDM TOT trained professionals were then mentors for health information technicians (HIT) at district and primary health care entities. A total of 677 HIT professionals were trained on UDDM from district o ces and primary health care units within Amhara, Oromia, SNNPR and Tigray regions to improve performances of HIS at the point of data generation. In addition, gap lling HIS inputs such as 667 push pin boards for data visualization were procured and distributed to primary health care entities. Moreover, the DHIS2 application was installed for HIT professionals for data collection, analysis, data visualization and reporting purposes. Program monitoring dashboards were created on the DHIS2 application by UDDM trained HIT professionals to improve data use and evidence-based practices at the primary health care level.
2. Mentorship and coaching: USIAD Transform: Primary Health Care provided continuous mentorship and coaching for HIT professionals at the primary health care level. An average of four coaching sessions were conducted using the CWS such as HIS capacity e.g. availability of adequate human resource, HIS budget, system structure and supportive supervisions; data quality e.g. timely completion and reporting, high quality data (highly detailed), and proper use; administrative data use e.g. a functioning performance monitoring team (PMT), data analysis and dissemination (coverage, equity, quality indictors) (7). See the Fig. 2 below on connected woreda strategy (CWS) assessment criteria.
In addition, on average, four mentorship and coaching sessions were delivered, and tailor-based support was provided for each district and primary health care unit on a quarterly basis. During each mentorship and coaching session, priority problems/challenges were identi ed using CWS assessment criteria, solutions were identi ed, and action plans were prepared and regularly reviewed. Average scores for each mentorship and coaching session were registered at a baseline score of 60% 'emerging'; a 1st session coaching score of 65% 'candidate'; a 2nd session coaching score of 68% 'candidate'; a 3rd session coaching score of 72%'candidate', and a 4th session coaching score of 78% 'candidate'. Moreover, USAID Transform: Primary Health Care developed and implemented an Excel based dashboard and created a data use forum through a phone application channel, (Telegram), to facilitate mentorship and coaching on CWS at the district level. Furthermore, the created forum contributed signi cantly to experience sharing of best practices on HIS performance among HITs at district and primary health care levels. Sharing of best practices: USAID Transform: Primary Health Care has been working to strengthen learning collaborative forums between 'emerging' and 'model' districts and primary health care entities through the Ethiopian primary health alliance for quality (EPAQ) initiative. EPAQ is an Ethiopian health reform initiative used as a learning collaborative forum and is designed facilitate experience sharing, learning forums and collaboration to improve service delivery at primary health care entities. The 'model' districts were the facilitators of the EPAQ forum, and the remaining four districts were members. Both lead and member districts identi ed thematic areas for collaboration and signed an agreement. This created linkages between high performing 'model' and low performing 'emerging' districts. As a result, various success stories on HIS capacity, data quality, and information use were compiled from 'model' districts were shared with 'emerging' ones within Amhara, Oromia, SNNPR, and Tigray regions. A total of eight EPAQ forums were organized at the district level in collaboration with partners working at the primary health care level (20,21).

Study setting
USAID Transform: Primary Health Care is being implemented in the four agrarian regions of Ethiopia: Amhara, Oromia, SNNP, and Tigray. Each of these four regions is divided into zones and each zone is divided into districts. A district health o ce is located at the lower level of the administrative structure which is key for primary health care performance. The Activity works in 400 districts; 95 (23.7%) of which are in Amhara, 162 (40.5%) are in Oromia, 120 (30%) are in SNNP, and 23 (5.8%) are in Tigray regions (10). See the Fig. 3: connected woreda strategy implementation (CWS) status by districts (woredas), Ethiopia from July 2019 -September 2020.

Study design and instruments
The study employed a repeated cross-sectional study design through pre and post testing of district health o ces using CWS assessment tools, assessing the effect of the connected woreda strategy implementation on improving HIS performance at the primary health care level. The study assessed data from 78 district health o ces. The selected CWS implementing districts for this study were those which had at a minimum, received three mentorship and coaching sessions from the Activity using the CWS assessment tools. Both baseline and post intervention data were collected using the CWS mentorship and coaching tool which compared data from 78 districts. The study outcome variables are data quality and information use that directly affect the performance of the HIS. The CWS assessment checklist is comprised of 54 questions all of which used numeric number response types. The 54 questions were categorized into HIS capacity (30%), data quality (30%), and administrative data use (40%), (Additional le 1) and districts were then further categorized as 'emerging' (below 65% of common set of criteria), 'candidate' (between 65% and 90% of common set of criteria) and 'model', (above 90% of common set of criteria) based on the total sum of all criteria met.

Data source
The CWS assessment tool is a set of questions related to HIS capacity, data quality and administrative data use, (Additional le 1), which was developed by the Ethiopian Ministry of Health of to guide eld level support as part of the health information reform agenda. In addition, the Activity has developed a CWS dashboard to support evidence-based practices and data driven decision making at the district level. The assessment tool is organized to frame a two-way discussion between the mentors and performance monitoring team (PMT) members at each district. Each question has a documentation section for de nitions, decision points and responses to inform improvement plans. This study employed the CWS program outcome monitoring data collected from 78 district health o ces during the January to December 2019 scal year.

Sample size
A total of 78 district health o ces were considered for this study. The sampled districts were from information revolution targeted districts which had received CWS coaching and mentorship by the Activity at least three times from January to December 2019, of which 35 were in Amhara, 21 were in Oromia, 17 were in SNNPR and 7 were in Tigray regions.

Data collection
During district health o ce mentorship and coaching support, data collection and entry was conducted onsite using a CWS dashboard and an online electronic data entry system; DHIS2 application (on CWS mentorship and coaching data). Both online and o ine data entry were performed on the public sector's DHIS2 application.

Data analysis
Data were managed using the project DHIS2 dashboard and a web-based system for mentorship and coaching data using DHIS2 and were then exported to SPSS version 25 for statistical analysis. Descriptive statistics were performed for all study variables. The mean and standard deviations were calculated for three categories and the differences in mean scores of HIS capacity, data quality and administrative data using pre and post tests were examined using dependent (paired test) sample t-test and Levene's test, with the level of signi cance being determined at a p value < 0.05.

Ethical considerations
Ethical clearance was obtained from JSI-IRB, (reference number IRB #20-39E) to use the Activity led District Health Information System2 (DHIS2) connected woreda coaching data, and permission was sought and obtained from USAID Transform: Primary Health Care Activity. Data privacy, anonymity and con dentiality was maintained throughout the analysis and report writing activities.

Characteristics of the study districts
A total of 78 districts were characterized, analyzed and used in the study to understand the status of sampled districts. On average, the population of the districts were 142,547, the number of kebeles were 26.2, the number of health centers (HCs) were 5.3, the number of health posts (HPs) were 24.2, the number of computers per HC were 2.1, trained staff per HC on HMIS were 1.7, 0.7 on UDDM, 1.5 on DHIS2 and 1.1 on CWS. In addition, the status of ICT infrastructures was described as the proportion of districts with electricity (87.2%) and internet connectivity (28.2%). See the Table 1 on demographic characteristics of districts.

CWS categorization of districts
In this study, a total of 78 districts were analyzed to compare the effects of the CWS implementation before and after the intervention. It was found that there were signi cant differences between before and after the intervention periods. Moreover, the number of 'emerging' districts decreased from 60-17%, 'candidate' districts increased from 40-73%, and a total of 10% 'model' districts were created after the intervention as compared to the baseline data. See the Fig. 4 below the performance of connected woreda strategy before and after intervention by category.

CWS implementation and HIS performance
In this study, a total of 78 districts were analyzed using paired sampled t-test to measure performance of districts on CWS implementation. Overall, there were statistically signi cant differences between before (baseline) and after the intervention on HIS capacity, data quality and administrative data use. HIS capacity scores after the intervention was data 21.6%± 4.67 standard deviation (SD) showing signi cant improvement from a baseline score of 17.7%±5.16 SD, with t (77) =-5.66 during the Activity's implementation phases (p = .001). Data quality in 23.6% ± 4.34 SD showed improvements form a baseline score of 18.3%±5.88 SD, with t (77) =-7.20 with p = .001. Moreover, district level data use for planning and decision making also signi cantly improved from a baseline score of 24.4 ± 7.7 SD to an after-intervention score of 28.1 ± 6.24 SD with t (77) =-3.60 and p = .001. Finally, overall improvements were observed after interventions (73.4%± 10.48 SD) from a baseline score of 68.1%±13.69 SD, with t (77) =-7.18 and was statistically signi cant (p = .001). See the Table 2 below on before and after intervention analysis using paired sample test.

Discussion
In this study, the CWS implementation was shown to be an effective tool in improving the performance of districts on the implementation of HIS. Furthermore, 10% of districts changed their status from emerging or candidate to model; the percentage of 'candidate' districts increased from 40-73%; and the percentage of 'emerging' districts signi cantly decreased from 60-17%. According to the World Health Organization (2008), the performance of health information systems is highly dependent on national strategies or standard tools that strengthen HIS system and capacities, quality data production and use of data for evidence generation for health service delivery at primary health care (11,17,24). Therefore, Ethiopia has introduced standard HIS tools to strengthen HIS system and capacity, data quality and data use (6,23). The country has gained signi cant improvements from the implementation of the connected woreda strategy towards its HIS performance.
In this study, the results have shown that implementing the CWS initiative signi cantly improves the performance of districts on HIS system and capacity. Following the CWS implementation, districts changed their status from limited HIS inputs to adequate HIS inputs, all districts assigned enough data managers or health information technician (HIT) professionals, all districts showed improvement on quarterly integrated supportive supervisions, and districts also understood the usefulness of the initiative and agreed to assign adequate budget for HIS activities including for HMIS printing and procurement of computers, etc. HIS system and capacity initiatives contribute signi cantly to the performance HIS at the primary health care level. This nding was consistent with similar research in other settings on performance of HIS in low-and middle-income countries indicating that continuous integrated strategies using HIS standard tools on health information system signi cantly improves HIS system and capacity (12,13,14). Furthermore, the performance of districts towards HIS system and capacity were signi cantly improved through integrated strategies at the primary health care level (15,16,19).
In terms of data quality, the ndings of this study indicate that there were signi cant improvements in terms of quality data production as compared to before the implementation of the CWS. The study re ected that both reporting timeliness and completeness were signi cantly improved, quality of data production was monitored using RDQA and LQAS techniques on a monthly basis, and data veri cation for primary health care entities was regularly performed. As data quality is not a one-time activity and needs regularity, the CWS implementation creates a culture of monitoring quality data production at the primary health care level which signi cantly improves the performance of HIS. Similarly, according to the Lancet, Global Health Report 2018 ndings, quality data production is a major contributing factor towards quality of health service delivery that needs major standard tools to support the use of quality data for evidence-based practices (1).
From the data use perspective, this study highlighted improvements on building sustainable and functional performance monitoring teams (PMTs). Moreover, PMTs were regularly organized and performance reviews were conducted on a monthly basis. Districts selected key equity and quality indicators, identi ed major gaps, provided solutions, and prepared and circulated action plans. Furthermore, districts signi cantly improved their status on use of data for planning, budgeting and decision making in health service delivery at the primary health care level. Similar results were presented by the Measure Evaluation 2018, which showed that data use is a milestone to strengthening evidencebased practices in health service delivery and patient care. Therefore, data use needs monitoring systems to create a culture of data use and keep regularity at all levels in the health care system (1,12). As lowand middle-income countries grapple with poor-quality data production and limited information use at the primary health care level, strong integrated supportive supervision and strategies are required to overcome problems in data use at district levels (12,17,18).
This study revealed the importance of the CWS towards data use culture at the district level. Moreover, use of quality data for performance management and improvement at the district level has a direct implication on in uencing the performance of RHIS and health system quality. Another study conducted in Kenya also showed that integrated health information system mentorship and coaching can improve e cient use of data for decision making (UDDM) by improving the capacity of districts in components of HIS, such data collection, data analysis, data production and dissemination and use (12,13,18).

Limitations
This study has some limitations that must be considered. The rst limitation is that it is di cult to discuss causality on repeated cross sectional (before and after intervention) study design due to lack of a comparison or a control group. Additionally, opinions of health information technicians on compliance with the standards were not included.

Conclusion
District health o ces are a place for data production for evidence-based practices and health service delivery at the lower levels of the health system in Ethiopia. The connected woreda strategy is a tool that improves the culture of information use at the primary health care level. There are clear performance differences between the baseline and post intervention stages that were uncovered using a pre and post analysis of the initiative. The provision of the connected woreda strategy implementation using its assessment tools resulted in clear differences on creating cultural transformation on the use of data for decision making that also improved the performance of HIS such as HIS capacity, data quality and information use. Therefore, scaling up the CWS implementation at the national level is recommended. The framework for the connected woreda strategy (CWS) pathway Connected woreda strategy implementation (CWS) status by districts (woredas)' Ethiopia July 2019 -September 2020. The ArcGIS map presents the location of Ethiopia, Amhara , Oromia, SNNPR and Tigray regions and 78 districts. In addition, the Connected woreda strategy implementation (CWS) status were presented with spatial analysis by districts (woredas). The map can be reproduced or used after acknowledging the authors. (Source: Shape le from Ethiopia Central Statistical Agency (CSA), 2016 at https://africaopendata.org/dataset/ethiopia-shape les) Note: The designations employed and the presentation of the material on this map do not imply the expression of any opinion whatsoever on the part of Research Square concerning the legal status of any country, territory, city or area or of its authorities, or concerning the delimitation of its frontiers or boundaries. This map has been provided by the authors.

Figure 4
Connected Woreda Strategy (CWS) categorization by Woreda -Before and After Intervention

Supplementary Files
This is a list of supplementary les associated with this preprint. Click to download.