Tools availability
A total of 115 healthcare facilities in 11 districts were assessed. Both urban and rural districts were included in the study. The urban districts were Dodoma (11), Igunga (10), Kahama (10), Kinondoni (18) and Njombe (10). The rural districts were Hai (10), Kibaha (8), Mbinga (10), Mbulu (8), Nkasi (10) and Tandahimba (10). Dodoma had no district hospitals, therefore, the respective regional hospital was included. Due to a large number of private facilities in Kinondoni district, additional private hospital and two private health centres were included. Of the 115 HFs, 58.3% (n=67) were dispensaries, 31.3% (n=36) health centres and 10.4% (n=12) hospitals. Of all the HFs, 114 had OPD, IPD (43), ANC (108), PNC (105), PITC (94), LnD (93) and FP (88) service-areas.
The overall median availability rate for registers was 91.1% (IQR:66.7%,100%) compared to 77.8% (IQR:35.6%, 100%) and 86.9% (IQR: 62.2%,100%) for the tally-sheets and report forms, respectively (Table 1). HMIS tools were mostly available at the dispensaries 91.1% (IQR:60%, 100%) than health centres 82.2% (IQR: 55.6%, 100%) and hospitals 77.8% (IQR:30%, 97.8%) (p-value<0.0001). Faith-based owned facilities had a significantly higher amount of tools available than the government and private-owned facilities (p-value<0.0001). The service-areas with high tool availability rates were ANC 95.6% (IQR:73.3%,100%), FP 93.3% (IQR:66.7%,100%) and LnD 91.1% (IQR:73.3%,100%). PITC had the lowest rate 53.3% (IQR:20%,88.9%). Hai, Kibaha, Mbinga, Mbulu (rural districts) had the highest availability rates while Kinondoni and Dodoma (urban districts) had the lowest levels (Table 1). A high variation in the range value was observed in the tally-sheet indicating a significant difference in its utilisation between facilities. A remarkable increasing trend in the availability of tools with lesser variation between HFs was observed from 2014 (median=83.3%; range=100%) to 2017 (median=100%; range=22%) (Table 1; Figure 3).
In terms of availability of tools, we categorised HFs into 4 groups: (i) >75%-100% (very high); (ii) 50%-75% (high); (iii) 25%-50% (average); and (iv) <25% (low). In all service-areas, with exception of PITC, over 50% of the facilities were able to locate up to >75% of the required registers (Figure 4), with high percentages observed in the ANC (82.4%), OPD (74.6%), LnD (73.1%) and FP (72.7%) service areas. PITC registers were rarely available with only 45% of the facilities been able to locate >75%. Over 15% of the facilities presented less than 25% of the expected PITC registers followed by PNC service-area (11.4%). Tally-sheets for ANC, LnD, and FP were available in larger proportions than those for IPD, OPD, and PNC (Figure 5). Report forms were highly available in all service-areas except for PITC which had 42.1% of facilities providing ≤25% of expected report forms (Figure 6). Urban districts of Igunga, Kibaha, Njombe, Kinondoni, and Tandahimba fell into average or low categories of tool availability particularly on tally-sheets.
At the district office, the overall median availability of submitted HF reports was 65% (IQR:48%,75%) indicating that a third of expected report forms were not found. The district-specific performance indicated that less than half of the expected reports were found in the urban districts of Dodoma (median=45%, IQR:25%,51%) and Kinondoni (median=46%, IQR: 41%-50%) (Figure 7). Rural districts had higher rates: Hai (75%, IQR:67%,82%), Igunga (74%, IQR:56%,81%), Mbinga (73%, IQR:67%,76%), Nkasi (72%, IQR:66%,81%), and Mbulu (71%, IQR:71%,86%). We matched availability of reports at district level and HF by service area for each of the four quarters included in the 12-month detailed review period. The findings indicate higher availability of reports at the HF level than at the district level with variations between service areas and HFs. Reproductive health service areas (PNC, LnD, ANC) presenting best performance. The PITC reports were difficult to find at both levels, especially at district office while the IPD reports were often missing at the district office. Challenges in transmission of reports, differences in programme reporting practices and weakness in the filing system at the district office were reported. There was no standard filing system of the received HF reports which hindered assessing if the report was received or not. In most districts, there was an increase in the availability of report forms over the years. However, the availability rate in the urban districts of Dodoma, Kibaha, Kinondoni and Njombe remained low during the period under review (Figure 8).
Completeness
Wrongly filled or empty cells in HF registers were common. Diagnoses were either not recorded or recorded without indicating disease severity (as instructed) or without laboratory results when available. This was common for malaria and anaemia. In OPD registers, it was a common practice for patient’ height and weight variables to be left blank, and occasionally sex and age were not filled.
Poor adherence to the coding procedures was frequent. For instance, instead of using “N-Ndiyo” and “H-Hapana” (Kiswahili words for “Yes” and “No”, respectively), several records were in the English version of the words “Y-Yes” and “N-No”. In other situations, instead of using a “tick” mark as instructed when the service was provided, a recorder would use “N” or “X”, or leave the entries blank or use a different code that meant a different thing altogether. Consequently, this resulted in changing the meaning of that particular record. In some cases, health workers couldn’t remember the meanings of some of the codes they used. Such practices were reported to complicate compilation of the report, especially if a different person (from the one who did the recording) is compiling the report.
Improper use of carbon papers was observed in HFs, to the extent that it was difficult to identify the value recorded in the report form. The use of worn-out carbon papers was common and resulted into a blank or very faint report copies. Such poor recording practices led to differences between recounted and reported data, hence low accuracy performance.
Data accuracy
Antenatal care service (ANC). At the HF phase, Diff1 indicates over 50% representation of data in tally-sheets while Diff2 shows extreme over-representation (of close to 3-folds) in the reports compared to registers’ records. A similar pattern was observed for Diff6 when registers counts were compared to the DHIS2 records. The transmission phase indicated consistency. The slight difference between Diff2 and Diff6 (with stable Diff3-Diff5) suggests that the reports transmitted to the district were manipulated (corrected/revised) before entered in the DHIS2, yet the changes were not documented. The over-representation levels decreased slightly over the years (Table 2).
The indicators for the provision of tetanus vaccine (TT2) and malaria intermittent preventive treatment (IPT2) performed poorly with lower registration than what had been tallied (>2-folds), compiled, and reported (>3-folds). This implies that there was intensive marking of clients in tally-sheets without registration. Indicators on gestation age and HIV testing for pregnant women were moderately matched in the tally-sheets and reports. The indicator for HIV testing among pregnant women <25 years old was highly under-represented in all phases (Diff1=0.31, Diff4=0.14; Diff5=0.16 and Diff6 =0.03). Data were found in registers but not reflected in the tally-sheets or report forms or DHIS2. There was variation in the ANC performance by districts in Diff1, Diff2, and Diff6 with much higher over-representation observed in Mbulu, Kinondoni, Kahama, and Nkasi districts.
Labour and Delivery service (LnD). At the HF level there was over-representation of data in the tally-sheets and reports as compared to what was recorded in the registers. In 2016, at the transmission phase, there was an over-representation of data in the district report as compared to the copy available at the facilities indicating that the reports were not comparable. There was no significant difference between Diff2 and Diff6. The Diff6 values decreased over time indicating an improvement in the data accuracy (Table 3). The first indicator had good matching levels at the HF phase. However, it was found to be revised at the transmission phase, thus more records were observed in the district copy than in HF (source) copies, DR=1.31. For the second LnD indicator, few data were observed in the registers than tally-sheets or reports. Although a large number of clients were indicated to deliver at the HF as marked in the tally sheet, almost none were marked of who assisted in the delivery. The values in a report for this period matched with those of who delivered at the HF. There was a little variation on the data quality performance by district on LnD.
Post-natal (PNC) service. For the PNC, the quality of data, especially in the filling of tally-sheets and compilation, improved significantly over the years. However, the results indicated that sometimes the data journey was not followed hence resulting in larger DR at report/register (Diff2) than tally sheet/register (Diff1). Diff2 and Diff6 were very similar indicating that data management at the transmission phase does not influence the quality of PNC data (Table 4). Although about half of postnatal indicators performed well in Diff1, there were variations in Diff2. The first indicator (attendance within 48 hours), had moderate over-representation, indicating that tally-sheets captured more attendees than those recorded in the registers (Diff1=1.30). However, data were extremely over-represented in report forms compared to data entered in the registers. The Diff2 of 3.09 implied that registers had less data compared to what was included in summary reports. This indicates that reports were not compiled using data from the tally-sheets and cases were not recorded in the registers but somehow summed up in the reports. District performance in PNC differed highly in Diff2 and Diff6. Health workers reported some of the PNC registers and indicators to be difficult to understand.
Family planning (FP) service. For the FP service area more data were found in the registers than in the tally-sheets (DR less than 1). Comparing earlier years (2014 and 2015) against recent ones (2016 and 2017), a high improvement was observed at the transmission phase (Table 5). However, the under-representation of data in tally-sheets did not improve. Overall, half of the indicators in FP services performed quite well with data presenting good matching between tally-sheets, registers and report-forms. An indicator on cervical cancer screening presented a DR less than 1 for Diff1 indicating more data were recorded in the registers than tally-sheets. The screening for breast cancer had a DR of 1.34 for Diff2 indicating that data were compiled in report forms but not indicated in registers. Most of the variations between district performance in FP were observed in Diff1.
Outpatient (OPD) service. This service area indicated the highest levels of mismatch in the HF and robust phases. Diff1 showed moderate over-representation in tally-sheets versus registers, which improved significantly over time suggesting adoption on the use of tally-sheets. Extremely large Diff2 and Diff6 values were observed in 2014-2015. It was observed that records in the reports could be up to 5-7-times higher than register records, but was better in 2017 suggesting an improvement in client registration. The transmission phase performed well suggesting moderate manipulation of HF reports before been entered in the DHIS2. The variation in this manipulation between indicator, HFs, and districts is captured by the slight differences observed between Diff2 and Diff6 (Table 6). Individual OPD indicators did not perform well. Data obtained from registers were much less than in the report forms. The indicator of mild/severe anaemia performed worse with DR value indicating a difference of over 6-times between the register records and the report form. Blood smear positive records were corrected in the report forms by inflating values in the copy at the district, with a plus that the changes were documented (Diff3/Diff4>1.25). The performance varied between districts, with Kinondoni and Kahama having high levels of data over-representation.
Inpatient (IPD) service. IPD was among the service areas that indicated a high level of mismatch between data sources, more particularly over-representation of data in the reports versus records in tally-sheets or registers. This was presented at the HF phase with Diff1 values of >2 in 2014 and 2016, and Diff2 reporting extreme representation from 2014 to 2017 indicating fewer records in registers versus reported values. Diff1 improved slightly in 2015 and then significantly during 2017 which indicates a better use of tally-sheets. For Diff2 there was no indication of improvement observed during the 4-year period under review (Table 7). At transmission phase, an improvement was observed as the data matched better for the 2016 and 2017. Most of the IPD indicators presented differences between data sources of at least 50%. Data on severe anaemia was extremely over-represented in the tally-sheets and report forms compared to register records (Diff1 =2.38 and Diff2=3.81). This implies that data were not found in registers but were marked in tally-sheets and recorded in the report forms. High difference between registers and reports were observed. There was high variation between district performance in IPD data accuracy mainly in Diff1 and Diff2 with Kinondoni and Dodoma under-utilizing tally-sheets and Mbulu over-representing data in the reports.
Provider-Initiated Testing and Counselling (PITC) service. In this service-area, Diff2 went from moderate (in 2014) to extreme (in 2017) report over-representation indicating weakness in the registration process. PITC reports were highly manipulated during transmission phase before data was entered in DHIS2, marked by large Diff3/Diff4. There were significant variations between HFs and districts for Diff2 and Diff6 (Table 8). Little improvements were observed over the years. Indicator on ‘number of new clients’ was 2-fold over-represented between the register counts and the reported records and its data was corrected before been entered in DHIS2. Kinondoni, Kibaha and Nkasi showed high levels of data over-representation. Mbulu had matched data for Diff2 but high Diff6 indicating the submitted reports were revised before data were entered in the DHIS2 though, the changes were not documented. Nkasi had the highest Diff4 and Diff5 indicating corrections made during the transmission phase.
An overall annual pattern indicated a slight improvement on Diff1 (from 1.37 in 2014 to 1.26 in 2017), but a high improvement on Diff2 (from 2.61 in 2014 to 1.70 in 2017). This indicates that although tally-sheets were not fully utilized, the reports were better prepared in 2017 than in 2014. Similarly, there was a marked improvement in values for Diff6 from 2.72 in 2014 to 1.76 in 2017, indicating less variation between register’ records and DHIS2 entries over the years. Data accuracy by HF levels categorized by service areas indicates high Diff2 and Diff6 for hospitals, and in OPD, IPD, ANC, and PNC (Table 9). Data accuracy was observed to vary between HFs even within the districts.