Availability
Data were extracted from nine centres in Honoria City Council and 64 health centres in Malaita Province. The MCMR records were not available for Gundacanal Province. Microscopists usually undertook blood examination in health centres while RDTs were performed by nurses. Some of the facilities had shortage of RDTs and adequate number of OPD registers [“(MCMR) book is there but sometimes need to photocopy extra copies for every table in OPD” FDG3]. The results were recorded, and then compiled and collated using MCMR forms by the nurses at the health centres and submitted to the provincial headquarter every month [“A lot of time, we come back at the end of the month [we try to come back] and fill up the forms and compile it. Those that were recorded in the OPD book it is easy for us to transfer to MCMR forms” FGD2]. After MCMR forms reach the provincial headquarters, malaria monitoring and supervisor officers enter the data into the online DHIS2 software [“So, my daily responsibility is for entering that (data) into the system (DHIS2)” IDI9].
Completeness
Only 28% of reports were 100% complete for all nine indicators. The respondents had a good understanding of the completeness of reporting, including the need for regular reporting even when there were no cases to report [“ ‘Forms completed’ means that each [form is] properly filled [with] no data missing,” FGD1] and [“ ‘completeness’ is filling all the things that are asked in the forms” FGD2 ].
Nearly half (47.2%) of all indicators were complete in Honoria City Council, while only 25.5% were complete in Malaita Province. The highest completeness amongst the nine indicators was ‘P. falciparum diagnosed with RDT’ at 90.3%, followed by ‘P. falciparum and P. vivax diagnosed with a microscope’ at 90.1% each. The ‘stock balance of drugs’ and ‘stock balance of RDTs’ were the least complete indicators with 45.1% and 38.5% completeness respectively. Honoria City Council had better completeness than Malaita Province for all nine indicators. In Honoria City Council, ‘total parasites’, and ‘P. falciparum and P. vivax diagnosed with RDT’ were 95.4% complete. ‘RDT stock balance’ was the least complete indicator with only 53.8% complete. However, in Malaita Province, ‘P. falciparum diagnosed with microscopy’ was the most complete indicator, with 89.7% completeness, followed by ‘P. falciparum diagnosed with RDT’ with 89.6% completeness. Similar to Honoria City Council, ‘RDT stock balance’ was the least complete indicator with 36.5% completeness (Table 3).
Table 3
Completeness of nine indicators from malaria case and morbidity reporting forms, Solomon Islands
Indicators | | Overall | | HCC | | MP |
| Number | % | | Number | % | | Number | % |
Repoot submission date | | 559 | 63.8 | | 86 | 79.6 | | 473 | 61.6 |
Total parasites | | 770 | 87.9 | | 103 | 95.4 | | 667 | 86.9 |
PF | RDT | | 791 | 90.3 | | 103 | 95.4 | | 688 | 89.6 |
PF | MIC | | 789 | 90.1 | | 100 | 92.6 | | 689 | 89.7 |
PV | MIC | | 789 | 90.1 | | 103 | 95.4 | | 686 | 89.3 |
Clinical malaria | | 782 | 89.3 | | 103 | 95.4 | | 679 | 88.4 |
Total test | MIC | | 499 | 57.0 | | 91 | 84.3 | | 408 | 53.1 |
Total test | RDT | | 611 | 69.8 | | 89 | 82.4 | | 522 | 68.0 |
Drug stock | | 395 | 45.1 | | 71 | 65.7 | | 324 | 41.2 |
RDT stock | | 337 | 38.5 | | 57 | 52.8 | | 280 | 36.5 |
Several factors facilitated the completeness of reporting. Nurses were assisted in filling in all fields of the reporting forms by other staff members in the health centre such as microscopists and laboratory technicians. As they shared the same office premises, nurses sought the help of microscopist and laboratory technicians when they needed clarification on the data recorded by them. Other enablers included training and supervision of nurses by the provincial supervisors, [“There should be … constant refresher [and] training in how the data should be documented because … those are the things [that] can improve [completeness]” IDI10].
The study respondents also identified some challenges for the completeness of reporting. The handwriting of nurses was often not clear enough to understand the information written in the registers. Heavy workload critically affected the completeness of reporting, [“There is a heavy workload, lots of times, …. there is lots of pressure, demand from patients and we run all over the place and in the end we forgot to write some [information] of these people in the record book” FGD1]. As a result of a heavy workload, nurses often multi-tasked, including providing regular care to the clinic patients whilst attending to administrative matters such as recording in the registers [“.. talking from my experience when a clinic does not have too many patients, we pay attention when filling up the forms. But when we have lots of patients and you are alone, that’s the problem” FGD2]. A lack of regular training and refresher courses also led to incomplete forms, [“In our unit, some of the staff [nurses] did not attend the training. So they lack the understanding of the filling up of forms [correctly]” FDG1], and [“[Provinical] officials should come and give us training. See that we are doing the right thing, [we] understand [filling] the form properly, then we will be able to [properly fill the forms]” FDG2]. The workload associated with reporting the individual records of all patients was seen as one of the reasons for incomplete reporting. This was true especially in some health centres with a large number of cases. In this regards, a participant in an IDI said, “…for a small country like Solomon [Islands]… entering 86,000 cases … is a huge task. … for the some of the provinces you’ll need at least two or three people just doing only this” [IDI6].
Use of technology such as computers that can automatically summarize the records for the month at the centre level (as opposed to the health centre level, where DHIS2 is used to complete this task) can help improve the completeness of reporting, [“ I feel I should have a computer or program that I fill in the data here and it summarises the data at the end of the month …[so that] I don’t have to do [a] tally” FDG2]. The nurses thought that supervisory visits also helped in improving the completeness of reporting. During those visits, a supervisor would have the opportunity to review the draft report and provide feedback to the nurses if they find any incomplete or inconsistency reporting. However, regular supervisory visits were not often observed in all areas due to inadequate funding and availability of supervisory level staff members [“[without supervisory visits] that’s when nurses don’t think seriously about the importance of these quality data collection because, like those at a higher level, they didn’t come down and visit us to give feedback, supervision. We take this information for granted” FGD2].
Timeliness
The interview respondents knew that the reports should reach the provincial headquarters by the 15th of the following month [“From the province, they set up a timely reporting period for us if we submit before first two weeks or 15th of each month, then we are timely reporting” FGD2]. However, there was a significant delay in submitting reports, with only 5.1% (45/976) of health centre-months submitting reports before the 15th of the subsequent month. The submission date was not recorded on 36.2% (317/876) of MCMR forms, making it difficult to determine the timeliness of these reports. The most common lag time of the reports was 2 weeks to 1 month (29.5%, 258/876) (Fig. 3). Despite the quantitative findings, study participants felt that there was timely reporting, [“The timeliness of reporting for the province is more than 80%” IDI10]. As for DHIS2 data entry at the province, VBDCP set a deadline that all data of a calendar month should be entered by four weeks of the following month, giving the provinces two extra weeks to enter the data from the date of receipt of data from the health centres (assuming data were submitted on time).
The FGD and IDI participants outlined many challenges to timely reporting. The use of physical reporting forms that needed to be transported to provincial offices meant that the remoteness of some health centres impeded timely reporting [“Some of the clinics are far like in the southern region, they could not do reporting on time” FGD1]. A respondent in an IDI said, “It’s a bit hard because some clinics or health facilities … are remote…… far away so that’s why the reports are coming in late” [IDI2]. The MCMR report was ready in health centres but due to a lack of transportation facilities, they could not send the report to the provincial headquarters.
Other challenges included workload, inadequate infrastructure facilities such as separate offices, computers, uninterrupted internet access and communication systems to collect reports from geographically remote health facilities. The workload issues were raised by several respondents, [“Yes, [because of] workload …. Sometimes they don’t have time [to submit the report]” IDI8].
In addition, they had to input too many variables in the reporting forms, “…..sometimes it is difficult to fill a variety of information into the form. … we found out [it] is time-consuming especially in the bigger health facilities” FGD1]. Besides malaria, nurses were involved in reporting to other programs and had to use several reporting forms. Finally, a lack of regular supervision and feedback affected timely reporting [“…. lack of feedback and supervision [means] we are not motivated to submit timely reporting” FGD3].
Study participants suggested ways of improving timeliness of reporting. They suggested good transportation facilities are critical to timely reporting [“We can improve the late reporting through improving transport” IDI2]. In areas where the transportation facilities were erratic, it was proposed that the provincial supervisor officer could collect the reports by visiting these health centres. Communication technologies can enhance timely reporting, for instance, telephones can be used to collect reports from remote health centres. In some areas, they also used two-ways radio for communication. However, budgets often limited use of these communication methods.
Reliability
Information about the date of report submission, RDT and drug balance are not captured in DHIS2 despite these items being reported in the MCMR. The most reliable indicator in DHIS2 was ‘clinical malaria’ with 52.2% reliability followed by ‘P. falciparum diagnosed with microscopy’ at 48.4% reliability. The least reliable indicator was ‘P. vivax diagnosed with RDT’ at 29.9% reliability. Data from Malaita Province had higher reliability as compared to Honoria City Council. In Honoria City Council, the most reliable indicator was ‘clinical malaria’ at 67.6% reliability and the least reliable indicator was the ‘total tested with microscopy’ with 20.4% reliability. In Malaita Province, the most reliable indicator was ‘clinical malaria’ at 50.0% reliability and the least reliable was ‘P. falciparum diagnosed with RDT’ at 30.2% reliability (Table 4).
Table 4
Reliability of five indicators in DHIS2 as compared to paper records
Indicators* | | Overall | | HCC | | MP |
| Number | % | | Number | % | | Number | % |
Total parasites | | 334 | 38.1 | | 38 | 35.2 | | 296 | 38.5 |
PF | RDT | | 262 | 29.9 | | 30 | 27.8 | | 232 | 30.2 |
PV | MIC | | 424 | 48.4 | | 44 | 40.7 | | 380 | 49.5 |
Clinical malaria | 457 | 52.2 | | 73 | 67.6 | | 384 | 50.0 |
Total test | MIC | | 361 | 41.2 | | 22 | 20.4 | | 339 | 44.1 |
Total test | RDT | | 296 | 33.8 | | 24 | 22.2 | | 272 | 34.4 |
* Three indicators namely report submission date, drug and RDT balance were not recorded in online DHIS2 database. |
The interview respondents recognized that reliability of the malaria case reporting system meant that the data should be true, accurate and trusted. The data should be consistent with the other variables in the form as well as when matched with the different data sources (i.e. OPD registers and RDT and microscopy record books). Some respondents reported that there might have been some variance between the data sources, but this should be minimal [“Of course there’s a bit of variance there, which you have to expect. You never get 100%”].
The respondents identified multiple challenges in reliability in malaria case reporting. Similar to availability and completeness, inadequate human resources affected data reliability. The local and tertiary-level health facilities were unable to verify or assess the quality of data [“…unfortunately, the medical statistics unit doesn’t have the human resources capacity. They have just four coordinators, even out of the four coordinators two have left- they are only left with one now” IDI6]. Availability of adequate supervisory-level staff members was critical, particularly to assess data quality on a regular basis.
The respondents linked data reliability to the availability of adequate logistics. There were some facilities with a scarcity of tally sheets and reporting forms [“…when the forms run out, [we are] not able to submit the reports” FGD2]. Another factor that affects the reliability of the data is programs frequently changing their forms, driven by donors who want to collect data on new indicators. Inadequate and inconsistent internet facilities were one of the main limitations of online DHIS2 reporting. Due to insufficient internet connectivity, the provincial surveillance office delayed entry of the data, leading to limited time to assess and verify the data.
The respondents made several recommendations to improve data reliability. Use of devices such as computers, mobile phones or tablets for data collection could address several reliability-related challenges [“…we don’t have computers and everything is manual that’s why it is challenging” FGD3]. These devices would not only help in data collection but also enable the staff members to connect with their supervisors when they need instructions or guidance on reporting, especially when filling out forms. Arranging refresher training at regular intervals may help staff members to ensure quality data collection and reporting. Increased use of the DHIS2 data system could help in improving data reliability. For example, if data from the system were used for policy decision making regularly by government authorities, errors would be more likely to be identified and fixed, with an increased investment to ensure adequate human resources, infrastructure and facilities. A respondent in an IDI said, “The best way to improve reliability is to improve the demand for their time. The more the data is used the more issues that they will encounter with the data, the more ways you will find out to actually fix these issues” [IDI6].