In assessing the usability of the system, three main themes 1) variations in use, 2) barriers to use, and 3) recommendations to improve use, were identified (Table 2).
- Variation in use:
The private facility and some CHVs used the system more than health workers at the public facilities. Generally, the system was well received by those who used it and appreciated for its benefit.
a. Users’ experiences’.
i.Positive experiences and or benefits of using the system
The system was appreciated as beneficial in improving work-life experiences as highlighted below:
‘I would like to use that phone even somewhere else leave alone this one for the phone even somewhere else I would like to use it because it makes my work easier. I see the work is good because right now I would be carrying around that big book of 514 and that 514 is on my phone so I think it is easy. I would be carrying the 100 for a referral I see it in my phone. The 513, I would still….so that one makes my work easier because carrying these 3 books, they are heavy carrying with a bag because they can’t fit in this bag. And now the phone, I will put it in my bag and walk with it you see now that makes my work easier.’ IDI CHV
‘…early submission of reports I would say because immediately they visit a household you can actually get the information right where you are. Personally, I can use my phone to actually check on what they are doing, by the fact that you can put in the GPRS coding, you can know that this person actually was doing this at a particular area at this particular time and not in their own household. So, when it comes to report transmission it is very prompt and you can actually make a decision concerning that particular house hold in good time. KII CHA
iii. Negative experiences in using the system
On the other hand the processes surrounding use were a deterrent to effect use as exemplified below:
‘…it has a negative because at the moment, okay there is a time I lost my phone. It was stolen and it had that line of (organisation name) and it took a lot of time 3 months for them to return for me the line, so there is no data I have been checking. Because you cannot check without bundles, that’s one thing because if I check for that one that desktop in the office, that one the bundles you find that it’s not even there. Another thing since they returned for me that line, it was last month I have not been able to access any data because the password I am using and anything it doesn’t open it keeps telling me your password is wrong or your password is wrong every time every time’. KII CHA.
b. Limited users’ experiences
Among those who failed to effectively use the system were the public health facility and some CHVs.
i. Type of facility
Here, insufficient human resource and the perception that the application was additional work brought forth by lack of appreciation of ICT were critical issues raised. The facilities that demonstrated this were those run by the government with public resources.
“On the other side on the man power I had one clinician who was trained that clinician stays on the other side. After seeing the clients maybe after some hours it is when she comes and opens the computer this is the computer for [Name of organisation] you see that movement something else to be done. Either one person to be assigned maybe the partner to provide the human resource it was consuming a lot of time for me the facility in fact if it one clinician she cannot manage that. Eh that’s what I can say.” KII Health care worker
ii. CHV attributes
At the beginning of the interventions, CHV who had never used an e-health application, those not accustomed to using smart phone were not able to use the system adequately and these led to some drop out of CHVs. These issues presented challenges in use as demonstrated below:
“At first it was a challenge, ‘because some of them had never used smart phones before. Okay it was excitement; it was (laughing) what do I say, excitement and mmm… anxiety at the same time. We had many when we were starting they were 11 or 13 but they dropped out coz others saw it as a challenge because others had a GPS and someone wants to report in their own house so others dropped out but those who took it they are positive.... the response is good, they are liking it “ KII CHA
- Barriers to use:
Several barriers to use of the system emerged. These were categorised into three sub-themes: (a) socio-political environment, (b) attitudes and behaviours of the users, (c) issues related to the system and (d) poor infrastructure.
a. The socio-political environment
A prolonged industrial action by health workers and the contentious presidential election in the year of implementation affected the use of the technology. As depicted by the excerpt below, the industrial strike affect the use of the system as there was increased pressure and workload on those who remained:
“[….]because of the nurses strike we have not been able to do a close follow up on the referral from the community to us, at times we even forget to open because of the work load, because right now we are very few indeed. So by the time we remember to open you find that the time is already gone because of the workload.” KII_Health provider
b. Attitudes and behaviour
From the observations made during implementation as well as discussion held with various key stakeholders such as health managers, healthcare worker attitudes towards the system limited use. Many development partners have in the past provided extra financial motivation to system users. Without this, the intended users preferred not to use the system. Most of the clinicians at the public facilities saw the system as additional work and not something to improve their work experiences. Unlike the private facilities that already had pre-exiting ICT/mhealth systems, the public facilities had none and this may have further affected the use of the technology. As stated above, the industrial unrest could also have negatively affected the behaviour of the health care workers. In addition, we observed that a number of users lacked basic ICT knowledge and skills which may have contributed to poor attitudes and behaviour. This was evident among the older CHVs (with the smart phones) who had challenges in navigating through the different features of the technology.
c. Issues related to the system
For some users, issues inherent to the system, including network connectivity and the phone model were major sources of concern:
‘Sometimes you get to a house and the phone hangs, you are like wah! Now what to do? You remove the battery and when you start again, before they put for us that draft input, you start adding information from the beginning. You start again’. FGD CHVs Kamukunji
d. Poor infrastructure
In general, this being a slum setting, it is affected by the intermittent power supply and the internet coverage was weak in some of the facilities.
“….Like right now we have been having power problems in Gikomba market from the previous fire. Now it is two weeks down the line and they haven’t gotten power coz a transformer busted something major happened ….” KII_CHA
The public health facilities lacked basic electric features and lack of security for gadgets resulted in the computers being kept in a store, inaccessible to a clinician.
[…] for the clinician it was hectic in fact and most of our rooms, you see this is a store it’s not even an office it’s a store so, because of the security that is why I brought the comp here but where the clinicians sits it’s just open the door is just windows are open so security was another issue so if possible where the clinician sits its where this thing is supposed to sit so that when the client comes now the clinician can see that client my opinion.” KII_Health provider
- Recommendations to improve the use of the system:
In many of the public health facility rooms that the clinicians used, there were no power sockets. In addition, the public facilities were not securely enforced and as a result, the desktops had to be placed in secure areas of the facilities which were different rooms from the clinician rooms. Participants suggested strategies to improve usage.
First, integrate the system with others in use and make it available on users’ regular phones.
‘If they could send that app to our phones...we just use one phone. Either they just unlock those apps; I can also use my sim card’. FGD CHVs
Second, provide extra financial motivation for users as well as performance based remuneration together with using local languages in the system.
‘Yes! But it will only solve part of the problem, the other part is how do you make them stay? So yes you can make them…you can digitize the referral tool you can digitize the coordination mechanism but how do you make them stay you must pay. So the tool needs…the tool is necessary but the tool will not succeed without additional support, yes.’ KII Sub-County Medical Officer of Health
Thirdly, even though the manual MoH reporting tools are in English language only. There was concern that the system being in English limited its usage by users who are not so conversant with the language. The language concerns were mainly raised by the CHVs as one CHV supervisor reiterates below;
‘…the tool was easily adopted by the CHV’s after subsequent trials that was done but maybe further we can also improve on the language because we work with CHV’s some are very illiterate, semi illiterate and some just never went to school so when you use the system is only in English, it blocks a huge number of people who could have utilized it. So the language barrier issue should also be taken into consideration because there are those aspects that you can easily select which language you want to use whether Kiswahili or English so that everybody is accommodated.’ KII CHA
The health managers also recommended inclusion of a course in the basics of computer use, prior to the introduction of such innovations. It was noted that the CHVs who were very good at their daily work did not possess knowledge and skills to interact with the phone. In addition, some clinicians were not conversant with the use of computers, including powering on the device.
To facilitate adoption of the system, there was a recommendation from both the uses and health managers to provide extra motivation for users, including performance based remuneration.
Table 2: User perspectives on implementing the CHV DST
Variation in use
Positive experiences/benefits of using the system
Negative experiences in using the system
Limited users’ experiences
Type of facility
Barriers to use
Health workers’ strike
Attitudes and behaviour
Lack of knowledge and skills-ICT
Issues related to the system
Nature of the gadgets
Nature of the system
Recommendations to improve the system
Suggestions for enhancing usage
Integrate the system with others
Provide extra motivation for users, including performance based remuneration
Provide basic ICT skills for users
Strengthen ICT infrastructure