Characteristic participant
a. Quantitative study
In total, 142 respondents participated in this study spread across five districts or cities in the DIY province. Most respondents came from Gunungkidul District, PHC, UPS, and DHO, 24.8%, 24%, and 25%, respectively. For all research units, the majority are women. At the UPS and DHO/CHO levels, most respondents aged 41–45 years, i.e., 28.3% and 75%, respectively, while at the UPS level, the majority aged 25–30 years (56.0%). For education level, PHC and UPS are dominated by Diploma 3 graduates, namely 86.7% and 80%, respectively, while in DHO/CHO, it is predominantly undergraduate graduates (75%) (Table 2)
Table 2
Characteristic respondent in three groups of respondents
Characteristic
|
PHC (n = 113)
n (%)
|
UPS (n = 25)
n (%)
|
DHO/CHO (n = 4)
n (%)
|
District/City
Bantul
Gunungkidul
Yogyakarta
Kulonprogo
Sleman
|
23 (20.4)
28 (24.8)
17 (15.0)
21 (18.6)
24 (21.2)
|
5 (20.0)
6 (24.0)
4 (16.0)
4 (16.0)
6 (24.0)
|
1 (25.0)
1 (25.0)
0 (0.0)
1 (25.0)
1 (25.0)
|
Sex
Male
Female
|
3 (2.7)
110 (97.3)
|
0 (0.0)
25 (100)
|
2 (50.0)
2 (50.0)
|
Age
< 25
25–30
31–35
36–40
41–45
46–50
> 50
|
0 (0.0)
3 (2.7)
30 (26,5)
19 (16.8)
32 (28.3)
18 (15.9)
11 (9.7)
|
5 (20.0)
14 (56.0)
3 (12.0)
1 (4.0)
0 (0.0)
0 (0.0)
2 (8.0)
|
0 (0.0)
0 (0.0)
0 (0.0)
0 (0.0)
3 (75.0)
1 (25.0)
0 (0.0)
|
Education
Master
Bachelor
Diploma 4
Diploma 3
Senior high school
|
0 (0.0)
5 (4.4)
9 (8.0)
98 (86.7)
1 (0.9)
|
1 (4.0)
1 (4.0)
2 (8.0)
20 (80.0)
1 (4.0)
|
1 (25.0)
3 (75.0)
0 (0.0)
0 (0.0)
0 (0.0)
|
b. Qualitative study
Nine informants were recruited to provide the required information to explore deeper into the quantitative study results. They hold roles as managers and staff at DHO/CHO, PHC, and UPS. Among the nine informants, 2 were men, and 7 were women. Three informants graduated from masters, one bachelor's, and five diploma graduates (Table 3).
Table 3
Informants’ characteristics for the qualitative study
Sex
|
Age (years)
|
Education
|
Position
|
Subject group
|
Informant’s code
|
Female
|
56
|
Magister
|
Head of disease prevention and control department at PHO level
|
Managerial
|
M 01
|
Male
|
57
|
Magister
|
The former of disease prevention and control section at PHO level
|
Managerial
|
M 02
|
Male
|
54
|
Bachelor
|
Immunization programmer at PHO level
|
Managerial
|
M 03
|
Female
|
47
|
Magister
|
IT Person
|
Managerial
|
M 04
|
Female
|
34
|
Diploma
|
Data entry at the PHC level
|
Staff
|
S 01
|
Female
|
25
|
Diploma
|
Data entry at UPS level
|
Staff
|
S 02
|
Female
|
31
|
Diploma
|
Data entry at UPS level
|
Staff
|
S 03
|
Female
|
42
|
Diploma
|
Data entry at the PHC level
|
Staff
|
S 04
|
Female
|
24
|
Diploma
|
Data entry at the PHC level
|
Staff
|
S 05
|
c. Finding
Findings from the study are organized and presented across the three core themes that emerged from the qualitative analysis, notably system strengths, potential threats, and opportunities for scale-up. Yet, data from both the qualitative and quantitative data fed into the analysis of these core themes, to cross-validate the findings (Fig. 2. Detailed findings from the survey are presented in Table Supplement 1.
System’s Strengths
Factors contributing to the success of SIMUNDU include management, system performance, people’s behavior, and resources.
Management
SIMUNDU arose due to concerns from the DIY health office immunization section around data quality, notably the need to address issues related to data inaccuracy, duplicate or missing data and lack of timely data, and the need of quality data to support follow-up and appropriate planning. The need for SIMUNDU arose from these challenges and needs.
To our knowledge, [SIMUNDU development] started with a problem: estimates of the target population varied depending on the data source.
Yes, I think [SIMUNDU management team] started to tire of managing a large volume of data with dubious validity. They need to know the situation in each district.
Effective management of SIMUNDU from development to implementation was highlighted as an essential determinant of its success across the critical functions of Planning, Organizing, Leading, and Controlling.
Careful Planning was ensured at each stage of SIMUNDU development and implementation. These stages included developing an initial business plan, providing training on and socialization to SIMUNDU, and developing a staff replacement plan to respond to turnover or retirement of staff in charge of operating the system or entering data. The parties involved in planning included the head of disease prevention and control department, IT personnel, and immunization program staff from the DIY health office.
Organizing - the organization of SIMUNDU is carried out at several levels. The top level is the DIY health office, the second level is the district/city health office, and the third level is health facilities (Fig. 2). A third party was also involved in developing the system interface.
At the beginning of SIMUNDU development, essential functions included database administrators, interface designers, and server administrators, and their interplay facilitated the smooth operation of the system. Training specific to SIMUNDU was integrated with other training, typically immunization-related training. This enabled to share resources with other programs, thus ensuring viability. The training was delivered in the district/city health office: 87.6%, 72% and 75% of survey respondents from PHC, UPS and DHO/CHO respectively had participated in in-house training. Training typically consisted of short sessions and included practice on the trainee's device on how to operate the system in both online and offline mode. Informants indicated that day-to-day operations were carried out autonomously by the staff, through flexibly adjusting their work to protect time to enter the data. And this seemed to work effectively.
Leading - the success of SIMUNDU implementation is arguably related to strong leadership. Informants noted that managers played a crucial role in bridging the needs of the immunization program with the system design, closely monitoring the initial implementation process, and creating an enabling environment.
I try to combine supporting and managing the people involved and monitoring them. Currently, I monitor whether [SIMUNDU] can run optimally as our users are health facilities. I also monitor program development and the system's output.
[SIMUNDU] was born from program managers, primary health centers, Districts, and DIY health offices wanting to build systems together. We – DIY health office - give them motivation in every meeting.
I see that [management] is very good at networking. Staff data entry in the field always said that these people are very kind.
The role of IT in developing SIMUNDU was also reported to be significant. They helped develop the system and supported correct data entry by assisting data entry operators whenever these encountered technical issues or helping resolve inconsistencies in the data records. Acknowledgment of staff efforts was also an important lever to maintain motivation and buy-in.
In the early days of SIMUNDU’s development, the system was challenging to operate, as it wasn’t as stable as it is now. I praise the enthusiasm and dedication of the users.
The controlling function - consisting in quality assurance management - was critical to avoid data duplication or missing entries, and ultimately ensure data quality. This process was not regulated by specific Standard Operating Procedures but was addressed during training and monitored monthly. In addition, the DIY health office provided negative incentives to health facilities that were not submitting complete records and provided regular feedback from monitoring and evaluation exercises.
Specifically, 94.2%, 100%, and 100% of survey respondents in PHC, UPS, and DHO respectively, reported their work had been subject to monitoring. More than half of the respondents in PHC and UPS facilities had been observed by supervisors while performing data entry at least once over the past year. At the PHC level, 48.3% of survey respondents had been subject to monitoring from the district/city office’s team, and 45.7% received monitoring from DIY health office’s staff. Conversely, 40% of respondents from UPS facilities were monitored by PHC’s staff. Almost all survey respondents reported receiving feedback from the monitoring, mainly from the District/City and DIY health offices. Forty percent of respondents from UPS facilities reported receiving feedback from PHC. Immunization coordinators from the District/City health offices received feedback from the DIY health office.
In a [evaluation] meeting, DIY health office or district health office showed the progress of our data entry – correct or not, proper or not
It is worth noting that DIY province is quite a small geographic area. Because it consists of only five districts and one city, this province is relatively easy to monitor across all phases, from planning through monitoring and evaluation.
System Performance
While SIMUNDU predominantly contains individual-level immunization records, it also serves as a source for aggregation and can synergize with other information systems. Notably, SIMUNDU can link to the DHIS2 and generate immunization-specific reports as per Ministry of Health’s requirements. These reports are sent to the upper levels automatically if SIMUNDU is operated online or submitted via email if SIMUNDU is operated offline. This functionality has had an essential role in ensuring the acceptability and adoption of the system.
Informants noted how transitioning from paper-based tools to an electronic system made data entry easier and reduced errors. It also facilitated the implementation of protocols for data storage and security. It enabled follow-up and defaulter tracking. Finally, integration with the DHIS2 meant reduced workload for the staff.
We can do faster tracking of children who may have received vaccinations in different locations. For example, when the first dose of a vaccine is given in Bantul and the second one in Yogyakarta, the record can be linked within SIMUNDU.
SIMUNDU makes it easier to detect what data and vaccinations are missing since we enter data from the children’s birth through the end of the immunization schedule. So, we will know where they miss any vaccine.
The benefit of using SIMUNDU is first: we know the situation of immunizations more accurately….so our vaccine forecasting is more accurate …. and our budget, staff, facilities can be more effective and efficient in providing services.
Colleagues from the mother and child health (KIA) program enter the data via the KIA "Sembada." So, this data will appear automatically in SIMUNDU because the two-system are connected.
SIMUNDU is user-friendly and can be flexibly operated either offline or online, allowing the responsible staff to maintain data entry irrespective of connectivity. 82.3%, 96% and 100% of survey respondents from PHC, UPS and DHO respectively reported to operate SIMUNDU online.
People’s Behavior
The interview showed that staff commitment was critical for the successful implementation of SIMUNDU, as indicated by their willingness to work overtime and bring home the data to enter into the system.
I take it [the data] home too, for example, after immunization sessions– in my clinic, immunization runs four times per month, every week. So, when the session is finished, we can take the data home, [and] do the entry at home while relaxing
This dedication was confirmed by the interviews, which spoke to a societal culture of helping others and responsibility and commitment to the team. This contributed to shape an environment where people approach SIMUNDU as a shared responsibility and a collective endeavor. Informants also noted the high motivation of dedicated staff.
That's all; we cannot judge by money [people kindness, culture, and behavior]; it's essential to explain how good people are in Yogyakarta. I was in another place before, and I could not find people's kindness like in Yogyakarta - different characters.
The second thing is that we need human resources concerned and love for data; otherwise, even though we have a good system, it will amount to nothing without good human resources. But when people are concerned about data, good implementation will come more easily.
Resource: Material, Human And Financial
Infrastructure and equipment emerged as critical factors to introducing and sustaining SIMUNDU implementation. Some desktops were specifically allocated to the immunization program, and some had to be shared with other programs’ staff. Other data entry officers reported using their own laptop or smartphone (36.3% of survey respondents from PHC). In UPS facilities, 40.7% reported using office desktops, and in the DHO, more than half of the respondents stated they used an office-supplied laptop. The majority of respondents – regardless the type of facility - said their current device was sufficient to perform their work on SIMUNDU. Regarding connectivity, 64.6% of PHC survey respondents and 67.7% of UPS’s reported operating SIMUNDU online relying on the office’s internet connection.
Management of financial resources was also crucial. According to the key informants, no special funds were allocated to SIMUNDU in the initial stages. Resources were leveraged through sharing activities – e.g. monitoring visits or transportation - with other programs, thus allowing cost efficiencies. Integration with other programs proved critical to ensuring sustainability.
SIMUNDU's budget comes from the state budget called as Anggaran Pendapatan dan Belanja Negara (APBN). Every year the APBN allocates a funding envelop for immunization to DIY and other provinces, where the budget is apportioned across the program [not explicitly written budget for SIMUNDU]
Human resources are critical to the operation of SIMUNDU. According to interview, SIMUNDU data entry clerks must have patience, work carefully and not rush, be interested in data, be responsible, and have basic computer skills such as Ms word and Ms excel. As shown by the survey, the large majority of SIMUNDU-operating staff was educated: at least 80% of data entry clerks in either PHC or UPS facilities have secondary education (> 80%), while at the managerial level (DHO), 75% of respondents have a bachelor’s degree (Table 2). Yet, 19.4% and 9.1% of respondents from PHC and UPS facilities, respectively have low computer literacy.
Various data entry clerks looked for strategies to resolve the obstacles they encountered when entering data to SIMUNDU. Based on the interviews, some clerks furthered their computer skills by taking private computer classes. Others learned from other colleagues at their offices, or reached out for help to the district person in charge. To deal with the accumulation of data needing to be entered in SIMUNDU, staff would sometimes work at home after office hours, as their busy schedule at work did not allow time for data entry.
If we found obstacles, we asked people in charge in PHC – asking for a solution or sharing by WhatsApp – or sometimes I asked the IT person in the DIY health office.
Potential threats
As of today, SIMUNDU can be said to be a successful experience. Yet, a number of obstacles were encountered and addressed during implementation. Potential threats to sustaining the system include individual capacity, technical or system issues, and high workload. Staff computer literacy was identified as one of the main challenges to sustainability. Internet connectivity was another obstacle, as not all health facilities were equally supported by a good network. The survey shows that 64.6% and 67.7% of PHC and UPS staff used office internet, while others had to rely on their home internet.
Further, incomplete and inconsistent records – such as differing child's date of birth or name spelling across relevant entries - make it challenging to consistently record immunization information. These challenges have arisen during implementation, and were promptly addressed. Yet, they had an impact on staff who was already juggling busy schedule in the office, causing delays in data entry. As shown by the survey, almost all respondents stated having other responsibilities besides operating SIMUNDU – notably 97.3%, 88% and 100% of participants from PHC, UPS and district and city offices respectively.
Opportunities
Informants appreciated SIMUNDU asa good system for immunization data. SIMUNDU has become necessary for program managers and policymakers because it allows to monitor coverage and can inform planning and programming. Currently, SIMUNDU is stable, thus is easier to manage than when it was in the development phase. It is also viable, and no longer requires heavy reliance on the core workforce that started the system. The hopes expressed by data entry clerks in the interviews are that SIMUNDU is easier to operate, and system errors are less frequent. Informants also stressed the need for refresher training to ensure knowledge and practice of the system is not lost.
In my opinion, SIMUNDU is the best program in DIY which is a collaboration between program managers and IT. It will continue to be implemented because it is a necessity. It has been stably used for more than five years, meaning this is needed.
If I have the tool, in this case, SIMUNDU, when it is stable, whoever will be able to run it, I am sure that anyone can operate it. It means that it doesn't matter if we have people shifting (jobs).
In the future, if SIMUNDU is still used, other reports are not necessary. Now we have two different reports: SIMUNDU and stock card of vaccine – each stand-alone and need a separate report.
Based on the key informants’ interviewerskwkkwkw, SIMUNDU is likely to be developed further / or expanded to other provinces. The DIY health office is open to support other provinces interested in introducing the system, for instance through lending staff for training and orientation. However, informants advised that successful introduction requires a strong commitment from both staff and management.