Implementing SMS reminders for routine immunization in Northern Nigeria: a qualitative evaluation using the RE-AIM framework

doi:10.21203/rs.3.rs-1113957/v1

Download PDF

Research Article

Implementing SMS reminders for routine immunization in Northern Nigeria: a qualitative evaluation using the RE-AIM framework

https://doi.org/10.21203/rs.3.rs-1113957/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

Background: SMS reminders have improved vaccine uptake in clinical and community settings in low- and middle-income countries (LMICs). However, the limited use of SMS reminders in LMICs requires evaluating the intervention’s internal and external validity to improve adoption and sustainability. Using the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework, we qualitatively assessed the impact of an SMS reminder intervention implemented in Kebbi State, Northwest Nigeria, between May 20, 2019, and May 31, 2020. This is to guide and inform future SMS reminder interventions to improve childhood immunization uptake in LMICs.

Methods: In June 2020, we conducted 14 focus group discussions, 13 in-depth interviews, and 20 key informant interviews among 144 purposively selected participants from five local government areas of Kebbi State. For analysis, we used a deductive approach to develop preliminary codes based on the RE-AIM framework and the inductive approach to generate themes that emerged from the interviews.

Results: The perceived importance and impact of the SMS reminder in improving demand and uptake for vaccination were the consistent contributing factors that encouraged participants’ participation. Other facilitators included the involvement of health workers in supporting SMS reminder registration and community gatekeepers using existing structures to convey messages on scheduled immunization services. Policymakers adopted the intervention because it aligns with the State’s priority to improve immunization coverage. Similarly, the SMS reminder appealed to health workers and program managers because it reduced their workload and served as a performance monitoring tool to track immunization and intervention defaulters. Despite these, low mobile phone ownership and the inability to read text messages due to the low literacy level were the main barriers during implementation. Finally, data availability on cost-effectiveness and the intervention’s impact to improve coverage was critical for scalability.

Conclusions: Our study demonstrated that SMS reminders in local languages could improve vaccination demand and uptake in resource-constrained settings due to its perceived importance and impact. Addressing the cited implementation barriers and promoting the facilitators is critical to its adoption and sustainability. Costing and impact data are needed to collaborate findings on the effectiveness of the SMS reminder to improve childhood vaccination uptake.

Vaccination

Immunization

SMS reminder

Text message

mHealth

Qualitative study

RE-AIM

Nigeria

Immunization remains one of the most cost-effective interventions available to public health today [1]. Globally, vaccines avert 2-3 million deaths every year and prevent morbidity and disability from common and rare diseases [2]. Beyond saving lives and averting diseases, the benefit of vaccination extends across the life course of a vaccinated person [3]. For instance, it prevents outcomes in the broader community by protecting the unimmunized population through herd protection, stabilizes health systems by providing opportunities for better primary healthcare (PHC) services, and promotes health equity by giving every child an equal chance and access to vaccination irrespective of their socio-economic status [3]. Furthermore, vaccination provides significant savings by avoiding the health costs associated with treating diseases and possible long-term disability and benefits national economies. In low- and middle-income countries (LMICs), for every US$1 invested in childhood vaccination stands to yield, for countries, between US$19.8 and US$52.2 return on investment from 2021 to 2030 [4]. However, countries attain these benefits when children receive all the scheduled recommended doses, crucial to achieving universal health coverage and the Sustainable Development Goals.

Despite introducing new and under-used vaccines, an estimated 20million children globally did not receive the recommended three doses of diphtheria, tetanus, and pertussis vaccine in 2019 [5]. In the same year, ten countries accounted for 12.2million of unprotected children worldwide, with Nigeria reporting 3 million unvaccinated children [5]. In Nigeria, not all children benefit from immunization as vaccine-preventable diseases account for an estimated 29% of childhood deaths annually [6]. While Penta3 coverage increased from 38% in 2013 to 50% in 2018 [7], the current immunization coverage rate falls short of national and global targets [5]. In 2018, about 31% of children received all vaccines by 23 months, and 19% had never received any vaccine from the RI program [7]. Furthermore, coverage is highly variable across the country, with the northwest and northeast regions having the worst performance [7].

Demand-related issues, including lack of awareness, mother’s literacy level, weak mobilization, and vaccine hesitancy, are reasons for non-vaccination in Nigeria [8, 9]. A national survey revealed that lack of knowledge of the benefits of immunization and the place or time of vaccination accounted for 42% of the reasons why children remain unimmunized [10]. In Northern Nigeria, lack of awareness was the consistent reason for the non-vaccination of children aged 12-23 months in North Central (50%) [11], North East (39%) [12], and North West (37%) [13]. Similarly, a study in Northern Nigeria found that lack of knowledge about vaccines and vaccination services (50%), mother forgetting or being too busy (16%), and suboptimal access to RI services (15%) were the commonly reported reasons for non-vaccination [14]. Hence, an effective and innovative strategy is needed to overcome these information and knowledge gaps and improve compliance with the immunization schedule to enhance vaccine demand and uptake.

Evidence has shown that reminder and recall systems improve health-seeking behavior for vaccination [15]. A Cochrane review found that patient reminder or recall interventions in PHC settings increased the number of people who received vaccinations, whether sent when vaccinations were due or overdue [16]. While postcards, text messages, and automated telephone calls were efficient, telephone reminders were more effective but more costly than other methods [16]. Thus, the need for effective centralized and text message interventions [16]. Mobile phone text messaging, also known as SMS, has been a cost-effective method of disseminating health information and reminders, especially among low-income populations [17, 18]. In addition, the use of mobile phones, including non-smartphones, is becoming widespread in LMICs given its low-cost nature [19], thus, making it a convenient tool to improve health outcomes [20, 21].

SMS reminders have successfully improved immunization uptake in LMICs in small-scale clinical and community settings [22–26]. While this is promising and has strengthened immunization programs in developing countries, its effectiveness varies with the implementation context [27]. Furthermore, the limited attention and use of SMS interventions in LMICs requires evaluating the intervention’s internal and external validity to improve sustainable adoption.

Studies have emphasized the extensive use of dissemination and implementation models, theories, and frameworks to understand, guide, and inform future interventions for scale-up [28]. Several frameworks to determine implementation success for translation into practice exist. One such is the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework used in several studies to design, implement, and evaluate interventions “with a higher likelihood for uptake and sustainability in a typical community or clinical setting.” [29, 30] The five RE-AIM dimensions are defined in Table 1.

Table 1

Definition of RE-AIM dimensions
Dimension	Definition
Reach	The number, proportion of the intended audience, and participants’ representativeness compared with the intended audience.
Effectiveness	The degree to which the intervention changes health outcomes and quality of life, including producing unintended or negative results.
Adoption	The number and proportion of settings and staff members who agree to initiate program or policy change and how representative they are of the intended audience regarding the location and the staff.
Implementation	The degree to which those settings and staff members deliver a program or apply policy as intended, including the adaptations made and the related costs.
Maintenance	The sustained effectiveness at the participant level and sustained (or adapted) delivery at the setting or staff level.

Studies have recommended qualitative methods alongside quantitative approaches to understand each RE-AIM dimension [30, 31]. Unfortunately, most studies using RE-AIM assess intervention outcomes quantitatively and lack qualitative contribution. Additionally, there is limited use of qualitative methods with RE-AIM [32, 33] thus, limiting the ability to understand why and how we obtained results.

To the authors’ knowledge, there is no evidence of using the RE-AIM framework on assessing SMS reminder interventions in LMICs. For this study, we qualitatively evaluated the impact of an SMS reminder intervention called Immunization Reminder and Information SMS System (IRISS) in Kebbi State, Nigeria, using the RE-AIM framework. This is to guide and inform the implementation of SMS reminder interventions to improve childhood immunization uptake in developing countries.

Intervention

IRISS, also known as Tunatar da ni (meaning “Remind me” in the Hausa language), is a cloud-hosted customized registration and message scheduling application that provides automated registration and delivers one-way immunization information to the targeted population via SMS. The application houses four databases: Contact information, Child’s data, Health facility information, and Message library. IRISS used SMS to inform and educate the public about the importance of immunization and remind caregivers/parents of their child’s immunization schedules, including health facility vaccination schedules in their locality. We designed the project as a two-arm cluster randomized controlled trial to assess the IRISS intervention’s impact on improving demand for and uptake of immunization. Fourteen (14) LGAs received the SMS intervention for one year between May 20, 2019, and May 31, 2020, while the remaining seven (7) LGAs did not receive the intervention.

IRISS delivered immunization information to the public and parents/caregivers in three ways:

1. General broadcasts

Every quarter, IRISS sent general broadcasts with informative messages about immunization and a prompt to opt-in to IRISS e-registry for more information to 190,000 active subscribers in the 14 intervention LGAs. The immunization information aimed to improve positive norms about vaccination. Individuals that responded to general broadcasts were called Leads and scheduled for the weekly targeted broadcasts.

2. Targeted broadcasts

IRISS sent two targeted broadcasts every week. Every Thursday, IRISS sent information on immunization benefits to 3,924 Leads, 1,010 community gatekeepers (District Heads, Village Heads, Mai-ungwas, Imams), and 406 health workers. Every Sunday, immunization clinic days for the nearest health center are sent to 1,010 community gatekeepers to pass on to their communities and congregations. IRISS sent 160,428 and 15,150 targeted broadcasts on the benefits of immunization and health facility schedules, respectively.

3. Individualized/personalized reminders

IRISS sent 36,722 individualized reminders a day before the immunization due dates to 21,906 children (approximately 61% of the targeted population) registered into the IRISS. Health workers assisted the registration of caregivers and community gatekeepers via direct or proxy registration. Direct registration was for caregivers or community gatekeepers who own a phone. In contrast, proxy registration was for those who did not have a phone but could access a relative/friend’s phone to receive messages.

IRISS disseminated all messages in the Hausa language. Table 2 provides an example of SMS used for general broadcast, targeted broadcast, and personalized reminders.

Table 2

Sample IRISS intervention messages by broadcast and language type
Broadcast type	Sample message in English	Sample message in Hausa
General broadcast	Immunize your child with a single dose of BCG vaccine at birth to prevent tuberculosis. To learn more, text YES to 07040055505	Ka/ki ba wa yaron ka/ki rigakafin BCG da zarar an haife shi/ta domin kariya daga tarin huka. Domin karin bayani game da rigakafi, tura yes ga 07040055505
Targeted broadcast to Leads, community gatekeepers, and health workers	Promoting the health benefits of vaccines is a collective effort. Do your part by telling relatives, neighbors, and friends to vaccinate their children.	Sanar da jama'ar mu amfanin kiyon lafiya abu ne da ya rataya a wuyan kowa. Ka bayar da taka gudunmawa ta hanyar sanar da yan'uwa da sauran alumma domin su kaia yaransu wurin rigakafi.
Targeted broadcast to community gatekeepers	Greetings! PHC Biri will be conducting an immunization session on Monday. Please inform your community members.	Barka! Akwai allurar rigakafin yara a ranar Litini a Phc Biri. Taimaka ka yada wanan sako a cikin unguwar ka.
Individualized/ personalized reminders	Greetings! Yusuff is due for Measles vaccination tomorrow; kindly visit Illela HF. They conduct immunization sessions on Thursday and Monday	Barka da warhaka! Yusuf ya isa karbar rigakafin Measles gobe, ka garzaya zuwa Illela HF. Su na rigakafi a ranar Alhamis,Litini.

Study design

We used Focus Group Discussions (FGDs), In-depth Interviews (IDIs), and Key Informant Interviews (KIIs) to evaluate the impact of the IRISS intervention on all RE-AIM dimensions following the conclusion of the intervention. In each RE-AIM element, we adapted qualitative questions from Glasgow et al. [29] and Holtrop et al.[31] to understand how and why outcomes occurred, their variation across settings, and how implementation context may influence generalizability and translation to other locations. Table 3 shows the qualitative questions for each RE-AIM dimension evaluated. We used the Standards for Reporting Implementation Studies [34] (Additional File 1) and the Standards for Reporting Qualitative Research [35] checklists as a guide in reporting this study.

Table 3

Applied qualitative questions for each RE-AIM dimension
Dimension	Qualitative questions
Reach	● What type of IRISS messages did people receive? ● What are the facilitators and barriers to registering into IRISS for community members? ● What are the reasons for not registering for IRISS across study sites? ● What was done to encourage participation?
Effectiveness	● Did the IRISS intervention improve uptake and demand for vaccination? ● What other factors contributed to the outcomes observed? ● How meaningful were the outcomes for different participants?
Adoption	● What factors contributed to the State Primary Health Care Development Agency (SPHCDA) and its staff, including health workers and program managers, participating in the IRISS intervention? ● What were the reasons for participating in the IRISS intervention or not? ● What were the barriers to successful adoption across health workers and their settings? ● Was there a partial or complete adoption?
Implementation	● How was the IRISS intervention implemented? ● What influenced implementation or the lack of it? ● What were the modifications to the intervention, and why did they occur? ● What were the contextual factors and processes underlying barriers at each setting to registering participants into IRISS, and how was it addressed? ● How did health workers improve registration?
Maintenance	● Was the IRISS intervention sustained, discontinued, modified- and why? ● What are the barriers to maintaining the intervention? ● Is the IRISS intervention being implemented (and adapted) after the intervention core period?

Study Setting

We conducted our study in Kebbi State, Northwest Nigeria, because of its low RI coverage and lack of awareness on vaccination [10, 13]. Kebbi State has a total population of about 4.2 million [36]. It is primarily rural and comprises 21 local government areas (LGAs), divided into four Emirate Councils: Argungu, Yauri, Zuru, and Gwandu, and 225 political wards. Their main economic activity is agriculture, their primary language is Hausa, and 99% are Muslim.

According to a recent survey, 36% of the households in Kebbi State are in the poorest quintile, and only 2% are in the richest [7]. Among individuals aged 15-49 years, only 15% and 35% of women and men, respectively, were literates^a [7]. Likewise, 22% of women and 63% of men own a mobile phone [7]. Immunization services are delivered via government hospitals, PHCs, and mobile/outreach stations. While 6% of infants 12-23months received all basic vaccination^b, a whopping 21% have never been vaccinated [7]. Furthermore, RI coverage in Kebbi State is low at 11% compared to the national at 50% [7].

In Kebbi State, the Study Advisory Group and the study team selected five LGAs for the qualitative study, namely: Aliero, Argungu, Fakai, Shanga, and Augie, to be representative based on the following criteria (see Table 4):

Table 4

Characteristics of study LGAs
S/N	Name of LGA	Emirate Council	Study Group	Demography	% RI performance^c	% Phone ownership^d
1	Aleiro	Gwandu	Intervention	Rural	23%	45%
2	Argungu	Argungu	Control	Urban	10%	42%
3	Fakai	Zuru	Intervention	Rural	35%	5%
4	Shanga	Yauri	Control	Rural	53%	3%
5	Augie	Argungu	Intervention	Rural	20%	2%

Representation from the four Emirate Districts and study group (intervention and control LGAs)

Demography: Inclusion of rural and urban areas

RI performance [37]

Phone ownership [37]

Study Participants And Sampling

We purposively selected participants for the interviews from the five LGAs. They included RI program managers (i.e., state-level policymakers, program managers at state and LGA levels, development partners), and PHC health workers with a minimum of one year of experience in their respective current positions. Also, community members (i.e., parents of children under five years of age, parents of newborns, pregnant women, opinion leaders (traditional and religious leaders, persons of influence), ward development committee (WDC) members, traditional birth attendants, young men and women, and male and female youth groups) who are current residents of the community for at least one year. Other information on the recruitment process and inclusion criteria have been published [38]. Invited participants were not required to have a phone or have experience using a phone. All participants were available and consented to participate in the interviews.

Data Collection

We developed the data collection instruments and conducted 14 FGDs, 13 IDIs, and 20 KIIs among 144 people in the study LGAs between June 22 and 27, 2020. Additional Material 2 and 3 present the interviews guides and a complete list of respondents and interview types, respectively. After obtaining verbal consent, trained research assistants and data collectors conducted the interviews using a pre-tested, semi-structured interview guide at the participant’s home or a convenient location. We conducted all 27 interviews at the community level in the Hausa language due to the literacy level of the study setting. We also collected participants’ socio-demographic information^e using a cover sheet. Each interview lasted between 60 and 90 minutes. The study team offered no compensation to respondents during the data collection. However, we provided FGD respondents 500Naira in travel per diems.

In addition, the data collection teams followed all COVID-19 preventive guidelines such as temperature testing using an infra-red thermometer and handwashing with soap and water before commencing interviews and maintaining 2 meters distance between each other before and during the interviews.

Data analysis

The audio-recorded interviews were transcribed verbatim and analyzed using a deductive and inductive approach. Interviews conducted in Hausa were transcribed by external vendors and then translated to English for validation and analysis purposes. We used a deductive approach to develop preliminary codes based on the RE-AIM framework and used the inductive approach to generate themes that emerged from the data [39]. First, ten transcripts were randomly selected from the study sites and participants and read independently by COJ and FA for content comprehension and coding. All codes with their definition and application rules were entered into a codebook. COJ, FA, and CW reviewed the codebook for verification and harmonization. The study team further reviewed the harmonized codes to form appropriate themes according to the data pattern. This review ensured that the codes within the themes connect meaningfully together while presenting a clear distinction between the themes. The remaining transcripts were coded using the Atlas.ti software, where emerging codes and themes were recorded and modified in the codebook. Using the Atlas.ti software, we queried the data for meaningful content and interpreted it based on identified themes captured with the evaluation questions. The Atlas.ti software analyzed the themes by respondent type, study group (intervention and control LGAs), and setting. The respondents’ socio-demographic information was analyzed in Microsoft Excel and presented as frequencies and percentages.

Study participants demographic characteristics

We interviewed 144 participants representing the target population at all levels. The majority were community members, 124(86%) and males 99(69%). Participants’ ages ranged from 19-77 years, with an average mean age of 39 ± 13.3 years. Educational status among our participants was slightly high. More than half of our participants, 113(78%), had a formal education; and about 72% attained secondary school or higher (Table 5).

Table 5

Socio-demographic characteristics of participants
Characteristics	Number (n=144)	Percentage (%)
Respondent type
Community members	124	86%
Government program managers	9	6%
Government health workers	10	7%
Policymaker	1	1%
Sex
Male	99	69%
Female	45	31%
Attended formal education
Yes	113	78%
No	31	22%
Highest level of formal education completed
None	31	22%
Primary School	9	6%
Secondary School	39	27%
Graduate	62	43%
Post-Graduate	3	2%

Reach

For the Reach dimension, we assessed receipt of IRISS messages, facilitators, and barriers to registering into IRISS among community members and activities conducted to increase participation.

Most participants from the intervention LGAs confirmed to have received IRISS messages as 1) a reminder on child’s vaccine due dates on their mobile phones or via proxy; 2) a message on immunization schedule of the nearest facility; 3) a message on the importance of vaccination, and 4) a message on COVID-19.

“Participant 1: Sometimes it’s a reminder to take the children to the hospital. Some other times its awareness about immunization. Participant 2: They also tell us to take these persons to the hospital on these dates for immunization; that’s why they send it. But some days back, we were sent messages on coronavirus. Participant 3: Honestly, I did not receive them because the messages come through my wives’ phone to take the child because the child was registered with her number.”- Opinion leaders

The key facilitators of involvement in the IRISS intervention were its perceived importance in reminding and educating community members on the importance of immunization and satisfaction with the message content, especially the health facility day and operation time accuracy.

“Participant 1: It is very good because if you have your phone and you get the message, it would remind you to go for immunization.Participant 2: It is very useful where you do not intend on taking the child, or you have forgotten, it’s a reminder even if it’s your husband that will be reminded.” – Mothers of under-five

Most participants cited the lack of mobile phones and the inability to read text messages as barriers to non-participation into IRISS. However, educated community members were perceived not to encounter such challenges.

“Participant 1: Yes, most of them are not educated; some do not have a phone or even never bought it. If the message is sent to him, it is of no use to him. Participant 2: Tunatar da ni has its disadvantage or advantage. The disadvantage is that we cannot read. We find it difficult to read the message and other villagers also have this problem.” - Fathers of under-five.

However, among mobile phone owners with a low literacy level, the majority sought help from relatives or other people who could read it for them.

Despite the low phone ownership and literacy levels, involving trusted stakeholders in the community, such as the health workers who served as proxies for receiving the SMS for both community gatekeepers and parents, encouraged participation.

“The major challenge we are facing is that most people do not have phones…You will have to register on behalf of most people here because they don’t have phones, both mothers and fathers.” - Health workers

Similarly, disseminating weekly performance reports to program managers at State and LGA levels for follow-up and the occasional supportive supervision where the study team and state program managers conducted on-the-job training for health workers encouraged registrations into IRISS.

Effectiveness

The effectiveness dimension assessed the perceived impact of IRISS in improving demand and uptake for vaccination and its usefulness for different stakeholders.

Many participants noted that personalized reminders reminded caregivers, especially those busy and forgot their child’s vaccination dates, and prompted them to go for vaccination.

“Before, we were not bothered with immunization, but now there is a constant reminder of when and where to be immunized. A woman in my compound would come to meet me and show me a text message, and she would ask me to read the content of the text message for her. I will read the message to her telling her that it is time to immunize her child. She will then take her child to go for immunization.” - Female Youth

The alluded perceived impact of the IRISS intervention in improving demand for vaccination and potentially immunization coverage in the State could have implications for scale-up and sustainability.

“IVAC too has come in to do the intervention where women and caregivers are reminded about routine immunization of their children using SMS. This impacted positively in creating demand for immunization which I believe has also contributed towards improving coverage for immunization.” –Policymaker

Receiving IRISS messages also increased awareness and knowledge about vaccination and triggered discussions about immunization in the community.

“I was not aware that those diseases could be cured, but because of the Tunater da ni message I later knew.” - Fathers of under-five

IRISS intervention helped reduce the workload of health workers and program managers from following up on clients on the child’s subsequent immunization due date.

“It has greatly reduced the stress I used to have. Those receiving the texts have been turning up promptly.” –Health worker

Informed on how the IRISS intervention works, program managers and health workers from the control LGAs also felt that the intervention would ease their workload and potentially improve health outcomes.

“Since it’s easier for them to know when it’s due time, and there won’t be any need to make an announcement, you will just receive a message reminding you to come for immunization.” –Health worker

Program managers at state and lower levels also mentioned that IRISS messages helped them monitor health workers’ performance in conducting registrations as instructed and tracking immunization defaulters as they were proxies.

“As you know, from time to time, I received a message from Tunatar da ni about the number of people registered and the number of hospitals in our local government. As soon as I receive the message, I call the RI providers and praise those that do well and also those that are behind.” - Program Manager

Adoption

We assessed the facilitators and barriers to adopting the IRISS intervention among the state institutions (SPHCDA) and implementing staff (health workers and program managers) for the RE-AIM adoption dimension. State decision-makers adopted the IRISS intervention because improving immunization coverage is a priority in Kebbi state and welcomed interventions to achieve this goal. The IRISS intervention was also incorporated as part of the activities assessed during RI supportive supervision at the health facilities.

Implementing staff of the agency and partners was committed to improving RI coverage in the State and providing support. For example, state program managers supported supportive supervision at health facilities to encourage registration, and health workers conducted registrations, sensitized community members about IRISS, and helped caregivers understand the messages.

“Whenever I access a parent, I try to collect their phone number and also ask them to be attentive to their phone because a message will be sent on when next to bring their kids for another vaccine. Sometimes the parents come to me for an explanation whenever a message is sent to them.”- Health workers

Among the 426 health facilities providing RI services in the intervention LGAs, only 361(85%) participated in conducting registrations. Health workers from the remaining 65 health facilities reported network issues and lack of mobile phones as barriers to successful adoption.

Although the IRISS intervention was implemented in 14 LGAs as a research project to inform policy actions, decision-makers expressed willingness to scale up to the remaining seven (7) LGAs given the perceived impact.

“We are thinking of scaling up …If you compare what is happening in these 14 LGAs, we will likely see a positive impact, so scaling-up is truly something we want to do.” –Policymaker

Implementation

The implementation dimension explored the intervention fidelity, including motivations for/barriers to some aspects of the intervention. The most frequently cited implementation barriers were the low mobile phone ownership, especially among rural dwellers, and the inability to read text messages due to the low literacy level. This may have implications on the impact of the intervention than initially planned, especially for the uneducated.

We also learned from participants interviewed that most community members who own mobile phones use it to receive calls only and ignore text messages. Either because they cannot read, are unaware when they receive these messages, or avoid paying attention due to unsolicited messages.

“That’s true. Most people only use the phone to receive and make calls and are not even bothered when messages come in.” –WDC members

Despite these challenges, those who opted to receive SMS via proxy noted that their proxies, e.g. health workers, relatives, or friends, informed them.

“He knows the importance because he tells me about it whenever he receives the same.” –Mother of newborn

The community gatekeepers used town announcers to convey messages on scheduled immunization services in the nearest health facility when informed by the health workers.

“When the time comes for immunization, the Mai-anguwa [Village Head/Chief] is informed. He will, in turn, inform the announcers to pass the information across the town..” - Opinion leaders

Beyond the commitment to the state target and avoiding sanctions, health workers’ willingness to serve as proxies was driven by the desire to improve health outcomes.

“The reason for carrying out this decision of mine is that immunization is important. I endeavour to provide enlightenment, so they know that vaccination is beneficial for their children’s wellbeing.” –Health worker

While health workers mentioned that they informed parents and community leaders who were registered on their phones, only one was clear on how she did it.

“I follow them up; house-to-house to get the message across to them.”- Health workers

Meanwhile, implementing the door-to-door/house-to-house vaccination by the SPHCDA provided another opportunity for health workers to register eligible children and sensitize their parents. We also learned that health workers conducted most registrations during an outreach session compared to fixed.

To further encourage registrations during the intervention delivery, the study team provided a modest incentive (6Naira) per successful registration to encourage RI providers to conduct proxy registrations. To promote and improve registrations, state program managers and the study staff conducted supportive supervision in health facilities with inconsistent registrations and a high target population. Health workers were urged to inform community gatekeepers and health workers when they received their messages.

While State program managers encouraged proxy registration using phone numbers of friends and relatives, they felt using a health worker was a limitation in the intervention implementation. They felt health workers benefit as they may not convey the information to the recipients.

State program managers also felt that incorporating Robocall as part of IRISS intervention for the uneducated would have been more effective due to the low literacy level in the study settings.

“ROBOCALL ahh… it will go along in disseminating the positive information because you can send a text message and they cannot read. But in those areas that can read, definitely so they can be ahh…sent SMS.” –Program manager

Community members agreed to have the educated ones receive the SMS reminder, and the uneducated get the messages via phone call or their town announcers.

“Participant 2: During registration only, the educated ones should be registered for SMS while the uneducated ones should be given a call of reminder. Participant 3: Those with phone and telecommunications service should be registered, those without a town crier should be provided to them.” - Fathers of under-five

Maintenance

We assessed the IRISS intervention sustainability and the reasons for continuation or discontinuation to inform future program design and scale-up. On what needed to happen to support the intervention’s sustainability, state decision-makers noted that it is dependent on the available data to show its impact on improving coverage and affordability by the state government.

“Yeah... uh… it’s data because we leverage on using data for action.” –Program Manager

The data will be used for decision-making by key stakeholders who would prioritize and approve its inclusion in the state budget for funding. Upon funding, the IRISS intervention will be situated and managed by the immunization department of the Kebbi SPHCDA.

While we discontinued the SMS-reminder dissemination on immunization after the project ended in May 2020, the Nigerian Centre for Disease Control used the IRISS application to send messages on COVID-19 preventive measures in the Hausa language across the 21 LGAs between June 5 and July 31, 2020. The majority of the participants in both study groups received messages from IRISS on COVID-19.

We found considerable facilitators to optimize the reach, effectiveness, adoption, implementation, and maintenance of the IRISS intervention within community settings to improve the demand and uptake of immunization services. The perceived importance and impact of IRISS intervention in improving demand and uptake for vaccination were the most contributing factors that encouraged participants’ participation. The IRISS intervention served as a reminder and prompted vaccination visits, increased awareness and knowledge about vaccination, and triggered immunization discussions among community members. This is consistent with other studies where SMS reminders increased vaccination appointment attendance and uptake [40, 41]. Similarly, embedding information on vaccination benefits has improved influenza vaccine uptake among low-income, urban, and minority populations [42], made parents feel encouraged and aware and helped them influence their attitude towards self-responsibility [43]. While our study did not investigate the particular message that had a substantial impact on parental behavior, messages informing and educating the public about the importance of immunization are needed to help community members move from lack of awareness to considering vaccination[44].

The community members’ desire to participate in the IRISS intervention due to its perceived impact aligns with the state decision-makers and service providers. State decision-makers adopted the IRISS intervention because it aligns with the State’s priority policy to improve immunization coverage. Interventions that address the target population’s health needs have been shown to promote acceptability and political will to devote resources for implementing and scaling up the program in LMICs [45]. This is particularly important in areas with competing health priorities and limited resources to ensure the intervention’s sustainability after project funding ends. While our study found that the staff and partners attached to Kebbi SPHCDA supported the implementation of the IRISS intervention and tracked registrations during supportive supervision, the sustainability is dependent on the availability of impact and cost-effectiveness data. This is similar to other studies in LMICs, where the availability of research and monitoring and evaluation data facilitated the scale-up of public health interventions [45].

Additionally, despite the desire to improve RI coverage in the State, the IRISS intervention appealed to health workers and program managers because they believed it reduced their workload and served as a performance monitoring tool to track immunization and project defaulters. Previous studies reported that SMS technology helped service providers monitor and provide accurate information on patients’ acne treatment adherence [46] and improve case-management practices [47]. It may be worth considering sending SMS reminders to service providers and patients as an essential addition to immunization reminders to improve health outcomes.

Other factors that encouraged participation at the community level were health workers’ involvement in supporting IRISS registration and sensitization. These individuals are trusted stakeholders and have been shown to influence the acceptability of a vaccination reminder intervention in Kebbi State [48]. Their participation in delivering intervention cannot be overemphasized to encourage community participation. Our health workers did not report that the use of incentives encouraged support to register community members. However, adding incentives for involvement in SMS reminder interventions has promoted participation [26, 49], although there may be cost implications in sustaining this approach in LMICs. As implementation progressed, program managers’ refresher training during occasional supportive supervision reinforced the initial training received by health workers and filled the gap for new health workers to encourage more registrations. The weekly performance SMS messages to program managers also served as an accountability measure to monitor different health workers’ participation. In addition, satisfaction with the accuracy of the health facility schedule sent directly to caregivers’ mobile phones encouraged participation in our study. This is similar to another study where a high level of satisfaction with SMS reminders’ precision was helpful and motivated parents to recommend the text messages to other parents [42].

Across the RE-AIM framework, we found that the low mobile phone ownership in rural settings and the inability to read text messages due to the low literacy level were the main barriers to non-participation in IRISS. The baseline assessment also reported this as a potential barrier to implementing the IRISS intervention [38]. These findings are similar to other studies in LMICs, where the lack of mobile phones, especially in rural settings and caregivers’ inability to read an SMS message, was an obstacle to implementing SMS reminder interventions [50–53]. This might have a direct impact on the efficacy of SMS messages in improving immunization coverage. While the IRISS application disseminated the SMS messages in Hausa, this study reveals that low literacy could be a determinant of not owning a mobile phone. In our research, people used the health worker, family member/relative, or friend’s phone to receive immunization messages and reminders where phone ownership was lacking. Incorporating proxy registration should be considered for future SMS-reminder interventions for childhood vaccination appointment reminders for caregivers who do not own a phone. Also, using town announcers to disseminate information on vaccination dates, times, and venues could be an opportunity to reach the unreached. Another suggestion would be for communities to establish a microcredit program where community members can purchase a group phone via loan [54]. However, its feasibility and acceptability will need to be assessed.

Similar to the baseline findings, we found that most community members who own mobile phones use it to receive calls only, ignore text messages, and do not know how to open SMS messages [38], consistent with barriers encountered in other developing countries [53]. This limitation could be due to low literacy and exposure to mobile technologies or unhappiness with unsolicited messages over time. In other studies, the reported drawbacks were people not reading their text messages and being annoyed at receiving multiple messages, but no SMS reminder has caused adverse iatrogenic results [41]. Phone calls or incorporating Robocalls in SMS reminder applications could be a simpler alternative for those who cannot read and do not know how to open text messages. Costing implications and its feasibility need to be assessed in low-resource settings. The use of two-way messaging that elicits a response from message recipients to address the challenge of ignoring text messages has been proposed [41]. Further research is recommended to ascertain the feasibility primarily for short-term interventions such as childhood vaccination appointments. This is because evidence has shown two-way messages to be more effective for life-long programs requiring counseling and communication with a health provider, e.g., HIV appointments [55].

Despite the perceived importance and impact of the IRSS intervention, addressing the cited implementation barriers and promoting the facilitators is critical to its sustainability. Likewise, data availability on the intervention’s impact to improve coverage and the cost-effectiveness for scalability.

Our study had several limitations. Given the study methodology that relies on a small sample size to explore an in-depth analysis of the IRISS intervention, our findings are based on a limited number of participants, subject to social desirability bias, especially the effectiveness dimension. Thus, our study findings are unique to settings and populations with similar contexts and cannot be generalized. Despite these limitations, we explain results obtained across the RE-AIM framework and how we achieved these results, including implementation barriers and facilitators. We also present lessons to overcome future failures and considerations for scale-up or adaptation in other comparable contexts.

We conclude that IRISS automated one-way reminders in local languages could improve vaccination demand and uptake in resource-constrained settings due to its perceived importance and impact. The barriers identified present learnings to inform future community-based SMS-intervention design and implementation to improve uptake and increase sustainability chances. In addition, costing and impact data are needed to collaborate findings on the effectiveness of the IRISS intervention to improve childhood vaccination uptake. Targeted interventions are required for the uneducated and those with no access to mobile phones to benefit from the intervention.

FGD

Focus Group Discussion

IDI

In-depth Interview

IRISS

Immunization Reminder and Information SMS System

KII

Key Informant Interview

LGA

Local Government Area

LMIC

Low- and Middle-Income Countries

PHC

Primary Health Care

RE-AIM

Reach, Effectiveness, Adoption, Implementation, and Maintenance

Routine Immunization

SPHCDA

State Primary Health Care Development Agency

WDC

Ward Development Committee

Ethical approval

Ethical approval for the study was renewed and obtained from the National Health Research Ethics committee of Nigeria (Approval Number: NHREC/01/01/2007-29/08/2019B); and the Johns Hopkins University Institutional Review Board (Approval Number: IRB00008732). We performed all methods according to the relevant guidelines and regulations by the National Health Research Ethics Committee of Nigeria and the Johns Hopkins University Institutional Review Board.

Additionally, we used a standard oral consent script to obtain consent for all individuals interviewed. The consent script indicated individuals’ consent to participate in the study, confidentiality of participants, protection of participants from harm, and permission to record the interview. The National Health Research Ethics Committee of Nigeria and the Johns Hopkins University Institutional Review Board approved the verbal informed consent given the low literacy level in the study setting, especially the inability of uneducated participants to read a written informed consent. After obtaining permission to record the interviews, we repeated the same verbal informed consent process to capture it as part of the study transcripts.

Consent for publication

Not applicable

Availability of data and materials

The dataset generated or analyzed during this study are included in this published article and available from the corresponding author on reasonable request

Competing interests

The authors declare that they have no competing interests.

Funding

This study was made possible through Grant # OPP1181095 from the Bill and Melinda Gates Foundation to CW from the Johns Hopkins International Vaccine Access Center and implemented by Direct Consulting and Logistics Limited. The Bill and Melinda Gates Foundation had no role in the study design, data collection, analysis, data interpretation, or manuscript writing.

Authors’ contributions

COJ contributed to the study design, led the data acquisition and analysis, and drafted the manuscript. CG contributed to the study design and reviewed the manuscript critically for important intellectual content. FA contributed to data acquisition and coding of transcripts. TB contributed to data acquisition. WD led the design of the IRISS application and contributed to data acquisition. AL contributed to the study design and the IRISS application and interpretation of results. CW led the conception and design of the IRISS study and contributed to the data analysis and interpretaton. All authors approved the final document for submission.

Acknowledgments

The project team acknowledges the National Primary Health Care Development Agency, members of the National Emergency Routine Immunization Coordinating Centre, Kebbi State Primary Health Care Development Agency, members of the Kebbi State Emergency Routine Immunization Coordination Center, and Traditional leaders (Emirs, District Heads, and Village Heads) from the 14 intervention LGAs – Aleiro, Augie, Bagudo, Birnin-Kebbi, Dandi, Fakai, Gwandu, Kalgo, Koko Besse, Ngaski, Sakaba, Suru, Wassagu Danko, and Yauri for their support during the study duration. Our special thanks also go to Mr. Abubakar Muhammad for supporting the study implementation process, including the supportive supervision at the health facility and community levels.

Footnotes

^a Attended schooling higher than the secondary level and can read a whole sentence or part of a sentence.

^b All basic vaccination is defined as receiving BCG, three doses of DPT-HepB-Hib, three doses of oral polio vaccine (excluding polio vaccine given at birth), and one dose of measles.

^c RI performance is defined as the proportion of children appropriately immunized for age assessed from the Q3 2018 Lots Quality Assurance Sampling survey. High performance >16% and Low performance <15%.

^d Phone ownership is defined as owning a phone or having access to a friend, relative, or neighbor who owns a mobile phone.

^e Socio-demographic information collected included respondent type, interview date, years in the position, years in immunization, years in health care, respondent sex, age, and education level

Rémy V, Zöllner Y, Heckmann U (2015) Vaccination: the cornerstone of an efficient healthcare system. J Mark access Heal policy 3:27041
Andre FE, Booy R, Bock HL, et al (2008) Vaccination greatly reduces disease, disability, death and inequity worldwide. Bull World Health Organ 86:140–146
Wilder-Smith A, Longini I, Zuber PL, Bärnighausen T, Edmunds WJ, Dean N, Spicher VM, Benissa MR, Gessner BD (2017) The public health value of vaccines beyond efficacy: methods, measures and outcomes. BMC Med 15:1–9
Sim SY, Watts E, Constenla D, Brenzel L, Patenaude BN (2020) Return On Investment From Immunization Against 10 Pathogens In 94 Low-And Middle-Income Countries, 2011–30: Study estimates return on investment from immunization programs against ten pathogens for ninety-four low-and middle-income countries from 2011 to. Health Aff 39:1343–1353
World Health Organization/UNICEF (2020) Progress and Challenges with Achieving Universal Immunization Coverage. In: 2019 WHO/UNICEF Estim. Natl. Immun. Cover.
World Health Organization (2018) Number of deaths in children aged <5, by cause. In: Glob. Heal. Obs. https://www.who.int/data/gho/data/indicators/indicator-details/GHO/number-of-deaths. Accessed 11 Oct 2021
National Population Commission (NPC) [Nigeria] and ICF (2019) Nigeria Demographic and Health Survey 2018. Abuja, Nigeria, and Rockville, Maryland, USA
National Primary Health Care Development Agency (2019) Nigeria Joint Appraisal 2019.
Oku A, Oyo-Ita A, Glenton C, Fretheim A, Ames H, Muloliwa A, Kaufman J, Hill S, Cliff J, Cartier Y (2016) Communication strategies to promote the uptake of childhood vaccination in Nigeria: a systematic map. Glob Health Action 9:30337
National Bureau of Statistics (NBS) and United Nations Children’s Fund (UNICEF) (2018) Multiple Indicator Cluster Survey Nigeria 2016-17, Survey Findings Report. 1–538
International Vaccine Access Center (2018) Nigeria National Immunization Coverage Survey Brief Template_NorthCentral_2017. https://www.jhsph.edu/ivac/wp-content/uploads/2018/04/Nigeria-NICS-Brief-North-Central-Zone-1.pdf. Accessed 11 Oct 2021
International Vaccine Access Center (2018) Nigeria National Immunization Coverage Survey Brief Template_NorthEast_2017. https://www.jhsph.edu/ivac/wp-content/uploads/2018/04/Nigeria-NICS-Brief-North-East-Zone-1.pdf.
International Vaccine Access Center (2018) Nigeria National Immunization Coverage Survey Brief Template_NorthWest_2017. https://www.jhsph.edu/ivac/wp-content/uploads/2018/04/Nigeria-NICS-Brief-North-West-Zone-1.pdf. Accessed 11 Oct 2021
Gunnala R, Ogbuanu IU, Adegoke OJ, Scobie HM, Uba B V, Wannemuehler KA, Ruiz A, Elmousaad H, Ohuabunwo CJ, Mustafa M (2016) Routine vaccination coverage in Northern Nigeria: results from 40 district-level cluster surveys, 2014-2015. PLoS One 11:e0167835
Manakongtreecheep K (2017) SMS-reminder for vaccination in Africa: research from published, unpublished and grey literature. Pan Afr. Med. J. 27:
Vann JCJ, Jacobson RM, Coyne-Beasley T, Asafu‐Adjei JK, Szilagyi PG (2018) Patient reminder and recall interventions to improve immunization rates. Cochrane Database Syst. Rev.
Kannisto KA, Koivunen MH, Välimäki MA (2014) Use of mobile phone text message reminders in health care services: a narrative literature review. J Med Internet Res 16:e222
Stockwell MS, Kharbanda EO, Martinez RA, Lara M, Vawdrey D, Natarajan K, Rickert VI (2012) Text4health: Impact of text message reminder-recalls for pediatric and adolescent immunizations. Am J Public Health. https://doi.org/10.2105/AJPH.2011.300331
Kay M, Santos J, Takane M (2011) mHealth: New horizons for health through mobile technologies. World Heal Organ 64:66–71
Free C, Phillips G, Watson L, Galli L, Felix L, Edwards P, Patel V, Haines A (2013) The effectiveness of mobile-health technologies to improve health care service delivery processes: a systematic review and meta-analysis. PLoS Med 10:e1001363
Pillay Y, Motsoaledi PA (2018) Digital health in South Africa: innovating to improve health.
Bangure D, Chirundu D, Gombe N, Marufu T, Mandozana G, Tshimanga M, Takundwa L (2015) Effectiveness of short message services reminder on childhood immunization programme in Kadoma, Zimbabwe-a randomized controlled trial, 2013. BMC Public Health 15:1–8
Domek GJ, Contreras-Roldan IL, O’Leary ST, Bull S, Furniss A, Kempe A, Asturias EJ (2016) SMS text message reminders to improve infant vaccination coverage in Guatemala: a pilot randomized controlled trial. Vaccine 34:2437–2443
Brown VB, Oluwatosin OA, Akinyemi JO, Adeyemo AA (2016) Effects of community health nurse-led intervention on childhood routine immunization completion in primary health care centers in Ibadan, Nigeria. J Community Health 41:265–273
Haji A, Lowther S, Ngan’Ga Z, Gura Z, Tabu C, Sandhu H, Arvelo W (2016) Reducing routine vaccination dropout rates: evaluating two interventions in three Kenyan districts, 2014. BMC Public Health 16:1–8
Gibson DG, Ochieng B, Kagucia EW, Were J, Hayford K, Moulton LH, Levine OS, Odhiambo F, O’Brien KL, Feikin DR (2017) Mobile phone-delivered reminders and incentives to improve childhood immunisation coverage and timeliness in Kenya (M-SIMU): a cluster randomised controlled trial. Lancet Glob Heal 5:e428–e438
Mekonnen ZA, Gelaye KA, Were MC, Gashu KD, Tilahun BC (2019) Effect of mobile text message reminders on routine childhood vaccination: a systematic review and meta-analysis. Syst Rev 8:1–14
Nilsen P (2020) Making sense of implementation theories, models, and frameworks. In: Implement. Sci. 3.0. Springer, pp 53–79
Glasgow RE, Estabrooks PE (2018) Peer reviewed: Pragmatic applications of RE-AIM for health care initiatives in community and clinical settings. Prev. Chronic Dis. 15:
Glasgow RE, Harden SM, Gaglio B, Rabin B, Smith ML, Porter GC, Ory MG, Estabrooks PA (2019) RE-AIM planning and evaluation framework: adapting to new science and practice with a 20-year review. Front public Heal 7:64
Holtrop JS, Rabin BA, Glasgow RE (2018) Qualitative approaches to use of the RE-AIM framework: rationale and methods. BMC Health Serv Res 18:1–10
Gaglio B, Shoup JA, Glasgow RE (2013) The RE-AIM framework: a systematic review of use over time. Am J Public Health 103:e38–e46
Shoup JA, Gaglio B, Varda D, Glasgow RE (2015) Network analysis of RE-AIM framework: chronology of the field and the connectivity of its contributors. Transl Behav Med 5:216–232
Pinnock H, Barwick M, Carpenter CR, Eldridge S, Grandes G, Griffiths CJ, Rycroft-Malone J, Meissner P, Murray E, Patel A (2017) Standards for reporting implementation studies (StaRI) statement. bmj 356:
O’Brien BC, Harris IB, Beckman TJ, Reed DA, Cook DA (2014) Standards for Reporting Qualitative Research: A Synthesis of Recommendations. Acad. Med. 89:
National Population Commission (NPC) (2018) 2006 Census Final Results Projected to 2018.
National Primary Health Care Development Agency (2018) Lots Quality Assurance Sampling (LQAs) survey. Q2 2018.
Obi-Jeff C, Garcia C, Onuoha O, Adewumi F, David W, Bamiduro T, Aliyu AB, Labrique A, Wonodi C (2021) Designing an SMS reminder intervention to improve vaccination uptake in Northern Nigeria: a qualitative study. BMC Health Serv Res 21:1–17
Elo S, Kyngäs H (2008) The qualitative content analysis process. J Adv Nurs 62:107–115
Kazi AM, Ali M, Zubair K, Kalimuddin H, Kazi AN, Iqbal SP, Collet J-P, Ali SA (2018) Effect of mobile phone text message reminders on routine immunization uptake in Pakistan: randomized controlled trial. JMIR public Heal Surveill 4:e7026
Schwebel FJ, Larimer ME (2018) Using text message reminders in health care services: a narrative literature review. Internet Interv 13:82–104
Stockwell MS, Hofstetter AM, DuRivage N, Barrett A, Fernandez N, Vargas CY, Camargo S (2015) Text message reminders for second dose of influenza vaccine: a randomized controlled trial. Pediatrics 135:e83–e91
Ginsburg AS, Delarosa J, Brunette W, Levari S, Sundt M, Larson C, Tawiah Agyemang C, Newton S, Borriello G, Anderson R (2015) mPneumonia: development of an innovative mHealth application for diagnosing and treating childhood pneumonia and other childhood illnesses in low-resource settings. PLoS One 10:e0139625
Prochaska JO, Velicer WF (1997) The transtheoretical model of health behavior change. Am J Heal Promot 12:38–48
Bulthuis SE, Kok MC, Raven J, Dieleman MA (2020) Factors influencing the scale-up of public health interventions in low-and middle-income countries: a qualitative systematic literature review. Health Policy Plan 35:219–234
Boker A, Feetham HJ, Armstrong A, Purcell P, Jacobe H (2012) Do automated text messages increase adherence to acne therapy? Results of a randomized, controlled trial. J Am Acad Dermatol 67:1136–1142
Zurovac D, Sudoi RK, Akhwale WS, Ndiritu M, Hamer DH, Rowe AK, Snow RW (2011) The effect of mobile phone text-message reminders on Kenyan health workers’ adherence to malaria treatment guidelines: a cluster randomised trial. Lancet 378:795–803
Obi-Jeff C, Rakhshani NS, Bello-Malabu JI, Nwangwu C, Nwaononiwu E, Eboreime E, Wonodi C (2020) Vaccine indicator and reminder band to improve demand for vaccination in Northern Nigeria: A qualitative evaluation of implementation outcomes. Vaccine 38:4191–4199
Wakadha H, Chandir S, Were EV, Rubin A, Obor D, Levine OS, Gibson DG, Odhiambo F, Laserson KF, Feikin DR (2013) The feasibility of using mobile-phone based SMS reminders and conditional cash transfers to improve timely immunization in rural Kenya. Vaccine 31:987–993
Bigna JJR, Noubiap JJN, Plottel CS, Kouanfack C, Koulla-Shiro S (2014) Barriers to the implementation of mobile phone reminders in pediatric HIV care: a pre-trial analysis of the Cameroonian MORE CARE study. BMC Health Serv Res 14:1–7
Ghadieh AS, Hamadeh GN, Mahmassani DM, Lakkis NA (2015) The effect of various types of patients’ reminders on the uptake of pneumococcal vaccine in adults: A randomized controlled trial. Vaccine 33:5868–5872
Siddiqui M, ul Islam MY, Mufti BAI, Khan N, Farooq MS, Muhammad MG, Osama M, Kherani D, Kazi AN, Kazi AM (2015) Assessing acceptability of hypertensive/diabetic patients towards mobile health based behavioral interventions in Pakistan: a pilot study. Int J Med Inform 84:950–955
Kruse C, Betancourt J, Ortiz S, Luna SMV, Bamrah IK, Segovia N (2019) Barriers to the use of mobile health in improving health outcomes in developing countries: systematic review. J Med Internet Res 21:e13263
Nhavoto JA, Grönlund Å, Klein GO (2017) Mobile health treatment support intervention for HIV and tuberculosis in Mozambique: Perspectives of patients and healthcare workers. PLoS One 12:e0176051
Linde DS, Korsholm M, Katanga J, Rasch V, Lundh A, Andersen MS (2019) One-way SMS and healthcare outcomes in Africa: Systematic review of randomised trials with meta-analysis. PLoS One 14:e0217485

AdditionalFiles.docx
Additional FilesAdditional File 1. Doc. Standards for Reporting Implementation Studies ChecklistAdditional File 2. Doc. Interview GuidesAdditional File 3. Doc. Full list of participants interviewed and type of interview conducted

Download PDF

Reviewers agreed at journal
25 Jul, 2022
Reviewers invited by journal
25 Mar, 2022
First submitted to journal
07 Dec, 2021

You are reading this latest preprint version

Implementing SMS reminders for routine immunization in Northern Nigeria: a qualitative evaluation using the RE-AIM framework

Status:

Version 1

Abstract

Background

Intervention

1. General broadcasts

2. Targeted broadcasts

3. Individualized/personalized reminders

Methods

Study design

Study Setting

Study Participants And Sampling

Data Collection

Data analysis

Results

Study participants demographic characteristics

Reach

Effectiveness

Adoption

Implementation

Maintenance

Discussion

Limitation

Conclusion

Abbreviations

Declarations

References

Supplementary Files

Status:

Version 1