Study Setting
The study took place in two district hospitals in Rwanda: Masaka hospital in Kigali, an urban province; and Nyamata hospital located in the eastern rural province [26]. The two hospitals were selected out of 12 district hospitals in the two provinces because both had a high number of deliveries per year [27]. The two hospitals have been offering BEmONC services for more than five years.
Study Participants
Nurses and midwives working in the maternity departments of the selected hospitals, who manage deliveries and newborn care were invited to participate in the study. The inclusion criteria for nurses and midwives were set as follows: having a work experience of at least 6 months in obstetric care, being full-time employed in the selected hospital, and willing to participate in the study. The study participants included 33 midwives and 21 nurses.
Study design
The study adopted a quasi-experimental design to compare knowledge and skills of nurses and midwives in the management of PPH and NR at two measurement points: pre and post the SDA intervention. The study was carried out in four phases: establishing the baseline, the educational sessions, the intervention and the end-line. The baseline and end-line surveys were conducted pre and post the intervention respectively. The six-month intervention period was preceded by training of the participants on how to use the SDA. The study took place between July 2019 and April 2020. Figure 1 provides more detail on this study design.
Description of the SDA intervention
The SDA is an mLearning application designed to provide training for health care providers in low-income countries on how to manage obstetric and neonatal emergencies. The SDA conveys knowledge and skills, such as how to ventilate a newborn in need of resuscitation or how to remove a retained placenta by means of visual guidance using animated instructional videos and a self-explanatory learning platform. The SDA also contains a catalog with essential obstetric drugs and equipment. The content of the SDA is based on WHO clinical guidelines on BEmONC and has been validated by an international group of global health experts [28]. All features and functions in the SDA are designed for low-literacy and low-income settings and work off-line once downloaded. It can thus, be adapted to local contexts and policies. The SDA can be downloaded free of charge for iPhone at https://itunes.apple.com/dk/app/safe-delivery/id985603707?mt=8 and for Android at https://play.google.com/store/apps/details?id=dk.maternity.safedelivery&hl=en .
Smartphones with the SDA were implemented at two district hospitals for a six months’ period. Nurses and midwives working in the maternity departments of the selected hospitals received a half-day SDA introduction training. The training encompassed a description of the SDA features and modules and an explanation of how to use the smartphone and SDA with joint video viewing and interactive exercises in small groups on using the SDA as a learning tool. Each of the study hospitals (Masaka and Nyamata) was allocated three smartphones, with pre-installed English and French versions of the SDA. Participants who owned smartphones, were encouraged to download the SDA for use during the training. The majority of participants (n = 47) downloaded the SDA on their personal smartphones for the training. During the six months’ SDA intervention, nurses and midwives used the SDA in a personalized learning journey to improve their knowledge and skills on how to manage obstetric and neonatal emergencies with the assistance of the SDA installed on their personal smartphones or the smartphones provided by the researcher. The research team ensured that the provided smartphones were available to the team on duty at all times. The intervention also entailed two visits per week by the researcher and four research assistants to each of the study hospitals. During the visits, the researcher and the research assistants observed how the participants were using the SDA and encouraged them to watch the videos while they were there, they also provided guidance on how to start their learning journeys, specifically for the PPH and NR modules.
Measurement and data collection
The primary outcomes of the study were knowledge scores and skills scores of nurses and midwives. Self-administered key feature questionnaires developed and tested by Maternity foundation were adopted, only questions regarding PPH and NR were considered. Knowledge and skills scores were measured for 2 BEmONC services, PPH management and neonatal resuscitation. The research instrument included questions on PPH management and NR knowledge and skills, demographic information, as well as exposure to in-service education on the topics under study. The knowledge questionnaire consisted of clinical content questions on causes, consequences, prevention, treatment and management of PPH and NR. The knowledge questionnaire includes 36 knowledge questions for PPH management and 37 knowledge questions for NR with single and multiple responses options. The skills questionnaire comprised of six clinical scenarios for PPH management and four clinical scenarios for NR with single and multiples responses options. The collected demographic data of the participants included age, gender, education level, years of experience in obstetric care, hours spent per week in providing obstetric care, and number of deliveries participated in monthly. In addition, we asked participants if they have received any in-service training about PPH management and/ or neonatal resuscitation during the 6 months prior or during the SDA intervention period. This question helped the researcher to understand more about the influence of the SDA or other available in-service training. We also asked participants if they have previously used smartphones to capture their experience with smartphones.
A total of 54 nurses and midwives completed either the English or the French form of self-administered questionnaires at baseline and at 6 months’ post-intervention. English and French languages are both professionally used in Rwanda. Baseline measures were taken prior to the training of nurses and midwives on the use of the SDA and included: demographic data; knowledge of nurses and midwives in the management of PPH and NR; and skills of nurses and midwives in managing PPH and NR. All data were collected on paper in a classroom setting. Six months after the SDA introduction, knowledge and skills were measured a second time in the classroom setting using the same data collection instruments.
Data management and analysis
The data were entered and checked for errors in Microsoft Excel and subsequently transferred and analyzed in Stata version 16 (StataCorp LLC). Descriptive summary statistics were computed on demographic data including age, gender, education level, years of experience, hours spent per week in providing obstetric care, delivery participated in monthly, previous use of smartphone, and in-service training for six months prior and during SDA intervention. Since the study targeted all nurses and midwives working in the maternity department of the selected hospitals, all the eligible participants in the maternity department, a sample of more than 30 participants was required to support the use of parametric statistical tests.
Knowledge and skills scoring were calculated by dividing the number of questions answered correctly by the total number of questions times a hundred. Respondents who scored 50% and above were considered knowledgeable. Further, Mean scores across all knowledge and skills scores were computed. A paired-sample t-test was used to measure the pre-post intervention, mean knowledge and skills scores differences (where the dependent variable was the mean difference in change on scores for the two measurements, baseline and endline). Confidence intervals (CIs) and effect size were calculated. To test for potential confounding, using a t-test, pre-post intervention mean knowledge and skills scores were compared and between group differences were calculated to examine the role of in-service training and previous smartphone use on test scores. One-way analysis of variance (ANOVA) was used to compare means scores across the three categories of experience in obstetric care (group 1: 1 year to 5 years, group 2: 6 years to 10 years, group 3: Above 10 years) for pre and post intervention results. Significance level was set at p<0.05.
Ethics approval
This study has been approved by the Human Research Ethics Committee of the University of the Witwatersrand (M190258) and the University of Rwanda, College of Medicine and Health Sciences’ Institutional Review Board (No.377/CMHS IRB/2018). Written consent was obtained from each participant. Confidentiality of the participants was maintained at all times. Questionnaires were number-coded thereby keeping the identity of the participants coded.