Data for this analysis come from an evaluation of the IntegratE Project, a pilot project that seeks to improve the quality of FP services delivered by CPs and PPMVs through a tiered accreditation system. Beginning in January 2019, 1,465 CPs and PPMVs located in Kaduna and Lagos States were trained by the IntegratE project. As part of the evaluation study, 559 CPs and PPMVs completed a self-administered questionnaire to assess their knowledge related to the provision of FP counseling, and injectable and implant contraceptive services depending on their tier at three points in time: 1) before the training; 2) post/immediately after the training; and 3) 9-months after the training. The assessments were developed from established knowledge assessment tests used by the Project’s training partners.
All questionnaires asked the same knowledge questions on FP counselling, injectable and implants contraceptives. The pre-test assessment also included questions on respondent characteristics and their cadre, state, experience providing injectable and implant contraceptives. The nine-month questionnaire included questions on services offered by CPs and PPMVs since the training, and their experience with the intervention including training received, job aids and monitoring visits. The results from the pre and posttest and 9 months follow up questionnaires were compared and analyzed in this paper.
All participants enrolled in the study were given detailed information about the study before consenting to participate. The study protocol was submitted and approved by the Population Council Institutional Review Board.
The IntegratE Project
The IntegratE Project, supported by Bill &Melinda Gates foundation and MSD for mothers was implemented by a consortium of partners led by Society for Family Health in Nigeria. Other partners include Marie Stopes International of Nigeria, PharmAccess Foundation, Planned Parenthood Federation of Nigeria, and the Population Council. The project adopted a tiered accreditation approach where providers were categorized into three tiers based on their health qualifications. Each tier was authorized to practice FP services consistent with their qualifications (see Table 1).
All providers were trained in FP counseling using the balanced counseling strategy plus (BCS+) toolkit. Sessions focused on (a) different contraceptive methods available in Nigeria; (b) instruction and practice sessions to offer client-centered FP counseling using BCS+; and (c) instruction on how and when to refer clients for FP to other sources of care based on client’s medical eligibility.
In addition to FP counselling training, CPs and Tier 2 and Tier 3 PPMVs received further training on administration of injectable contraceptives and administration and removal of implant contraceptives. Providers practiced inserting implants on models and inserting injectables and implants on clients who voluntarily choose either of these two methods. CPs and PPMVs were referred to the social marketing sector to procure additional commodities. Supervisory monitoring visits were conducted within three months of the training to gauge compliance with protocols related to FP and infection prevention and control.
After the FP training, CPs and PPMVs were given the BCS+ toolkit, which includes: a) BCS+ counseling cards; b) BCS+ algorithm; c) FP flip chart; and d) FP method brochures. In addition to the BCS+ toolkit, Tier 2 and Tier 3 PPMVs and CPs were given the Contraceptive Medical Eligibility Criteria (MEC) wheel check. An explanation of each tool is available in Table 2.
Three dependent variables were used in this paper: 1) general FP knowledge retention, 2) injectable knowledge retention, and 3) implant knowledge retention. General FP knowledge retention was assessed among Tier 1 and Tier 2 PPMVs and CPs, while injectable and implant knowledge retention were assessed only among Tier 2 PPMVs and CPs, as these tiers were trained in the provision of these methods. General knowledge and injectable knowledge categories each included 11 items, while 7 items were included in implant knowledge. Knowledge retention was measured for each item from posttest to the 9-month follow-up survey. Those who correctly answered a knowledge assessment question at posttest and answered it correctly at 9 months had the knowledge after the training and retained it over time. Respondents who did not know the item at posttest or at 9 months, who did not know it at posttest but did at 9 months, and who did know it at posttest and at 9 months were not considered to have retained knowledge from posttest to 9-months.
Overall summary knowledge retention scores were developed for general FP knowledge, injectable knowledge, and implant knowledge. Scores were created for each of the three outcome variables separately. The total number of items that each respondent retained from posttest to 9 months was added. For general and injectable knowledge this is 11 items and for implant knowledge this is 7 items. Retention includes those who answered the question correctly at posttest and 9 months, while all other combinations are not counted as retention. For each outcome variable, the mean number of items that were retained was calculated. Those who scored above the mean were coded as 1 for high knowledge retention, and those who scored below the mean were coded as 0 for low knowledge retention. The mean scores were 5.7 out of 11 for general knowledge, 7.9 out of 11 for injectable knowledge, and 2.7 out of 7 for implant knowledge. Those who scored 6 or higher for general, 8 or higher for injectable, and 3 or higher for implant were considered to have high knowledge retention.
The key independent variable for this analysis were the five job aids given to all providers trained by the project. The job aids were categorized into three groups for their similarities: category 1 included BCS+, BCS+ algorithm, or FP flip charts (an alternative to BCS+), category 2 were the FP method brochures, and category 3 was the MEC wheel. For each of the three categories of job aids, responses were categorized as those who received the job aid during or after the training and those who did not receive the job aid.
Additional covariables considered were gender, age, tier, state, previous health facility experience, and receipt of specific job aids. Education was also considered but because it was used in the determination of tiers, it was highly correlation with tier and excluded from the analysis. For implant and injectable knowledge, a variable on whether the provider had offered these specific services in the 30 days preceding the interview was also included.
Descriptive statistics were calculated for respondent characteristics at pretest and are presented by tier. Descriptive statistics for knowledge retention for general FP (among all providers), injectable (among Tier 2 PPMVs and CPs), and implant (among Tier 2 PPMVs and CPs) were conducted. The proportion who correctly answered each item at posttest was calculated, and among those who answered correctly at posttest, the proportion that retained the knowledge at 9 months is presented. The denominator for each item is dependent on the proportion who answered correctly at posttest.
Three multivariate logistic regression models were used to assess the effect of provider characteristics and receipt of job aids on general FP knowledge retention, injectable knowledge retention, and implant knowledge retention 9 months after the training. The outcome variable in each of the three models was high versus low knowledge retention according to the mean scores (as described above). The model for general knowledge was conducted among all respondents, while the implant and injectable models were conducted among Tier 2 PPMVs and CPs.