Strengthening the quality of paediatric primary care: A process evaluation of a complex health systems intervention in South Africa

BACKGROUND: Innovative strategies like WHO’s Integrated Management of Childhood Illness (IMCI) have resulted in progress in addressing infant and child mortality. However, the needs of children continue to present a burden upon primary healthcare services. The capacity of services and quality of care offered require greater support to address these needs and extend integrated curative and preventive care, specifically, for the well child, the child with a long-term health condition and the child older than 5 years, not included in IMCI. In response to these needs, the PACK Child intervention was developed, based on a similar successful approach in adults, that expands the scope and reach of integrated management and training programmes for paediatric primary care. We report findings from the process evaluation of the integration of PACK Child intervention within the existing primary health care system. METHODS: A mixed methods process evaluation was completed in ten primary health care facilities in the Western Cape province in South Africa, where clinicians were trained to integrate PACK Child into routine practice. Qualitative data included interviews with managers, clinicians, caregivers and policymakers; observations of training, consultations and clinic flow. Quantitative data included training logs and clinicians’ questionnaires. RESULTS: Impact of PACK Child on clinical practice: Longer consultations, improved clinical knowledge and practice, better teamwork, strengthening of appropriate referrals and inclusion of psychosocial risk screening. Implementation fidelity of PACK Child was hindered by over-reliance on documentation and embedded checklist approach to consultations limited to acute episodic illnesses. PACK Child

In the Western Cape province of South Africa almost every public sector primary care facility employs an IMCI-trained nurse, and it is these nurses who attend to the majority of children's healthcare care needs. At a series of meetings with key stakeholders in provincial paediatric health -primary care nurses, doctors, managers and educators, hospital-level paediatricians and policy makers -the growing gaps in knowledge and expertise for children at primary care level were recognised as well as a need to integrate well child routine care into the delivery of everyday paediatric primary care.
This prompted the development of an expanded programme to address a larger remit of paediatric care. Led by the University of Cape Town's Knowledge Translation Unit (KTU), the PACK Child intervention was based on the Practical Approach to Care Kit (PACK) Adult programme that has supported the delivery of comprehensive, integrated adult primary care in the province for the past 13 years. (8,9) Implementation of a health system strengthening intervention like PACK in a health system is a complex activity, requiring an understanding of how it will interact with varying contexts of delivery. The Department of Health was especially keen that we address stakeholder concerns of its integration with existing programmes and policies, particularly IMCI. To explore these issues and address concerns, a process evaluation was conducted alongside a pilot of PACK Child to determine what refinements are needed at intervention and health system levels to optimise its implementation.
The process evaluation provided many rich insights on the tension between contextual features of South African paediatric primary care and how clinicians attempted to integrate the PACK Child intervention within primary care consultations. These are addressed in more detail in another paper providing indepth understanding of the complexities of paediatric primary care, and the challenges of embedding an intervention within this context.
In this paper, we provide an overview of the process evaluation findings as the PACK Child intervention evolved during the pilot, including how the health system interacted with intervention delivery, and perspectives of how PACK Child impacted clinical practice and the needs of caregivers. In the discussion, we provide suggestions on how to optimise delivery of PACK Child for wide-scale implementation in primary care with consideration of barriers and facilitators to change.

Setting
The pilot and process evaluation took place in 10

PACK Child Intervention
The PACK Child programme comprises a clinical decision support tool (guide), staff training and health systems strengthening elements to enable its use in everyday practice. The guide is a comprehensive, integrated and policy-aligned clinical decision support tool providing an approach to 63 common symptoms, and 16 priority long-term health conditions for children from birth to age 13). (11) It also contains several 'Routine Care' pages [see Additional file 2]-standardised checklists for the provision of well child routine care which covers aspects of nutrition, growth, immunisations, HIV and TB screening, and psychosocial risk of both carer and child.
Following the implementation strategy of the PACK Adult programme,(12) PACK Child employs key methods to enable scalability and sustainability. Governmentemployed staff serve as trainers to deliver, via a cascade model, onsite, educational outreach sessions to the clinical team using a training methodology underpinned by adult education principles. (12) The 1½ to 2-hour training sessions focus on priority content and key messages, using a curriculum of case scenarios to convey these messages and embed use of the guide into everyday clinical practice [see Additional File 3]. Health systems strengthening components include: clarification of prescribing roles among cadres of clinicians, strengthening referral pathways with secondary and tertiary level hospitals through sensitisation training, and assessment of patient clinic flows and referral pathways. A KTU trainer, who has extensive experience with children and has worked both as an IMCI nurse and trainer (MS), trained clinicians on-site during the first two phases and then trained facility trainers in a five-day workshop to train staff in the remaining six facilities include in Phase Three. Additional file 4 shows the evolution of the intervention across the three phases of the pilot, integrating feedback from the process evaluation and directly from staff at facilities, including the addition of health systems strengthening components.

Design
We used a linguistic ethnographic methodology, (13,14) which provides theoretical and methodological tools for analysing social action within context. We have previously adapted this approach, to facilitate detailed investigation of complex healthcare interventions across macro-, meso-and micro-contextual levels. (15) Detailed methods are described elsewhere. (16) The process evaluation used a mixed method approach included quantitative and qualitative data collection methods in all facilities. Qualitative methods included observations of training sessions; semi-structured interviews with caregivers; clinician, policymaker and paediatric manager focus groups, and ethnographic observations of consultations and non-clinical areas in each facility. Quantitative methods included auditing of training attendance logs and a clinician' questionnaires completed six months after finishing the PACK Child training programme.

Study population
Participants included in this study were caregivers, and their children where appropriate; clinicians working at the selected facilities, which included nurses (professional nurses (PN), clinical nurse practitioners (CNP), enrolled nurses (EN), enrolled nursing assistants (ENA)), doctors and pharmacists; PACK Child facility trainers; facility managers, local district structure representatives and paediatric programme co-ordinators and policymakers.
To be eligible for inclusion, nurses and doctors needed to receive PACK Child training and caregivers and children to be receiving paediatric primary care at the selected facilities. Children needed to be aged 0-13 years to receive paediatric services. Policymakers and managers needed to be responsible for delivery of public sector primary care in South Africa.

Sampling
Data collection for the process evaluation occurred concurrently with the three phases of the pilot, enabling analysis of Phase One data to inform the sampling strategy in Phases Two and Three. Purposive sampling was planned in Phase One to select and recruit managers, clinicians, caregivers and children. Sampling of children in Phase One was intended to be informed by diversity of conditions, level of deprivation and the age of the child. However, in practice, we responded to children presenting on days during which data collection took place, with clinic nurses identifying and approaching caregivers of eligible participants in clinic waiting rooms. Findings from the analysis of Phase One qualitative observation and interview data, (for example, children's presenting conditions, or challenges to using the PACK Child guide), informed theoretical sampling (17) of clinicians, caregivers and children and timing of data collection in Phases Two and Three. In addition, the type of consultation observed in Phase One included mostly children under 5 years presenting with acute symptoms. In Phase Two and Three, we tried to include more consultations with older children and those with long term health conditions, however due to embedded patterns of caregivers/children seeking care, this proved difficult to implement.
We observed all training sessions in Phase One to understand how each session was delivered by trainers and received by clinicians. Sessions in which we identified tensions, or difficulties in delivery were observed in Phases Two and Three such as difficulties in using different documentation and orienting to symptoms as potential markers of long-term health conditions. In Phase One, ethnographic observations of consultations were conducted after each session of the PACK Child training. This highlighted that clinicians needed time to familiarise themselves with the PACK Child guide. In Phases Two and Three we therefore decided to conduct observations of consultations towards the end of the training programme. We invited all managers at facilities to be interviewed and we observed non-clinical areas in all facilities. We conducted a stakeholder workshop with 23 representatives from the pilot facilities, local district structure representatives and paediatric programme coordinators to present the pilot findings and facilitate discussions on the implications of PACK Child for wider implementation. Children over seven years old were asked to give assent to their participation.
Caregivers and children were asked to consent to interview and observation on the day they attended the clinic. Facility managers provided consent for observations of training sessions and non-clinical areas. All participants were provided with written information about the research, informed that their participation was voluntary and that they could withdraw from participation at any time.

Data analysis
Qualitative interviews and focus group data, and field notes of our observations of waiting rooms and reception areas were analysed to understand how facilities were organised to provide paediatric primary care, identify contextual features likely to impact on intervention delivery, and to obtain perspectives of the existing primary healthcare system and the function of PACK Child within that system. We also identified which documents clinicians used during paediatric primary care delivery To understand how the context of primary care interacted with delivery of PACK Child we analysed observations of training sessions and consultations. To assess fidelity we drew on MRC guidance that understanding adaptation to the local context is more appropriate than a strict assessment of fidelity. (18) Unlike studies that assessed nurse fidelity to IMCI guidance in a large sample of consultations (19,20) using a fidelity checklist, we chose to incorporate an understanding of the interaction between clinician's use of the guide and the organisational and social context in which PACK Child was being introduced. This required in-depth qualitative investigation in a relatively smaller number (n = 53) of consultations and sites. We analysed the audio-recordings, transcriptions and researcher field notes of consultations to understand how contextual features shaped nurse's interactions with caregivers and children. (16) We then triangulated all qualitative and quantitative data in order to obtain indepth insight into the relationship between PACK Child and the context of paediatric primary care, allowing us to generate hypothetical propositions for optimising implementation of the intervention more widely.

FINDINGS
The findings are presented in three broad areas considering how delivery of PACK Child changed as the intervention evolved across the three phases of the pilot.  (Table 1).  Length of consultations Analysis of consultation observations suggested that where clinicians followed the PACK Child guide in detail, which included routine care for the well child, consultations were longer than where clinicians only partially used the guide. Other factors appearing to influence consultation length included the number of symptoms the child presented with, previously unidentified problems now identified through the clinician's use of the guide, and staff using PACK Child who did not routinely attend children. Table 2 shows the average consultation length in Phase Three was 7-8 minutes longer than the other two phases and included an average of 10-13 more questions per consultation. However, there were also a few extreme cases in Phase Three, with one consultation comprising 84 questions and lasting 52 minutes due to the child presenting with multiple problems that required assessment and which the nurse dealt with in turn. Although use of PACK Child appeared to entail longer consultations, some staff reported that increased familiarity with the guide, through regular use over one or two months, would reduce the duration of consultations.
"Sometimes it takes too long, because I work alone in the room, and I have about 30 patients, sometimes more than 30 that I see in a day, then it takes a while, but the more you familiarize yourself with the guideline it will go quicker, because you'll know the questions to ask, but for now it's taking a little bit of time." (Nurse, Focus group, Phase 3)

Improving clinical knowledge and practice
Eighty-seven per cent of surveyed clinicians reported that the PACK Child guide and training programme improved their ability to both diagnose and treat children even though 85% of them were already trained in IMCI.
One area where this was especially pronounced was on growth monitoring. One of the training sessions is dedicated to embedding correct monitoring and interpreting growth in children. Using case scenarios, clinicians are asked to plot children's growth measurements on a chart and interpret their growth using the PACK Child guide. Many nurses initially struggled to determine where to plot measurements on growth monitoring charts. After this session, staff reported growing confidence in their ability to plot and interpret the growth of children.
"And the assessing and interpreting the growth let alone the minor things, plotting the growth chart which we were doing, but now we know how to do it in the right way. But now that we have done PACK Child, we know the way to do and assess whether they are implementing it right. It starts with weigh, measuring and plotting.
If you plot wrongly, it will be interpreted wrongly. Many clinicians also reported that the algorithms were clearer in PACK Child than in IMCI despite having the same or similar information.
"What we have noticed is that the algorithm we have got from PACK Child is more clearer than IMCI, although the information is the same or similar, but it is more clearer." (CNP, Phase 2) However, clinicians experienced some difficulty in shifting the focus of care from an acute episodic paradigm to one of long-term health conditions, especially in the management of recurrent respiratory conditions. Several doctors reported being hesitant in supporting nurses to prescribe inhaled medication, potentially discouraging nurses' from taking advantage of the expanded content of PACK Child.
"In the asthma case presented in the session, the child presented with a recurrent wheeze for five days. The child was given a trial of an inhaler, but the clinicians omitted checking the bronchodilator response before prescribing. The "health system strengthening session", which primarily focused on the flow of children through the clinic, facilitated changes in some facilities. Where verticalization of care was predominant such as one nurse weighing all the children, nurses in consultation rooms began to alleviate the burden of tasks traditionally limited to certain cadres, which promoted task-sharing amongst clinicians.  Improvements in psychosocial risk screening According to clinicians, use of PACK Child in consultations led to more psychosocial risk issues being identified in consultations, resulting in some referral and resolution of these disclosures.
"It prompted you now with that section to ask for social problems. I also had one child: she didn't have an ID. Mum didn't have an ID that's why she didn't register the child, and she can't apply for a grant, and I helped her. So that section is really good. It prompts you to ask those questions. In the past we overlooked it." (Nurse, Focus Group, Phase 2) However, introduction of routine psychosocial risk questions, in an embedded checklist approach to consultations, appears to result in nurses asking the questions in such a way that limit disclosure of psychosocial problems. Due to the volume of questions to be covered in a consultation, questions are framed in a way to rule out problems instead of encouraging disclosure. (21) Furthermore, in some cases, where disclosures were made, the clinician could be seen to minimise its importance or not address the disclosures made by caregivers.
This excerpt describes the lack of human resources and one of the reasons why issues that were disclosed may have been minimised.
"We have a problem with psychiatrist. If you get the problem of abuse, then you must send the child to the hospital, because we don't have a person here every day, that's also a problem. Sometimes when you book the people, for that then the guy cancels his visit." (Nurse, Focus Group, Phase 3)

Implementation fidelity of PACK Child
The PACK Child training encouraged clinicians to approach a consultation in a standard way using the PACK Child guide, starting with an assessment for urgent symptoms and in their absence proceeding to provide well child routine care.
Clinicians were trained to navigate the guide for the relevant symptom/s, or longterm health conditions (LTHC) only once they had completed routine care activities.
Our findings revealed several contextual issues which interacted with clinicians' use Navigating PACK Child within current routine practice Implementation of the PACK Child approach during consultations was variable and each facility's requirement to use multiple documents resulted in clinicians alternating between different documents and consultation activities. Table 3 illustrates how the PACK Child approach to consultations was interrupted as clinicians switched between different routine care and symptom-based activities and various sections of the guide, whilst also completing necessary documentation. In particular, clinicians typically struggled to follow the consultation sequence recommended by PACK Child guide and training. Following the PACK Child guide as a script The distribution of question types shown in Table 4  When the Routine Care page was used, nurses were sometimes not returning to the child's reported complaint following routine care, leaving the complaint unaddressed, or not resolved according to PACK Child recommendations. This demonstrates the difficulty clinicians had in attending to both routine care and the presenting symptoms, with some clinicians reporting a tendency to forget the presenting symptoms.
"You ask symptoms, it may be related to the skin and no urgent signs whatever the case may be and then we have to go routine care, and then we need to note we have to go back and then continue from there, which is where Sister picked up the weight and things which was more alarming because her initial thing that was noted. So that is where Sister also forgot about the symptoms, which was the skin.
So, the Routine Care page is a big help, it is a big help, but it is just the back and forth as Sister say." (Nurse focus group, Phase 3) The impact of PACK Child on caregiver/child participation in care The focus on acute symptoms was identified by caregivers as the core element of most paediatric primary care consultations. Although sometimes attending with recurrent symptoms, caregivers reported that clinicians rarely asked about the child's previous history or referred to documentation, instead treating each episode separately.
"Interviewer: And when you come for the same problem, do they check that she had it before, or do they just treat it like it's new?
Caregiver: They never do, they never like look on her records before, normally she does have tonsils, then they give tablets or something."(Caregiver Interview, Phase 3) The caregiver view that nurses do not investigate the child's medical history, lack the knowledge to diagnose and adequately treat their child, and that the caregiver feels uninformed.
"Interviewer: Have you brought him to the clinic for the tight chest.
Caregiver: I have come many times Interviewer: Have they given him anything?
Caregiver: They give an inhaler. The chest is too tight.

Interviewer: Have they said what it is?
Caregiver: They don't know, it's like they don't know, and I don't know.
Interviewer: Did they refer you to anybody.
Caregiver: No, they don't. They don't want to check what is wrong, he is still using the inhaler." (Caregiver Interview, Phase 1) However, interviews with caregivers emphasised the impact of PACK Child's broadened scope of content on the clinical assessment process. They, frequently reported a marked shift in the questions they were asked in PACK Child consultations compared with that prior to the pilot, which they linked to the clinician's use of the PACK Child guide.
"It has changed because there was not that book, like there were not those kinds of questions because the last time I was here, I also came for the same problem, but it was not the same he has sore wounds, like there was a wound here, that I came for, but there was not this book asking such questions, they didn't ask so many questions, like they just gave me medication." (Caregiver Interview, Phase 3) Caregivers also reported that they were not routinely given scheduled visits for their children's long term health conditions, but with PACK Child an appointment was given.
"I just bring (her) in when I see that the nebulizer at home isn't helping, but now the sister gave me three-month supply with the medication, and she gave me an appointment for after the 3 months; to see us which didn't happen previously." just that one thing. Like this was now nice. Everything was asked, and they have the patience to explain everything. And feel free to explain everything. Sometimes you go to the doctor, you just get cut you off because they rush you to get to another patient. Then that happens all the time." (Caregiver Interview, Phase 3) Caregivers valued being given the space and time to talk and ask things that were important to them, a sentiment frequently expressed following PACK Child consultations.
"And it was something good that I've learnt today, and I hope they keep it this way; to ask all those questions, and they took their time with us. It was not like before, where the time was really short." (Caregiver Interview, Phase 3) "Yeah, the way the doctor handled it. It was nice for me, because just for the fact that I can talk a lot of things ask lot of things. He come for his nose, but I could ask for this… she saw the marks of the eczema, almost like eczema." (Caregiver Interview, Phase 3) Explanation was defined as key to participatory care by caregivers, without the caregiver needing to ask for information. Caregivers described being unable to question nurses when they did not understand the jargon they used. In order to foster participation of caregivers in consultations, using language which caregivers understand is essential, and caregivers stated this occurred when PACK Child was used.
"The only difference is, like now with this consultation. She explained to me properly. A lot of the time the sisters were doing stuff, or give you something, but they don't explain to you what is going on. You must always ask, and you feel so stupid sometimes. You must always ask to explain something to me properly, because you don't know the jargon. So now you have to ask every detail and stuff like that. And sometimes they will look at you like you silly, and so that's the only difference, that they explain to you stuff." (Caregiver Interview, Phase 3) Although many caregivers reported improvements in their level of participation in consultations, this was not reflected in how several clinicians viewed the impact of PACK Child on the clinician-caregiver interaction.

DISCUSSION
The PACK Child intervention was developed to address the limited scope of IMCI, by expanding a focus in the provision of paediatric primary care from under-fives to children aged up to 13 years and those living with long term health conditions. (11) However, this study illustrates the challenges of implementing PACK Child within a Unsurprisingly clinicians perceived an increase in consultation length as a consequence of introducing new routine care questions. While staff felt that with practice and consistent use of the guide, consultation times would shorten; they experienced difficulty following the guide meticulously within a time-pressured consultation. Both our observations and interviews revealed that clinicians occasionally forgot to return to the presenting symptom after providing routine care. This was identified as a potential harm and calls into question the need for and feasibility of routine care questions, and whether these can be reduced or tailored to individual needs or included in a different way. completion of documentation appears to be a strong driver and shifts the focus from the caregiver and the child. Only through addressing these tensions is optimal implementation of PACK Child likely and any other programmes that aim to improve the quality of paediatric primary care.

Strength and Limitations
This process evaluation was the first study in LMICs to conduct qualitative observations of consultations in paediatric primary care that applied conversational linguistic analysis (21) to audio recordings which were then triangulated with observed use of documentation and interviews with clinicians and caregivers. Whilst our findings do not offer statistical generalisability, this in-depth analysis functioned to provide critical insights on the current state of paediatric primary care in the Western Cape, South Africa in a low income setting and the generation of hypothetical propositions for optimising intervention delivery.
Caregiver interviews were often conducted in facility waiting rooms either before or following the child's consultation, which limited the ability to have extended confidential discussions with caregivers. Sampling of caregivers and children was also limited by those who attended on the day, which may have restricted the broader view of other caregivers who may have attended on different days and attended the clinic regularly. Our findings are also limited by needing to collect data both during and immediately following completion of the PACK Child training programme, which allowed little time for the intervention to be embedded into everyday practice. Whilst our observations of clinicians using PACK Child may have been subject to the Hawthorne effect, (27) arguably affecting the extent to which clinicians used the PACK Child guide, the researcher's presence was less likely to affect how clinicians asked questions and the sequence to the consultation that they delivered.

CONCLUSIONS
In this study, despite many barriers to optimal care, the PACK Child intervention had a positive impact on the delivery of paediatric primary care. It expanded the scope offered, fostered team work and improved the patient/caregiver experience. With further refinement of the programme at a guide, training and system level there is potential for PACK Child to streamline paediatric care in health resource scarce settings globally.