Realist evaluation
Realist evaluation (RE) is a theory driven evaluation that starts and ends with a detailed hypothesis called programme theory developed on the basis of existing knowledge and tested through iterative process of theory development, testing and refinement using empirical research(31, 32)(see figure 1).
According to Merton, 1967, programme theories are “ theories that lie between the minor but necessary working hypotheses that evolve in abundance during day-to-day research and the all-inclusive systematic efforts to develop a unified theory that will explain all the observed uniformities of social behaviour, social organization and social change.” (33);
We consider regional taskforces as complex social systems characterised by a dynamic network of interacting agents and structures and by the emergence of unpredictable outcomes depending on context and time periods (35). The realist evaluation (RE) proved appropriate in deciphering such complex social systems (32, 36).
Developing an initial programme theory
Following guidance from (Westhorp,2012) and (Shearn,2017)(36) (37), we combined a rapid review of theoretical frameworks of health system governance structures and policy processes (18) (38) with policy document analysis (25, 39-41) and interviews with policy makers to develop a programme theory, that explain how, and under which conditions a regional taskforce is effective in the specific context of Morocco.
Based on the analysis of policy documents (25), an effective regional taskforce (RTF) need to accomplish the following goals: 1) Coordinating actions between different types of actors from different health systems organisations (Health regional office, teaching referral hospital, regional, provincial hospitals and primary healthcare centres (PHC)(25). 2) Strategic management of MNH programmes by analysing health system performance in relation to MNH and aligning national MNH strategies to the context of the health regions and provinces. 3) Technical support to MNH programme implementation:RTF supports provincial health teams in identifying implementation gaps and allocating required technical support, regulations and resources to alleviate them. 4) Monitoring MNH key performance indicators (e.g. Proportion of certified child birth centres, proportion of births attended by skilled health personnel, antenatal care coverage, caesarean section rate) (42) and ensuring the quality assurance of health information management systems at provincial level (e.g. maternal mortality surveillance systems (SSDMAR). 5) Engaging with partners : an effective taskforce needs to be able to engage with local partners (Governor, funders, socio-professional organisations, community representatives…etc).
In health policy and system research, many health system governance frameworks exist(17, 27). Most of these are inspired from economic and political theories (agency theory, policy networks, etc). In this study, in line with (booth, 2012) (43) and (Brinkerhoff and Bossert 2014)(17), we posit that effective governance systems are characterised by a best fit between their intrinsic capacities and the context of policy processes (complexity of health problems, leadership and engagement of actors, and the institutional statutory rules).
Intrinsic capabilities are the causal powers, (mechanisms in realist terms (44)),that are triggered in specific contexts which bring about the outcome of interest (in our case, degree of goal attainment) (38). These capabilities refer to the dynamic abilities of governance systems and organisations to coordinate a set of tasks and to utilise their internal resources to achieve specific end results(45) According to (Contandriopoulos,2004)(18),intrinsic capabilities includes technocratic, political, democratic and learning processes. Technocratic capabilities refer to the ability of the taskforce to support policy implementation by delegation of responsibilities, priority setting and ensuring accountability to central administration. Political capabilities refer to the ability to engage with key stakeholders. This depends on the decision space and degree of autonomy of members, their capacity to set rules and procedures, to negotiate and distribute power among its members. Democratic capabilities refer to the ability of members to reach consensus and implement agreed upon collective decisions. Finally, learning capabilities mean the ability to stimulate learning processes, use of evidence, and knowledge sharing among actors (Contandriopoulos,2004)(38).
By context, we mean the complex nature of health problems (technical difficulties, diversity of actors, the nature of required social change), the pre-existing healthcare statutory rules and legal frameworks (decision making processes, role attribution, objective coherence and resource allocation), the leadership and engagement of actors (senior managers and frontline workers), and finally, the general context of health regions (organisation of regional health systems, staff availability, pre-existing relationships)(18).
We formulate our programme theory as follows ”a taskforce is effective whenever there is a best fit between its intrinsic technocratic, political, democratic and learning capacities and the characteristics of the institutional context, the complexity of health problems and the engagement and leadership of actors (see figure 2.)
Multiple embedded case study design
We will adopt a multiple embedded case study design (46) that allows a flexible exploration of different levels of analysis and takes into consideration the role of context (47).
The multiple embedded case study design is a suitable approach for the exploration of complex social phenomena and is flexible enough to allow the cross case comparative analysis (47) and the testing of our programme theory in two negative and positive cases.
Case definition
The case here corresponds to the dynamic interactions between actors and structure within the MNH regional taskforce.
Case selection
We selected six health regions :Tangier Al Hoceima Tetouan (TTA), Marrakech Safi (MS), Benimellal Khenifra (BMK), Casablanca Settat (CS), Deraa Tafilalt (DT) , Rabat Salé Kenitra (RSK). We purposefully chose the six case studies to allow maximum variation in regional contexts (rural versus urban, with or without financial support, high versus low population density). The selection processes was informed by discussions between research team members and with commissioners of the evaluation (Ministry of health, direction of population and the UNICEF) (meeting, 22 oct 2020).
Selection of participants
The selection of participants is guided by the programme theory (31, 48) and the statutory regulations (49). We included two types of participants :
- taskforces members : this group includes ex officio members (regional health officer (RHO), provincial health officers (PHO), hospital directors, heads of maternity wards and primary healthcare networks), and professional experts in the following disciplines: neonatology, gynaecology and obstetrics, anaesthesiology and resuscitation, neonatal resuscitation, neonatology, paediatrics, midwifery (25).
- Policymakers and consultants who have been involved in the design and the implementation of the RTF.
Data collection
We based the choice of the data collection methods on our initial programme theory (figure 2). We will follow specific guidelines in carrying our individual interviews (50), focus group discussions (FGDs)(51), a document review. Interviews and FGD, will be carried out in Moroccan dialect or in French (see supplementary files 1-3). All interviews and FGD will be audio recorded and transcribed.
During the research, we will collect relevant policy documents (taskforce meeting minutes, regional action plans, regional strategic plans, maternal death audit reports etc).
We will adopt a sequential data collection (see figure 3) to allow the refinement of our programme theory (programme specification in realist terms)(32, 34, 48, 52). Data collection will be organised in three phases:
Phase 1: We will carry out 10 to 12 individual interviews (see supplementary) with policy makers funders, experts who designed or participated to the implementation of RTF intervention. Snowballing technique will be used to identify additional key informants. We will also collect policy documents, reports and statutory regulations that informs “taskforce” policy formulation, and implementation processes. The end result of this phase is a refined programme theory.
Phase 2: We will conduct 4 to 6 individual interviews in each regional taskforce with taskforces members (regional health officer, health managers at regional and provincial level, health professionals). At the end of this cycle, we will use comparative case analysis that allows us to identify sufficient and necessary conditions that led the desired outcome. This phase will guide the selection of one high performing and one low performing taskforce)
Phase 3: We will conduct two in depth case studies (taskforces with contrasting performance). In each case study, we will carry-out additional 6 to 12 face to face or online (due to the Covid19 pandemic) individual interviews and three focus group discussions with managers (RHO, PHO, hospital directors, head of maternity) and experts (physicians, professional association representatives, midwifes). At the end of this phase, we will be able to confirm or refute our programme theory (53) (see figure 3).
Data analysis
In case analysis
During the initial coding phase, we will use concept and nvivo coding(54). Initial coding is guided, but not restricted by our programme theory. We will use NVivo 11 to manage the qualitative data (transcripts, policy documents, summary contacts) (55). In a second phase, we will use the causation coding method(54) by identifying respondents individual reasoning about the causal powers underlying the effectiveness of taskforces.
During this iterative process, we will focus specifically on the causal linkages between the intrinsic capacities of taskforces and the contextual elements identified during in case empirical data analysis. We will use the Intervention-Context-Actors-Mechanism-Outcome configuration (ICAMO) as a heuristic tool to describe and identify plausible causal configurations (56, 57).
Cross case analysis
To compare different ICAMOs configurations between the 6 different case studies, we will use qualitative comparative analysis(58).
Qualitative comparative analysis
To compare each taskforce in the six health regions, we will use qualitative comparative analysis (QCA)(58). QCA is a case-based method that allows for a systematic comparative analysis, description, interpretation, categorisation and explanation of cases (58-62). QCA proved appropriate in identifying typologies of cases and set theoretical relations (63-65).
In this study, QCA will allows us to compare cases by identifying necessary or sufficient conditions ( in this case,[ taskforce core functions, see figure 2 and supplementary file 3] to bring about the desired outcome (strengthened MNH health system governance).
Using QCA, we will be able to categorise each case by assessing the degree of membership in each specific set of subfunctions (in QCA terminology this process is called calibration)(65). We will use specifically QCA crisp set that differentiate qualitatively between a set of cases on the basis of case full membership scores ( 0 or 1)(65). For instance, a taskforce, that is not carrying meetings on a regular basis will be scored (0). 0 is a membership score that is a qualitative attribute that describe full non membership to a subset [Taskforce with regularity of meetings]). A taskforce that is carrying situation analysis of MNH in the health region will be scored (1). 1 means full membership to a subset of TSF carrying situational analysis)(see table supplementary file 3). This will allow us to test the sufficiency or necessity of contextual conditions derived from our initial programme theory and cross case analysis. Plausible conditions include, for instance, the existence of financial support, actors’ engagement, leadership of regional health officers, mobilisation of extrabudgetary resources, involvement of frontline workers…etc (see figure 2 and supplementary file 3).
At the end of the analysis, we will construct a truth table that is a visual description of the configurations of cases (membership scores in different set of conditions (subfunction). In practice, we will use crisp set QCA using Excel QCA add-in software(66). Following the comparison between the different ICAMO configurations across different cases, we will synthetise them into a refined programme theory.