Realist evaluation is a theory-driven approach which considers what is working, for whom, under which circumstances, and how . It is used within healthcare evaluation because it uncovers a deeper understanding of the issues present and potential solutions to mitigate them [24, 25].
The findings reported in this paper are part of a mixed-methods realist evaluation: ‘Evaluating effectiveness, safety, patient experience and system implications of different models of using GPs in or alongside Emergency Departments’ (HS&DR Project 15/145/04) to examine changes in service delivery . We will report results from qualitative data collection, describing the opinions of staff and patients. Further quantitative analysis exploring changing attending rates at EDs with GP-ED models is ongoing and will be reported elsewhere.
Generation of the study sample
In 2017, we distributed a survey to clinical directors (CDs) of all type 1 EDs (Consultant – led departments, open 24 hours with full resuscitation facilities) in England (n = 171) and Wales (n = 13). We received 77 responses and chose a sample of 30 EDs with different primary care models as seen in Table 1  to conduct follow up qualitative interviews with CDs ; 21 EDs were included in follow up interviews. From these, 13 case study site were purposely selected based on the criteria listed (below) considering the types of primary care models identified in our taxonomy of primary care services in EDs (Table 1) . The 10 EDs included in this paper had GPs and other primary care clinicians working within them, three as ‘inside-integrated’ models (Hospitals, 3, 8 and 14,) four as ‘Inside-parallel’ (Hospitals 4, 6, 7, 9) and three as ‘outside-onsite’ (Hospitals 10, 11,13) . The three EDs not included did not have GPs/primary care clinicians working within them.
All interviews were audio-recorded and transcribed verbatim.
Criteria for case study site selection
Primary care service in ED since 2010.
Variation in service model - delivering separate primary care services, inside or outside EDs or a primary care service integrated with the ED.
Spread across England and Wales.
Variety of contexts – including rural/urban locations, small/large hospitals, higher/ lower attendances.
Variation in streaming method – who, streaming criteria and guidance.
Variation in the physical layout of ED.
Variation in relationship with the GP out-of-hours services.
Inside - integrated services
Primary care services fully integrated within EDs. Staff review primary and emergency care patients (n = 3).
These were not visible to patients/patients generally unaware of GPs working in EDs (Hospitals 3, 9, 14).
Separate primary care service within ED for patients with primary care type problems (n = 4), were not visible (Hospitals 4, 8), or separate and patients were streamed by ED/111 (telephone service): patients were unaware of them (Hospital 7), or services accessible from ED that the public were aware of (Hospital 6).
Primary care services on site (n = 3) were visible, offering walk-in services that the public were aware of (Hospitals 10,11). Or primary care services within different part of the hospital and patients streamed from ED/111 (Hospital 13).
Data collection (see Appendix 2 for Interview Guide)
Pre-visit interviews with CDs
Telephone or in-person interviews took place between February 2018 and March 2019 (~ 60 minutes) . During these interviews CDs were asked about service operation, perceived successes, and challenges in providing and delivering services and how their experiences related to our theories. Ethical approval for the survey and follow-up interviews with CDs was given by (anonymised) Ethics Committee (ref: 17/45).
Observations and interviews with staff at study sites
Researchers (ME and AC) undertook 2–3-day visits to study sites between February 2018 - April 2019. Visits took place when primary care clinicians were present within ED over approximately 8 hours on average.
Patients were observed throughout their care journey from arriving at the reception, triage and streaming assessments and formal and short informal interviews were conducted with nurses and other clinicians. The realist teacher-learner interview technique was used to present initial theories and explore how different mechanisms in different contexts may result in intended and unintended outcomes .
As described by Price et al.  we also carried out semi- structured interviews with 24 patients/carers who visited EDs for one of six conditions (chest pain, cough and breathlessness, abdominal pain, back pain, headache and fever in a child under 10 years old). These were considered by stakeholders (academics, primary care and ED clinicians and patient and public contributors) as conditions that could be managed by primary care clinicians or ED clinicians, and were identified using literature on ambulatory care sensitive conditions [22, 28–35] and our stakeholder group . Patients were purposively sampled and contacted via post within 12 weeks of their visit to EDs or by members of NHS staff during site visits, to inform about eligibility to take part in the study and request their consent for interviews. Interviews were conducted by telephone by ME between February 2018 and March 2019 (over ~ 20 minutes).
Despite experiencing difficulties with recruitment and access to patient participants , the purposive sample included adults of different ages, parents of children, and people with different conditions from the three primary care models (Table 1).
We analysed data from observations and telephone interviews with CDs and case study visits. We used a realist approach, generating ‘context-mechanism-outcome’ (CMO) configurations  from the data. We did this by identifying mechanisms that relate to influences on demand and the contextual factors that influence those mechanisms. We then mapped CMO configurations against different primary care service models  and factors perceived to influence demand based on Pawson’s theory-building processes (juxtaposition, reconciliation, adjudication, and consolidation). We incorporated expert knowledge of primary and emergency care academics and public contributors in theory refinement and development by discussing early findings within the study team and co-investigators and refined analysis based on feedback.
We presented our theories at a stakeholder workshop  with 56 attendees including ED staff, GPs, service managers, policymakers, patients, and public contributors. They provided feedback and suggested additional contexts and mechanisms for consideration. In the final stage of the analysis, we identified relevant middle range theories which we used as a lens to interpret our results . These informed the development of our programme theory which summarises the findings of this work.
Interpreting results through a theoretical lens – using middle range theory and generating a programme theory
We used Richardson’s analysis of supply and demand in health care  as a middle range theory to interpret the findings and theories emerging from our study
. We aimed to integrate these theories as a ‘Programme Theory’ to explain and summarise why using primary care clinicians in or alongside EDs may or may not lead to provider-induced demand, for whom, and in what specific circumstances. A programme theory is an overall high-level theory summarising how the intervention works, developed using the theories refined from the data .
Patient and public involvement
Patients and public members were involved in the study design  and as co-applicants in the funded study in line with best practice , discussing their experience as NHS patients to contribute to this research. They advised on data collection tools and patient recruitment when the team experienced difficulties . They supported involvement of public and patient contributors to the stakeholder event and were involved in discussing draft data and paper preparation .