Challenges of patient recruitment within emergency departments for qualitative research: lessons learnt from the GPs in EDs study

Background: At times of increasing pressure on emergency departments, and the need for research into different models of service delivery, little is known about how to recruit patients for qualitative research in emergency departments. We aimed to collect evidence on patients’ experiences of different models using general practitioners in emergency departments. Challenges were faced at all stages of patient recruitment, from identifying and inviting eligible patients, consenting them for participation, and finally to engaging them in interviews. Methods: By analysing processes, amendments and outcomes of recruitment methods, as well as conducting a thematic analysis of field-notes taken during research visits, this paper aimed to describe the factors which influenced patient recruitment in the GPs in EDs study, and make recommendations based on our learning. Results: We found the following factors influenced the success of patient recruitment in the emergency department setting: complicated or time-consuming electronic health record systems for identifying patients; narrow participant eligibility criteria; limited NHS research nurse support; and lack of face-to-face communication between researchers and eligible patients. Conclusions: This paper adds to the growing evidence for improving patient recruitment in different settings, with a particular focus on qualitative research in emergency departments. Our findings have implications for future studies attempting to recruit patients in similar settings.


Introduction
Increasing demand on emergency departments has led to the development of different models of service delivery [1]. Research is urgently needed to understand the impact of these service models on patient experience, but there is little evidence in the literature about effective patient recruitment for qualitative research in the emergency department setting.
Challenges to patient recruitment and engagement in research can occur at all stages of the process: initially identifying and inviting eligible patients; gaining their consent; and successfully engaging them in data collection [2]. Limited success in identifying and inviting eligible patients has been associated with poor collaboration between hospital sites and researchers [2]. Hospital staff may also have doubts about their involvement in the research if they are unsure of the purpose of the research, or have concerns about their level of skill in identifying and inviting eligible patients [3].
Furthermore, narrow eligibility criteria may also contribute to insufficient numbers of patients being identified and invited to participate in research [4].
Once approached to take part, patient participation in research studies is an individual choice based on several factors, such as the purpose of the study, what participation involves, how the findings will be used and who will benefit from the findings [5]. Impersonal communication between researchers and patients (such as postal or email invitations, telephone interviews or online surveys) can hamper attempts to recruit patients, whereas sitting down with a patient and their family to explain the research conveys trust and openness, increasing the likelihood of patients engaging with a study [6,5]. While payment for participation in research is highly debated and can be viewed as presenting ethical challenges [2], a lack of monetary recognition for participation in research has also been identified as a reason for low recruitment rates [6].
Our study, "GPs in EDs", recruited 13 hospital emergency departments as "case study sites". We used mixed methods to evaluate the effectiveness of different models of general practitioners working in or alongside emergency departments. One key objective was to understand the impact on patient experience, by conducting qualitative interviews with patients. In line with best practice, public and patient members are study co-applicants alongside multi-disciplinary and specialist partners [7]. Two public members are also members of our independent Study Steering Committee and have worked alongside stakeholder and academic colleagues to oversee study implementation and contribute to decisions at all stages of the research, including enhancing data collection through patient interviews.
The literature demonstrates a range of potential barriers and solutions to recruiting patients for research. However, existing research has generally focussed on patient recruitment into large clinical trials, as opposed to smaller-scale qualitative research in defined care settings. We aim to describe the factors which influenced patient recruitment in the GPs in EDs study, by exploring the key challenges and consequent amendments made to our processes, and from our learning we propose recommendations for future research that seeks to recruit emergency department patients for qualitative research.

Patient recruitment methods
We aimed to recruit a purposive sample of 60-120 patients from our 13 case study sites with conditions appropriate for management by either emergency department clinicians or general practitioners working in the emergency department setting. We planned to conduct semi-structured telephone interviews, of 20-30-minute duration, following the emergency department visit to explore patients' reflections on their experiences. During the early phase of our study at a stakeholder event, a group of expert clinicians, policymakers, public contributors and researchers took part in a consensus exercise to choose five "marker" conditions which were deemed the most suitable for a comparative analysis. These marker conditions comprised of a presenting complaint (e.g. back pain) with associated exemplar diagnoses (e.g. sciatica) with a low acuity score (see Appendix 1). The aim was to understand how patients with non-specific symptoms (but similar end diagnoses) were managed in the different services by different staff, in terms of use of acute investigations, observation times and referral to other acute hospital services. We aimed to recruit 5-10 patient participants at each participating hospital site (covering the range of marker conditions. Our initial strategy included two recruitment methods to invite patients into the study, discussed below. Before agreeing to be included as a case site for our study, departments had to state that they were able to support our research at their site, which included the capability and capacity to support patient recruitment. However, researchers often had to negotiate the level of support for each aspect of the study, offering flexibility if departments were unable to give the ideal level of support needed for identifying and inviting patients. It was important that we were flexible with what we requested of departments, as we needed to recruit sufficient and appropriate sites and collect a range of data via different research methods for all aspects of the study (observations, staff interviews etc.).
Initial recruitment method 1: inviting patients via post The first method aimed to recruit patients via postal invitations sent out by a member of hospital staff (i.e. a NHS-appointed delivery research nurse), who would use the electronic health record (EHR) system to identify up to 50 patients who had attended the emergency department in the last three months, with the marker conditions in our recruitment framework (see Appendix 1). Eligible patients were mailed a patient study pack containing: a study invitation letter, a participant information sheet two consent forms and a stamped and addressed envelope. The patient information sheet asked patients to sign and return their consent forms to the university if they wished to take part in a telephone interview. Once these were received, the study team would contact the patient by telephone to arrange an interview.
Initial recruitment method 2: inviting patients at the emergency department The second method aimed to recruit patients during the study visit in the emergency department. A member of hospital staff (i.e. a research nurse) would identify up to 50 eligible patients when they were in the emergency department, informing them of the study and providing them with the patient study pack. Patients would be asked to take home the materials they had been given and consider whether they would like to take part, returning the consent form (to the university) if they wished to participate. Once consent forms were received by the study team, the same process for contacting patients would be followed as method 1.
Amendments to our patient recruitment methods Due to low initial patient recruitment figures from both methods, the study team sought advice from study co-applicants, study steering committee members and public and patient involvement representatives, on potential changes to the recruitment methods. The following amendments were made in May 2018 to help with patient recruitment: Inviting up to 100 participants at each study site rather than 50, to increase returns.
Printing study invitation letters on hospital headed paper rather than Cardiff University headed paper so that patients were familiar with the sender.
Giving participants a quicker and easy option of registering their interest by texting 'yes' to a mobile number, to then be followed up with a phone call from a member of the research team to request the consent form.
Specifying the desired marker conditions on the participant information sheet so that patients could recognise their eligibility for the study.
Offering an incentive of a £20 high-street shopping voucher to participants who were interviewed.

Analysis of patient recruitment methods
Progress reports created at weekly research team project meetings were reviewed to analyse the process and progress of recruiting patients throughout the study period with the different research methods. 12 sets of co-applicant, study steering committee and patient and public involvement meeting minutes were also reviewed to identify suggested changes to recruitment procedures, and to understand perceived reasons for low patient recruitment.
A thematic analysis was carried out (DP) on 26 sets of in-depth field notes taken by two researchers (ME and AC) at each hospital visit, to identify themes from the data [8]. Field notes were analysed for information and reflections relating to patient recruitment at each site. Key themes were identified and coded using NVivo 12 (QSR International V.12) to understand the challenges facing patient recruitment for qualitative research in the emergency department setting and inform recommendations for future research. Meetings were had with the two researchers (ME and AC) to validate these themes.

Results
Total number of patients invited, consented and interviewed A total of 748 patients were invited to take part in a telephone interview for the study, 43 (6%) patients consented and 24 (3%) were subsequently interviewed. We successfully recruited patients at nine sites. Following the first visit to a 'control' site (hospitals without a model of using general practitioners at the emergency department), the decision was made not to recruit patients at control sites, therefore no patients were recruited at the following two control sites. We were unable to recruit from two sites because of limited staff availability (such as research nurses, clinicians or administrators) and local research and development protocols which restricted data sharing. We included patient data from a 14th site which had initially been planned as a full case study site, but at the time of visit did not meet the eligibility to be included as a case study site.  The amendments made to improve patient recruitment did not have a notable impact on patient recruitment. 304 patients were invited before the amendments, resulting in 5 interviews (2%) and 444 patients were invited after amendments, resulting in 19 interviews (4%). Patient recruitment for method 1: Some hospital sites were unable to allocate research nurse support to invite patients to the study via post (n = 6), and therefore at these hospitals we could not use this recruitment method. From those hospitals that were able to assist with inviting patients by post (n = 8), the following numbers of patients were invited, consented and interviewed: Despite the small sample size for recruitment method 2, the success rate was higher with recruitment method 2 (inviting patients in person; n = 33.3%) than method 1 (inviting patients via post; n = 2.5%).

Integration of experiences and documentary findings
From reviewing recruitment results and meetings with study colleagues, in addition to analysing detailed field notes taken by both researchers (AC and ME) at each hospital, the following factors were identified as contributing to low patient recruitment in the GPs in EDs study, and suggestions for future research attempting to recruit patients were made. Table 3. Theme 1: complicated or time-consuming electronic health record systems Findings Evidence Suggestions for future research In some departments, it was difficult to identify patients using the EHR systems in place. Some departments used multiple systems for different areas of the department (e.g. registration, triage assessment, discharge notes etc.), meaning that all systems had to be looked at separately to identify eligible patients. At one hospital, the EHR system was not set up to retrieve data by specific details such as presenting complaint, and so the task of identifying eligible patients was too difficult. As a result, no patient recruitment could take place at that site. Often, EHR systems were very slow and identifying even a small number of eligible patients took much longer than anticipated, for example due to having to switch between multiple EHR systems. This slowed down the process of identifying and therefore inviting patients via both recruitment methods.
[ED Consultant] informed me that the computer system does not enable them to pull up details by presenting complaint... She seemed to think that there are IT problems and it would not be easy to pull up a list of patients to send invitations to... [she] did not seem to have a lot of enthusiasm for another visit or to find a way of identifying patients on their system.
-(Field notes -hospital 12) I spent from 10am − 1.15 [pm] with the research nurse searching for patients on the Maxims system… After over three hours of looking through the system to screen for eligible patients we had only found 13.

-(Field notes -hospital 3)
Broadening eligibility criteria (for example, just searching by presenting complaint rather than presenting complaint and diagnosis) would reduce the reliance on multiple electronic systems for identifying eligible patients (see theme 2). Evidence Suggestions for future research Because of the narrow eligibility criteria, and the need to identify eligible patients by a specific diagnosis associated with the presenting complaint, often it was difficult to identify sufficient numbers of eligible patients, particularly for recruitment method 2 during the research visit (see appendix 1), for example if the diagnosis was made at the end of the patients' visit. Some emergency departments did not see children or streamed them to a separate paediatric assessment unit at the hospital, rather than the GP service in the emergency department. Furthermore, public health education has encouraged patients with chest pain to phone an ambulance, and for those who do self-present in the emergency department, many departments had strict guidelines which meant chest pain patients were automatically seen by an emergency department doctor. Thus, local protocols made it difficult to identify children who had been seen in the emergency department, and patients with chest pain who had been seen by a general practitioner.
There were not enough patients coming through the department with marker conditions during the time we were there. I couldn't find one patient on Saturday afternoon.

-(Field notes -hospital 4)
We did not find any [patients] who had seen a GP with chest pain as they usually go to ED doctor.
-(Field notes -hospital 9) While all research needs appropriate eligibility criteria to answer its research question(s), consideration should be given to how eligible patients will be identified. Using broader initial eligibility criteria (for example, just searching by presenting complaint rather than presenting complaint and diagnosis) may result in more patients being identified. Evidence Suggestions for future research Research nurse support was needed not only to identify eligible patients, but also to prepare and post the research packs (for method 1) or approach eligible patients on behalf of the researchers (for method 2). While the study team conducted as much of the research pack preparation as possible, labelling and distributing the packs could only be carried out by staff at the hospital sites for both recruitment methods, due to data protection guidance and ethical approvals of processes. One department stated that they were more likely to allocate their research nurses to larger studies which brought in more patient participants. The availability of support from research nurses or other staff in the department to help facilitate recruitment was varied. We could invite and therefore interview many more patients (via both recruitment methods) in those departments where a research nurse or other staff member was able to offer support than in those where one was not, and research nurses greatly influenced the success of a research visit.
Having a research nurse with us for 3 days was invaluable.

-(Field notes -hospital 3)
At one-point, reception told me that there were some patients waiting to go to minors, but we could not approach them and we could not find anyone to assist us to approach them. We did not have a research nurse to be able to approach patients on our behalf.

-(Field notes -hospital 14)
Good contact before the visit helps to inform and prepare staff members for the research visit, improving their understanding of the study and how they might help. If practical preparations (such as preparing patient packs) can be carried out in advance of a research visit, this can lessen the burden on research nurses involved in the study, thus generating more willingness to offer support to research. Our patient and public involvement representatives felt that patient wariness was a likely reason for low patient recruitment. For example, patients could be wary about taking part in an interview which might ask them to justify their reasons for seeking emergency care, especially if they were aware that they had been seen by a general practitioner, or were told by a clinician that their condition could have been managed more appropriately by their own general practitioner. Our co-applicants and patient and public involvement representatives also considered how face-to-face recruitment makes the research more memorable for the patient, thus improving recruitment and retention.
Because most patients were invited by post, there was little opportunity for researchers to reassure patients about the purpose of the interviews, and this lack of face-to-face communication may have resulted in fewer patients recruited.
The total returns rate for face-toface invitations (method 2) was 33%. In contrast, the total returns rate for postal invitations (method 1) was 2.5%.
While it may take more time and require more complex ethical approvals, future research should consider research designs which utilise face-to-face recruitment methods, for example using more informal interviewing methods. Further consideration should be given to the process of consenting patients. Ensuring that this is a smooth process, for example by allowing patients to consent during their time in the department, rather than requiring consent in the post, may improve patient recruitment.

Discussion Principal Findings
We found that the following interdependent and interacting factors contributed to low patient recruitment for this qualitative study in the emergency department setting: (1) complicated or timeconsuming EHR systems; (2) narrow eligibility criteria; (3) limited contact with/ availability of research nurses or other support staff; and (4) lack of face-to-face communication between researchers and patients. Amendments made to our methods did not improve recruitment substantially: 2% of patients invited before amendments were interviewed, and 4% of patients invited after amendments were interviewed.
Narrow eligibility criteria limited the number of patients who could be identified and invited. This made searching for eligible patients on already complicated EHR systems even more time consuming.
Both factors made our recruitment processes time consuming, meaning research nurses could not always commit enough time to supporting patient recruitment, making engagement in patient recruitment less feasible for case study sites. Limited availability of research nurses was also due to the small size of our study, as departments are more likely to allocate resources to large clinical trials and commercial studies where more patients will be recruited, and therefore more accruals (monetary credits) will be obtained. These factors all interacted to limit the number of patients identified and invited to take part in the study.
In terms of recruiting patients once they had been invited, we believe that a lack of face-to-face communication between researchers and patients meant that invitations to participate were impersonal, easily ignored and could lead to patient wariness about participation.
Strength and weaknesses of this paper The experience reported in this paper is helpful for understanding the reasons for low patient recruitment in the GPs in EDs study and wider studies in similar settings. By using mixed methods of evaluation to analyse meeting minutes, in-depth field notes and recruitment methods, figures and amendments, this paper gives an insight into the reasons for low recruitment, and highlights ways to improve patient recruitment in future studies using a similar setting. However, we were unable to obtain data from those who ignored invitations or consented but then declined, to understand reasons for not participating. The findings in this paper are based on one study's experiences, and further evidence is needed.

Context of other literature
The findings from this paper fit with the current literature surrounding patient recruitment in research. Studies have found that narrow eligibility criteria can restrict patient recruitment, as it limits the number of patients who can be invited and thus interviewed, as well as slowing down and often complicating the process of identifying eligible patients [4]. This is consistent with our experience in the GPs in EDs study, as it was often difficult to find suitable numbers of eligible patients within time constraints available to us.
Studies have identified that if preparatory work (e.g. helping to screen and identifying appropriate patients, preparing appropriate recruitment materials, informing relevant staff members about the study) can be carried out by study members (e.g. research staff, study support staff), then the burden on hospital staff is lessened, and this can be key to ensuring successful engagement of staff and recruitment of patients [2]. Furthermore, having other priorities and not having much time to dedicate to a study is a known barrier to hospital staff being able to help with patient recruitment [3]. In the GPs in EDs study, support of research nurses or other staff at hospitals was key to identifying and inviting appropriate numbers of patients.
Face-to-face communication is valued by research participants, and informing patients (and their family members) of the research in person allows rapport to be built, in an open and trustful manner, and will increase the likelihood of the patient engaging in the research [6,5]. Again, this is supported by our experiences, as we found higher participation among patients invited in person rather than by post, albeit with a small sample size. Previous research has successfully used "informal interviewing" as a practical technique for gaining patient perspectives, in person, in busy emergency departments [9]. Informal interviewing involves informal conversations with participants to enable more open discussions than formal interviewing, making the process of gathering data on patient experience easier and faster than formal interviewing methods [9].

Future research indicated after this study
While we have been able to identify key factors which restricted the ability to identify and invite patients into the study, challenges were faced in terms of identifying patients' reasons for declining participation once invited. Future research could explore emergency department patients' possible reasons for not taking part in research, to develop patient recruitment methods that encourage participation. Furthermore, the learning from this paper comes only from one study's experiences and could be formally evaluated in further larger studies or clinical trials. Further research is also needed into how researchers can best work with patient and public involvement representatives to increase patient recruitment in different settings [10].

Conclusion
This paper adds to the growing evidence for improving patient recruitment in different settings, with a particular focus on qualitative research in emergency departments. We found that patient recruitment in the emergency department setting was influenced by slow or time-consuming electronic health record systems, narrow eligibility criteria, the support (or lack of support) from research nurses or