General practitioners’ views on use of patient reported outcome measures in primary care: a cross-sectional survey and qualitative study

Abstract

Background Patient reported outcome measures (PROMs) are increasingly used to assess impact of disease and treatment on quality of life and symptoms; however, their use in primary care is fragmented. We aimed to understand how PROMs are currently being used in primary care, the barriers and facilitators of this use and if appropriate how it might be optimised.Methods Cross-sectional survey and semi-structured interviews among general practitioners (GPs) in England. GPs’ opinions were explored using an electronic, self-completed questionnaire disseminated to 100 GPs via an online doctors’ community and semi-structured qualitative interviews with 25 GPs.Results Most GPs surveyed (77/100; 77%) reported using one or more PROM, primarily to aid clinical management (n=66) or as screening/diagnostic tools (n=62). Qualitative interviews highlighted challenges in identifying and selecting PROMs; however, some GPs valued PROMs for shared decision making and to direct patient discussions. The interviews identified key barriers to PROM use including: time constraints; insufficient knowledge; lack of integration into clinical systems; and PROMs being mandated without consultation or explanation. Evidence of the benefit of PROMs is required to promote uptake and use of PROMs in primary care.Conclusion Implementation of PROMs in primary care requires integration with clinical systems, a bottom-up approach to PROM selection and system design involving meaningful consultation with patients and primary care clinicians and training/support for use.

Introduction

Patient Reported Outcome Measures (PROMs) are increasingly used to measure patients’ own experience of their health, such as symptoms, mobility, mental health and social function.(1) Capturing PROMs alongside traditional clinical outcomes can offer a range of benefits: at an individual patient level PROMs can facilitate communication, identification of problems and help tailor care to needs;(2-4) at aggregate level these data can be used as real world evidence of treatment effectiveness and can be used for audit/benchmarking purposes and to assess service delivery/reconfiguration.(5)

In the United Kingdom, there are a few cases of PROMs use being implemented in primary care. For example, completion of the Patient Health Questionnaire-9, a depression screening PROM, was incentivised between 2006 and 2013 as part of the pay-for-performance quality and outcomes framework (QOF).(6) In addition, completion of the Oxford Knee Score is a requirement of a referral pathway for knee replacement.(7) However, beyond mandated use, anecdotal evidence suggests PROM use in primary care is fragmented and ad hoc. The heterogeneous nature of primary care, such as the diverse range of conditions, symptoms and outcomes, may present challenges for PROM selection and use in this setting.(8) We aimed to understand the current use of PROMs in primary care, barriers and facilitators, and if appropriate how their use might be optimised.

Methods

The study was a mixed methods design comprising a survey and qualitative interviews. The study was approved by the University of Birmingham the Science, Technology, Engineering and Mathematics Ethical Review Committee (Reference ERN_16-0568S).

Online survey

An electronic, self-completed questionnaire was disseminated to general practitioners (GPs) in England via an online doctors’ community (www.doctors.net.uk) in December 2017. A sample size of 100 was pre-specified based on funding limitations. The survey explored GPs’ current use of PROMs; barriers to PROM use; and how PROMs could ideally be used in primary care (Tables 2 and 3) and would take an estimated ten minutes to complete.

Qualitative study

GPs were recruited from amongst those that had completed the online survey and by using convenience sampling of known contacts and snowballing.(9) Semi-structured interviews were conducted by telephone. All interviews were conducted by IL, a Research Fellow employed by the University of Birmingham. The interviewer did not have a relationship with any of the participants. A semi-structured topic guide was developed based on the online survey findings. Use of the topic guide ensured key topics were consistently covered including: how PROMs were used, by whom, in which circumstances, and barriers/ facilitators including influences of patient, clinician, and system factors (see appendix). Interviews were conducted between June and September 2018. Data was collected and analysed until data saturation was reached.(10)

Interviews were audio recorded using a digital recorder, and recordings were transcribed verbatim by a professional transcription service. NVivo v12 was used to manage, sort, code and organise the transcribed data.

Data analysis

Quantitative survey data was summarised using descriptive statistics and free text comments were categorised into themes. Qualitative interview data was thematically analysed.(11) Transcripts were read several times to enable the researcher to familiarise themselves with the data. IL (experienced qualitative researcher) coded all transcripts and OLA independently coded a subset (10%). Open coding was applied to the transcripts, codes were then reviewed, organised in categories and, after being refined, overarching themes. The final analysis and interpretation was discussed with the wider team.

Results

Online survey

The online survey was completed by 100 GPs across England (Table 1). Seventy seven GPs reported using one or more PROM; however, 17% (38/224) of the measures reported were not PROMs (for example cardiovascular risk scores). The majority of PROMs reported were for mental health (n=85 PROMs), urology (n=37 PROMs), sleep apnoea (n=25 PROMs) or orthopaedics (n=19 PROMs). A description of the most commonly used patient-reported outcome measures in primary care can be found in eTable 2 in the Appendix.

The most common reasons for PROM use were to aid clinical management (n= 66) or as a screening/ diagnostic tool (n= 61) (Table 2). GPs most frequently accessed PROMs through clinical systems (n=56), clinical templates (n=49) or online (n=47). PROMs were usually completed during the consultation (n=72), using paper (n=68) or GP administered interviews (n=51), and were reviewed by GPs (n=84). The main barriers reported for PROM use were time constraints (n=77) and being mandated to use without consultation or explanation (n=55).  When asked “how do you think your patients feel about completing PROMs?”, most GPs responses were positive or neutral (n= 47 and n=31, respectively).

Similar to current PROM use, the most common areas GPs considered PROMs could provide the most benefit were to aid clinical management (n=66), as a screening/ diagnostic tool (n=62) or facilitate shared-decision making (n=60) (Table 3). GPs would prefer to access PROMs through clinical templates (n=65) or clinical systems (n=63) and considered patients would prefer to complete PROMs during the consultation (n=34). The preferred format for patients to complete PROMs was considered to be paper (n=43) or online (n=21), and the preferred format for GPs recording PROM results was electronic: as part of the electronic health record (n=78). GPs viewed that doctors would be the most appropriate people to interpret PROM results (n=78). The patient groups/conditions where GPs considered there would be the most benefit from completing PROMs was mental health (n=20), all/ most patients (n=15) and patients with chronic conditions (n=14). GPs considered PROMs should be collected for chronic disease monitoring annually (n=36) or as clinically indicated (n=27). Most GPs felt development of PROM systems should be designed to meet clinician/ patient needs (n=68) rather than designed primarily for audit, benchmarking or commissioning (n=3). Integrated clinical systems (n=29), more time (n=12) and easy access (n=12) were the most common features GPs reported that could facilitate/ support the use of PROMs in primary care.

Qualitative study

The final sample comprised 25 GPs, participant characteristics are detailed in Table 4 and eTable 1 (see appendix). Interviews lasted between 18 and 59 minutes.

Current use of PROMs

PROMs were considered useful to aid shared decision making, the ability of PROMs to provide an objective measure that could be used in subsequent discussions of treatment was described.

“It does help direct the discussion regarding future management, especially the mental health patients because it allows them to objectively score how they feel and what’s going on, and allows me to help discuss treatment options with them.” GP12

By completing PROMs with patients, one GP described how they facilitated the discussion with a patient about their symptoms.

“I have sometimes filled in the questionnaires with the patients, and actually I see the value of that because we actually get a better picture with the discussion around filling the form with the patient as opposed to just getting a figure just attached to the referral letter.” GP13

Facilitators to PROM use

GPs knowledge and understanding of the value/benefits of PROMs to their clinical practice was an important factor in their use with some GPs considering that better communication of the evidence-base for PROMs could be an important driver for their take-up.

“For me I am quite evidence based personally, and if someone was to show me like you’re doing, if I’m the outlier and most GPs love PROMs and I would actually be thinking hand on I’m the outlier here, actually maybe I’ll just get more on board.  If there was a study saying this particular PROM if they said PHQ-9 shortened a ten minute consultation down to five minutes, improves on patient outcomes, reduces re-attendance rates, improve compliance to medications, then I would say right we’ve got to get on board and do that.” GP18

Related to the reliability of the evidence base one GP described how they were more likely to use PROMs if they heard about them from multiple sources, corroborating perceptions of their utility.

“… I am unlikely to go and start using some new coeliac disease PROM when I have just been to a talk from a private gastroenterologist or something like that.  I am more likely to use something that is appearing to me in lots of different areas of my CPD or medical education.  So if I might see a paper about it, and then I might hear a colleague talking about it, and then I might see something on GP Notebook or something like that.  So you’re getting over exposed to it, and then try it out and see how well it resonates, and how useful it is and how quick and easy to remember it is.” GP24

Barriers to PROM use

Some GPs were unconvinced of the benefits of PROMs instead placing the onus on clinical data. This appeared at least in part due to concerns of the reliability of patients whose responses might be influenced by attempts to manipulate the output for their own purposes.

“…sometimes the patient can fill them in with what they think the clinician might want them to say rather than what they actually feel.  So sometimes patients can underplay their symptoms, and equally sometimes patients can overplay their symptoms if there might be some perhaps secondary gain for them in terms of certification from work or whether they want some help with some other part of their care.  So I think they can potentially be a bit skewed by that.” GP19

Though the use of PROMs can be encouraged at a policy level, this top-down approach for mandatory PROM completion was objected to by GPs and did not convince them of the value of PROMs. For example one GP felt obliged to use a PROM solely due to the financial incentive offered by the National Health Service (NHS) pay-for-performance scheme Quality Outline Framework (QOF) in the absence of any clinical value.  This is supported by its presence on the dashboard of their clinical management software.

“so the big one is PHQ-9, it’s pushed very hard and for example with people with chronic diseases as well it flags up in the QOF box on EMIS.  But in reality it’s irrelevant to assisting you that much in terms of referral and management, so there’s no point in doing it.”  GP18

Lack of time in the consultation to complete, analyse and integrate an additional source of information was described and one GP felt this was a barrier to their routine use.

“In a pressurised rushing surgery and you’ve only got ten minutes the person usually would need at least 20 minutes to solve their issues, and if you were to include a questionnaire on top of that you would be definitely talking about 30 minutes at least, and you can’t afford to be doing that on a regular basis.  You can do it as a one off thing and then you have an idea, but you would be pressurised to just do things quickly...” GP2

Lack of integration with clinical systems was also identified as a potential barrier; in some cases PROMs were integrated in clinical systems but they were not easy to use.

“So we do have some which are integrated into the system, but they are not quite integrated enough to be user friendly, … I’m actually thinking here of the dementia screening test which isn’t really a PROM I guess, but you have to input the data and then the score is added up wrong because the template is set up wrong, so you end up having to override it and do it yourself anyway which makes it a bit of a waste of time it being integrated, and I think sometimes with the clunky way that clinical computer systems sometimes work it be difficult during the consultation to use that tool there and then.  So I think there could be better integration, it could be more fluid, and I suppose depends on which clinical system is used, which PROMs are used locally and how easy they are to integrate into the system, because I guess some are going to be more objective than others.” GP19

Though large numbers of PROMs have been produced there appeared no systematic method of communicating their identity and availability to practice staff. For one GP the subsequent lack of awareness significantly reduced their uptake.

“To make me want to use a PROM first of all I would have to hear about it, and that is the main problem that we wouldn’t hear about it and therefore people get them on committees as mandatory things to put down on referrals.  That happens because we don’t hear…”  GP20

Discussion

Summary

Most GPs surveyed reported using one or more PROM, primarily to aid clinical management or as screening/diagnostic tools. Qualitative interviews highlighted challenges in identifying and selecting PROMs; however, some GPs valued PROMs for shared decision making and to direct patient discussions. Key barriers to PROM use included: time constraints; insufficient knowledge; lack of integration into clinical systems; and PROMs being mandated without consultation or explanation. Understanding of the value/benefits of PROMs to clinical practice and hearing about PROMs from multiple different sources were facilitators for PROM use. Whilst PROMs may offer a range of benefits, a more systematic awareness of the elements affecting their successful implementation is needed.(12) In this way the co-design of systems and processes incorporating PROMs and the development of a coherent evidence base can successfully underpin their uptake and roll out.

Strengths and limitations

Key strengths of our study include the mixed methods design and national sampling of GPs across England. However, the survey sample size was restricted to 100 GPs due to funding constraints.  Similarly we recognise the importance of patient perspectives on the use of PROMs but it is beyond the scope of this study. Only members of the online doctors’ community could access the survey which may limit generalisability of survey results. Furthermore, 17% of measures reported in the survey were not PROMs; therefore, some GPs survey responses may be in reference to risk scores. We interviewed GPs from across England with a balance of genders and a range of experience and though broad and descriptive in nature this preliminary analysis has allowed us to begin to understand the nature of the influences on PROMs adoption.

Comparison with existing literature

We found that PROMs were considered to impair communication with patients by some GPs. Similarly in other studies, GPs have reported that PROMs can disrupt the flow of consultations through ‘mechanistic’ questions which ‘trivialise’ patients’ emotions.(13-15) In contrast, patients have reported that they found PROMs facilitated communication with GPs and were helpful prompts to share issues.(16) In addition, PROMs completion can change how patients think about their condition; therefore, are more than just modes of information collection.(17) A recent (2018) systematic review of reviews exploring facilitators and barriers to implementing PROMs in organisations delivering health related services highlighted the important of clinicians valuing PROMs and understanding their validity.(18) Therefore, effective implementation of PROMs in requires time and resources invested appropriate training of primary care clinicians.(18)

We identified that challenges in selecting PROMs and concerns about relevance and reliability of PROMs in a primary care setting were barriers to their use. Similarly Dowrick et al (2009) reported some GPs felt that PROMs do not adequate represent the dynamic nature of patients’ conditions or capture the complexity of symptoms.(16) Selection of appropriate PROMs for general practice has been debated.(8, 19) Condition and symptom specific PROMs present challenges given the diversity of conditions and symptoms seen in primary care and presence of multimorbidity. However, generic PROMs are often developed for long-term conditions or refer to a disease, which may not reflect all health problems in general practice.(20) Some PROMs have been developed specifically for primary care including the Patient Enablement Instrument(21) and the Primary Care Outcomes Questionnaire.(22, 23) However, choice of measure should be based on rationale for assessment. This was highlighted by GPs in our study who valued PROMs to aid clinical management and as screening/diagnostic tools.

Implications for research and/ or practice

Lack of knowledge and training about PROMs were important barriers to their use. Systematic training and exposure to PROMs is necessary to motivate GPs to integrate PROMs into routine care and empower them to make informed decisions regarding when to use PROMs and which measures to select. However, even with appropriate training, organisational barriers need to be addressed, particularly integration within clinical systems to this end we have produced a systematic analysis of the factors affecting the current implementation of PROMs.(12)

PROMs collected routinely in primary care could subsequently be used for audit and benchmarking; however, our research suggests that this should not be the primary purpose of PROM collection. Optimal implementation of PROMs into routine clinical practice requires a bottom-up approach driven by needs of primary care clinicians and patients. Our study was restricted to GPs, future research should explore experiences of other primary care healthcare providers, particularly practice nurses who conduct long-term conditions reviews, and patients.

Conclusion

PROMs have a potential to aid clinical management and diagnosis in primary care. However, current use is fragmented and there is a lack of knowledge regarding what PROMs are available, when to use them and their evidence base. Implementation of PROMs in primary care requires integration with clinical systems, a bottom-up approach to PROM selection and system design involving meaningful consultation with patients and primary care clinicians and training/support for use.

List of Abbreviations

GPs: General Practitioners

NHS: National Health Service

PROMs: Patient Reported Outcome Measures

QOF: Quality and Outcomes Framework

Declarations

Ethics approval and consent to participate

The study was approved by the University of Birmingham the Science, Technology, Engineering and Mathematics Ethical Review Committee (Reference ERN_16-0568S). Written or recorded verbal consent was obtained from the participants.

Consent for publication

Not applicable.

Availability of data and material

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Competing interests

MC has received personal fees from PCORI, Astellas, Takeda, Glaukos, and Merck outside the submitted work. SF receives income from freelance writing and lecturing and is a fellow for Evidence and Values with the Royal College of General Practitioners. GT, IL and OLA have no competing interests.

Funding

This study was funded by the Wellcome Trust Institutional Strategic Support Fund (ISSF) (Grant number: 1516ISSFIRA26).

GT is funded by a National Institute for Health Research (NIHR) Postdoctoral Fellowship Award. This article/paper/report presents independent research funded by the NIHR (and Health Education England if applicable). MC receives funding from the NIHR Birmingham Biomedical Research Centre, NIHR Surgical Reconstruction and Microbiology Research Centre at the University Hospitals Birmingham NHS Foundation Trust and the University of Birmingham, Health Data Research UK, Innovate UK and Macmillan Cancer Support. SF receives funds from the University of Birmingham and the West Midland Clinical Research Network. IL’s work on this study was funded by the Wellcome Trust ISSF grant (grant number: 1516ISSFIRA26). OLA is funded by the Health Foundation an independent charity committed to bringing about better health and healthcare for people in the UK. He also receives funding from the NIHR Birmingham Biomedical Research Centre.

The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR, the Department of Health and Social Care or Innovate UK.

Authors' contributions

GMT, MC, IL and SF contributed to the study conception and design. GMT drafted the survey, analysed the results and led the survey study. IL designed the topic guides, conducted the interviews and led the qualitative study. Qualitative data were analysed by IL and OLA. GMT, MC, IL, SF and OLA were involved in the interpretation of quantitative and qualitative results. GMT drafted the manuscript and MC, IL, SF and OLA provided feedback. All authors read and approved the final manuscript.

Acknowledgements

We thank all participants who contribute to this study. In addition we thank and acknowledge the work of Dr Tom Keeley, Dr Clare Taylor and Dr Helen Stokes-Lampard for their contribution to the protocol and funding application.

References

1. Black N. Patient reported outcome measures could help transform healthcare. BMJ : British Medical Journal. 2013;346:f167.
2. Basch E, Deal AM, Kris MG, et al. Symptom Monitoring With Patient-Reported Outcomes During Routine Cancer Treatment: A Randomized Controlled Trial. J Clin Oncol. 2016;34(6):557-65.
3. Luckett T, Butow PN, King MT. Improving patient outcomes through the routine use of patient-reported data in cancer clinics: future directions. Psychooncology. 2009;18(11):1129-38.
4. Snyder CF, Jensen RE, Segal JB, Wu AW. Patient-reported outcomes (PROs): putting the patient perspective in patient-centered outcomes research. Med Care. 2013;51(8 Suppl 3):S73-9.
5. Calvert M, Thwaites R, Kyte D, Devlin N. Putting patient-reported outcomes on the ‘Big Data Road Map’. Journal of the Royal Society of Medicine. 2015;108(8):299-303.
6. Quality and Outcomes Framework guidance for GMS contract 2009/10 2009 [Available from: https://www.nhsemployers.org/-/media/Employers/Documents/Primary-care-contracts/QOF/2010-11/Quality-and-Outcomes-Framework-guidance-for-GMS-contract-200910---Delivering-investment-in-general-practice.pdf.
7. NHS Scarborough and Ryedale Clinical Commissioning Group (CCG). Knee replacement referral criteria 2015 [Available from: https://www.scarboroughryedaleccg.nhs.uk/data/uploads/rss2/orthopaedics/2015_10_13__update/knee-replacement-march-2015.pdf.
8. Murphy M, Salisbury C, Hollinghurst S. Can the outcome of primary care be measured by a Patient Reported Outcome Measure? British Journal of General Practice. 2014;64(629):647.
9. Marshall MN. Sampling for qualitative research. Family Practice. 1996;13(6):522-6.
10. Guest G, Bunce A, Johnson L. How many interviews are enough?: An experiment with data saturation and variability. Field Methods. 2006;18(1):59-82.
11. Braun V, Clarke V. Using thematic analysis in psychology. Qualitative Research in Psychology. 2006;3(2):77-101.
12. Litchfield I, Turner GM, Finnikin S, Calvert M. in press. 2019.
13. Leydon GM, Dowrick CF, McBride AS, et al. Questionnaire severity measures for depression: a threat to the doctor-patient relationship? Br J Gen Pract. 2011;61(583):117-23.
14. Pettersson A, Bjorkelund C, Petersson EL. To score or not to score: a qualitative study on GPs views on the use of instruments for depression. Fam Pract. 2014;31(2):215-21.
15. Mitchell C, Dwyer R, Hagan T, Mathers N. Impact of the QOF and the NICE guideline in the diagnosis and management of depression: a qualitative study. Br J Gen Pract. 2011;61(586):e279-89.
16. Dowrick C, Leydon GM, McBride A, et al. Patients’ and doctors’ views on depression severity questionnaires incentivised in UK quality and outcomes framework: qualitative study. BMJ. 2009;338:b663.
17. Greenhalgh J, Gooding K, Gibbons E, et al. How do patient reported outcome measures (PROMs) support clinician-patient communication and patient care? A realist synthesis. Journal of Patient-Reported Outcomes. 2018;2(1):42.
18. Foster A, Croot L, Brazier J, et al. The facilitators and barriers to implementing patient reported outcome measures in organisations delivering health related services: a systematic review of reviews. Journal of Patient-Reported Outcomes. 2018;2(1):46.
19. Murphy M, Hollinghurst S, Turner K, Salisbury C. Patient and practitioners' views on the most important outcomes arising from primary care consultations: a qualitative study. BMC Fam Pract. 2015;16:108.
20. Murphy M, Hollinghurst S, Salisbury C. Identification, description and appraisal of generic PROMs for primary care: a systematic review. BMC Family Practice. 2018;19(1):41.
21. Rööst M, Zielinski A, Petersson C, Strandberg EL. Reliability and applicability of the Patient Enablement Instrument (PEI) in a Swedish general practice setting. BMC family practice. 2015;16:31-.
22. Murphy M, Hollinghurst S, Salisbury C. Agreeing the content of a patient-reported outcome measure for primary care: a Delphi consensus study. Health Expect. 2017;20(2):335-48.
23. Murphy M, Hollinghurst S, Cowlishaw S, Salisbury C. Primary Care Outcomes Questionnaire: psychometric testing of a new instrument. British Journal of General Practice. 2018;68(671):e433-e40.

Tables

Table 1: Characteristics of GPs completing the survey (n=100).

Table 2: Survey questions about GPs current use of PROMs (n=100).

Table 3: Survey questions about how PROMs could ideally be used in primary care (n=100).

Table 4: Participant characteristics (n=25).