Main study design
Prospective cohort study of patients undergoing major surgery in NHS hospitals. Observational study at patient level; interventional action research study using time-series analysis at hospital level.
Aims
A1. To measure and improve outcome from major surgery in the UK NHS.
A2. To implement and evaluate a complex intervention aiming to enhance the use of data for improvement by clinical teams
A3. To create a national resource for the collection, management and analysis of high-quality perioperative data, to support collaborative research and efficient study design.
Objectives
Obj 1. To comprehensively measure and report structures, processes and risk-adjusted complications, patient-reported outcome and mortality rates after major surgery
Obj 2. To evaluate the relationships between structures, processes and outcomes after major surgery
Obj 3. To develop bespoke, regularly updated and non-proprietary risk models for the purposes of case-mix adjustment and perioperative risk prediction for different outcomes
Obj 4. To support local quality improvement through feedback of data to clinicians and managers using near-real-time feedback and regular comprehensive reporting.
Obj 5. To develop and evaluate the effectiveness of a theoretically underpinned complex intervention involving novel methods of data feedback, analysis and support mechanisms to aid local improvement initiatives.
Obj 6. To support the development and delivery of collaborative nested studies
Research Questions
RQ1. What is the level of compliance with evidence-based structures and processes for patients undergoing major surgery?
RQ2. What are the rates and predictors of complications, failure to rescue (FTR, defined as mortality within 30 days occurring after the presence of morbidity on postoperative day 7), prolonged length of stay, short- and long- term mortality in patients undergoing major planned surgery in NHS hospitals?
RQ3. How do the rates of adverse outcomes vary over time and between institutions?
RQ4. What are the structural or process related predictors of these adverse outcomes?
RQ5. What is the relationship between different measures of postoperative outcomes (e.g. different measures of morbidity; short and longer-term outcomes, health-related quality of life and disability)?
RQ6. What is the validity and predictive performance of previously published risk prediction models for different perioperative outcomes in the PQIP population?
RQ7. Can existing risk models be improved, or more parsimonious versions developed to support temporally responsive risk adjustment and clinically useful risk evaluation?
RQ8. How representative is the sample of patients recruited to PQIP of the total cohort of patients undergoing potentially eligible procedures in PQIP hospitals?
RQ9. Is there an outcome benefit to hospital and/or patient involvement in PQIP which can be differentiated from secular variation?
RQ10. Can quality of care be improved through a theoretically underpinned complex intervention supporting the use of data for improvement at local level, and do patient outcomes also improve?
Ethics
The study has been approved by the Health Research Authority (HRA) as a research study, following review at the South-East Coast - Surrey Research Ethics Committee (REC reference: 16/LO/1827; IRAS project ID: 215928). All hospitals provide Caldicott Guardian approval in addition to confirmation of capacity and capability in accordance with usual HRA requirements.
Participants
All NHS hospitals which undertake any of the included procedures are eligible to participate. Hospitals are approached via the Royal College of Anaesthetists and its Health Services Research Centre, and via the National Institute for Health Research’s Clinical Research Networks. Patients aged 18 years or older undergoing planned major surgery (as defined by a list of index procedures requiring inpatient stay) are eligible for recruitment and patient consent is sought pre-operatively. Patient consent is required as the study seeks additional data to that collected as part of routine care (specifically the patient-reported outcome data). The consenting process is undertaken by staff locally trained and approved. The participant information provides detail on how data are collected, the request for patient reported data, and the linkage of hospital data with external databases (see below, dataset).
Hospitals may choose to recruit patients from all specialties or specify particular specialties where they wish to focus at local level. Each hospital is offered the opportunity to either approach all eligible patients for consent, or all patients within a particular specialty or plan to recruit between one and five patients per week, approaching patients for written consent based on a random sampling strategy. This will involve an eight-day rolling sampling cycle (i.e. for week one the first five patients starting on Monday morning, followed by the first five patients starting from Tuesday in week two, et cetera). If any of the first five patients approached decline to consent, then consecutive patients will be approached for consent until the target recruitment number has been achieved. Patients can choose to be withdrawn from the study at any time.
Sample size
Target recruitment is 70,000 patients. It is expected that this will take at least 5 years to achieve. The long observation period will facilitate the study of how data collection and feedback impacts upon patient outcomes over time.
Dataset
The dataset has been informed using the best available evidence. Recorded outcomes include: postoperative morbidity defined using the Post-Operative Morbidity Survey, collected on day 7 after surgery; 14 POMS-defined major morbidity; 15 complications graded according to the Clavien-Dindo classification; 16 failure to rescue (FTR); 1718 resource utilisation (critical care admission (planned/unplanned); critical care length of stay, hospital length of stay, hospital readmission within 30 days of index procedure); mortality at 90 days; days alive and out of hospital 1920 censored at 30days, 60 days, 90 days and one year postoperatively; disability-free survival at one year; change in patient reported health related quality of life (HRQOL) between baseline and 6 and 12 months after surgery; the responses to the “Ask2Questions” brief questionnaire about complex pain. 21 The choice of outcome measures has been based on formal validation studies 14, 22, 23 and expert consensus. 20, 24–30 To calculate FTR, we will follow previously described methodology, but using the presence of a Clavien-Dindo grade II or above complication as our definition of postoperative complications. 17, 18 Candidate variables for risk adjustment were selected from the results of a systematic review, and subsequent original research 31–33. Structure and process measures have been informed by systematic review 34 and Delphi consensus process. An expert panel (the Clinical Reference Group) of stakeholders from different medical and surgical specialties provided further input to refine the long-list of potential measures. In addition to prospectively collected data, we will also link patient-level data with NHS Digital Hospital Episode Statistics and the Office of National Statistics mortality register. We will also consider patient-level linkage with other registries such as the National Cancer Registry, the Intensive Care National Audit and Research Centre’s Case-Mix Programme and relevant National Clinical Audits, in order to provide a comprehensive dataset at lowest local data collection burden.
Study flow
Patients are approached for participation either by mail, at the preoperative assessment clinic or on the day of surgery. A minimum of 1 hour is allowed for patients to consider the information on the participant information sheet before they are asked whether they wish to consent. At the time of consent, patients are asked to indicate whether they wish their longer-term follow-up to be by email or telephone. Following consent, the patient is asked to complete baseline demographic and HRQOL data. On the day of surgery, clinical teams or research staff complete preoperative, intraoperative and recovery room data. Further objective data capture occurs on day 2 or 3 (the Drinking Eating and Mobilising process measure 35), day 7 (the Post-Operative Morbidity Survey (POMS) 14) and at hospital discharge (the Clavien-Dindo complications grading scale, 16, 36 and information about the length of stay and post-discharge destination of the patient. In addition, patients are asked to complete a patient satisfaction with anaesthesia questionnaire 37, 38 within 24h of surgery, a quality of recovery questionnaire at baseline (at the time of consenting to participate in PQIP) and postoperative day 3, 39, 40 and longer-term HRQOL and disability-assessment questionnaires at baseline, 6 and 12 months after surgery. 2223.
Loss to follow-up
We are conscious that with a large pragmatic study of this nature without substantial funding but with multiple follow-up points and hospitals participating, that there is the potential for significant loss to follow-up. We have considered this as follows:
Inpatient morbidity and complication follow-up rates are predicted to be high, as they are collected in-hospital. We have created logic checks within our online data entry tool to enable us to differentiate between missing data and negative responses. Hospitals with large amounts of missing data will be contacted and offered support.
Mortality follow-up rates are likely to be high, as availability of the information does not depend on patients being contactable or willing to respond to a questionnaire (through linkage with NHS databases). There will be a small number of patients for whom data linkage is not possible (for example if they have left the NHS) but we will receive information about this type of loss to follow-up from NHS Digital.
For longer-term patient-reported outcomes, we will take measures to maximise response rates, for example through multiple reminder letters & phone calls. Additionally, and where required and appropriate, we will conduct sensitivity analyses to understand if there are any important differences between responders and non-responders and to assess for potential biases
Co-enrolment
There is no barrier to PQIP patients being co-enrolled into other studies. We actively seek collaboration with other researchers to facilitate co-enrolment with other studies while paying attention to the need to minimise redundant data collection and data collection fatigue of patients and investigators.
Analysis Plan
Descriptive statistics (RQ1, RQ2)
Descriptive statistics will be used to describe the basic demographics of participants. We will report process measures of engagement with PQIP (case-ascertainment rates, data completion) and compliance with processes of care at local and national level. (RQ1) Both unadjusted and adjusted outcomes will be reported (see inferential statistics for details of risk adjustment). (RQ2) All hospital level data will be presented anonymously.
Missing Data
For each statistical analysis, we will document the number and rate of missing observations on all variables involved. For each statistical model, we will assess the likely process that lead to missing observations, and whether data are likely to be missing at random (MAR) or missing not at random (MNAR). This assessment will inform a decision about the appropriate method of analysis. For example, multiple imputation or imputation of normal values may be considered for predictors in statistical models, depending on the likely process of missingness. Patients who are missing outcome data will be omitted from the relevant analysis; sensitivity analyses will be conducted to evaluate differences between patients with and without missing outcome data.
Temporal and between-hospital variation in outcomes (RQ3)
We will assess the variation over time and between hospitals on all primary and secondary outcome measures. Risk-adjustment will be based on logistic regression or other regression models, as appropriate for each outcome measure. Both patient-level and operation-level predictors will be included, such as age, gender, operation type and the constituent variables of previously published and validated risk adjustment models such as the Portsmouth Physiological and Operative Score for the enumeration of morbidity and mortality (P-POSSUM) and the Surgical Outcome Risk Tool (SORT) 41; 32.
Association between outcomes and hospital structures and processes of care (RQ4)
To describe the relationship between outcomes and hospital structures and processes, we will use mixed-effects models, employing random coefficients to assess the variation between hospitals. Patient-level covariates will be included as appropriate to distinguish the effect of case-mix from the effect of hospital characteristics on outcomes.
Relationship between different postoperative outcomes (RQ5)
We will evaluate the association between different outcome measures which purport to assess similar constructs (e.g. short-term morbidity) and between outcomes which may be associated with each other. As an example for the latter, the relationship between short-term complications and long-term survival has been demonstrated previously; 578 however, there are fewer data evaluating the relationship between a complicated postoperative course and health-related quality of life or disability free survival. We will use odds ratios, Pearson correlations and statistical measures of agreement as appropriate.
Evaluation of existing risk prediction models (RQ 6)
We will evaluate discrimination (using the area under receiver-operator-characteristics curves) and calibration (using Hosmer-Lemeshow or Pearson correlation statistics) of known risk prediction models including the P-POSSUM, SORT and Surgical Risk Scale.
Developing new models for the prediction of risk (RQ 7)
We will develop and internally validate new models for the prediction of risk, using logistic or other regression models as appropriate. Penalised regression models will be considered to reduce the risk of overfitting. These analyses may lead to a modification of the dataset with the aim of reducing data collection burden, if we find parsimonious models that are able to do without some variables previously considered important for risk prediction.
Evaluating sample validity (RQ8)
One of the objectives of PQIP is to measure and report processes and outcomes from surgery for the purposes of quality improvement. While it is widely accepted that data from a non-random sample of patients within an institution may provide important information about reliability of processes and systems, 42 sampling of patient outcomes should be statistically and conceptually robust so as not to provide biased estimates. Clinical trials and cohort studies which have specific inclusion and exclusion criteria, or which require patient consent, can be criticised as non-representative (and therefore the findings are of limited generalisability). 43 A potential solution to this is to use administrative data to evaluate outcomes; however, in the UK, such data are limited to mortality, hospital readmission and length of stay.
We will address these issues through an evaluation of our sampling strategy using administrative data from Hospital Episode Statistics (HES). For included procedures, an anonymised extract of HES data will be requested from NHS Digital for each hospital. This will enable us to conduct sensitivity analyses comparing patient characteristics (for example age, comorbidity, socioeconomic status using the Index of Multiple Deprivation) and outcomes (mortality, hospital readmission, length of hospital stay) between patients included in PQIP and those who are not. We will do this at individual hospital level and at aggregate (national) level.
Evaluating impact of PQIP (RQ9)
We will compare risk-adjusted patient outcomes which are available from administrative data (e.g. mortality, length of stay, readmission to hospital, ‘happy days’ censored at 90 days and one year after surgery) in hospitals and patients who are enrolled in PQIP and those who are not. Temporal trends will be analysed using a difference in differences approach. In order to avoid bias associated with different hospital types which may affect patient outcomes irrespective of involvement with PQIP, we will apply coarsened exact matching, 44, 45 matching PQIP and non-PQIP hospitals using pooled data from organisational surveys conducted by the Health Services Research Centre for PQIP and other studies (e.g. SNAP2-EPICCS 46 and the National Emergency Laparotomy Audit 47) as well as open-access external sources such as NHS England and Department of Health datasets.
Embedded further research
PQIP evaluation (RQ10)
One of the core aims of PQIP is to enhance the use of data for improvement by clinical teams, and in so doing, improve processes and outcomes for patients. PQIP may be considered as a complex intervention, which has been developed based on two theoretical frameworks (Normalisation Process Theory 48, 49 and the Theoretical Domains Framework 50). The intervention involves every part of the measurement, audit and feedback cycle and beyond, and includes collaborative events, regular reporting, dashboards of key measures and the use of social media and educational media to disseminate knowledge and learning. We are evaluating the effectiveness and processes of implementation of PQIP (RQ9) using both qualitative and quantitative methods. Quantitative methods will include the temporal analysis described to address RQs 3 and 9, and longitudinal survey work which will evaluate engagement with the use of PQIP data. The qualitative research will take the form of a multi-sited ethnography, using interviews and observations with frontline staff across PQIP and non-PQIP sites and PQIP project team members. The protocol for this evaluation and a description of the development of the intervention will be published separately.
Nested studies and secondary use of PQIP data
The multi-centre nature of PQIP provides an opportunity to develop and conduct multiple further studies in sub-groups of patients or hospitals, to pilot and evaluate new interventions, and then consider wider implementation. Such interventions may be aimed at improving processes and outcomes of care for patients (core aim 1 of PQIP) or improving the use of data for improvement (core aim 2 of PQIP). Two examples are briefly outlined below. In addition, PQIP has the potential to be used as a clinical trials platform, investigating new treatments or pathways using a randomised design. The PQIP project team will continue to consider requests from external clinicians and researchers collaborate and modify the PQIP dataset to maximise its usefulness for QI or research. After our planned analyses have been completed, a process will be established to enable external clinicians and researchers to apply for access to an anonymised dataset to explore research hypotheses.
pomVLAD
Post-Operative Morbidity reporting using Variable Life Adjusted Displays (pomVLAD), is a nested study funded by the Health Foundation which will develop, implement and evaluate the effectiveness of a near real-time reporting system for risk-adjusted morbidity, mortality and FTR rates. Originally developed to monitor observed versus expected mortality in cardiac surgery, VLAD provides a graphical display of risk-adjusted outcome data over time. 51 We plan to develop an recommendation bundle which will be paired with the VLAD display. The enhanced recovery bundle and VLAD dashboard will be trialled in ten early adopter hospitals and evaluated using a difference-in-differences analysis to evaluate its effectiveness and feasibility 52.
ERAS Plus
The ERASPlus initiative (erasplus.co.uk) is an expansion of a quality improvement programme aimed at reducing postoperative pulmonary complications. It was initially implemented, evaluated and found to be clinically effective in a single centre 53 and is now being rolled out throughout the Greater Manchester area. The intervention consists of a bundle of care supported by technological innovation (an app for patients to use) including exercise advice and training and targeted interventions to improve pulmonary function and reduce the risk of complications. The expanded ERASPlus initiative is also being funded by the Health Foundation and is working with PQIP to combine data collection efforts to reduce burden on local teams; additionally, this will facilitate comparisons with the non-ERAS+ population of PQIP patients.
Data management and linkage
All investigators and study site staff comply with the requirements of the Data Protection Act 1998 with regards to the collection, storage, processing and disclosure of personal information and will uphold the Act’s core principles. Patient-level data is entered by local investigators into a secure, electronic, web-based database. This is hosted on servers managed by UK Fast on behalf of the RCoA. Local investigators have access to their own full datasets. An anonymised dataset will be used by the PQIP Project Team for analysis. In this dataset: NHS number is replaced by a unique study patient identifier; date of birth is converted to age on date of surgery; postcode is converted to primary care trust (PCT), strategic health authority (SHA) of residence and the ONS Lower Super Output Area to allow the allocation of the Index of Multiple Deprivation.
The minimum amount of patient identifiable data (including PQIP unique identifier, NHS number, date of birth, gender and postcode) will be extracted from the study database by the PQIP Project Team to allow data linkage to ONS mortality data and Hospital Episode Statistics data from NHS Digital, so that we may track post-discharge outcomes (e.g. hospital readmission) and long-term survival.
Data sharing
Local data can be exported by approved local investigators at each hospital at any time. We will invite PQIP collaborators to apply for multi-centre data to undertake their own secondary analyses which are outside our initial analysis plans. These applications will be reviewed by the PQIP project team and assistance offered for analysis and interpretation, subject to capacity. Applications for access to PQIP data by non-PQIP collaborators will be considered when the study and the planned analyses are completed. Fully anonymised datasets will be made available on data sharing resources once the study and all follow-up is complete.
Patient and public involvement
The PQIP patient study addresses areas prioritised by four James Lind Alliance Priority Setting Partnerships (JLA-PSPs): anaesthesia/perioperative care, intensive care, dementia & pressure ulcers. We received detailed structured feedback on our protocol from members of the PCPIE group at the National Institute for Academic Anaesthesia's Health Services Research Centre (NIAA-HSRC). Our project team has two lay members, as does our clinical reference group which provides support and advice on request. Patient representatives are full members of the study team and as such, are invited to comment on all aspects of continuing study development and implementation.
Impact and dissemination
A report of key findings will be produced annually. Quarterly study reports will also be disseminated to participating sites and published on the study web site. Research resulting from the study will be disseminated by presentations and publications in open-access peer-reviewed journals. Wider dissemination to various stakeholders, including the public, will be achieved via social media, podcasts, short videos, press releases, and written and electronic communications.