Consumer Engagement in Health Care Policy, Research and Services: Methods and Effects.

To assess the effects of consumer engagement in health care policy, research and services. Methods Selection criteria included randomised controlled trials assessing consumer engagement in developing health care policy, research, or health services. The International Association for Public Participation, Spectrum of Public Participation was used to identify, describe, compare and analyse consumer engagement. Outcome measures were effects on people; effects on the policy/research/health care services; or process outcomes. We included Most consumer adopted to health Based on four large cluster-randomised controlled trials, there is evidence that consumer engagement in the development and delivery of health services to enhance the care of pregnant women results in a reduction in neonatal, but not maternal, mortality. From other trials, there is evidence involving consumers patient information material results in that is more relevant, readable and and can improve knowledge. Mixed effects are reported of consumer-engagement on and/or health There is some evidence that using consumer interviewers instead staff in satisfaction surveys can have a small inuence on the results. There is some evidence consumers may have a health care priorities that are complementary to those from professionals. There is some evidence that evaluating health


Background
Engaging consumers in health care decisions is widely recognised as being important in health care policy, research and services.Consumer participation can be viewed as a goal in itself, by encouraging participative democracy, public accountability and transparency.Consumers may offer different and complementary perspectives and priorities to those of professionals; furthermore, they may not have the same con icts of interest and loyalties as professionals.The concept of consumer engagement is founded on the principle that health care policy, research and health services are in the public interest [1] and people have a moral right to contribute to decisions which will affect them and have been publicly resourced through government funds and charitable donations [1,2].In health policy, input from consumers is typically sought to promote democratic accountability, to ensure that diverse and divergent perspectives are introduced into policy discussions, and to test the suitability of different deliberative techniques to generate evidence for policymaking [3].In health research, evidence supports the notion that consumer engagement leads to research of greater quality and clinical relevance [4] and greater uptake of ndings [5].There is also some evidence that input from consumers in planning health care can lead to more accessible and acceptable health services [6].That said, there is a lack of research that reliably investigates whether consumer engagement achieves these intentions and if so, which methods of consumer engagement are most effective, and how these effects might be measured.
Despite the development of policy to support consumer engagement, there is evidence of widespread national and international variation in the extent to and the manner in which consumers are engaged.For example, membership of peak grant committees has been found to be dominated by academics and clinicians in over 70% of eleven nationally-based research funding organisations recently surveyed, and only one organisation provided public access to full protocols for completed or ongoing research [7].Furthermore, and notwithstanding the availability of well-established standards for consumer (stakeholder) engagement in creating clinical practice guidelines [8][9][10], there are considerable inconsistencies and gaps in practice [11][12][13].
Conversely, several consumer engagement strategies have led to tangible improvements across a range of metrics.The Guidelines International Network sought to operationalise their published standards for guideline development [9] as a toolkit [14] employing a mixed approach of literature reviews and stakeholder panels, they speci cally focused on ways to meaningfully involve patients and members of the public in developing guidelines.In addition, a number of initiatives have been implemented by organisations such as Planetree International (e.g.consulting and training programs), The King's Fund (e.g.involving patients and carers in research, conference planning and collaborative leadership) [15], and the UK National Health Service project aimed at exploring how patients and carers can act as leaders, provide feedback and improve the experience of healthcare (e.g.'patients as leaders' which resulted in the generation of building blocks for success and identi cation of key roles across systems) [16].
Many health professionals, consumers and organisations are calling for hard evidence and robust evaluations of many factors around consumer engagement including impact, how it leads to bene ts, the best method for translation and implementation and even how it should be conceptualized [17][18][19][20][21][22].Not all of these gaps in knowledge and application can be addressed by this review.However, we can aim to identify good practice to support and advise uptake in areas that are currently active, as well as provide evidence to support engagement in settings where it is not as common.A review was rst published on this topic in 2006 and updated in 2009 [23].Since 2009 there have been numerous developments regarding consumer engagement (sometimes termed 'patient and public involvement') in health care services, policy and research.This has been accompanied by considerable growth in people and organisations undertaking consumer engagement [24,25], shifts in terminology [26], and new conceptual models and frameworks to explore, explain and evaluate consumer engagement in health [27][28][29][30].In our review, we aim to report on the methods and effects of consumer engagement strategies used in health care policy, research and services.
To assess the effects of consumer engagement on health care policy, research and services.
Secondary objectives were to explore whether differences between studies might explain any differences between the effects.We were speci cally interested in differences in the: methods (levels) of consumer engagement (e.g.fact sheets, focus groups, patient advisory committees representing the levels of inform, consult, involve, collaborate or empower); stages (i.e.development, implementation, monitoring, evaluation) of health care policy, research and services in which consumers are engaged; and characteristics of consumer or professional participants (e.g.background, experience or training in consumer engagement).

Patient and Public involvement statement
A new author team was formed to undertake an update on the 2009 review [23] that included researchers and a consumer representative.In addition, we elected to partner with a stakeholder advisory group (including ten consumers) to enhance the relevance and currency of the review to potential readers and users.Further details of the stakeholder group members and the engagement process are included in additional le 1.Given the signi cant developments since the last review, the author team as advised by the stakeholder group, deemed the publication of a new review protocol and results to be relevant and necessary [31].At key stages of the systematic review process, stakeholders were invited to provide perspectives and feedback which were used to: craft and re ne the research question(s) and de nitions for the population, intervention, comparator/control, outcome [PICO] criteria; contextualize initial analyses of results from included studies; and ensure the appropriateness of interpretations from the study ndings in the draft nal review report.
Criteria for considering studies for this review Our review protocol was published a priori in PROSPERO [31]; for further details of our methods please refer to this listing.For our working de nitions of consumer engagement, health care consumer, health care professional/researcher/policy-maker, health care policy, health care services, health care research and further descriptors of potential outcomes, see additional le 2.

Types of participants
We included studies investigating the effects of consumer engagement on health care services, policy or research.There are two layers of participants in this review: (1) 'Engagement participants' who are involved in the engagement process and (2) 'Intended recipient participants' of the health care policy, research or services that have been developed, implemented, monitored, and/or evaluated using the consumer engagement strategy.'Engagement participants' are health care consumers and professionals (meaning health care practitioners, researchers or policymakers).

Types of interventions
We de ned consumer engagement as "an informed dialogue between an organisation and consumers, carers and the community which encourages participants to share ideas or options and undertake collaborative decision making, sometimes as partners" [32], and used the IAP2 Public Participation Spectrum to help us determine what should be included as a consumer engagement study [28].We included studies which compared consumer engagement to no consumer engagement or compared one method of consumer engagement to another method of consumer engagement, in the context of health care policy, research or services, where that engagement met the criteria for consult, involve, collaborate or empower (i.e.all items with the exception of 'inform' as it does not t with our de nition of consumer engagement being a two-way 'informed dialogue').

Types of outcome measures
To be included, a trial must have had a quantitative measure, based on the following three broad outcome categories to describe the range of effects: effects on people; effects on health care policy/research/ services; and process outcomes.

Search methods for identi cation of studies
We revised previous search strategies (additional le 3) and searched the following databases without language restriction: The Cochrane Central Register of Controlled Trials (CENTRAL, The Cochrane Library, February 2020); MEDLINE (OvidSP) (2009 to We searched the following additional sources and places for published and unpublished studies: websites of relevant organisations; clinical trials registries; Google Scholar; reference lists of included studies; and citation tracking of included studies.In addition, we liaised with our stakeholder group and contacted experts in the eld directly.We also promoted our review on Twitter and Facebook, inviting people to send us studies.All revised and updated search strategies are available from the principal author.

Data collection and analysis
Two authors (two of LW, AW, JL, SH) independently screened all titles and abstracts identi ed from searches to determine which met the inclusion criteria, with the assistance of Covidence systematic review software [Covidence systematic review software, Veritas Health Innovation, Melbourne, Australia].We retrieved in full text any papers identi ed as potentially relevant.Two review authors independently screened full-text articles for inclusion or exclusion, with discrepancies resolved by discussion and consulting a third author, if necessary, to reach consensus.A pilot screening of 100 papers was conducted to ensure all criteria were being applied consistently across studies by the four authors.All potentially relevant papers excluded from the review at this stage are listed as excluded studies, with reasons available from the authors.We also noted citation details and any available information about ongoing studies and collated and reported details of duplicate publications, so that each study (rather than each report) is the unit of interest in the review.Studies with more than one reference were identi ed by the year the study was conducted or completed.We report the screening and selection process in an adapted PRISMA ow chart ( gure 1) and checklist (additional le 4) [33].

Data extraction and management
Two review authors (two of LW, JL, SH) extracted data independently from included studies using a standardised data extraction form.Any discrepancies were resolved by discussion until consensus was reached, or through consultation with other authors where necessary.We piloted the data extraction template to ensure su cient information about the study design, participants, consumer engagement and study methods, and outcomes measured to inform the interpretation of the results.All extracted data were entered into RevMan (RevMan 2012) by two review authors (JL and SH) and were checked for accuracy against the data extraction sheets by a third review author working independently.In addition to data items to be extracted, the author team assessed the extent of the consumer engagement in each included study.To do this, the method of consumer engagement used in included studies was mapped onto one of the four items of the IAP2 Spectrum of Public Participation (i.e.consult, involve, collaborate or empower), and scored (Yes/No/can't tell) according to the seven principles of the IAP2 quality assurance standard [28] (table 1).independently assessed the risk of bias of included studies, with any disagreements resolved by discussion with a senior (SH) and consumer author (DK), to reach consensus.
We also contacted study authors for additional information about the included studies when necessary to clarify methods, data or expected completion dates.Studies were categorised as 'awaiting classi cation' if additional information was not able to be sought, or 'ongoing' if the trial was deemed in progress.

Data synthesis
We decided whether to meta-analyse data based on whether the interventions in the included trials were similar enough in terms of participants, settings, intervention, comparison and outcome measures to ensure meaningful conclusions from a statistically pooled result.
Where we were unable to pool the data statistically using meta-analysis, we had clear reasons for this decision (outlined in the Results section) and presented the data in tables and narratively synthesised results.We have presented the major outcomes and effects, organised by intervention categories according to the major types and/or aims of the identi ed interventions.
Within the data categories, we explored the main comparisons of the review: consumer engagement to no consumer engagement; one method of consumer engagement to another method of consumer engagement.
If studies compared more than one consumer engagement intervention (e.g. two or more interventions and a nointervention/control group), we would have compared each intervention separately to the no intervention/control group; and with one another.

Measures of treatment effect
For dichotomous outcomes, we analysed data based on the number of events and the number of people assessed in the intervention and comparison groups.We used these to calculate the odds ratio (OR) or risk ratio (RR) and 95% con dence interval (CI).For continuous measures, we analysed data based on the mean, standard deviation (SD) and number of people assessed for both the intervention and comparison groups to calculate the mean difference (MD) and 95% CI.If the MD was reported without individual group data, we used this to report the study effects.If more than one study measured the same outcome using different tools, we would have calculated the standardised mean difference (SMD) and 95% CI using the inverse variance method in RevMan 5.
The range of outcomes considered in this review was developed collaboratively with our stakeholder panel and is substantial.
Where studies recorded outcome data at multiple time points, we reported the data from the nal follow-up, because longerterm outcomes were considered most important in practice by our stakeholder panel.Therefore, we did not report outcome data recorded at other time points.

Unit of analysis issues
If cluster-RCTs were included, we checked for unit-of-analysis errors.If errors were found, and su cient information was available, we would have re-analysed the data using the appropriate unit of analysis, by taking account of the intra-cluster correlation (ICC).We planned to obtain estimates of the ICC by contacting authors of included studies or imputing them using estimates from external sources (e.g.similar studies, resources that provide examples of ICCs, or ICC patterns for particular types of cluster or outcome) [34].If it was not possible to obtain su cient information to re-analyse the data we would have reported effect estimates and annotated with a 'unit-of-analysis error' message.

Assessment of heterogeneity
Where studies were considered similar enough (based on consideration of the populations, consumer engagement activities, or other factors such as outcomes) [35] to allow pooling of data using meta-analysis, we assessed the degree of heterogeneity by visual inspection of forest plots and by examining the I² test for heterogeneity.We reported our reasons for deciding that studies were similar enough to pool statistically.Heterogeneity was quanti ed using the I² statistic.An I² value of 50% or more was considered to represent substantial levels of heterogeneity, but this value was interpreted in light of the size and direction of effects and the strength of the evidence for heterogeneity, based on the p-value from the I² test [34].Where heterogeneity was present in pooled effect estimates we planned to explore possible reasons for variability by conducting subgroup analysis.

Assessment of reporting biases
We assessed reporting bias qualitatively based on the characteristics of the included studies (e.g. if only small studies that indicate positive ndings were identi ed for inclusion), and if information that we obtained from contacting experts and authors or studies suggested that there were relevant unpublished studies.If we identi ed su cient studies (at least 10) for inclusion in the review we planned to construct a funnel plot to investigate small study effects, which may indicate the presence of publication bias.We would formally test for funnel plot asymmetry, with the choice of test made based on advice in Higgins et al [34],and bearing in mind that there may be several reasons for funnel plot asymmetry when interpreting the effects.

Subgroup analysis and investigation of heterogeneity
Given the nature and quantity of trials, it was unlikely we would be able to conduct a formal sub-group analysis; we therefore explored any potential effect modi ers narratively.We considered whether the differences in any of the following factors might explain any differences in the effects: the methods of consumer engagement per IAP2 participation level [28] (recorded verbatim from included studies; e.g.focus groups and patient surveys, patient advisory committees and Delphi processes, and citizen panels or consumer managed projects and services) [23,36]; IAP2 Public Participation Spectrum (with the four items: consult, involve, collaborate, and empower) [28], the stages (i.e.development, implementation, monitoring, evaluation) of health care policy, research and services in which consumers are engaged; differences between consumer or professional participants; and context (health care policy, research, services).

Sensitivity analysis
Where meta-analysis was possible, we considered a sensitivity analysis to determine the effects of including studies at highest risk of bias.This meant comparing the effects of studies deemed at highest risk of bias (as outlined in Assessment of risk of bias of included studies) with those rated at lower risk of bias.By default, the highest risk of bias studies would include quasi-RCTs.

Description of included studies
The prior reviews [23] reported on a total of six included trials.The revised and updated searches for the 2020 version identi ed an additional 14,540 citations, with full-text copies of 156 articles retrieved for further assessment.Seventeen new trials, described in 41 papers, were then added to the six trials from the original review (see gure 1 for ow chart of search results).We identi ed 16 trials through database searches, one from hand searching the reference lists of relevant publications, and three ongoing studies from searches of trial registries (ISRCTN41083256; NCT02319967; KT Canada 87776; ACTRN12614000457640).

Interventions
To describe the interventions involving consumer engagement, we classi ed trials according to the area (context), level (methods), stage and characteristics of the consumers.Table 2 provides a summary of these features.
Areas of consumer engagement: The majority of interventions were in the area of health services [15], predominantly in the development (and/or implementation) of the service itself; although two were more speci cally involved in developing patient information material [49,55], two in developing training material for health professionals who deliver health services [56,57].Four studies engaged consumers in health-related research, spanning development, implementation and monitoring within the research process [42,43,53,54].The nal four trials employed consumer engagement in the area of health policy, predominantly in developing priorities [39,47,48,58] but also in the development of a clinical practice guideline [11].
Stages of consumer engagement: Consumer engagement occurred mostly at the stage of development (n=18), with 11 at the stage of implementation, four monitoring and one evaluating.Ten trials engaged with consumers at two or more stages (table 2).
Characteristics of Consumers: In all trials, the consumer engagement participants had the relevant background as the intended recipient participants -whether that was related to a particular cultural or ethnic characteristic, or diagnostic or sociodemographic grouping (e.g.age or community-dwelling or service user).No consumers in the engagement process were reported to have had prior training in engagement, but the majority received relevant training as part of the trial process (13 trials) [37-41, 44-47, 50, 51, 53, 54] which ranged from an hour or two familiarising the consumers with the task at hand, through to several days spent in training, discussion and development.

Excluded studies
Of the 180 full texts excluded in this update, 43 studies used an intervention that did not t our de nitions for consumer engagement, and 58 used the wrong study design (i.e. were not RCTs), nine were conference abstracts, and one did not relate to health research, policy or services (i.e.wrong subject area).The most common reason for exclusion (n=64 studies) was the use of a design that could not differentiate the effect of consumer engagement intervention from other effects (for a full list please contact authors).

Risk of bias in included studies
Overall, the risk of bias in the included studies was high, confounded by poor (unclear) reporting particularly in the earlier studies as noted in the 2006 and 2009 reviews.Only two studies achieved four criteria as low risk [39,43]; the average number of criteria judged as low risk was only 1.3 per trial.Figure 2 shows individual ratings for each risk category.Consumer engagement versus no consumer engagement in health services (15 included trials)

Effects of interventions
Health service delivery: We were able to nd four studies [38,41,44,45] su ciently similar across the PICO domains to allow meta-analyses.These studies all investigated women giving birth who received a comprehensive health services intervention for healthy birthing that was developed and implemented with consumer engagement at an empowering level, compared to birthing education and support developed and delivered without consumer engagement (across three different countries) [38,41,44,45].All four trials used cluster randomisation and all four performed appropriate analyses at both the individual and cluster levels.All stated their intent to account for the cluster design in their protocols and all carried this intention out in the nal reports; therefore, no unit of analysis issues were present in the outcome data.For the outcomes of neonatal mortality, we extracted data for 83,925 births, with a total of 1,028 deaths for the intervention group and 1,282 for the control group; this gave an odds ratio of 0.8 in favour of the intervention group (95% CI 0.77, 0.91; p<0.0001) (see gure 3).The heterogeneity (as assessed by the I 2 ) was high at 72% and mostly explained by the individual ORs that ranged from 0.66 to 0.92.The second outcome (maternal deaths) that afforded meta-analysis returned a non-signi cant nding from the same four studies [38,41,44,45] and same number of births with 80 maternal deaths in the intervention group versus 78 in the control (OR of 1.10, 95% CI 0.81,1.51;p=0.54) (see gure 4).Again the heterogeneity was high at 83% and this time was predominantly explained by the result of one study [38] which reported an unforeseen maternal death rate of 63 in the consumer engaged group versus 35 in the control; in addition, heterogeneity by chance may have been exaggerated by the small number of events (maternal deaths).The authors could not explain the reason for this and felt it was not attributable to the intervention as there was no common cause of death.The other three studies [41,44,45] in contrast, all reported lower maternal death rates in the intervention groups (two of the three non-signi cant).
Based on these four trials [38,41,44,45] there is evidence that consumer engagement in developing and implementing health services, to enhance the care of mothers and their babies, results in a reduction in neonatal, but not maternal, mortality.
Three other trials [40,51,52] investigated the effect of consumers in developing health services.Choi et al [40] collaborated with consumers from a culturally appropriate community group to develop community-based smoking cessation program, nding in favour of increased self-reported quitting for the consumer engagement intervention, but no effect on saliva-based testing for smoking abstinence.Coker et al [51] consulted parents to develop parent coaches for children from lower-income groups, and reported favourable effects on improved receipt of services and satisfaction with consumer engagement-developed services compared to non-consumer engagement.Early et al [52] consulted with consumers in the development of respiratory outpatient services but found no difference in outcomes between the services that were consumer-consulted versus not.Based on these ndings there is some evidence that some aspects of health service development may be improved by consumer engagement, but the evidence is not clear as to what aspects these may be in any consistent way.
One trial [50] investigated the effect of consumer engagement on implementation only of health services.Corrigan et al [50] used an empowerment model with consumers to implement services for people who were both homeless and with mental health issues.They reported that all outcomes were in favour of the consumer engagement-empowered services including health status, quality of life, and rates of homelessness.One trial [37] collaborated with consumers in the monitoring and evaluation of health services for safety, quality and health staff motivation.They reported that patient safety and staff motivation improved with the consumer-collaboration processes, and no change for quality items.Van

Patient information
Two trials evaluated products (patient information lea ets) which were developed following consumer consultation [49,55].
The lea ets were compared with patient information developed without consumer consultation.Based on these two trials [49,55] there is evidence that consumer consultation prior to developing patient information material probably results in material that is more relevant, readable and understandable to patients.Further, based on these same two trials, there is evidence that there may be little or no difference in worries or anxiety associated with procedures for patients receiving information material developed following consumer consultation, compared with patients receiving material developed without consumer consultation.Both studies had some unclear risk of bias issues.

Health professional training
Two trials [56,57] engaged consumers in the development of education material for medical practitioners.Fujimori et al [57] consulted with consumers in the development of communication training with oncologists and reported that consumer-based training led to improved objective performance and con dence in communication by the doctors and possible improvement in patient trust, though there were no differences for patient anxiety or satisfaction.Jha et al [56] collaborated with consumers to develop and implement training for junior doctors in patient safety and reported no difference in junior doctors' attitudes to safety but favourable changes in their positive affect.Based on these two trials [56,57] there are mixed effects of consumerengagement (consultation) on the development and/or implementation of health professional training.

Consumer versus no consumer engagement in health care policy ( ve included trials)
Two trials [39, engaged consumers in priority setting processes for health services.Abelson et al [48] compared three different methods of consulting consumers, reporting that consumers did have some different perspectives with a greater focus on environmental, local education and employment conditions.Boivin et al [39] involved consumers in setting priorities for chronic disease prevention and management, nding that consumers and professionals were in broad agreement although differed on speci cs for example were more focused on community whilst the professionals were more focused on the technical aspects.Carman et al [58] collaborated with consumers in exploring ethical decision-making in health care -they compared different ways of deliberating with the consumer group via face to face or online options compared to no consumer engagement and reading material only, and found that all options which involved consumer engagement were superior to the control group for knowledge and attitudes towards decision-making, as well as attitudes to hospital use.Based on these studies there is some evidence that consumers may have a role in identifying a broader range of health care priorities that are complementary to those identi ed by professionals.There is also some evidence of consumers contributing positively to identifying need and developing mental health service directions, as well as supporting decision-making processes in health.
The fourth policy-based trial [47] collaborated with consumers with previous mental health issues in identi cation, planning, development (and implementation) of community-based services.The services developed with consumer collaboration were superior to services developed with no consumer engagement in all measured items (mental health scores, service use and so forth), except employment and medication use.
The remaining policy-related trial [11] convened two parallel guideline development groups, one with and one without consumers, to craft Population, Intervention, Comparator, Outcome, Time (PICOT) questions, and identify key bene ts and harms on the topic of using amyloid positron emission tomography in the diagnosis of dementia.The proposed guideline questions, and bene ts and harms, were largely similar between the two groups; however, only the group that collaborated with consumers proposed outcomes around the future development of cognitive impairment at certain time points and proposed rate of progression (rather than considering the development of dementia as a binary [yes/no] outcome).Consumer collaboration also in uenced the conduct of guideline development, scope, inclusion of patient-relevant topics, outcome selection, and planned approaches to recommendation development, implementation, and dissemination.

Consumer engagement versus no consumer engagement in research (four included trials)
Two trials compared consumers (former patients) with professionals as data collectors in patient satisfaction surveys in mental health services [53,54].They compared the data collected, to investigate if responses given to consumer interviewers differed from responses given to staff (professional) interviewers.Clark et al [53] found that those who were surveyed by consumer interviewers gave signi cantly more 'extreme negative' responses, de ned as a score of 0 on any question (on a scale from 0 to 4), compared to those surveyed by staff interviewers (p = 0.02).There was no signi cant difference between the two groups in the number of 'extreme positive' responses, de ned as a score of four on any question.Overall, participants reported high levels of satisfaction with mental health outpatient services regardless of whether the interviewer was a consumer or staff member.Polowczyk et al [54] also found that participants reported high levels of satisfaction with mental health outpatient services regardless of whether the interviewer was a consumer.In this trial the average satisfaction score was a little lower in the consumer (client) interviewed group than it was in the staff interviewed group (0.16 on a scale from 1 to 4, p = 0.05).When the results of these two trials [53,54] were pooled, the overall difference was similar (0.14 on a scale from 0 to 4, p = 0.001), ( gure 5).Clark et al [53] asked for patients' views on satisfaction with case management services and with physicians' services.We have reported results from satisfaction with case management services only.The conclusion would not have differed if scores from satisfaction with physicians' services had been used.Based on these same two trials there are small differences in satisfaction survey results when consumer interviewers are used instead of staff interviewers.
Two trials [42,43] engaged with heath researchers in other research processes.Guarino et al [42] compared an informed consent document developed with consumer involvement (potential trial participants) to a consent document developed by professionals (trial investigators).Overall, there were no statistically signi cant differences between trial groups for understanding of the parent trial at any time point for any of participant characteristics.Hughes-Morley et al [43] involved consumers in recruiting participants for a mental health care trial and found no difference between the number recruited using consumer engagement-based processes versus non-consumer engagement involved methods.Based on these trials, consumer consultation in the development of consent documents or recruitment may have little if any impact on participant's selfreported understanding of the trial described in the consent document, satisfaction with study participation, adherence to the protocol or recruitment to participate.

Statement of principal ndings
The primary objective of this updated review was to assess the effects of consumer engagement on health care policy, research and services.There has been a rapid increase in the number (and, in some instances, quality) of randomised controlled trials investigating the effectiveness of consumer engagement.The most noteworthy rise in the number of trials is in the area of health services (up from 2 to 15), with only a single addition in each of the policy and research domains.Typically, the trials compared a form of engagement with no engagement.The most common objective of the health service trials was to improve patient outcomes and the primary meta-analysis gave evidence that empowering consumers in the development and implementation of community-based services for pregnant women can reduce neonatal mortality, although there was no effect on maternal death rates.Coupled with other single trials, there is emerging evidence that consumer engagement in health service delivery affords bene ts (and no evidence of harms to date) and these seem to be particularly evident in community health care settings.It should also be noted that the outcomes of the health services trials were wide ranging beyond health effects on the intended trial participants and included satisfaction, health behaviour adoption, knowledge and service utilisation.There is some evidence that consumers can have a positive role in the training of health professionals and in providing information for patients to inform their decision-making, although there is still no trial evidence for the role of consumers in guideline production.
Investigations into the effect of consumer engagement in health policy were less prevalent but did include some trials demonstrating potential bene ts of collaborating with consumers in health priority-setting where they can broaden the perspectives otherwise gained from health professionals alone.There remains a dearth of investigation into the effect of consumer engagement in policy decision-making.The investigations of the effect of consumer engagement in health research remains limited to process indicators like recruitment, adherence, information and satisfaction, with no trials investigating the bene ts of consumers being engaged in setting a research agenda or research methodology.
Secondary objectives were to explore whether differences between studies might explain any differences between the effects.We were speci cally interested in differences in the: methods (levels) of consumer engagement (e.g.fact sheets, focus groups, patient advisory committees); stages (i.e.development, implementation, monitoring, evaluation) of health care policy, research and services in which consumers are engaged; and characteristics of consumer or professional participants (e.g.background, experience or training in consumer engagement).Such a broad range of trials did not allow for rigorous investigation of the in uence of these differences using sub-group analyses.However, there are patterns that warrant commentary.Firstly the strongest evidence (community-based antenatal care for pregnant women) lay in the method of engagement lies within the de nition of empowerment -using the de nitions from the IAP2 of placing "the nal decision-making in the hands of the public" and with the tacit understanding "we will implement what you decide" [28].The four studies [38,41,44,45] that provided the evidence for improved neonatal mortality all empowered consumers in this decision-making, with the consumer supported project and sustainable community development.The majority of the remaining studies used collaborative methods such as advisory groups, special meetings and committees; and smaller numbers using involving methods (via public meetings) or consulting (via focus groups and surveys).See table 1 and table 2 for details.
The predominant stage of consumer engagement was in development (over 75%), with a third of these [38,41,44,45,47,56] (including the four comprehensive health service interventions for healthy birthing) [38,41,44,45] undertaking engagement in both developmentAND implementation.This seems a logical paired process and again was most common in the community settings, to ensure buy-in and relevance for service delivery in most examples.Only one trial [46] collaborated with consumers at all stages from development through to evaluation.However, the numbers are too small to allow a statistical analysis of the import of this factor on effectiveness.
Finally, we considered the characteristics of the consumers in the trials.Again, there was a consistent nding that the consumers were not formally trained in engagement methods prior to the engagement, but the majority received requisite training in the study methodology or intervention to be delivered as part of the engagement process.In three studies [46,53,54], health professional participants were reported to receive this same training.Only eight trial consumer cohorts [11,43,48,49,52,55,56,58] received no training before or during the trial -bringing their lived experience as su cient expertise for the purposes of the evaluation.This is an important nding for future trials -to consider the engagement of all participants -professionals and consumers -in health care stages across policy, research and services.
We found no evidence of adverse effects in engaging with consumers; most trials did not collect data on this aspect.The trials with patient health level data (such as neonatal mortality) [38,44] considered unintended effects in terms of examining the risk ratios for all outcomes between the two groups.None of the included trials addressed possible other adverse effects of consumer engagement, such as tokenism or time and other resource impacts.Seven studies [37,[39][40][41]44,45,47] did consider cost-effectiveness, reporting favourably on the cost per life saved and/or cost of years of life saved in the case of the neonatal care studies, or the cost of the intervention itself, but with no bene ts or effectiveness analysis [47].
Strengths and weaknesses of the study including a strong stakeholder engagement at key stages in the review process we believe we have mitigated against any major biases.
Our included trials were wide ranging internationally and across diverse health populations.In particular, we were struck by the number of trials engaging with consumers who were most likely to experience disparity or disadvantage in health services.The settings were predominantly in the community -this seems to lend itself to participatory processes.However, there were some examples of consumer engagement in acute and outpatient settings that augur well for ongoing investigations across the health services spectrum.Positive ndings across this range suggests that consumer engagement has a role in many settings, however this does not negate the need for consideration to always be paid to these contextual factors when developing, implementing, monitoring and evaluating consumer engagement processes.
Whilst we note the increase in number of more strongly designed (randomised controlled) trials in this important area, most still had a relatively high risk of bias.It should be acknowledged that it can be hard to blind participants -consumers and professionals -in a health care intervention.However, more achievable aspects like adequate randomisation and allocation should be more stringently adhered to, as well as better reporting of outcomes and attrition.

Strengths and weaknesses relation other studies
Treweek et al [59] published a Cochrane regarding strategies to improve recruitment to randomised trials.Their key results relating to consumer engagement in recruitment are similar to the results we have found; "Using a particular, bespoke, user-testing approach to develop participant information lea ets.This method involved spending a lot of time working with the target population for recruitment to decide on the content, format and appearance of the participant information lea et.This made little or no difference to recruitment; absolute improvement was 1% (95% CI −1% to 3%) (p.2)" [59].Crawford et al [6] published a systematic review with the aim of examining the effects of involving patients in the planning and development of health care.They reported a low level of studies (mostly case studies) which support the notion that involving patients does contribute to changes across a range of settings, however they found no evidence base for the effects of this involvement on various outcomes such as health, service usage and/or quality.

Meaning of the study: possible explanations and implications clinicians and policymakers
Health professionals can be con dent that engaging consumers can have a positive effect in health care policy, research and services, however there are no standard metrics to guide evaluation of this effect.The factors that underpin successful consumer engagement can be hypothesised from the included trials as involving consumer empowerment processes, in the development and implementation phases; there is a lack of evidence arising from the trials regarding the effect of other elements of engagement (inform, consult, involve, collaborate and empower) across all phases.Our recommendation is to use a tailored approach to consumer engagement with an IAP2 level that is as high and appropriate to the goals and promises to the public, and embedded evaluation.
The effect of engaging consumers in health care policy, research and services is gaining attention in the literature.The 23 trials included in this review demonstrate that randomised controlled trials of consumer engagement are feasible.Variation in practice re ects the complex nature of consumer engagement and a climate of innovation rather than evidence-based The evidence from this review suggests that the best methods to achieve effective engagement are likely to vary and will need to engage participants (professionals and consumers).It is our contention that more work needs to be done to implement consumer engagement strategies and solutions speci c to each problem.For instance, metrics used in assessing effectiveness and overall quality improvements associated with a health care service delivery project may be very different to that of a clinical trial, versus that of a more translational research-based project.The small number of studies available have forced the comparison of methods and outcomes across different contexts and metrics.Trials are needed to evaluate the effects of different methods of: identifying participants (health consumers and professionals) determining and utilising the elements and phases of engagement participant engagement training and support distinguishing purposefully between consumers with lived experience versus the community at large (acknowledging that health professionals have a unique and privileged knowledge of the health sector and therefore, while consumers of health policy, research and services, cannot be seen to contribute a consumer voice representative of the wider community) timing the engagement (which included studies did not evaluate) engaging individuals, groups and organisations resourcing engagement including customised models of nancial support evaluation.

Conclusion
There is growing evidence from randomised controlled trials of the effects of consumer engagement on the relevance and positive outcomes of health policy, research and services; however, there are no standard metrics to guide evaluation of this effect.The factors that underpin successful consumer engagement can be hypothesised from the included trials as involving consumer empowerment processes, in the development and implementation phases.Our recommendation is for health care consumers, providers, researchers and funders to continue to employ evidence-informed consumer engagement in their jurisdictions, using a tailored approach with an IAP2 level that is as high and appropriate to the goals and promises to the public, and embedded evaluation.

Figures
Figures

Table 1
Interventions with consumer engagement as de ned by the IAP2 Public Participation Spectrum allocation sequence concealment; blinding (participants, personnel); blinding (outcome assessment); completeness of outcome data; selective outcome reporting; and other sources of bias.Using Covidence systematic review software, two authors (JL, LW)

Table 2
Results -included trials, with summary of population, intervention details, comparisons, outcomes and ndings.
Malderen et al [46] collaborated with residents of an aged care facility to develop, implement, monitor and evaluate several training interventions versus an intervention with no consumer input.They reported no difference between the various interventions except that the consumer-collaboration interventions favoured improved quality of life.