Description of included studies
The prior reviews [23] reported on a total of six included trials. The revised and updated searches for the 2020 version identified an additional 14,540 citations, with full-text copies of 156 articles retrieved for further assessment. Seventeen new trials, described in 41 papers, were then added to the six trials from the original review (see figure 1 for flow chart of search results). We identified 16 trials through database searches, one from hand searching the reference lists of relevant publications, and three ongoing studies from searches of trial registries (ISRCTN41083256; NCT02319967; KT Canada 87776; ACTRN12614000457640).
Twelve inclusions were RCTs, and the remaining 11 were cluster RCTs [37-47]. Six studies included 100 or fewer participants [11, 39, 46, 48-50], nine studies had between 100 and 1,000 participants, and eight had more than 1,000 participants including three studies with over 20,000 participants [38, 41, 45].
Participants and setting
Included studies were conducted in ten countries (USA, UK, Canada, Bangladesh, Norway, Ghana, Japan, Nepal, Vietnam, Belgium), in both urban and rural settings. Nine studies were set in (self-described) low socioeconomic countries and/or within disadvantaged communities [37, 38, 40, 41, 44, 45, 47, 50, 51]. The interventions for most studies took place within outpatient [51-54] or local community health clinics [37, 39, 43] or social support hubs [38, 41, 44, 45] and two studies were conducted in hospital inpatient settings [49, 55] one in a medical school [56], and one in aged care facilities [46].
The interventions were directed at people within various diagnostic groupings including mental illness [43, 47, 50, 53, 54]; pregnant and birthing women [38, 41, 44, 45]; and chronic diseases [39, 52] including smoking [40]. Other targeted participants were users of health care services such as participants undertaking hospital procedures [49, 55], attending Oncology clinics [57], or attending Paediatric clinics [51]. Other participants represented general health care users in their community [37, 48, 58], military veterans [42], nursing home residents [46], advocates for patients with Alzheimer’s and their caregivers [11], and medical students [56].
Interventions
To describe the interventions involving consumer engagement, we classified trials according to the area (context), level (methods), stage and characteristics of the consumers. Table 2 provides a summary of these features.
Areas of consumer engagement: The majority of interventions were in the area of health services [15], predominantly in the development (and/or implementation) of the service itself; although two were more specifically involved in developing patient information material [49, 55], two in developing training material for health professionals who deliver health services [56, 57]. Four studies engaged consumers in health-related research, spanning development, implementation and monitoring within the research process [42, 43, 53, 54]. The final four trials employed consumer engagement in the area of health policy, predominantly in developing priorities [39, 47, 48, 58] but also in the development of a clinical practice guideline [11].
Levels of consumer engagement: Using the IAP2 descriptors of consumer engagement, five trials were judged to be consulting [49, 51, 52, 55, 57], four were involving [39, 42, 43, 54], nine were collaborating [11, 37, 40, 46-48, 53, 56, 58] and five were empowering [38, 41, 44, 45, 50].
Stages of consumer engagement: Consumer engagement occurred mostly at the stage of development (n=18), with 11 at the stage of implementation, four monitoring and one evaluating. Ten trials engaged with consumers at two or more stages (table 2).
Characteristics of Consumers: In all trials, the consumer engagement participants had the relevant background as the intended recipient participants - whether that was related to a particular cultural or ethnic characteristic, or diagnostic or sociodemographic grouping (e.g. age or community-dwelling or service user). No consumers in the engagement process were reported to have had prior training in engagement, but the majority received relevant training as part of the trial process (13 trials) [37-41, 44-47, 50, 51, 53, 54] which ranged from an hour or two familiarising the consumers with the task at hand, through to several days spent in training, discussion and development.
Table 2
Results – included trials, with summary of population, intervention details, comparisons, outcomes and findings.
Study and population
|
Area of CE
|
Level of CE
|
Stage of CE
|
Outcomes
|
Findings
|
Aabakken et al. [55] 1997
Endoscopy patients (n=235)
|
Health Services
|
Consult
|
Development
|
Levels of anxiety
Patient satisfaction
|
Favours CE
No difference
|
Abelson et al. [48] 2003
Community groups
CE1vs CE2 vs CE3 (n=46)
|
Health Policy
|
Collaborate
|
Development
|
Prioritising health concerns
Rating importance of strengths
Ranking health determinants
|
CE priorities more likely to change
CE greater environment/local education
CE greater employment/conditions
|
Alhassan et al. [37] 2016
Health staff (n=234 staff; 64 health facilities)
|
Health Services
|
Collaborate
|
Monitoring
Evaluation
|
Safe-care essentials – patient safety, quality etc
Motivation levels
|
Favours CE (overall risk score p<0.05)
No difference for some sub items
Favours CE (overall motivation score p<0.0001)
|
Armstrong et al. [11] 2018
Guideline development groups (n=18 participants)
|
Health Policy
|
Collaborate
|
Implementation
|
Descriptive comparison of proposed PICOT questions, benefits, and harms between groups
Qualitative analysis of discussion themes from audio
recordings of the question development retreat discussions
|
Proposed guideline questions, benefits and harms largely similar between groups
Only the CE group proposed outcomes related to the importance of being able to plan for the future.
CE influenced the conduct of guideline development, scope, inclusion of patient-relevant topics, outcome selection, and planned approaches to recommendation development, implementation, and dissemination.
|
Azad et al. [38] 2010
Bangladesh women giving birth (n=43 717 births)
|
Health Services
|
Empower
|
Development
Implementation
|
Neonatal mortality rate
Maternal death
Health services
|
No difference
Favours no CE (RR1.91; 95%CI 1.27,2.9)
No difference
|
Boivin et al. [39] 2014
Patients with chronic conditions (n=17 patients, 44 professionals, 6 cluster sites)
|
Health Policy
|
Involve
|
Development
|
Level of agreement between patient and professional priorities
Changes in priorities/prof intentions/cost
|
CE priorities in agreement
Different changes – CE more community; prof more technical
No difference in intentions or cost
|
Carman et al. [58] 2015
Health consumers
CE1 vs CE2 vs CE3 vs control (n=1774, 76 groups)
|
Health Policy
|
Collaborate
|
Development
|
Participant knowledge
Attitudes toward decision-making
Attitudes towards hospital use
|
Favours any CE (p<0.05) vs no CE
Varied attitudes mostly not different
|
Choi et al. [40] 2016
American Indians who smoke (n=624)
|
Health Services
|
Collaborate
|
Development
|
Smoking abstinence
|
Favours CE at 12 weeks and 6 months for self-report of quitting
No difference for salivary tests
|
Chumbley et al. [49] 2002
Surgical patients (n=100)
|
Health Services (PIM)
|
Consult
|
Development
|
Clarity and knowledge of PCA
Worries about using PCA
|
Favours CE for clarity of information and knowledge of PCA; no difference for worries
|
Clark et al. [53] 1999
Patients with mental health diagnosis (n=120)
|
Health Research
|
Collaborate
|
Implementation
Monitoring
|
Patient satisfaction
Negative responses
Positive responses
|
No difference
Favours CE
No difference
|
Coker et al. [51] 2016
Children (lower income) with parent coach (n=251)
|
Health Services
|
Consult
|
Development
|
Receipt of services
Parent experiences
Service use – healthcare utilisation
|
Favours CE
Favours CE
Aspects in favour; others no different
|
Corrigan et al. [50] 2017
Homeless with mental health diagnosis (n=67)
|
Health Services
|
Empower
|
Implementation
|
TCU-HF – health status and QoL
Homelessness
SF-36; Recovery scale; QoL Scale
|
All outcomes favour CE
|
Early et al. [52] 2015
Respiratory outpatients (n=165)
|
Health Services
|
Consult
|
Development
|
Satisfaction; Confidence; Outcome; Consultation time
|
All outcomes no difference
|
Fottrell et al. [41] 2013
Bangladeshi women giving birth (n=13459 pregnancies/neonatal events)
|
Health Services
|
Empower
|
Development
Implementation
|
Neonatal mortality
Maternal death rate; process indicators; maternal psychology
|
Favours CE (OR 1.91; 95% CI 0.55,0.8)
No difference or qualified differences only for other outcomes
|
Fujimori et al. [57] 2014
Oncologists having communication training (n=601 patients, 30 oncologists)
|
Health Services
(training)
|
Consult
|
Development
|
Objective performance
Confidence communication
Patient distress (HADS A and D); satisfaction; trust
|
Favours CE
Favours CE
Favours CE for patient trust/depression; No difference for anxiety or satisfaction
|
Guarino et al. [42] 2006
Military veterans (n=1092, 10 medical centres)
|
Health Research
|
Involve
|
Development
|
Participant understanding
Satisfaction
Adherence/participation
|
No difference for all outcomes
|
Hughes-Morley et al. [43] 2016 Recruitment into mental health trial (n=8182 patients)
|
Health Research
|
Involve
|
Development
|
% recruited through CE
% responded/recruited with telephone follow-up
|
No difference
No difference
|
Jha et al. [56] 2015
Junior Drs in patient safety (n=283)
|
Health Services
(training)
|
Collaborate
|
Development
Implementation
|
Attitude to patient safety
+ve & -ve affect scales
|
No difference for attitudes to safety
Favours CE for +ve affect
|
Manandhar et al [44] 2004
Nepalese women giving birth (n=28931, 6380 pregnancies)
|
Health Services
|
Empower
|
Development
Implementation
|
Neonatal mortality rates
Maternal deaths
Still births; Service uptake; Home care practices; Infant morbidity
|
Favours CE (OR 0.7; 95%CI 0.52, 0.94)
Favours CE (OR 0.2; 95%CI 0.04, 0.91)
No difference except for uptake of services favours CE
|
Persson et al. [45] 2013
Vietnamese women giving birth (n=22561 births)
|
Health Services
|
Empower
|
Development
Implementation
|
Neonatal mortality rates
Frequencies live/still births; staff knowledge; resources and usage
|
No difference overall (favours CE 3rd year)
No difference except for increased care usage favours CE
|
Polowczyk et al. [54] 1993
Patients with mental health Dx (n=530)
|
Health Research
|
Involve
|
Implementation
Monitoring
|
Patient satisfaction
|
Favours control (no CE in treatment)
|
Van Malderen et al. [46] 2017 Residents of ACF
CE1 vs active vs passive controls (n=88)
|
Health Services
|
Collaborate
|
Development
Implementation
Monitoring
Evaluation
|
Active ageing survey;
QoL;
Participation and autonomy scale
|
No difference
Favours CE and active control
No difference
|
Wells et al. [47] 2013
Patients with mental health diagnosis (n=1246)
|
Health Policy and Services
|
Collaborate
|
Development
Implementation
|
Mental health scale
Services use; socioeconomic factors
|
Favours CE all items
Favours CE except no difference for employment and use of medication.
|
Key: CE = consumer engagement; CI = confidence interval; PICOT = Population, Intervention, Comparator, Outcome, Time; QoL = quality of life; OR = odds ratio; PCA = patient controlled analgesia; RR = relative risk; SF-36 = Short Form (36) Health Survey; TCU-HF = Texas Christian University Health Form.
|
Outcomes
Table 2 summarises the outcome measures used across the trials. Twenty-one trials [37-47, 49-58] considered the effects on the intended recipient participants; only one trial [42] measured the effects of the intervention on the engagement participants themselves; five [11, 39, 46, 48, 51] measured the effects on research, policy or health care service itself and nine [11, 37-41, 44, 45, 47] measured process outcomes.
Excluded studies
Of the 180 full texts excluded in this update, 43 studies used an intervention that did not fit our definitions for consumer engagement, and 58 used the wrong study design (i.e. were not RCTs), nine were conference abstracts, and one did not relate to health research, policy or services (i.e. wrong subject area). The most common reason for exclusion (n=64 studies) was the use of a design that could not differentiate the effect of consumer engagement intervention from other effects (for a full list please contact authors).
Risk of bias in included studies
Overall, the risk of bias in the included studies was high, confounded by poor (unclear) reporting particularly in the earlier studies as noted in the 2006 and 2009 reviews. Only two studies achieved four criteria as low risk [39, 43]; the average number of criteria judged as low risk was only 1.3 per trial. Figure 2 shows individual ratings for each risk category.
Effects of interventions
Twenty-one [11, 37-47, 49-57] of the 23 included trials were comparisons of consumer engagement versus no consumer engagement. Two trials involved comparisons of different methods or degrees of consumer engagement, all compared with no consumer engagement [48, 58]. No trials were identified that only compared one type of engagement with another. Subgroup analyses were not able to be performed with insufficient numbers of trials in any sub-group of interest, as expected.
Consumer engagement versus no consumer engagement in health services (15 included trials)
Health service delivery: We were able to find four studies [38, 41, 44, 45] sufficiently similar across the PICO domains to allow meta-analyses. These studies all investigated women giving birth who received a comprehensive health services intervention for healthy birthing that was developed and implemented with consumer engagement at an empowering level, compared to birthing education and support developed and delivered without consumer engagement (across three different countries) [38, 41, 44, 45]. All four trials used cluster randomisation and all four performed appropriate analyses at both the individual and cluster levels. All stated their intent to account for the cluster design in their protocols and all carried this intention out in the final reports; therefore, no unit of analysis issues were present in the outcome data. For the outcomes of neonatal mortality, we extracted data for 83,925 births, with a total of 1,028 deaths for the intervention group and 1,282 for the control group; this gave an odds ratio of 0.8 in favour of the intervention group (95% CI 0.77, 0.91; p<0.0001) (see figure 3). The heterogeneity (as assessed by the I2) was high at 72% and mostly explained by the individual ORs that ranged from 0.66 to 0.92. The second outcome (maternal deaths) that afforded meta-analysis returned a non-significant finding from the same four studies [38, 41, 44, 45] and same number of births with 80 maternal deaths in the intervention group versus 78 in the control (OR of 1.10, 95% CI 0.81,1.51; p=0.54) (see figure 4). Again the heterogeneity was high at 83% and this time was predominantly explained by the result of one study [38] which reported an unforeseen maternal death rate of 63 in the consumer engaged group versus 35 in the control; in addition, heterogeneity by chance may have been exaggerated by the small number of events (maternal deaths). The authors could not explain the reason for this and felt it was not attributable to the intervention as there was no common cause of death. The other three studies [41, 44, 45] in contrast, all reported lower maternal death rates in the intervention groups (two of the three non-significant).
Based on these four trials [38, 41, 44, 45] there is evidence that consumer engagement in developing and implementing health services, to enhance the care of mothers and their babies, results in a reduction in neonatal, but not maternal, mortality.
Three other trials [40, 51, 52] investigated the effect of consumers in developing health services. Choi et al [40] collaborated with consumers from a culturally appropriate community group to develop community-based smoking cessation program, finding in favour of increased self-reported quitting for the consumer engagement intervention, but no effect on saliva-based testing for smoking abstinence. Coker et al [51] consulted parents to develop parent coaches for children from lower-income groups, and reported favourable effects on improved receipt of services and satisfaction with consumer engagement-developed services compared to non-consumer engagement. Early et al [52] consulted with consumers in the development of respiratory outpatient services but found no difference in outcomes between the services that were consumer-consulted versus not. Based on these findings there is some evidence that some aspects of health service development may be improved by consumer engagement, but the evidence is not clear as to what aspects these may be in any consistent way.
One trial [50] investigated the effect of consumer engagement on implementation only of health services. Corrigan et al [50] used an empowerment model with consumers to implement services for people who were both homeless and with mental health issues. They reported that all outcomes were in favour of the consumer engagement-empowered services including health status, quality of life, and rates of homelessness. One trial [37] collaborated with consumers in the monitoring and evaluation of health services for safety, quality and health staff motivation. They reported that patient safety and staff motivation improved with the consumer-collaboration processes, and no change for quality items. Van Malderen et al [46] collaborated with residents of an aged care facility to develop, implement, monitor and evaluate several training interventions versus an intervention with no consumer input. They reported no difference between the various interventions except that the consumer-collaboration interventions favoured improved quality of life.
Patient information
Two trials evaluated products (patient information leaflets) which were developed following consumer consultation [49, 55]. The leaflets were compared with patient information developed without consumer consultation. Based on these two trials [49, 55] there is evidence that consumer consultation prior to developing patient information material probably results in material that is more relevant, readable and understandable to patients. Further, based on these same two trials, there is evidence that there may be little or no difference in worries or anxiety associated with procedures for patients receiving information material developed following consumer consultation, compared with patients receiving material developed without consumer consultation. Both studies had some unclear risk of bias issues.
Health professional training
Two trials [56, 57] engaged consumers in the development of education material for medical practitioners. Fujimori et al [57] consulted with consumers in the development of communication training with oncologists and reported that consumer-based training led to improved objective performance and confidence in communication by the doctors and possible improvement in patient trust, though there were no differences for patient anxiety or satisfaction. Jha et al [56] collaborated with consumers to develop and implement training for junior doctors in patient safety and reported no difference in junior doctors’ attitudes to safety but favourable changes in their positive affect. Based on these two trials [56, 57] there are mixed effects of consumer-engagement (consultation) on the development and/or implementation of health professional training.
Consumer engagement versus no consumer engagement in health care policy (five included trials)
Two trials [39, 48] engaged consumers in priority setting processes for health services. Abelson et al [48] compared three different methods of consulting consumers, reporting that consumers did have some different perspectives with a greater focus on environmental, local education and employment conditions. Boivin et al [39] involved consumers in setting priorities for chronic disease prevention and management, finding that consumers and professionals were in broad agreement although differed on specifics for example were more focused on community whilst the professionals were more focused on the technical aspects. Carman et al [58] collaborated with consumers in exploring ethical decision-making in health care - they compared different ways of deliberating with the consumer group via face to face or online options compared to no consumer engagement and reading material only, and found that all options which involved consumer engagement were superior to the control group for knowledge and attitudes towards decision-making, as well as attitudes to hospital use. Based on these studies there is some evidence that consumers may have a role in identifying a broader range of health care priorities that are complementary to those identified by professionals. There is also some evidence of consumers contributing positively to identifying need and developing mental health service directions, as well as supporting decision-making processes in health.
The fourth policy-based trial [47] collaborated with consumers with previous mental health issues in identification, planning, development (and implementation) of community-based services. The services developed with consumer collaboration were superior to services developed with no consumer engagement in all measured items (mental health scores, service use and so forth), except employment and medication use.
The remaining policy-related trial [11] convened two parallel guideline development groups, one with and one without consumers, to craft Population, Intervention, Comparator, Outcome, Time (PICOT) questions, and identify key benefits and harms on the topic of using amyloid positron emission tomography in the diagnosis of dementia. The proposed guideline questions, and benefits and harms, were largely similar between the two groups; however, only the group that collaborated with consumers proposed outcomes around the future development of cognitive impairment at certain time points and proposed rate of progression (rather than considering the development of dementia as a binary [yes/no] outcome). Consumer collaboration also influenced the conduct of guideline development, scope, inclusion of patient-relevant topics, outcome selection, and planned approaches to recommendation development, implementation, and dissemination.
Consumer engagement versus no consumer engagement in research (four included trials)
Two trials compared consumers (former patients) with professionals as data collectors in patient satisfaction surveys in mental health services [53, 54]. They compared the data collected, to investigate if responses given to consumer interviewers differed from responses given to staff (professional) interviewers. Clark et al [53] found that those who were surveyed by consumer interviewers gave significantly more 'extreme negative' responses, defined as a score of 0 on any question (on a scale from 0 to 4), compared to those surveyed by staff interviewers (p = 0.02). There was no significant difference between the two groups in the number of 'extreme positive' responses, defined as a score of four on any question. Overall, participants reported high levels of satisfaction with mental health outpatient services regardless of whether the interviewer was a consumer or staff member. Polowczyk et al [54] also found that participants reported high levels of satisfaction with mental health outpatient services regardless of whether the interviewer was a consumer. In this trial the average satisfaction score was a little lower in the consumer (client) interviewed group than it was in the staff interviewed group (0.16 on a scale from 1 to 4, p = 0.05). When the results of these two trials [53, 54] were pooled, the overall difference was similar (0.14 on a scale from 0 to 4, p = 0.001), (figure 5). Clark et al [53] asked for patients' views on satisfaction with case management services and with physicians' services. We have reported results from satisfaction with case management services only. The conclusion would not have differed if scores from satisfaction with physicians' services had been used. Based on these same two trials there are small differences in satisfaction survey results when consumer interviewers are used instead of staff interviewers.
Two trials [42, 43] engaged with heath researchers in other research processes. Guarino et al [42] compared an informed consent document developed with consumer involvement (potential trial participants) to a consent document developed by professionals (trial investigators). Overall, there were no statistically significant differences between trial groups for understanding of the parent trial at any time point for any of participant characteristics. Hughes-Morley et al [43] involved consumers in recruiting participants for a mental health care trial and found no difference between the number recruited using consumer engagement-based processes versus non-consumer engagement involved methods. Based on these trials, consumer consultation in the development of consent documents or recruitment may have little if any impact on participant’s self-reported understanding of the trial described in the consent document, satisfaction with study participation, adherence to the protocol or recruitment to participate.