Theme 1: We all bring unique qualities to the table
The lead research team was initially uncertain how best to involve the public in Priority III, a methodology PSP set in the specific context of rapid reviews. Building on their experience from previous methodology PSPs (which focused on recruitment and retention in clinical trials), the lead research team engaged a group of public partners with differing levels of experiences concerning the context of Priority III: some had prior experience of PSPs, some had evidence synthesis experience, and some had experience in neither PSPs nor evidence synthesis. This theme highlights the importance of diverse perspectives in shaping the PSP's operationalisation.
- PERSPECTIVE: Coming from different perspectives towards shared-decision making
When they were first approached, public partners spoke of their uncertainty about how they could contribute. They stated that they felt outside their comfort zone given the methodological topic. When reflecting after the completion of Priority III, some public partners recognised that their lack of experience brought a fresh perspective. By asking clarifying questions and seeking accessible language, public partners initiated discussions that challenged the assumptions held by the methodologists.
Interview, Public Partner 003 “I feel what I can contribute is that question around whether a layperson should be involved in conversations that are unrelated to their specific experience. In terms of my own work, I look at things from a communications perspective. And sometimes, if I even just ask the right question, it might get the researcher to think in a different way. So even if I don’t have any idea but I just seek clarity, sometimes it creates a conversation that wouldn’t have happened otherwise.”
Researchers recognised that they held assumptions that the public partners challenged. This influenced their thinking, how they articulated their discussions in meetings, and their communication with the broader participants of Priority III. Ultimately, public partners' challenges led to decisions that made the research process clearer for all.
Researchers stated that they valued the public partners perspectives and noted that, at times, the direction of the study changed due to the discussions (see Appendix 3 for a description of activities and where public partners influenced the project). An example of constructive discussion recalled by the participants focused on when some researchers assumed that all understood the traditional definitions around evidence synthesis. Public partners highlighted that these definitions were not universally accessible and then worked with researchers to reach a consensus over meetings, email, and working documents until a shared understanding was reached.
However, shared decision-making did not always mean agreement, and all participants of this case study referred to this. This was not a point of criticism: what was viewed as more important was that all opinions were considered, that decisions were transparent and were clearly, honestly and frequently communicated with the Steering Group.
Focus Group, Public Partner 004. ‘When we said, well - what do you actually mean by rapid review? There was a deep breath and there was an hours lesson which, in the end of it I was thinking, what if we hadn’t asked that fundamental question? Now, we didn’t get a perfect answer, but actually we got involved in the discussion around why there isn’t a perfect answer.’
Throughout this study, public partners acknowledged differences but stressed their respect for the researchers’ perspectives - they also commented that they thought this respect was mutual. This regard for the opinion of others was thought to contribute to the decision-making process: all voices were heard, all opinions were considered, and there was transparency in decision-making.
Focus Group, Public Partner 003: “I’ve learned that everyone has some perspective to share and, even if I spent a lot of time at the start just listening and learning and growing my knowledge of the context, that it’s a really respectful atmosphere. No one should feel that they don’t have enough to say or enough background to not become part of the process.”
- GROUNDING: Public partners bring pragmatism and grounding in reality.
A key quality that public partners brought to this project was pragmatism - bringing solutions and grounding the project in reality. This was recalled in focus groups, interviews and noted during the participant observations. Public partners brought accessibility in relation to language and communication, and solutions that helped how Priority III evolved. This down-to-earth perspective often led to frank conversations.
An example occurred at the start of Priority III: public partners were concerned with some of the language and explanations that were part of the PSP information. They wanted to spend time teasing out the issues to ensure clarity. This altered timeframes and meant that original milestones were not met. Though these “delays” were perceived as unexpected by researchers, they acknowledged that addressing this issue expedited the study later through improved clarity.
In addition to the practical solutions that public partners brought to or prompted in the operationalisation of the project itself, researchers noted that public partners had an insightful understanding of the tension between what was “optimal” and what was “feasible” during Priority III in terms of progress of the project.
Interview, Researcher 002: “...there’s almost a surprising contribution of the public partners to the Steering Group in terms of a focus on feasibility or pragmatism. I didn’t expect that. You’ve got public partners saying, well hang on a second, we need to take a step back from this, it would be ideal if we could do this, but [lead research team] have said that the impact of that would be four, five weeks. So let’s think about could we all live with this?”
Such tensions were sometimes noted, for example, when some researchers wanted the “right” solution to what constituted an answered question in the literature, contrasting with pressures of needing to reach project milestones with limited resources. Both groups acknowledged a tendency of researchers to bury down into, and get lost in, the detail, and that the public partners, with an eye to the bigger picture, helped move the process along, keeping it under time and budget.
Steering Group, Public Partner 001: “Okay, so I’m one of the public partners and I’m struggling with this. It seems to me that since the kind of questions this whole process is about, is to do with trying to pull out those unresearched areas. Surely we don’t need to go any further than reviewing what systematic reviews already exist? Because you’d then be beginning to answer the kind of questions that we wish to put out there. And it just seems to me that there’s a bit of a danger of going in a never ending circle here.”
In this way, public partners understood the time limitations and the project's need to add value. They suggested and supported practical ideas to ensure progress.
(2) Theme 2: We need support and space at the table
The lead research team strived to support the public partners to facilitate meaningful and purposeful involvement in Priority III. There were various types of support that the researchers provided (Table 1). These were reported to have provided space, which facilitated meaningful contribution, enabled public partners to learn from each other and grow their confidence in methodology, and built trust with the research team and with each other.
Table 1. Description of various types of support that the lead research team developed, with feedback from the public partners, to facilitate meaningful involvement in Priority III.
Support provided
|
Description
|
Group of public partners
|
There were five public partners on the PSP. The public partners felt that having more than one or two public partners was particularly needed given the methodology context.
|
Payment policy
|
The research team with public partners developed a clear and transparent payment process that highlighted tasks the public partners would contribute to and how much they be paid (Appendix 4).
|
Pre-meetings
|
Separate PPI group meetings were held with the five public partners, the lead research team, and a representative of the JLA. The lead researcher chaired these “pre-meetings”. These meetings were one hour with a break between the pre-meeting and Steering Group meeting. Initially, these meetings provided information around rapid reviews and evidence synthesis and allowed space for questions and discussion. The agendas for these meetings mirrored the main steering group agendas in addition to any topic the partners wished to discuss.
|
Individual email support
|
The lead research team offered a point of contact to the public partners and invited ad-hoc one-on-one support and group conversations in email threads to facilitate shared learning.
|
PPI Item on Steering Group agenda
|
Initially, the group decided together not to have a PPI item on the agenda, lest it feel like a tick-box exercise. The public partners later requested to return the item to the agenda to facilitate transparency at the Steering Group of their activities and to provide an opportunity for questions. Each public partner presented these updates in a rotating manner.
|
Pairing methodologists and public partners to review questions
|
A methodologist and public partner volunteered to review and refine the interim survey questions and then sent their collated feedback to the research team. All five public partners volunteered and participated in this exercise.
|
- SUPPORT: Define and develop support needed for meaningful involvement
All participants thought that processes needed to be established early in the project's lifetime to support purposeful engagement, given the abstract nature of the study. The lead research team attempted to facilitate this by establishing several types of support (Table 1). The public partners also brought peer support, governance, and sharing of resources, for example. Researchers sometimes struggled with balancing attempts to share knowledge rather than reinforcing the idea that as ‘the experts’ their knowledge was the ‘right’ knowledge.
Interview, Researcher 002: “I’m afraid of it not being that level playing field. I’m afraid of it being disproportionate power differentials going on. I’m afraid that the public partner won’t have a voice with the strong methodologists. I’m afraid of inadequate chairing of the groups and not giving voice to them. I’m afraid that no matter how good the chair is, that it’s impossible to create that level playing field or make that meaningful, purposeful.”
All of the participants of this qualitative study spoke of the specific and contextual information support that was needed to facilitate involvement. Flexibility was needed in terms of what specific, contextual support was needed for each stage of the PSP and for each person, as all had varying levels of needs throughout. Though some public partners felt that more context and background could have been given at the start, the personalised support was viewed as helpful and necessary. It gave context to otherwise abstract methodological concepts.
Interview, Public Partner 001: “I was very much a novice, and when [researcher] approached me, I did ask for some guidance and help and they pointed me in the direction of a couple of examples of rapid reviews. That was very useful. I felt I understood - well not enough - but sufficient to get by.”
The “pre-meetings” were discussed at length during interviews and focus groups. Initially, the pre-meetings helped the public partners understand terminology and concepts in evidence synthesis and oriented them to the context of the PSP itself. Later, they were viewed as dynamic - a venue for sharing resources and knowledge, seeing the big picture and reflecting on learning.
Another process to facilitate involvement noted in interviews and focus groups was pairing people from the methodologist group of the Steering Group with public partners to work on a part of the PSP. Each pair reviewed a portion of the questions generated from the initial online survey. The pair were tasked with reviewing if the submissions were in scope, interpreted and grouped correctly, or if the meaning of each summary question was unambiguous, for example.
Public partners spoke of challenges and benefits from this process, such as being initially intimidated by the volume of data, feeling unsure how to contribute, experiencing challenges with perceived power dynamics, and moving towards learning, finding it a fun, rewarding and useful process.
Focus Group, Public Partner 004: ‘I said [to myself] why did I agree to this, and I closed down the file, breathed, went back and then thought, let me concentrate for an hour and get my responses together. It was only then that I felt confident enough to speak to [researcher]. Then suddenly it made sense. But even with my experience and knowledge, I’m going bloody hell, I’ve overstepped the mark.’
Methodologist participants also found this pairing exercise a useful process, where each pair came from very different perspectives and needed to work together to reach a consensus. Like the public partners, they also found it fun, interesting, and a learning experience.
In addition to these processes, the public partners often spoke of peer support: the learning from each other that occurred during their interactions. Many references were made to the pre-meetings as being vital to support this. Having five public partners with varying levels of experience meant that different skills supported personal development within the group. Reference was made to how the pre-meetings helped build confidence, created a sense of togetherness, and built trust within the group. It was felt that this extra peer support was particularly needed given the methodology context but may also have a place within other research activities.
Focus group, Public Partner 005: ‘When you’re in something which is perhaps outside the comfort zone - you can’t bring a specific lived experience to methodology research. So it’s thinking more what do I bring? That can seed doubts and then you might not speak up. Whereas we’re encouraged by others, we’re all very frank. So I think it’s [peer support] possibly more important in areas that aren’t related to your lived experience than it is in areas that are.’
In terms of support, researchers stated that in hindsight, they might spend more time planning at the start. Researchers felt tensions around ensuring specific types of support had purpose and were not overburdensome to public partners. Researchers believed that the extra time it took to establish clarity on the project was essential to enable meaningful involvement. Though intensive and unexpected, it had the additional benefit of familiarising the group with the terms and language of the project and helped establish a shared sense of purpose.
Interview, Researcher 002: “[regarding pre-meetings] I think it’s made a big difference… I see increased confidence in the public partners in engaging and contributing to the Steering Group. I see that they seem to be more prepared to engage in conversations around some of the nuances of rapid reviews, with the methodologists in particular.”
Developing these processes required anticipating challenges, and planning time and space to allow for communication while also needing flexibility and iteration throughout.
- SPACE: Creating safe space to listen, challenge and learn.
The types of support listed in Table 1 and described above created space, not just for public partners, but for the research team. The concept of space, both tangible space “at the table” in meetings (to ask clarifying questions, to challenge researchers), and the space created outside of meetings (email conversations, time and space allowed for changing direction), created an environment in which trust was established. There was metaphorical space in terms of openness, flexibility, room for all opinions, and willingness to change.
Focus group, Public Partner 001 “I felt there was enormous receptiveness of public involvement in this. It certainly wasn’t a question of having to vociferously advocate. One of the lovely things about it was right across this international group there seemed to be a ready acceptance of a need to listen to a wide range of stakeholders.”
Public partners spoke to the importance of the space to learn, listen and reflect, in ensuring they could contribute meaningfully. This allowed the public partners to learn from each other and grow confidence in contributing to the wider project.
Researchers spoke of the benefits of the thoughtful pace set by public partners. The space also allowed dialogues to develop, constructive disagreements to be aired, and transparent conversations to be held, all leading to shared decision-making.
Interview, Researcher 002. “The PPI partners have been official brakes onto the Priority III process overall. They have slowed us down to the benefit of the product at the end of the day, the quality within it and what will happen at the end. It’s undoubtedly been better.”
Though such a pace would not be the norm from a researcher's perspective (and was acknowledged as challenging), it was felt it was required given the context and brought authentic involvement to the project.
(3) Theme 3: We all benefit from working together
Through valuing different perspectives and creating space to discuss, everybody gains from co-producing research. There is mutual learning with unexpected benefits. The public partners requested the establishment of this case study to capture the learning and share with the wider community, highlighting the leadership from the public partners relating to shared learning in this project.
- RECIPROCITY: Mutual learning and capacity building
Reciprocity was evident between researchers and public partners and within the group of five public partners. In terms of their learning, public partners said they ultimately learned the importance of methodology research to people’s lives. They gained a greater awareness of evidence synthesis. There were examples of impact beyond the project, where a public partner linked up with a methodologist on the Steering Group to develop training on rapid reviews to support public involvement in their own country. Some public partners spoke of having more confidence when contributing to other methodological projects. Given the right context, they could not only contribute but also learn.
Meeting: Public Partner 002: “I learned a lot about the methodology about rapid reviews, it gives you confidence to get involved in other methodological studies like PRISMA or core outcomes sets. In the right environment you realise, I can really learn and I can make a substantial contribution.”
The project benefited from strong recruitment of public and patient participants to the survey (for example, 17% response rate from the public) and increased diversity of voices at the consensus meeting. Researchers gained new perspectives to improve communication, data collection and interpretation.
Interview, Researcher 002: “Simple things like we now have a much clearer definition of what a rapid review is because of the public partners. We’re now able to communicate much more clearly because of the public partners. We re-think, they continuously push us in that direction.”
Researchers highlighted the unexpected benefits of public voices in forward action of research. Researchers also spoke of their learning as something they gained personally from the project:
Interview, Researcher 002 “I personally have learnt an awful lot on this journey. We tried to be open to that learning; that’s been one of the biggest steps we’ve had to take. And to bring everybody with us.”
As the public partners group consisted of varying levels of experience, this allowed for mutual learning within the group:
Focus Group, Public Partner 003: ‘If I was a lone wolf on this group I don’t think I would have lasted. Even just having the understanding interpreted through your eyes has helped me. When something is too big for my beginner’s mind to even wrap my head around, when I hear the other [public] members ask something, it puts it in a different context for me than the researchers would. And it might click, ah that’s what they’re trying to say, that’s what everyone else seems to understand’.
Capacity building was considered from the start to ensure fresh voices would be included, who could be involved in future studies and contribute to wider methodology research. The team recruited new public partners and public partners they had worked with before to the PSP.
Interview, Researcher 002: “Because there’s a risk of becoming dependent on a smaller number of people who are known to you. But it also gives the opportunity to meet some new people that you’d work with again.”
Overall, everyone gained from working together, with two-way exchange observed and highlighted by participants.
- RELATIONSHIP: Partners in research, with a feeling of togetherness
Public partners spoke of mutual respect, feeling valued, and the importance of developing trust, which led to a sense of being “partners in research” with a feeling of togetherness, which was observed and expressed by the public partners and researchers.
For example, when discussing the payments, public partners said that being asked for their feedback on the proposed payment helped them feel valued. They also found it useful to see the tasks comprehensively broken down, time estimated and costed individually, and suggested that other researchers would benefit from this example (Appendix 4).
Focus group. Public Partner 002: ‘[in terms of the payment process] I would add the breakdown of the tasks but also the opportunity to comment on it. So here is what we propose and what do you think about it, is it fair. That was a first for me, usually there’s no opportunity to comment. So I thought that was very respectful. Breaking it down [by tasks] - you feel as if somebody is taking the trouble to think of what your needs are”.
Some public partners stated that they felt in safe hands to get involved with this project based on positive expectations from previous experience with the research team, highlighting the importance of continued relationship building.
Interview, Public Partner 001: “I guess it's also down to having had previous contact with you all, I had a sense that this would be a comfortable setting in which to try something which is well out of my comfort zone, because let’s face it, I know very little about methodology in research and specifically very little about evidence synthesis.”
The partnership approach observed was illustrated with extensive “behind the scenes” negotiation by the lead research team with the publishing platform for the PSP protocol. The article submission form required mandatory completion of institution. The public partners felt strongly that there should be the choice of having solely “patient and public partner” that was not affiliated to an institution. They felt this would make clear that their roles were as public or patients, not researchers, and expand the idea that publishing is for academics only.
Pre-Meeting, Public Partner 002: “It’s really important for me to be identified as a patient partner. I would never entertain putting down an institution because I’m not an academic and I don’t want people to identify me as an academic. When I look at an article I always look to see who is the PPI in it, and sometimes I can’t tell. And I think that’s really bad. I’m not doing this to be melted down into a sea of academics. I want to be identified for what I’m doing.”
The lead research team advocated on behalf of the public partners to change this, which was granted by the journal. This example illustrated the public partners’ leadership and served to demonstrate the partnership approach in the team.
There was an observed and expressed sense of co-production, with leadership also observed from the public partners in building capacity in others, prompting reflection and the sharing of learning, recruiting participants, and moving forward the project. All participants of this qualitative study described a sense of relational openness that led to mutual learning between public partners and researchers, with reciprocity evident.
Public involvement approaches, as well as processes such as priority-setting activities, aim to balance power dynamics between researchers and public partners. However, it was also acknowledged that it can never be a level playing field, as everyone has different levels of experience. Even though in this project, the processes were laid out by the lead research team, their openness to change led to a mutually beneficial experience whereby public partners shaped the project's direction influenced the methodological quality, relevance, appropriateness and accessibility of the PSP.
Focus Group, Public Partner 004 “I must speak to the leadership of [research team] who are open and willing to change. I remember [researcher] going oh I need to think about that and then came back with a very open and transparent discussion about how we get to the next point. And I think that’s still quite unique and that’s been vital, it makes us feel a sense of belonging and trust that we’re part of something”.
This study highlights the learning gained in relation to public involvement in a methodology PSP from the specific context of Priority III. Underpinning the findings are the concepts of communication and trust: communication between researchers and the public and within the public group; trust in each other, and in the process. From the outset of Priority III, a research agenda was set, an endpoint concerning the study’s outputs was promised, and the lead research team was responsible for meeting the research goals. However, rather than researchers driving the project's trajectory, the experiences shared during this qualitative study speak to the iterative learning and processes that were put in place to guide everyone to share space at the research table.
Focus Group, Public Partner 003. “Listening to all of the different PPI people and the researchers was a huge learning experience for me. It definitely served to grow my confidence as well. Including not be afraid of looking foolish when asking something. And the kindness and respect that everyone has shown me throughout. It helped grow my trust in the system and the people involved for future groups.“