An evaluation of the Race Equality in public INVolvemENT for research (Reinvent) project

doi:10.21203/rs.3.rs-3782662/v1

Download PDF

Research Article

An evaluation of the Race Equality in public INVolvemENT for research (Reinvent) project

https://doi.org/10.21203/rs.3.rs-3782662/v1

This work is licensed under a CC BY 4.0 License

Journal Publication

published 19 Mar, 2024

Read the published version in Research Involvement and Engagement →

You are reading this latest preprint version

Background

Young people from Black African, Asian and Caribbean heritage communities are less likely to be involved in Public Involvement (PI) activities than their Caucasian counterparts. The involvement of young people in PI is important as it allows them to contribute to decisions that could influence research and the healthcare provided to them. The Reinvent project aimed for academics to work with a community group, the Eloquent Praise & Empowerment Dance Company, to develop a community partnership with young people from Black African, Asian and Caribbean heritage communities. The aim of this paper is to evaluate the Reinvent project for key learnings on how to engage with young people from Black African, Asian and Caribbean heritage communities.

Methods

Reinvent developed a steering group which consisted of five young people, one academic, a Race Equality Ambassador and the Director of Eloquent. The steering group co-produced an agenda for two workshops and the evaluation tools used. The content of the workshops included drama exercises, discussions on physical and mental health, nutrition and school-life, short introductions to the concepts of research and PI, and group work to critique and improve a video currently used to promote PI in health research to young people. The evaluation tools included using the ‘Cube’ evaluation framework, video-blogging and collecting anonymous feedback.

Findings

Whilst the workshops were co-designed by the steering group, some of the planned activities changed on the day of each workshop in response to the young peoples’ agendas and preferences, highlighting the need for academics to remain flexible during such engagement activities. A recurring observation from the collective evaluation methods was the value of the workshops in increasing young peoples’ confidence in sharing their thoughts and opinions about health and PI.

Conclusion

Whilst Reinvent forms the basis of a new partnership, more work is needed to now build upon this so that young people can feel confident and supported to get involved in PI activities relating to research, and so researchers can continue to develop their skills in creative engagement activities.

Patient and Public Involvement

evaluation

under-served communities

young people

Black African and Caribbean heritage

Why are we doing this project?

Young people from Black African, Asian and Caribbean heritage communities are less likely to be involved in Public Involvement (PI) activities than Caucasian young people. The Reinvent project involved researchers working with the Eloquent Praise & Empowerment Dance Company to develop a partnership with young people from Black African, Asian and Caribbean heritage communities.

What were the aims of this project?

The aim is to evaluate the Reinvent project for key learnings on how to engage with young people from Black African, Asian and Caribbean heritage communities.

How did we do this project?

To plan the project we had a steering group with membership from young people from the dance company working alongside researchers. The steering group worked together to plan two workshops and agreed the tools to evaluate them. The content of the workshops included drama exercises, discussions on health, nutrition and school-life, short introductions to research and PI and group work to consider and improve a video promoting PI. The evaluation tools included answering four questions, video-blogging and collecting anonymous feedback using a post-box.

What did we learn?

Working with young people in an environment in which they were comfortable, and by researchers joining in with the activities that the young people enjoyed (such as dance), enabled more informal and open conversations to develop. More work is needed to now build upon this project so that young people can feel confident and supported to get involved in PI activities relating to research.

Public Involvement (PI) is when research is carried out “with” or “by” patients and public contributors rather than “to”, “about” or “for” them (1). PI is put into practice through people with lived experiences helping to make decisions and conducting research in order to enhance study relevance, design, analysis, dissemination and governance (2, 3, 4). By involving the public in research the quality and appropriateness of research can be improved for the wider target populations to whom the research is relevant (6). It is becoming increasingly important to evaluate PI to improve the quality, evidence the impact and to learn best PI practice (7). Through the process of evaluation, public contributors can consider if there have been opportunities for meaningful involvement and researchers can gain insights into the contributors’ experiences (8).

A challenge of PI is to ensure the inclusion of under-served communities, defined as people who use or might potentially use health or social services and who are less likely to be heard by these service professionals and decision-makers (9). Whilst the optimal approach to achieving representation from these communities remains a topic of debate amongst the academic community (10), the United Kingdom’s (UK’s) National Institute for Health and Care Research (NIHR) suggests that the presence of diverse communities in health research fosters an inclusive environment, introduces a range of fresh skills and perspectives, and enhances research (11). As part of their Equality Diversity and Inclusivity strategy (12), the NIHR developed a Race Equality Public Action Group (REPAG) for public involvement in research (13). Hosted by the NIHR Centre for Engagement and Dissemination, the group aims to give people from Black African, Asian and Caribbean heritage communities - the terminology they chose - a stronger voice in shaping research priorities, the design and delivery of research, the recruitment of participants from these communities into studies, and the mobilisation of evidence into practice. REPAG developed a Race Equality Framework for Public Involvement in Research and we followed these principles of good practice throughout. REPAG suggested that people from these communities stress the need for organisations to develop trusting relationships with them to build their confidence in being able to contribute to research (13). Research organisations can begin to build trust with under-served groups through community engagement. Community and charity groups can play a vital role in connecting researchers with under-served communities (14). These groups have already established trust-based relationships and often work creatively to support the communities with whom they work (15).

Layers of exclusion and intersecting inequalities can mean that young people from Black African, Asian and Caribbean heritage communities are even less likely to be involved in research and PI activities than their Caucasian counterparts (16). The involvement of young people in research and PI is important as it creates opportunities for them to influence the healthcare provided to them (17). Involving young people in PI activities could be beneficial as their insights and expertise could contribute to addressing the needs of the community they represent (18).

Partnership development

The Reinvent project, a term coined by the young people from the Eloquent, Praise & Empowerment Dance Company (henceforth referred to as Eloquent), was developed in partnership between Keele University, Sheffield University, Edge Hill University and Eloquent. The dance company provides a variety of activities ranging from dance classes, performing arts workshops, events, empowerment workshops, and mentoring. Over 150 young people, aged between four and eighteen years old, are currently enrolled in Eloquent.

Reinvent was also supported by a public contributor who is a member of the REPAG group and drama performer and teacher (NM), Keele University’s Race Equality Ambassador (NK) and the director of Eloquent. NK is also an established community representative at Eloquent and volunteers there on a weekly basis. Through these existing links, a funding application was developed and awarded (NIHR205207) to explore approaches to developing and evaluating a community partnership.

The Reinvent project aimed for academics to work alongside Eloquent to develop a community partnership with young people from Black African, Asian and Caribbean heritage communities. Within Reinvent researchers worked with the Eloquent staff and the young people to co-design, run and evaluate two workshops. This paper presents the evaluation of the Reinvent project for key learnings on how to engage with young people from the Black African, Asian and Caribbean community.

Steering group

A steering group was convened and included five young people, one academic (AM), NK and the Director of Eloquent. This group met periodically throughout the project with NK also keeping in contact via instant messaging on a weekly basis. Instant messaging was the way in which staff at Eloquent communicated with the young people. At the beginning of the project the steering group met to agree the best way to establish a working relationship and decided that a co-productive approach could be taken. Co-production is a specific approach to PI whereby researchers and the public share power and decision making (19). The notion of co-production was explained by NK and this approach seemed to fit the values which the young people thought were important. A key principle of co-production is reciprocity, which the steering group thought should be at the centre of this project. The young people were expected to gain new knowledge and would be remunerated for their involvement in the project. The steering group thought that taking a co-productive approach was important to provide a platform to be heard. The steering group co-produced a structured agenda for each workshop (Tables One and Two). The group also helped to decide and co-produce the methods to evaluate each workshop.

The shorter version of the Gripp 2 (Additional File One) provides more details on working with the steering group and also offers a critical reflection of the project.

Ethical considerations

Whilst UK ethics committee approval is not required for most PI activities (20), the academic researchers approached all contact with the young people and the workshops in an ethical manner. Every effort was made to provide a safe environment where the young people were heard and supported as needed. We followed safeguarding practices established within Eloquent. Additionally, within each workshop, the ethical approach was as followed:

Preparing carefully each session in advance, with specific information, tasks and/ or questions for the group in clear, accessible English.

Making it clear that the young person is under no obligation to take part in any element of the project, and could leave the session at any time.

Gaining consent from each young person and their guardian for each workshop.

Providing written information about the project, nature of the activity and contact details of NK.

Encouraging public contributors to discuss their involvement with their peers within and outside of Eloquent.

Young people were made aware that they could withdraw from any activity at any point. To support each other, researchers and NK met to debrief and reflect on the activities after each workshop.

Workshop design

The content of each workshop was co-produced with the steering group. Tables One and Two describe the planned structure and content of Workshops One and Two respectively. Tables Three and Four describe the actual structure and content of the workshops. Modifications were made on the day in light of ‘real-time’ feedback from the young people. The content of the workshops included drama exercises, discussions on physical and mental health, nutrition and school-life, short introductions to health research and PI, and group work to consider and improve a video currently used to promote PI.

Involvement strategy

The young people were recruited to the workshops via NK. Within her voluntary capacity, NK verbally told the young people who regularly attended Eloquent about the workshops and a written invitation was sent out to all young people via Eloquent’s monthly newsletter. The steering group also discussed the project with their peers. To register their attendance for each workshop the young people had to verbally express an interest to NK and complete and return a consent form. The consent form was the standard consent form used by Eloquent.

Evaluation tools

The evaluation methods were co-produced by the steering group and included: completing the ‘Cube’ evaluation framework, video-blogging (vlogging) and posting anonymous feedback into a post-box within Eloquent’s headquarters during Workshop Two.

The ‘Cube’ evaluation framework

The ‘Cube’ evaluation framework (21) is a theoretical model which takes into account the dynamic and fluid nature of social interactions (21). The four dimensions are detailed in Table Five. Working with the steering group, questions relating to each four of the dimensions were operationalised.

Table One: Workshop One planned agenda

Time	Title of session	Description of the session	Whole group or small group activities	Facilitator/s
11:00 – 11:10	Introductions	NK to introduce the academic members of the team to the young people, and provide a brief description of why the workshops were being conducted.	Whole group	NK
11:10 – 11:50	Drama exercises	As a ‘warm up’ session NM to introduce the young people to a number of drama exercises and games (e.g. Splat).	Whole group	NM
11:50-11:55	What is health and well-being?	NK to facilitate discussions on health and well-being.	Whole group	NK
11:50 – 12:25	Discussions on school life	NK, NM, AM, TK to facilitate discussions on school life.	The group splits into two groups based on age (12-15 years old and 16 years old and above). AM and NK to facilitate the younger group, TK and NM facilitated the older group.	NK NM AM NK
12:25-12:30	What is mental health?	NK to facilitate discussions on mental health.	Whole group	NK
12:30 – 13:30	Break (lunch provided)
13:30 – 13:50	Drama exercises	As a ‘warm up’ session after lunch, NM to facilitate drama exercises and games (e.g. Splat).	Whole group	NM
13:50-13:55	What is research and how do we do it?	NK to facilitate discussions on research.	Whole group	NK
13:55 – 14:30	Discussions on nutrition and smoothie making	NK, NM, AM, TK to facilitate discussions on nutrition. The young people will also have to chance to make their own smoothies.	The group splits into two groups based on age (12-15 years old and 16 years old and above). AM and NK to facilitate the younger group, TK and NM facilitated the older group.	NK NM AM TK
14:30-14:35	What is Patient and Public Involvement and Engagement (PPIE?)	NK to facilitate discussions on PPIE.	Whole group	NK
14:35-14:55	Evaluation of the workshop	AM and NK to conduct the ‘Cube’ evaluation with the young people. The young people will also have to chance to video-blog their experiences.	Whole group	AM, NK
14:55 – 15:00	Close of the workshop	NK to thank the young people	Whole group	NK

Table Two: Workshop Two planned agenda

Time	Title of session	Description of the session	Whole group or small group activities	Facilitator
11:00 – 11:10	Introductions	NK to re-introduce the academic members of the team to the young people, and provide a brief description of why the workshops were being conducted.	Whole group	NK
11:10 – 11:50	Drama exercises	As a ‘warm up’ session NM to facilitate drama exercises and games (e.g. Splat).	Whole group	NM
11:50-12:00	What is Patient and Public Involvement and Engagement (PPIE)	NK to introduce the concepts of PPIE to the young people. NK to show a three minute video explaining what PPIE is.	Whole group	NK
12:00 – 12:30	Discussions on PPIE	NK, NM, AM, TK to facilitate discussions on PPIE.	The group splits into two groups based on age (12-15 years old and 16 years old and above). AM and NK to facilitate the younger group, TK and NM facilitated the older group.	NK NM AM TK
12:30 – 13:00	Break (light refreshments provided)
13:00 – 14:00	Drama exercises	As a ‘warm up’ session after the break, NM to facilitate drama exercises and games (e.g. Splat).	Whole group	NM
14:00 – 14:30	Re-visiting discussions on nutrition and smoothie making	NK, NM, AM, TK to facilitate discussions on nutrition. The young people will also have to chance to make their own smoothies.	The group splits into two groups based on age (12-15 years old and 16 years old and above). AM and NK to facilitate the younger group, TK and NM facilitated the older group.	NK NM AM TK
14:30-15:00	Evaluation of the workshop	AM and NK to conduct the ‘Cube’ evaluation with the young people. The young people will also have to chance to video-blog their experiences.	Whole group	AM, NK
15:00 – onwards	Close and a hot meal	NK to thank the young people. A hot meal was provided for all who attended.	Whole group	NK

Table Three: Workshop One actual agenda

Time	Title of session	Description of the session	Whole group or small group activities	Facilitator
11:15 – 11:20	Introductions	NK introduced the academic members of the team to the young people, and provided a brief description of why the workshops were being conducted.	Whole group	NK
11:20 – 12:20	Drama exercises	As a ‘warm up’ session NM introduced the young people to a number of drama exercises and games (e.g. Splat).	Whole group	NM
12:20-12:30	What is health and well-being?	NK facilitated discussions on health and well-being.	Whole group	NK
12:30 – 13:10	Discussions on school life	NK, NM, AM, TK facilitated discussions on school life.	The group splits into two groups based on age (12-15 years old and 16 years old and above). AM and NK to facilitate the younger group, TK and NM facilitated the older group.	NK NM AM TK
13:10 – 13:30	Break (light refreshments provided)
13:30 – 13:40	What is mental health?	NK facilitated discussions on mental health.	Whole group	NK
13:40 – 14:10	Drama exercises	NM facilitated drama exercises and games (e.g. Splat).	Whole group	NK
14:10 – 14:40	Discussions on nutrition and smoothie making	NK, NM, AM, TK to facilitate discussions on nutrition. The young people will also have to chance to make their own smoothies.	The group splits into two groups based on age (12-15 years old and 16 years old and above). AM and NK to facilitate the younger group, TK and NM facilitated the older group.	NK NM AM TK
14: 40 – 15:00	Evaluation of the workshop	AM and NK conducted the ‘Cube’ evaluation with the young people. The young people had the chance to video-blog their experiences.	Whole group	AM, NK
15:00 – onwards	Close and a hot meal	NK thanked the young people. A hot meal was provided for all who attended.	Whole group	NK

Table Four: Workshop Two actual agenda

Time	Title of session	Description of the session	Whole group or small group activities	Facilitator
11:00 – 11:10	Introductions	NK to reintroduced the academic members of the team to the young people, and provided a brief description of why the workshops were being conducted.	Whole group	NK
11:10 – 12:00	Drama exercises	As a ‘warm up’ session NM facilitated drama exercises and games (e.g. Splat).	Whole group	NM
12:00 – 12:10	What is Patient and Public Involvement and Engagement (PPIE)	NK introduced the concepts of PPIE to the young people. NK showed a three minute video explaining what PPIE is.	Whole group	NK
12:10 – 12:30	Discussing the PPIE video	NK, NM, AM, TK facilitated discussions on the PPIE video. The young people were invited to share their views on how the video could be improved.	The group splits into two groups based on age (12-15 years old and 16 years old and above). AM and NK to facilitate the younger group, TK and NM facilitated the older group.	NK NM AM TK
12:30 – 13:00	Break (light refreshments provided)
13:00 – 14:00	Drama exercises	As a ‘warm up’ session after the break, NM facilitated drama exercises and games (e.g. Splat).	Whole group	NM
14:00 – 14:30	Re-visiting discussions on nutrition and smoothie making	NK, NM, AM, TK facilitated discussions on nutrition. The young people had chance to make their own smoothies.	The group splits into two groups based on age (12-15 years old and 16 years old and above). AM and NK to facilitate the younger group, TK and NM facilitated the older group.	NK NM AM TK
14:30-15:00	Evaluation of the workshop	AM and NK conducted the ‘Cube’ evaluation with the young people. The young people had the chance to video-blog their experiences.	Whole group	AM, NK
15:00 – onwards	Close and a hot meal	NK thanked the young people. A hot meal was provided for all who attended.	Whole group	NK

Table Five: The four dimensions of the ‘Cube’ evaluation framework

Dimension	Description	Questions asked within the Workshops
Voice	Strong voices discuss issues and influence decision-making. Weak voices may discuss issues, but have little influence on decision-making.	Did you feel like your voice was heard?
Contribute	Knowledge can take on different forms, which may not be equally valued. A single involvement approach is likely to privilege one social/cultural group over another, thus perpetuating inequality.	Did you feel like there were enough activities to get involved in?
Agenda	Public concerns are in the context of social action, e.g. public opinion, norms and values, as well as individual experiences and behaviours. Organisation’s concerns are, e.g. bureaucracies and markets.	Did you feel like the event was based on things that mattered to you?
Change	Decision-makers’ willingness and ability to respond to issues raised by participants in knowledge spaces depend on contextual factors, e.g. economic resources and national policies.	Did you feel like the facilitators listened to you?

Within each workshop, NK introduced the framework and questions. The young people were then asked to map their experiences of being involved within the workshops along the four dimensions. Each dimension was separately represented on a wall chart. The steering group suggested offering the young people the response categories of ‘yes’, ‘no’ or ‘maybe’ along a continuum on the wall chart. The young people were asked to use a sticky note to indicate which category they felt best represented their experiences. The young people were also asked to write comments on their sticky notes explaining their choice. NK made it clear that the young people did not have to complete this activity if they did not wish too. Within Workshop One the planned allocated time for completing this activity was 20 minutes, however, the prior activities within the workshop overran and the evaluation had to be completed in 10 minutes. Within Workshop Two 30 minutes was dedicated to completing the ‘Cube’ activity.

Video blogs

The steering group suggested having one young person as the ‘interviewer’ who asked four questions of other young people. The four questions were: i) What did you learn from today’s workshop? ii) Do you have any new insights from today’s workshop? iii) What was your favourite part of the workshop? iv) What would you like to see us do at the next workshop? The same questions were asked within both workshops.

Anonymous feedback

Following Workshop One, and in light of the feedback from the steering group, an anonymous post-box was placed within the room in which Workshop Two was taking place. The young people were invited to write feedback and reflections on both of the workshops and post them in the post-box.

The ‘Cube’ evaluation

After each session the wall charts displaying each question relating to each domain of the ‘Cube’ were photographed and collated. Researchers counted each sticky note in the ‘yes’, ‘no’ or ‘maybe’ categories relating to each question.

Video blogging

The content of the vlogs relating to each workshop were collated. The key messages from the vlogs were translated into written form and discussed as a team.

Anonymous feedback

The anonymous feedback was collated into a Microsoft Word document. Key messages from the feedback were discussed as a team.

Findings

The first workshop was held on the 30th May 2023 and was attended by 25 young people; six males and 19 females. The ages ranged from between 13 and 17 years old. Two academics (AM, TK), a public co-applicant (NM), NK and the Director of the Eloquent attended.

The second workshop was held on the 26th July 2023 and was attended by 16 young people; three males and 13 females. The ages ranged from between 13 and 17 years old. All of the young people who attended the second workshop had been present at the first workshop. Two academics (AM, OB), a public co-applicant (NM), NK and the Director of the Eloquent attended.

Summary of the ‘Cube’ results

A low proportion of young people who attended Workshop One completed the ‘Cube’ (six out of 25). The scores in Table Six indicate that most young people felt listened to and had ample opportunity to have their voice heard. All young people who completed the ‘Cube’ within Workshop One thought that there were several ways that enabled involvement in the workshop and that the facilitators were willing to change, rather than be resistant. The scores also suggest that the young people believed that Workshop One

was dominated by activities and conversations which mattered to them. The low response rate may have been due to a lack of time to complete the ‘Cube’ activity.

A high proportion of young people who attended Workshop Two completed the ‘Cube’ evaluation (15 out of 16). All of the young people who completed the ‘Cube’ thought that their voice was heard and that the facilitators were willing to change. The scores indicate that most young people felt that there were several ways that enabled involvement and that the activities were relevant to them. The increase in the completion of the ‘Cube’ may have been because there was increased time dedicated to this activity. Table Six details the responses relating to each domain. No comments were written on the sticky notes within either workshop.

Table Six: Responses to the ‘Cube’ framework
Workshop One	The ‘Cube’ dimension	Yes	Maybe	No	Total number of responses
	Voice	5	1	0	6
	Contribute	6	0	0	6
	Agenda	4	1	0	5
	Change	6	0	0	6
Workshop Two	Voice	15	0	0	15
	Contribute	13	2	0	15
	Agenda	12	3	0	15
	Change	15	0	0	15

Summary of video-blogs

Within Workshop One, one young person was the interviewer and four young people answered questions. The young people discussed learning that mental health and well-being are related to physical health and both can influence school-life. One young person discussed learning the importance of meaningful friendships. A few young people suggested that they have new insights into how to express emotions. All of the young people described their favourite activities being the drama exercises and discussions focused on school and nutrition. The young people stated that Workshop Two should consist of more drama activities and discussions on nutrition; both were incorporated.

Within Workshop Two, one young person was the interviewer and six young people were interviewed. All of the young people indicated that they had developed self-confidence when participating in the whole and smaller group activities, and had been able to use their voice to speak their mind and to be their authentic self. A few of the young people described having a better understanding of the meaning and practice of PI. All of the young people described that their favourite activities were the drama exercises as these permitted them to bond as a group and to gain a better understanding of everybody’s personalities. To improve the workshops, the young people recommended including dance and singing exercises, as well as drama. All of the young people described that they enjoyed the format of the workshop and would attend another; they proposed a topic for the group work could focus on how to manage a hobby (such as dance) along-side school-life. The topics of discussion could allow for the younger members to learn from the older ones and academics. All young people also thought that the workshops should be whole day events.

Summary of anonymous feedback

There were a total of 25 notes which contained anonymous feedback. The anonymous feedback mainly centred on what the young people had learnt during the workshops. Learning was discussed in terms of them being a performer and gaining personal skills. As a performer, the activities facilitated by NM helped the young people to think about being louder, having a better physical stance and how to present themselves when in a group. On a personal level, the young people described that the workshops helped them to communicate, increased their confidence in using their voice and improved their teamwork.

The feedback also included a number of comments relating to the discussions focusing on school-life within the first workshop. Many of the comments included references to feeling more understood about their struggles relating to school. A few comments indicated that the young people felt safe to discuss potentially sensitive topics; one comment described how a young person had been bullied at school and how the workshops made them feel like they did not have to change who they are.

Although the length of each workshop was decided by the steering group, a few comments proposed that the workshops should have been whole day events as some of the activities seemed rushed.

This manuscript has presented an evaluation of the Reinvent project. The evaluation tools included using the ‘Cube’ evaluation framework, video-blogging and collecting anonymous feedback. Whilst the workshops were co-designed by the steering group, some of the planned activities changed on the day of each workshop in response to the young peoples’ agendas and preferences, highlighting the need for academics to remain flexible during such engagement activities. A recurring observation from the collective evaluation methods was the value of the workshops in increasing young peoples’ confidence in sharing their thoughts and opinions about health and PI.

The partnership between the universities involved in this project and Eloquent was strategic in nature as NK has been a volunteer at Eloquent for over a decade. NK had a boundary spanning role both working in the Impact Accelerator Unit, Keele, and being a community representative for Eloquent. Due to time resources that would be needed to begin to develop a community partnership, partnering with Eloquent offered pragmatic advantages for engaging with young people from an under-served community. Collaborating with a community representative external to the research team has been previously recommended (22), however, we found that NK’s boundary spanning role enabled her to have both knowledge of the academic environment and community partner, thus providing a strong communication link between the two. The role adopted by NK in this project may also have helped to foster ‘ethically conscious’ activities by helping to establish relationships and rapport, enabling the young people to experience safe boundaries and a space to disclose personal experiences.

Community interests

Whilst the workshops were co-designed with a steering group, it is clear that the planned activities changed on the day of each workshop. This was in response to feedback from the young people. Previous research has suggested that academics should avoid overly rigid and inflexible activities (23). Researchers with limited experiences of conducting such workshops may feel that they need to heavily structure activities to feel confident that all information gathering will be covered (23). This project has shown that academics should approach such activities with an openness to respond to what is heard from the young people and build in a contingency plan if anything unexpected happens. By doing this researchers are ensuring that the community’s topics of interest are central.

The impact upon the young people

Within the PI field there is a recognition of the need to understand the impact of involvement activities (24). Impact is one of the six UK Standards for Public Involvement in research which aim to help researchers and organisations improve the quality of the public involvement (25). A recurring observation from the collective evaluation methods was the impact and value of the workshops in increasing young peoples’ confidence in sharing their thoughts and opinions in a group setting. More work is needed to assess if the impact of these workshops results in young people having the confidence to pursue their own involvement in PI activities and research.

Evaluation methods

Previous studies have undertaken different approaches to evaluating PI activities primarily at the end of their research activities (26). The steering group wished for each workshop to be evaluated separately, but via multiple methods. By doing this and sharing learning from the first workshop (e.g. that the young people wanted more drama exercise activities), it maximised the impact on shaping the delivery of the second workshop. It is clear that multiple methods are emerging to support the evaluation of public involvement; a growing confidence in the field realises that there is no ‘one size fits all’ method. By using multiple evaluation methodologies the project has gone some way to optimise learning both individually for the young people, and collectively as a research team. The use of video blogs to capture the experiences of the young people was innovative and novel, however, this method needed time to set-up and finances to purchase the relevant equipment. Vlogging was also time-consuming during the workshops meaning that not all young people were able to be involved. More research is needed to understand how vlogging could work in practice whilst offering inclusive opportunities for all involved within the activity.

Previous research has used the ‘Cube’ framework to evaluate PI activities (27). The benefits of using this framework were that it enabled cross-sectional comparisons between the two workshops and the results were immediately available allowing for activities to be modified in a timely manner. The lack of responses in the first workshop was an important indicator that the young people needed more time to complete the exercise in the second workshop. Studies which have asked adults to write narrative feedback when completing the ‘Cube’ suggested that it encouraged public contributors to reflect about their involvement and experiences (21). The young people in this project provided no narrative feedback when completing the ‘Cube’, but did provide a number of comments via an anonymous post-box showing the importance of using multiple evaluation methods. This project suggests that young people may value the anonymous nature of evaluating PI activities. The completion of the ‘Cube’ did permit the academics to reflect on the purpose and strategic directions of the workshops and could help to inform the planning of future PI activities.

Strengths and limitations

Although this project demonstrates several strengths with respect to co-designing the methodological approach taken to evaluating the workshops, there are also limitations that should be identified. As suggested by the young people when evaluating the project, only having four hours to conduct the workshops made some of the activities (e.g. the Cube) feel rushed. Furthermore, the gender balance between the young people was skewed in favour of females; more efforts may be needed to engage young males. All of the young people were aged 13 years and over, involving younger children may require a different model of working.

Reinvent has shown that academic institutions and young people from an under-served community can partner to co-design workshops and apply evaluation tools. Whilst Reinvent formed the basis of a partnership, more work is needed to now build upon this so that young people can feel confident and supported to involve themselves within PI activities and research.

NIHR - National Institute for Health and Care Research

PI –Public Involvement

REPAG - Race Equality Public Action Group

UK – United Kingdom

Conflict of interests

The authors declare no conflicts of interest.

Ethics approval and consent to participate

We followed the NRES/NIHR Involve Statement and ethical approval was not required for this patient and public involvement piece ʻethical approval is not needed for the active involvement element of the research, (even when people are recruited via the NHS), where people are involved in planning or advising on research e.g. helping to develop a protocol, questionnaire or information sheet, member of the advisory group, or co‐applicant’.

Consent for publication

Not applicable

Availability of data and materials

Not applicable

Competing interests

The authors declare that they have no competing interests.

Funding

NIHR funding scheme Public Health and Social Care Delivery (PDG)

KD is part funded by the National Institute for Health and Care Research (NIHR) Applied Health Research Collaboration (ARC) West Midlands (NIHR 200165) and is an NIHR Senior Investigator (ID NIHR 200259). The views expressed are those of the author(s) and not necessarily those of the NIHR or the Department of Health and Social Care.

Authors' contributions

The first author (Alice Moult) was responsible for the overall paper, incorporating input from the wider team. All authors were involved in the conceptualisation of the paper and provided feedback on draft versions.

Acknowledgements

The views expressed are those of the author(s) and not necessarily those of the National Health Service, the NIHR, or the Department of Health and Social Care. The authors acknowledge the help and support of each public contributor for their time and efforts throughout the duration of this work. We would like to thank Dr Sarah Knowles and Mrs Jeevanthika Rubasinghe for their guidance throughout this project. We would also like to thank the Directors of Eloquent and the young people who attended each workshop.

INVOLVE. Briefing notes for researchers—Public involvement in NHS, health and social care research. 2022. https://www.nihr.ac.uk/documents/briefing-notes-for-researchers-public-involvement-in-nhs-health-and-social-care-research/27371 Accessed 16 August 2023.
Heywang-Köbrunner SH, Hacker A, Sedlacek S. Advantages and disadvantages of mammography screening. Breast Care. 2011;6(3):199–207.
Brett JO, Staniszewska S, Mockford C, Herron-Marx S, Hughes J, Tysall C, Suleman R. Mapping the impact of patient and public involvement on health and social care research: a systematic review. Health Expect. 2014;17(5):637–50.
Ennis L, Wykes T. Impact of patient involvement in mental health research: longitudinal study. Br J Psychiatry. 2013;203(5):381–6.
Domecq, J.P., Prutsky, G., Elraiyah, T., Wang, Z., Nabhan, M., Shippee, N., Brito, J.P., Boehmer, K., Hasan, R., Firwana, B. and Erwin, P. Patient engagement in research: a systematic review. BMC health services research. 2014;14(1), pp.1-9.
Staley, K. and Doherty, C. It’s not evidence, it’s insight: bringing patients’ perspectives into health technology appraisal at NICE. Research Involvement and Engagement. 2016; 2(1):1-12.
Kok M. Guidance document: evaluating public involvement in research. 2018. UWE Bristol e‐Prints Repository. Accessed 15^th March 2023.
Boivin A, Richards T, Forsythe L, et al. Evaluating patient and public involvement in research. BMJ. 2018; 363-5147.
Healthwatch. How to co-produce with seldom heard groups. 2020. https://network.healthwatch.co.uk/guidance/2020-10-26/how-to-co-produce-seldom-heard-groups. Accessed 28^th November 2023.
Lawton, M., Crane, J., Killen, V. and Patterson, J.M. Strategies for expanding patient and public involvement into under-served head and neck cancer communities. Current opinion in otolaryngology & head and neck surgery. 2023; 31(3), pp.180-184.
NIHR. Equality, Diversity and Inclusion. 2022. https://www.nihr.ac.uk/about-us/our-key-priorities/equality-diversity-and-inclusion/ Accessed 12^th August 2023.
NIHR. Equality, Diversity and Inclusion Strategy 2022-2027. 2022. https://www.nihr.ac.uk/documents/equality-diversity-and-inclusion-strategy-2022-2027/31295 Accessed 12^th August 2023.
NIHR. Race equality in public involvement. 2020. https://www.nihr.ac.uk/about-us/our-key-priorities/equality-diversity-and-inclusion/race-equality-public-involvement.htm Accessed 12^th August 2023.
Redman, S., Greenhalgh, T., Adedokun, L., Staniszewska, S., Denegri, S. and Co-production of Knowledge Collection Steering Committee. Co-production of knowledge: the future. Bmj. 2021; 372.
Provan, K.G., Veazie, M.A., Staten, L.K. and Teufel‐Shone, N.I. The use of network analysis to strengthen community partnerships. Public administration review. 2005; 65(5), 603-613.
Campbell, C. and McLean, C. Ethnic identities, social capital and health inequalities: factors shaping African-Caribbean participation in local community networks in the UK. Social science & medicine. 2022; 55(4), 643-657.
Rouncefield-Swales, A., Harris, J., Carter, B., Bray, L., Bewley, T. and Martin, R. Children and young people’s contributions to public involvement and engagement activities in health-related research: A scoping review. PloS one. 2021; 16(6), p.e0252774.
Ahmed, S., Calhoun, K., Corbie-Smith, G., Kleinman, L.C. and Wallerstein, N., Defining and Measuring Community Engagement and Community-Engaged Research: CTSA Institutional Practices.
Tembo, D., Morrow, E., Worswick, L. and Lennard, D. Is co-production just a pipe dream for applied health research commissioning? An exploratory literature review. Frontiers in Sociology. 2019; 4 -50.
Health Research Authority. Do I need ethical approval? 2020. https://www.hra.nhs.uk/planning-and-improving-research/best-practice/public-involvement/what-do-i-need-do/ Accessed August 22^nd 2023.
Gibson A, Welsman J, Britten N. Evaluating patient and public involvement in health research: from theoretical model to practical workshop. Health Expect. 2017;20(5):826–35.
Holmes, L., Cresswell, K., Williams, S. et al. Innovating public engagement and patient involvement through strategic collaboration and practice. Res Involv Engagem. 2019; 5 - 30 https://doi.org/10.1186/s40900-019-0160-4
Halsall, T., McCann, E. and Armstrong, J. Engaging young people within a collaborative knowledge mobilization network: Development and evaluation. Health Expectations. 2022; 25(2), 617-627.
Crowe, S., Adebajo, A., Esmael, H., Denegri, S., Martin, A., McAlister, B., Moore, B., Quinn, M., Rennard, U., Simpson, J. and Wray, P. ‘All hands-on deck’, working together to develop UK standards for public involvement in research. Research Involvement and Engagement. 2020; 6,1-9.
Aiyegbusi, O.L., McMullan, C., Hughes, S.E., Turner, G.M., Subramanian, A., Hotham, R., Davies, E.H., Frost, C., Alder, Y., Agyen, L. and Buckland, L. Considerations for patient and public involvement and engagement in health research. Nature Medicine. 2023; 1-8.
Hinton, E.C., Fenwick, C., Hall, M., Bell, M., Hamilton‐Shield, J.P. and Gibson, A., Evaluating the benefit of early patient and public involvement for product development and testing with small companies. Health Expectations. 2023.

No competing interests reported.

AdditionalFileOneTheGRIPP2ShortForm.docx

Download PDF

Journal Publication

published 19 Mar, 2024

Read the published version in Research Involvement and Engagement →

Editorial decision: Revision requested
09 Feb, 2024
Reviewers agreed at journal
04 Jan, 2024
Reviews received at journal
03 Jan, 2024
Reviews received at journal
30 Dec, 2023
Reviewers agreed at journal
21 Dec, 2023
Reviewers agreed at journal
21 Dec, 2023
Reviewers invited by journal
21 Dec, 2023
Editor assigned by journal
21 Dec, 2023
Submission checks completed at journal
20 Dec, 2023
First submitted to journal
20 Dec, 2023

You are reading this latest preprint version

An evaluation of the Race Equality in public INVolvemENT for research (Reinvent) project

Status:

Journal Publication

Version 1

Abstract

Plain English Summary

Background

Partnership development

Methods

Steering group

Ethical considerations

Workshop design

Involvement strategy

Evaluation tools

The ‘Cube’ evaluation framework

Video blogs

Anonymous feedback

Analysis

The ‘Cube’ evaluation

Video blogging

Anonymous feedback

Findings

Summary of the ‘Cube’ results

Summary of video-blogs

Summary of anonymous feedback

Discussion

Community interests

The impact upon the young people

Evaluation methods

Strengths and limitations

Conclusion

Abbreviations

Declarations

References

Additional Declarations

Supplementary Files

Status:

Journal Publication

Version 1