Impact guaranteed? Involving potential recipients, implementers and wider stakeholders in the co-production of two school-based relationships and sex education interventions to prevent teenage pregnancy and dating and relationships violence

doi:10.21203/rs.2.18606/v1

Download PDF

Research article

Impact guaranteed? Involving potential recipients, implementers and wider stakeholders in the co-production of two school-based relationships and sex education interventions to prevent teenage pregnancy and dating and relationships violence

https://doi.org/10.21203/rs.2.18606/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

Background: The benefits of involving intended recipients, implementers and other stakeholders in the co-production of public health interventions are widely promoted. Practical accounts reflecting on the process and value of co-production in intervention design, however, remain scarce. We outline our approach to the co-production of two multi-component, school-based relationships and sex education interventions. We reflect on the utility of involving school staff, students, and other youth, professional and policy stakeholders in intervention design and on some of the challenges we encountered during the process.

Methods: Seven consultations were conducted in southeast and southwest England involving 75 students aged 13–15 and 22 school staff. A group of young people trained to advise on public health research were consulted on three occasions. Twenty-three sexual health and sex education practitioners and policy makers shared their views at a stakeholder event. Written summaries of activities were prepared by researchers and shared with the specialist provider agencies for each intervention. Negotiated consensus between researchers and providers was reached about how participant views should inform intervention content, format and delivery models.

Results: Consultations confirmed acceptability of intervention aims, components and delivery models, including curriculum delivery by teachers. They sensitised us to the need to ensure content reflected the reality of young people’s experiences; include flexibility for the timetabling of lessons; and to develop prescriptive teaching materials and robust school engagement strategies to reflect shrinking capacity for schools to implement public-health interventions. Accessing and prioritising stakeholder feedback was not always straightforward, however, where specific expertise or capacity for participation was limited or when participant views contradicted best practice, budget or the randomised controlled trial design.

Conclusions: Involving potential recipients, implementers and wider stakeholders as co-producers in intervention design can bring valuable insights that can help reduce research waste. Successful co-production can be complex and challenging and requires careful consideration of the topics participants can most usefully speak to; the representativeness of those involved; the capacity available for participation; and how participants will be compensated. Findings also alert us to the importance of having well-defined, transparent procedures for deciding how stakeholder input will be incorporated.

Sexual & Reproductive Medicine

Health Economics & Outcomes Research

co-production

complex health interventions

evaluation

teenage pregnancy

sexual health

dating and relationships violence

sexual harassment

public health

relationships and sex education

There is much emphasis on the benefit of involving potential recipients, implementers and other stakeholders as co-producers in the development of complex public health interventions, but there exist few practical accounts reflecting on the value and challenges of co-production in intervention design.
We reflect on the practice of involving school staff and students and other youth and practice and policy stakeholders in the development of two multi-component relationships and sex education interventions for English secondary schools.
We demonstrate the potential for improving intervention acceptability and implementation through co-production, while highlighting some of the challenges of eliciting and prioritising stakeholder feedback in intervention design. These findings can usefully inform the work of others carrying out similar activities.

There have been growing calls to pay greater attention to the development of complex health interventions to maximise their potential feasibility, acceptability and effectiveness in a given context prior to costly implementation and evaluation studies. [1–6] Complex interventions are defined as those with multiple interacting components, targeting multiple organisational levels, behaviours and outcomes [5]. As such, they can be challenging to understand and implement [7, 8].

A number of frameworks have emerged to support the development of complex health interventions.[5, 9–19] Most propose a phased, iterative approach involving identifying similar effective interventions and/or mechanisms in the extant literature, developing intervention theory, and pilot-testing delivery models and materials. The importance of stakeholder involvement across phases is also emphasised, with potential beneficiaries and intervention providers viewed as having unique insights into the construction and maintenance of health problems and the local context in which interventions will be delivered [1, 20, 21]. Stakeholders are thus recognised as having a valuable contribution to make as ‘co-producers’ of interventions by, for example, identifying appropriate and relevant intervention aims and content; contributing to the delineation of theories of change; highlighting facilitators and barriers to implementation and acceptability; and identifying potential unintended consequences and ways of addressing these [1, 13, 14, 22].

Increasing interest in co-production in intervention design reflects a broader trend toward greater involvement of policy-makers, practitioners and the wider public in research, motivated by a range of concerns from democratising and improving the transparency of research, to enhancing relevance, quality and uptake in policy and practice.[23–31] Yet while co-production in its initial intended sense implies a level of collaboration and parity in power between researcher and co-producer, the term has come to describe a diverse set of activities ranging from stakeholders merely being informed or consulted about research, through to them having the authority to make decisions and shape its direction and content [30, 32, 33].

Moreover, while there has been much emphasis on the benefit of involving potential recipients, implementers and wider stakeholders in the development of complex health interventions, there are few practical accounts reflecting on the process and value of co-production in intervention design.[25, 34, 35]. Such accounts are critical for furthering understanding of the impact and utility of co-production and for informing practical strategies for carrying out such work. Here we report our approach to involving potential recipients (students and other youth), implementers (school staff) and wider practitioner and policy stakeholders in the co-production of two multi-component, school-based relationships and sex education (RSE) interventions for secondary schools in England: ‘Positive Choices’ and ‘Project Respect’. We reflect on what co-producing with school staff, students, and other youth, professional and policy stakeholders contributed to intervention design and on some of the key challenges and dilemmas we encountered during the process. We also consider the how far we can claim to have involved potential recipients, implementers and wider stakeholders as ‘co-producers’ in our research. The main aim of the paper is to provide a useful account of stakeholder involvement in intervention design to inform the work of others considering carrying out similar co-production activities. Our findings may also be valuable for those planning health interventions for delivery in secondary schools.

Initial intervention design

Positive Choices and Project Respect were both designed as new, rather than replications of existing evidence-based interventions. Design began by defining primary and secondary outcomes, a theory of change and set of components for each intervention based on existing evidence.

Positive Choices aimed to reduce unintended teenage pregnancy (primary outcome). Secondary outcomes included delayed sexual debut, reduced numbers of sexual partners, use of contraception, reduced non-volitional sex and increased educational attainment. Intervention components included: a report for schools on student sexual health needs informed by student surveys; a School Health Promotion Council (SHPC) involving at least six staff and six students to coordinate intervention activities and tailor the intervention to local needs; a teacher-delivered classroom curriculum for year-9 students (aged 13–14); parent newsletters and homework; student-led social-marketing campaigns; and a SHPC-led review of school and local sexual-health services. Training and a manual were included for staff facilitating the council, curriculum and campaigns.

Project Respect’s primary outcome was to prevent dating and relationships violence (DRV). Secondary outcomes included reduced sexual harassment, unintended pregnancy and sexually transmitted infections, delayed sexual debut, reduced numbers of sexual partners, and improved use of contraception, psychological functioning and educational attainment. The intervention comprised: a manual and training for key staff to coordinate intervention activities; training by these staff for other staff on preventing DRV; staff and student mapping of ‘hotspots’ for DRV on site and revision of staff patrols to address these; review of school policies to address DRV; a teacher-delivered classroom curriculum for year-9/10 students (aged 13–15); providing students with the ‘Circle of 6’ app for seeking support when experiencing or at risk of DRV; and parent information about DRV.

Initial design of both interventions was informed by studies of previous interventions reported as effective in promoting various sexual-health outcomes in randomized trials from the USA and Australia. Positive Choices was informed by Safer Choices, a whole-school approach to sexual health [36–38], particularly in terms of the centrality of the SHPC; by the Children’s Aid Society Carrera youth development intervention [39], in terms of the emphasis on viewing RSE as part of broader student development and by the Gatehouse whole-school health intervention, [40] particularly in terms of modifying the overall school environment to support health improvement. Project Respect was informed by the Safe Dates curriculum intervention [41–43] and by the Shifting Boundaries approach to DRV prevention via modifying school environments.[44, 45]

Positive Choice’s theory of change (Fig. 1) was informed by social-marketing theory,[46, 47] [48] models of school change,[49] social influence theory [50] and social cognitive theory,[51] and focused on achieving positive sexual-health outcomes by improving contraceptive knowledge; self-efficacy to communicate about sex; skills and competence; communication at home about relationships and sex; and school-wide social norms supporting positive relationships/sexual health.

Project Respect’s theory of change (Fig. 2) was underpinned by the theory of planned behaviour[52] and the social development model,[53] which informed a focus on challenging student attitudes and perceived social norms about gender, appropriate behaviour in relationships and violence, and promoting sense of control over behaviour. This approach was also supported by reviews which suggest that DRV prevention should both challenge attitudes and perceived norms concerning gender stereotypes and violence, and support the development of skills and control over behaviour.[54]

Initial design also involved consultation with staff from five schools and with young people from the ALPHA (Advice Leading to Public Health Advancement) group, convened by the Centre for Development and Evaluation of Complex Interventions (DECIPHer) based at Cardiff University, involving young people aged 14–25 trained to advise researchers on public-health research.

Involvement of potential recipients, implementers and wider stakeholders in co-production

Overview

Following initial design, research funding was obtained for ‘optimisation’ and piloting of each intervention prior to formal feasibility testing. In this case, optimisation involved the further specification and development of the interventions led by researchers in collaboration with specialist agencies who were to provide each intervention and involved consultation with secondary school staff and students; and other youth and policy stakeholders to produce fully elaborated interventions with materials appropriate for secondary schools in England.

The Sex Education Forum (SEF) was the specialist provider for Positive Choices and the National Society for the Prevention of Cruelty to Children (NSPCC) for Project Respect. Part of the National Children’s Bureau (NCB) charity, SEF advocates and provides resources for delivery of quality RSE in England. The NSPCC is also a charity, focused on preventing child abuse.

Optimisation aimed to involve: review by researchers and SEF/NSPCC of evaluation reports and, where available, intervention materials from the interventions that informed Positive Choices and Project Respect; initial consultation with staff and students from secondary schools in England on intervention content, delivery and materials; drafting by SEF/NSPCC of intervention materials in collaboration with research staff; further consultation with schools, other young people (ALPHA) and policy stakeholders on intervention format and materials; and intervention refinement prior to piloting. In practice, however, as discussed below, the alignment of stakeholder consultation with intervention development was rather messier than this, particularly for Positive Choices.

Consultation with schools

For Positive Choices, initial consultation with students and staff holding a range of roles in one London secondary school was carried out in June 2017 prior to the development of intervention delivery models and materials, which were to be piloted for feasibility and acceptability in the same school from September 2017. The session involved teachers and students from year-8, focused on: acceptability of intervention aims, components and proposed modes of delivery; preferences for the content and format of the student needs report, the manual guiding the intervention; and identifying any perceived challenges to implementation. In the case of Positive Choices, further planned consultation on intervention materials was not possible due to limited capacity for participation from the school.

For Project Respect, consultation involved two sets of sessions at four schools (two in south-east and two in south-west England). The first sessions were conducted in three of the schools in May 2017 and involved a mix of staff and students (Table 1). These focused on acceptability of intervention aims, components and delivery models, and content and format of the intervention, including training, the manual and the curriculum. This was supplemented by a telephone interview with a staff-member at the fourth school.

The second sessions occurred in July 2017, involving staff and students in consultations in three schools. These: sought feedback on draft curriculum materials; explored appropriate terminology for relationships and abusive behaviours; considered the role of social media in the conduct of young people’s relationships and DRV; and identified challenges to implementation.

All sessions were facilitated by two researchers (JC, GM, RM, RP, NT, TT) and a representative from the specialist agency who was to provide the respective intervention, with teachers and students placed into small working groups for discussion following introductions. The Positive Choices and the second wave of the Project Respect sessions were audio-recorded. Field notes were also taken during or directly after each session. Based on this, summary reports were then prepared. Recordings were not transcribed of subjected to formal qualitative analysis. For Project Respect, findings from each region were reviewed and synthesised to identify areas of consensus and disagreement.

Consultation with ALPHA group

Two sessions were held with the ALPHA group in July 2017 and April 2018, to explore young people’s perspectives on parent engagement in the Positive Choices intervention, and the acceptability and potential challenges of implementing student-led social-marketing campaigns in schools. For Project Respect, the ALPHA group were consulted on draft lesson plans in October 2017. All ALPHA sessions were facilitated by trained public involvement officer (PG). RP and HY also attended the first Positive Choices session. All ALPHA sessions were audio-recorded and summaries drafted by the group’s professional facilitator.

Consultation with practitioners and policy-makers

In March 2018, we convened a meeting of sexual health and sex education practitioners and policy-makers from governmental and non-governmental organisations. Following presentations of each intervention, participants provided feedback via small-group discussion on questions set by researchers, focusing on intervention design and practical challenges to implementation. Drawing on facilitator notes, researchers drafted a summary of the event, again with no formal qualitative analysis.

Due to time constraints within the project, this consultation took place towards the end of the Project Respect study, so could not inform optimisation prior to pilot. For Positive Choices, this consultation informed refinement of intervention materials as outlined below.

Ethics

Ethics approval for co-production procedures was granted by the London School of Hygiene and Tropical Medicine research ethics committee on 25th January 2017 for Project Respect and 5th June 2017 for Positive Choices. Students and staff participating in consultations were treated as research participants and provided with written information about the research one week beforehand, as well verbally just prior to the research. Participants were informed that they could stop taking part at any time or choose not to answer any questions. All completed written opt-in consent/assent forms. Parents of participating students were provided with information and could opt out their children.

ALPHA participants gave written consent for their participation as research advisors on DECIPHer affiliated studies and for their contributions to be shared anonymously for all general purposes in relation to DECIPHer’s work. Consultation with practitioners and policy-makers was treated as public engagement rather than research, so ethical review and consent were not sought. Participants were made aware of how their contributions would be used and received a summary of discussion, to which they could suggest amendments.

Incorporation of findings from consultation into intervention design

The summaries prepared for each of the above activities were shared with the specialist provider agencies for each intervention. Providers and researchers discussed the summaries arriving at a negotiated consensus about how these should inform models of delivery and materials.

In the following sections we report the findings from consultations with school staff, students and other youth and policy stakeholders and how these informed intervention design. These are also summarised in Table 1.

INSERT TABLE 1

Consultation with schools

Eight staff and nine students (five girls, four boys) from year 8 (age 12-13) participated in the Positive Choices consultations. Fourteen staff and 66 students (34 girls, 32 boys) from years 9-10 (age 13-15) participated in the Project Respect consultations (Table 1).

INSERT TABLE 2

For both Positive Choices and Project Respect, staff and students generally confirmed the acceptability of intervention aims, approaches and components. DRV, sexual harassment and unintended teenage pregnancy were recognised as salient issues for schools to address.

With Positive Choices, the tailoring of the intervention to student needs and the student-centred approach was particularly welcomed. Several school staff, nevertheless, raised concerns about student responses being used to inform curriculum content because they felt students might be unable to foresee what they needed to learn about relationships and sex. This feedback resonated with our planned approach to ensure a core recommended set of RSE topics were covered by all schools while maintaining a level of student choice by demarcating ‘essential’ lessons, which all schools would deliver, and ‘add on’ lessons, the selection of which would be informed by student data.

Staff and students were positive about Project Respect components. Parent engagement, a classroom curriculum, hotspot-mapping and the Circle of Six app were perceived as appropriate and useful. Project Respect proposed that NSPCC would train key staff who would then cascade training to other staff. Teachers supported this approach as a means of capacity development and limiting need for training cover. However, they also highlighted that it could be difficult for schools to release even these key staff for training and that this needed to be well planned out with each participating school. The scheduling of hour-long curriculum lessons was also highlighted as a potential issue and staff suggested that there was a need for curriculum lessons to be adaptable for split delivery over shorter (usually around thirty minute) tutor-time slots or longer ‘off-timetable’ days. An element of flexibility was built into both interventions, to enable delivery in shorter lessons. However, SEF advised against recommending delivery through single ‘off timetable’ (or ‘drop-down’) days in Positive Choices, even if this might cover the full number of specified curriculum hours or be convenient for schools. Such an approach was considered inconsistent with best practice in RSE delivery where skills, competence and knowledge should be developed and revisited across time (See [55, 56], for example).

With regard to intervention materials, staff in both Positive Choices and Project Respect consultations reported that, because there was so little time for planning for RSE, manuals needed to be comprehensive, containing everything they needed to know but concise, ‘sticking to the essentials’ necessary for delivery. Similarly, teaching staff in Project Respect reported a preference for ‘plug-and-play’ curriculum materials that provided detailed lesson plans, scripts to help guide classroom discussion and PowerPoint slides, so staff with limited confidence, experience or time could deliver an effective lesson. These recommendations posed some dilemmas in Positive Choices, however, they did not align with what is recognised within the sector as best practice for delivery of quality RSE in schools (in which highly trained, competent and confident professionals develop their own curriculum and / or tailor available resources to their context and students). After some discussion, however, researchers and SEF agreed that in order to ensure quality and fidelity of delivery it was a priority that the materials fitted with what was likely to be the current reality of teaching RSE in many English secondary schools, where the subject is afforded little priority, time for preparation is limited and less experienced teachers may be required to deliver.

In contrast some staff, nevertheless, requested flexibility in the curriculum design to allow more experienced staff to adapt activities including where topics had already been covered in earlier RSE provision. Options for adaption were, therefore, built in via suggestions for additional or extension activities beyond what was specified as essential. Excluding core content on the basis it had been previously covered was to be discussed with researchers and the specialist provider and agreed on a case by case basis.

Staff reported they favoured materials provided electronically, and interactive PDF documents where they could easily locate relevant resources from live links. Both interventions were therefore designed with this in mind. A handful of participants, nevertheless, suggested that they still liked hard copies and these were made available at staff trainings.

Although for both Project Respect and Positive Choices, students suggested that RSE should be introduced before year 9 we could not reflect this in our design as this, based on earlier consultation with teachers in the initial design phase, had already become embedded in our agreed study protocols.

Students were adamant that the intervention should reflect the reality of their experiences and recognise their emerging sexuality and involvement in intimate relationships. In terms of curriculum format for Project Respect, they supported proposed pedagogical approaches including the use of role-play and small-group activities particularly for discussing sensitive topics and recreating real life scenarios. Students also indicated that it would be important to cover what might be considered ‘subtler’ or less obvious forms of abuse, such as controlling and coercive behaviours, and emotional abuse. They also highlighted the need for training on how to respond if friends disclose DRV as well as the importance of ensuring that lessons covered the role of social media in DRV and sexual harassment. Staff and students also offered a range of terms to describe DRV and relationships, and suggested that terms should be introduced and defined early in lessons. These suggestions were reflected on and incorporated into the design of curriculum materials as described in Table 2.

In Project Respect consultations students had mixed views about the acceptability of teacher-delivered RSE proposed in both interventions. Some saw benefits to delivery by staff with whom they already had trusting relationships, suggesting this could promote reporting of safeguarding issues. However, students also associated teacher-led delivery with risk of breaching student confidentiality. Lessons led by teachers with whom students had more negative relationships were perceived potentially to compromise learning. Some suggested that an external provider might allow more honest conversations and increase confidentiality. Students suggested that, most importantly, whoever taught lessons should be non-judgmental, able to respect confidentiality and engage with the ‘reality of young peoples’ lives’. Staff explained that in practice the selection of which teachers taught the lessons would largely depend on timetabling and availability.

Across both interventions, teachers proposed that involving outside specialists could usefully cover topics teachers felt ill-equipped to teach, such as sexual violence and female genital cutting/mutilation. Based on these responses for both interventions we opted to continue with teacher delivery as a potentially acceptable model to students, but with clear instruction on the selection criteria for teachers to deliver the curriculum. However, budget constraints meant that we were unable to include an option for specialist external educators to deliver parts of the curriculum in either intervention.

In consultations for both interventions, some students and staff commented that lessons discussing sensitive issues should be taught in single-sex groups. One teacher’s preference for single-sex teaching was rooted in an explicitly feminist desire to provide female students with a supportive environment for active participation, where they would not be marginalised by boys. A suggestion from some staff and students was to teach some content in single-sex classes, but bring groups together at the end of a lesson to share learning. Ultimately, in general single-sex teaching in co-educational settings was not recommended as both interventions were intended to be universal, covering the same topics for all students. Discussion of different views and perspectives across genders was also perceived to be an important part of the learning and to be more reflective of real-life encounters. Preferences to deliver in single-sex classes because of cultural or religious sensitivities were, however, to be discussed with individual schools on a case-by-case basis.

Consultation with students and staff on the most appropriate models of engaging with parents revealed that schools already had a variety of approaches, such as: sending letters home with students; communicating via email, text or social media; posts on school websites; and parent events. While it was agreed adaptable templates to send information home would be useful, participants suggested the content of parent materials should be sensitive to different home cultures and that schools were best placed to tailor these themselves. Templates for parent communication were therefore included in both interventions but precise content and mode of delivery was left to the discretion of the school.

Consultation with the ALPHA group

A total of 12 males and 10 females participated across three ALPHA consultations (Table 3).

INSERT TABLE 3.

ALPHA members were generally supportive of the student-led social-marketing element of Positive Choices, on the grounds that student-led campaigns could ensure messaging was relevant to young people, complementing teacher-led lessons. Participants raised the importance, however, of ensuring campaigns were genuinely student-led and that messages were consistent with the programme aims. This informed our decision to ensure student-led social-marketing groups were accountable to the joint staff-student School Health Promotion Councils (SHPCs).

Participants broadly supported the parent component of the Positive Choices intervention, recognising the value of informing parents about the RSE being taught in school and involving them in supporting their children’s learning at home. Some participants, however, were more sceptical about resources (like homework assignments or newsletters) aiming to prompt conversations at home and felt that many students would avoid carrying out homework activities due to the risk of embarrassment or breaching existing child/parent boundaries. In line with SEF’s intended plan, homework activities were selected with these considerations in mind and aimed to ease into parent/child discussions at home, focussing initially on the universal, relatively less sensitive topic of ‘rites of passage’ progressing to focus on ‘abusive and healthy relationships’ in a later assignment.

In the Project Respect ALPHA consultations, participants supported the use of small group and scenario-based learning activities that enabled students to reflect on ‘real-life’ situations, and the inclusion of same-sex relationships in these. As with the student consultations, ALPHA also raised some concerns about the sensitivity of some of the Project Respect lesson plans and ensuring appropriate support for students who have experienced or witnessed DRV or other abuse. They suggested that, across lessons, greater attention to the use of online and social media in the conduct of young people’s relationships was important. These suggestions were reflected on and incorporated into the curriculum.

Consultation with practitioners and policy-makers

Twenty-three practitioner and policy-maker stakeholders from governmental and non-governmental organisations in the field of education and health attended the consultation event.

Stakeholders anticipated that one of the major challenges to implementation would be ensuring schools prioritised the interventions, given other pressures, and made suggestions to address this. These included: increased engagement with head teachers and/or senior leadership teams; dissemination of programme information to all school staff; seeking ‘buy-in’ from school governors and parents; involving local partners with long-standing relationships with schools, such as those in public-health departments or school networks; and maintaining regular contact with a named strategic lead with enough seniority to drive implementation.

Participants recommended that to ensure school commitment, the researchers should highlight what schools might gain from the interventions: free staff training to support continued professional development; specialist-designed curriculum materials; improved safeguarding procedures; meeting statutory obligations to support students’ social and emotional wellbeing; contributing to meeting national school-inspectorate criteria; and potential for improved pupil attendance and attainment.

Stakeholders also suggested signing service-level agreements with schools, highlighting expectations for intervention providers, schools and researchers. These strategies were incorporated in to the design of Positive Choices, but due to the time at which the stakeholder meeting took place, they were unable to inform Project Respect in the same way prior to piloting.

Summary of key findings

Involving teachers, students and other youth, professional and policy stakeholder in the co-production of Positive Choices and Project Respect brought valuable insight about the potential feasibility and acceptability of our planned approaches to inform the optimisation of both interventions. In some cases, feedback confirmed the acceptability and feasibility of our planned approaches and, in others, helped identify important refinements to enhance applicability to the current English school context. Consultation with schools, ALPHA and practitioner/policy stakeholders supported intervention aims, components and models of delivery. Contrary to much of the existing literature,[57, 58] students confirmed the potential acceptability of teacher-led delivery, but the need for careful selection of which staff delivered lessons. Students and ALPHA also sensitised us to the need to ensure content and materials reflected the reality of young people’s lives, including their digital cultures, which is a common concern in the RSE literature. [58–60]

Consultation with school staff, practitioners and policy-makers highlighted the competing priorities for school leaders’ and teachers’ time and hence their shrinking capacity to implement public-health interventions. This prompted us to develop our materials in line with what school staff felt was workable in this context and to adopt strategies suggested by practitioners and policy makers to ensure school buy-in. The consultations also suggested the need to build flexibility into our intervention design, for example enabling schools to adapt how lessons were timetabled and delivered, and how parents were engaged.

Importantly, including diversity of stakeholder groups ensured participants could speak with authority to different aspects of intervention design. Students, for example, were able to inform us about their preferences for content and delivery, enabling us to improve the relevance of the format and content of our interventions. Teachers provided insight into the current school climate and ‘what would work’ practically in terms of implementation in these settings. ALPHA members drew on their experiences of school and their training as advisors on public-health research to provide authoritative views on intervention design. Practitioners and policy-makers could advise on the broader context of the English education system, particularly relating to securing commitment and ensuring delivery.

Challenges, dilemmas and limitations of co-production in intervention design

While seeking and prioritising stakeholder feedback into intervention design is often presented as being quite straightforward and inherently beneficial to researcher and participant, as we discovered, these processes can also be complex and challenging, and potentially have harmful consequences.[28–30]

We have already noted where tensions emerged between stakeholder views and existing best practice, the budget or the RCT design. Rather than simply absorbing stakeholder feedback, deciding when and how it could be incorporated required careful deliberation among the research team and specialist providers about where it was reasonable and feasible to incorporate stakeholder views.

Such dilemmas raise important questions not only about the extent to which stakeholder feedback can reasonably be incorporated into intervention design when that feedback threatens to undermine the logic of interventions, but also about the skills necessary to manage complex inter-institutional and inter-personal processes to resolve disputes [30].

Whether we achieved a sufficiently diverse cross section of participants in the staff and student consultations is also open to question. In many cases teachers self-selected based on their interest in the topic following invitation from school leaders. In other cases, staff may have been requested to come along to represent their department or school. Although we requested a representatively inclusive and diverse sample of students, in some cases higher-attaining, more articulate students (who are also likely to be more affluent) may have been selected to represent their school. Personal relationships with teachers and, quite simply, which students were available on the day may also have shaped these decisions. This raises important considerations about incorporating stakeholder views that may not be representative of the intended recipients of interventions and consequently about the potential for co-production to lead to equity harms where interventions are developed in line with the cultures and preferences of some groups at the expense of others who may be at more risk.[61]

At times we also experienced some challenges eliciting what we felt were useful responses to some of the questions posed about intervention design, which may have placed limits on the utility of the views expressed. Presentation of the intervention components and theories, for example, generated little discussion or critical commentary among participants, with most simply agreeing they were appropriate, perhaps because such questions appeared to participants to be quite abstract or the to be already fixed. Other topics of discussion on which they were more expert, like the challenges of delivering RSE in schools or whether teachers were appropriate RSE educators, generated more useful insight.

Pressures on school timetables and staff time affected the feasibility of scheduling of face-to-face consultation. Indeed, the potential burden stakeholder involvement in research can place on participants, who may already have very full workloads, has been acknowledged and there is a need to ensure that participants are appropriately acknowledged and compensated for their contributions to co-production.[28, 30]

Finally, acknowledging that ‘co-production’ varies as to the authority possessed by stakeholders,[14, 30] we accept that there are limits to how far we can claim our own approach empowered youth and professional stakeholders to control the process of intervention optimisation.[30, 32, 62, 63]. As such, the aspects of co-production discussed in this paper were necessarily instrumental and researcher-led.[29, 34]. However, the active involvement of specialist provider agencies in the elaboration of both interventions did resemble a more collaborative approach providers drafted the materials and with researchers feeding in to ensure materials aligned with the theory of change and intended outcomes, albeit with the research team leading the work and having ultimate responsibility over decision making as contractors and owners of any new intellectual property.

Implications for further research

Our work suggests that consultation must first consider what questions and aspects of intervention design require the involvement of stakeholders, and which participants can most usefully speak to which questions. Careful consideration over the selection of participants to ensure diversity of views and experiences are accounted for in intervention design is also important.

In introducing co-production, it is important ethically to be transparent and clear to participants from the outset about what their role involves, the how far their contributions can be incorporated into intervention design and under what circumstances this may not be possible. Researchers should be prepared and trained to enable different participants to express their views, to represent them fairly and sympathetically in reports, and to give fair and due consideration to these different views in deciding if and how to act upon them.

Co-production also needs to take account of the potential for additional burden on participants and ensure that participants are acknowledged and adequately recompensed for their contributions. In school research specifically, steps need to be taken to build flexibility into timelines for intervention design (and to encourage funders to allow this) to take account of the current pressure on school timetables. A range of consultation methods is also essential to ensure that stakeholders can contribute in other ways besides face-to-face meeting. Employing multiple methods could also help to increase representation of different views and ensure all participants feel able to voice their concerns. This could include the use of anonymous consultations with broader groups using online Delphi methods, for example[64].

Finally, it is also important to consider the potential for the involvement of intended recipients to go beyond passive consultation to have more of an active, empowered partners in the design process. This depth of involvement may give greater assurances of the relevance of intervention aims, approaches and materials, but will bring its own challenges in terms of stakeholder burden and how to balance power in decision making to ensure interventions are locally relevant and context specific, while maintaining the opportunity to build on evidence-based approaches.

In conclusion, involving the potential providers and recipients as co-producers in intervention design can yield valuable insights that can maximise the applicability of interventions to local settings before they are formally piloted and evaluated. This can reduce research waste and provide assurances that any subsequent lack of observed effect is less likely to be down to intervention failure. However, these processes can be complex and challenging and successful co-production requires careful consideration of the topics participants can most usefully speak to; the representativeness of those involved; the capacity available for participation; how participants will be recompensed; and depth of involvement required to meet intended co-production aims, including any emancipatory ones. From the outset, well-defined, transparent structures and procedures for deciding how stakeholder input is to be prioritised and incorporated are also essential. Without such consideration the positive impact of co-production on intervention design or individuals themselves not be guaranteed.

ALPHA - Advice Leading to Public Health Advancement

Centre for Development and Evaluation of Complex Interventions (DECIPHer)

DRV – Dating and Relationships Violence

NCB – National Children’s Bureau

NSPCC – National Society for Prevention of Cruelty to Children

PC- Positive Choices

PR – Project Respect

RSE – Relationships and Sex Education

SHPC – School Health Promotion Council

SEF – Sex Education Forum

Ethics approval and consent to participate

Ethical approval for collection of the data on which this article is based was granted by the London School of Hygiene and Tropical Medicine on 25^th January 2017 for Project Respect and 5^th June 2017 for Positive Choices.

This manuscript does not contain any individual person’s data in any form.

Consent for publication

This manuscript does not contain any individual person’s data in any form.

Availability of data and materials

The dataset supporting the conclusions of this article are available on request.

Competing interests

The authors declare that they have no competing interests

Funding

This paper presents independent research funded by the National Institute for Health Research (NIHR) in England under its Public Health Research Board (14/184/02 and 15/03/09). The views expressed in this publication are those of the authors and do not necessarily reflect those of the National Health Service (NHS), the NIHR or the Department of Health for England.

Authors' contributions

RP led on data collection for Positive Choices, contributed to data analysis and interpretation, and drafted the paper. RM and JC led on data collection for Project Respect and contributed to analysis and interpretation of the data. SB contributed to data collection, analysis, and interpretation of data for Positive Choices. LE led on Positive Choices intervention development for SEF. TT, GM and NT contributed to collection, analysis, and interpretation of data for Project Respect. PG and HY led on work with ALPHA. RC contributed to the design and led on the Project Respect study for the SW of England. CB conceived of and led on both studies, and contributed to the drafting of the paper. All authors commented on and approved the final version of the manuscript.

Acknowledgements

We are grateful to all participants for their contributions to intervention design. We are also grateful to Kathryn Oliver who provided comments on an earlier draft of the manuscript.

Moore, G.F. and R.E. Evans, What theory, for whom and in which context? Reflections on the application of theory in the development and evaluation of complex population health interventions. SSM - Population Health, 2017. 3: p. 132-135.
O'Cathain, A., et al., What can qualitative research do for randomised controlled trials? A systematic mapping review. BMJ Open, 2013. 3(6): p. e002889.
Hardeman, W., et al., A causal modelling approach to the development of theory-based behaviour change programmes for trial evaluation. Health Education Research, 2005. 20(6): p. 676-687.
Craig, P., et al., Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ, 2008. 337: p. a1655.
MRC, Developing and evaluating complex interventions:. 2008, Medical Research Council: Available online.
Fletcher, A., et al., Realist complex intervention science: Applying realist principles across all phases of the Medical Research Council framework for developing and evaluating complex interventions. Evaluation (London, England : 1995), 2016. 22(3): p. 286-303.
Power, R., et al., Developing complex interventions for rigorous evaluation—a case study from rural Zimbabwe. Health Education Research, 2004. 19(5): p. 570-575.
Sermeus, W., Modelling processes and outcomes in complex interventions, in Complex interventions in health: and overview of research methods, D.a.H. Richards, IR., Editor. 2015, Routledge: OXon. p. 408.
Collins, L.M., S.A. Murphy, and V. Strecher, The multiphase optimization strategy (MOST) and the sequential multiple assignment randomized trial (SMART): new methods for more potent eHealth interventions. American journal of preventive medicine, 2007. 32(5 Suppl): p. S112-S118.
Collins, L.M., et al., A strategy for optimizing and evaluating behavioral interventions. Annals of Behavioral Medicine, 2005. 30(1): p. 65-73.
Michie, S., M.M. van Stralen, and R. West, The behaviour change wheel: A new method for characterising and designing behaviour change interventions. Implementation Science, 2011. 6(1): p. 42.
Wight, D., et al., Six steps in quality intervention development (6SQuID). Journal of Epidemiology and Community Health, 2016. 70(5): p. 520-525.
De Silva, M.J., et al., Theory of Change: a theory-driven approach to enhance the Medical Research Council's framework for complex interventions. Trials, 2014. 15(1): p. 267.
Hawkins, J., et al., Development of a framework for the co-production and prototyping of public health interventions. BMC Public Health, 2017. 17(1): p. 689.
NICE, Behaviour Change at Population, Community and Individual Levels. NICE Public Health Guidance. 2007., National Institute of Health Research: London.
O’Cathain, A., et al., Taxonomy of approaches to developing interventions to improve health: a systematic methods overview. Pilot and Feasibility Studies, 2019. 5(1): p. 41.
Levati, S., et al., Optimisation of complex health interventions prior to a randomised controlled trial: a scoping review of strategies used. Pilot and Feasibility Studies, 2016. 2(1): p. 17.
O'Cathain, A., et al., Guidance on how to develop complex interventions to improve health and healthcare. BMJ Open, 2019. 9(8): p. e029954.
Turner, K.M., et al., Understanding successful development of complex health and healthcare interventions and its drivers from the perspective of developers and wider stakeholders: an international qualitative interview study. BMJ Open, 2019. 9(5): p. e028756.
Popay, J., What will it take to get the evidential value of lay knowledge recognised? International Journal of Public Health, 2018. 63(9): p. 1013-1014.
Popay, J., et al., Theorising Inequalities in Health: The Place of Lay Knowledge. Sociology of Health & Illness, 1998. 20(5): p. 619-644.
Bonell, C., et al., ‘Dark logic’: theorising the harmful consequences of public health interventions. Journal of Epidemiology and Community Health, 2015. 69(1): p. 95-98.
Oliver, K., T. Lorenc, and S. Innvær, New directions in evidence-based policy research: a critical analysis of the literature. Health Research Policy and Systems, 2014. 12(1): p. 34.
Ponsford, R., et al., Competing for space in an already crowded market: a mixed methods study of why an online community of practice (CoP) for alcohol harm reduction failed to generate interest amongst the group of public health professionals at which it was aimed. Implementation Science, 2017. 12(1): p. 91.
Boaz, A., et al., How to engage stakeholders in research: design principles to support improvement. Health Research Policy and Systems, 2018. 16(1): p. 60.
Oliver, K., et al., Broadening public participation in systematic reviews: a case example involving young people in two configurative reviews. Research Synthesis Methods, 2015. 6(2): p. 206-217.
GREENHALGH, T., et al., Achieving Research Impact Through Co-creation in Community-Based Health Services: Literature Review and Case Study. The Milbank Quarterly, 2016. 94(2): p. 392-429.
Foutaki, M., Co-Production Under the Financial Crisis and Austerity: A Means of Democratizing Public Services or a Race to the Bottom? Journal of Management Inquiry, 2015. 24(4): p. 433-438.

Beresford, P., User Involvement in Research and Evaluation: Liberation or Regulation? Social Policy and Society, 2002. 1(2).
Oliver, K., A. Kothari, and N. Mays, The dark side of coproduction: do the costs outweigh the benefits for health research? Health Research Policy and Systems, 2019. 17(1): p. 33.
Berriman, L., Howland, K, Recipes for co-Production with Children and Young People, in Researching Everyday Childhoods: time technology and documentation in a digital age., L.B.a.S.B. R. Thomson, Editor. 2018, Bloomsbury: London. p. 139-162.
Arnstein, S.R., A Ladder Of Citizen Participation. Journal of the American Institute of Planners, 1969. 35(4): p. 216-224.
Cornwall, A., Unpacking ‘Participation’: models, meanings and practices. Community Development Journal, 2008. 43(3): p. 269-283.
Vennik, F.D., et al., Co-production in healthcare: rhetoric and practice. International Review of Administrative Sciences, 2016. 82(1): p. 150-168.
Clarke, D., et al., What outcomes are associated with developing and implementing co-produced interventions in acute healthcare settings? A rapid evidence synthesis. BMJ Open, 2017. 7(7): p. e014650.
Basen-Engquist, K., et al., School wide effects of a multicomponent HIV, STD and pregnancy prevention program for high school students. Health Education and Behavior, 2009. 28(2).
Coyle, K., et al., Safer choices: reducing teen pregnancy, HIV, and STDs. Public Health Reports, 2001. 116(Suppl 1): p. 82-93.
Coyle, K., et al., Short-term impact of safer choices: a multicomponent, school-based HIV, other STD, and pregnancy prevention program. J Sch Health, 1999. 69(5): p. 181-8.
Philliber, S., et al., Preventing pregnancy and improving health care access among teenagers: an evaluation of the Children’s Aid Society-Carrera Program. Perspectives on Sexual and Reproductive Health, 2002. 34(5): p. 244-251.
Patton, G., et al., Promoting social inclusion in schools: group-randomized trial of effects on student health risk behaviour and well-being. American Journal of Public Health, 2006. 96(9): p. 1582-1587.
Foshee VA, B.K., Ennett ST, Fletcher Linder G, Benefield T, Chirayath Suchindran MS. Am J Public Health.19-24., Assessing the long-term effects of the Safe Dates program and a booster in preventing and reducing adolescent dating violence victimization and perpetration. 2004. 94(6): p. 19-24.
Foshee VA, B.K., Arriaga XB, Helms RW, Koch GG, Linder GF. , An evaluation of Safe Dates, an adolescent dating violence prevention program. American Journal of Public Health, 1998. 88(1): p. 45-50.
Foshee VA, B.K., Ennett S, et al. . Prevention Science Assessing the effects of the dating violence prevention program “safe dates” using random coefficient regression modeling. 6, 2005. 245: p. e58.
Taylor BG, M.E., Stein N. , Effectiveness of ‘Shifting Boundaries’ teen dating violence prevention program for subgroups of middle school students. Journal of Adolescent Health, . 2015. 56(2): p. S20e6.
Taylor BG, S.N., Mumford E, Woods D. 2013;14:64-76., Shifting Boundaries: An experimental evaluation of a dating violence prevention program in middle schools. Prevention Science., 2013. 14: p. 64-76.
Hastings, G. and M. Stead, Social Marketing, in Health Promotion Practice, M. Macdowall, C. Bonell, and M. Davies, Editors. 2006, Open University Press: Maidenhead.
Simiyu Wakhisi, A., et al., The effectiveness of social marketing in reduction of teenage pregnancies: a review of studies in developed countries. Social Marketing Quarterly, 2011. 17(1): p. 56-90.
Fletcher, A., et al., Interventions addressing the social determinants of teenage pregnancy. Health Education, 2008. 108(1): p. 29-39.
Marsh, D., et al., Building Effective Middle Schools: A study of middle school implementation in California schools. 1988, Los Angeles: University of Southern California.
Fisher, J.D., Possible effects of reference group-based social influence on AIDS-risk behaviors and AIDS. Am Psychol 1988. 43: p. 914-20.
Bandura, A., Social Foundations of Thought and Action: A Social Cognitive Theory. 1986, Englewood Cliffs NJ: Prentice Hall.
Ajzen, I., From intentions to actions: A theory of planned behavior., in Action-control: From cognition to behavior., J. Kuhl and J. Beckman, Editors. 1985, Springer: Heidelberg, Germany. p. 11-39.
Hawkins, J.D. and J.G. Weiss, The social development model: An integrated approach to delinquency prevention. Journal of Primary Prevention, 1985. 6: p. 73-97.
Wolfe, D.A. and P.G. Jaffe, Emerging strategies in the prevention of domestic violence. Domestic Violence and Children, 1999. 9(3): p. 133-144.
Forum, S.E. Principles of good RSE. 2019 [cited 2019 11th September 2019]; Available from: https://www.sexeducationforum.org.uk/sites/default/files/field/attachment/A4%20RSE%20POSTER%20SCHOOLS%202017_0.pdf.
Pound, P., et al., What is best practice in sex and relationship education? A synthesis of evidence, including stakeholders’ views. BMJ Open, 2017. 7(5): p. e014791.
Pound, P., R. Langford, and R. Campbell, What do young people think about their school-based sex and relationship education? A qualitative synthesis of young people's views and experiences. BMJ Open, 2016. 6(9): p. e011329.
David, P.A.a.m., Get real about sex: The politics and practice of sex education. 2007, Maidenhead: Open Univeristy Preaa and mcGraw Hill Education.
Allen, L., Closing Sex Education's Knowledge/Practice Gap: The reconceptualisation of young people's sexual knowledge. Sex Education, 2001. 1(2): p. 109-122.
Whittington, E.a.T., R. , Educating for Consent: Beyond the Binary’, in Critical Pedagogy, Sexuality Education and Young People: Issues about democracy and active citizenry, F.a.Y. Sanjakdar, A. (eds), Editor. 2018, Peter Lang: New York.
Lorenc, T. and K. Oliver, Adverse effects of public health interventions: a conceptual framework. Journal of Epidemiology and Community Health, 2014. 68(3): p. 288-290.
Ocloo, J. and R. Matthews, From tokenism to empowerment: progressing patient and public involvement in healthcare improvement. BMJ Quality & Safety, 2016. 25(8): p. 626-632.
Bate, P. and G. Robert, Experience-based design: from redesigning the system around the patient to co-designing services with the patient. Quality & safety in health care, 2006. 15(5): p. 307-310.
van Urk, F., S. Grant, and C. Bonell, Involving stakeholders in programme theory specification: discussion of a systematic, consensus-based approach. Evidence & Policy: A Journal of Research, Debate and Practice, 2016. 12(4): p. 541-557.

[Due to technical limitations, table 1 could not be displayed here. Please see the supplementary files section to access the table.]

			Positive Choices	Project Respect
				Wave 1		Wave 2^a
				South-east England	South-west England	South-east England	South-west England
Year-8	Girls		5	0	0	0	0
Year-8	Boys		4	0	0	0	0
Year-9	Girls		0	6	2	6	5
Year-9	Boys		0	3	4	6	6
Year-10	Girls		0	5	4	6	0
Year-10	Boys		0	6	1	6	0
Total students		Girls	5	11	6	12	5
		Boys	4	9	5	12	6
		All	9	20	11	24	11
Staff			8	6	3	4	2

^a In Project Respect, some of the wave 2 participants had also participated in wave 1

Table 2. School consultation participants

Age in years	Positive Choices		Project Respect
Age in years	Girls	Boys	Girls	Boys
14	2	1	0	0
15	3	2	0	1
16	1	1	0	0
17	1	0	2	2
18	1	3	0	1
19	0	1	0	0
Total	8	8	2	4

Table 3. ALPHA Participants

Download PDF

Version 1

posted

You are reading this latest preprint version

Impact guaranteed? Involving potential recipients, implementers and wider stakeholders in the co-production of two school-based relationships and sex education interventions to prevent teenage pregnancy and dating and relationships violence

Status:

Version 1

Abstract

Figures

Contributions to the literature

Background

Methods

Initial intervention design

Involvement of potential recipients, implementers and wider stakeholders in co-production

Overview

Consultation with schools

Consultation with ALPHA group

Consultation with practitioners and policy-makers

Ethics

Incorporation of findings from consultation into intervention design

Results

Discussion

Summary of key findings

Challenges, dilemmas and limitations of co-production in intervention design

Implications for further research

Conclusion

Abbreviations

Declarations

References

Tables

Status:

Version 1