Study design
The prioritisation process involved multiple stages and methods. These included identification of a broad range of stakeholders, face-to-face meetings to generate initial research questions, and a one-day workshop to prioritise the research questions generated. The overall process was based on a modified Nominal Group Technique (NGT) [20-22]. The NGT is usually conducted with homogenous groups [23].
Stage 1: Generation of preliminary research questions
The first stage involved the identification of key stakeholders within the GW4 Alliance institutions, e.g. academics with expertise in pharmacy and pharmacology, medical and other healthcare professionals, and patient and public involvement representatives i.e. health service users and organisational representatives. In August 2018, a face-to-face stakeholder meeting was convened (by the corresponding author), to undertake a modified Nominal Group Technique (NGT) [20-22]. Participants were provided background information and asked to address thequestion: ‘What are the priority topics/areas that need to be addressed so that medicines optimisation can be realised?’. During the meeting, participants, including the research team, were encouraged to generate as many questions as possible in response to the research question; these were recorded as individual written responses and collated on flip-charts. No discussion was permitted until the generation process was complete. Discussion then followed for the purpose of clarification of questions, removal of duplicates and the identification of common themes. Following the meeting, the questions were reviewed by the research team and refined to produce a distilled list of research questions for consideration in Stage 2. Each question was assigned to one of four categories that reflected common themes: ‘patient concerns’, ‘polypharmacy’, ‘non-medical prescribing’ (NMP), and ‘deprescribing.
Stage 2: Consultation with wider stakeholder group
The purpose of Stage 2 was to seek input from a wider stakeholder group regarding the original research questions identified in Stage 1. In addition to Stage 1 participants, an email invitation was sent to 80 individuals identified from relevant literature and policy documents and via the professional networks of the core research team and which included a wide range of local, regional, national, and international stakeholders (e.g. pharmacists, academic pharmacists, physicians, National Health Service (NHS) Trust directors, patients, physicians, health workers, and advocacy organisations (including Age UK and the Patients’ Association)). All questions from Stage 1 were presented in a document, using the four categories, and emailed to all prospective participants. The task for participants was to rank the Stage 1 questions according to their perceived importance, and to add new questions from their own ideas/experiences. For each original question, a mean rank was calculated using the Excel rank function. Additional questions suggested by participants were sense checked with duplicate questions removed or combined and then assigned to one of the original four question categories according to its content.
Stage 3: Final Prioritisation
Stage 3 comprised a one-day prioritisation workshop in November 2018, held on University of Bath campus and facilitated the lead author (MCW). All respondents in Stages 1 and 2 were invited. Participants and the research team were purposively assigned to one of four groups comprising eight individuals, to ensure that each group included a range of participants, e.g. at least one lay representative, and representatives from each stakeholder group.
The research questions in each of the four categories were discussed within each group with each category being assigned 45-minute discussion session. Discussion included the opportunity to reflect on the mean rank of questions from Stage 1.
Following discussion session, the participants rated the importance of each question. TurningPoint software was used, which facilitates live polling and as well as the curation and simple statistical analysis of results [24]. Each question was presented alongside a Likert scale, ranging from one (‘extremely important’), to seven (‘extremely unimportant’). Participants rated each question independently. The process was then repeated for three remaining categories of research questions.
Following the meeting, the questions were presented in ordered rank to derive a definitive list of prioritised research questions (Additional file 1; a list of all 92 questions alongside their rank, how they were rated by PPIs and non-PPIs, and the percentage of ‘extremely important’ and ‘important’ ratings).
Patient and public participants received a participation fee and their travel expenses were reimbursed. Non-PPI participants had their travel expenses reimbursed if requested, but received no additional payment for their involvement.
Data analysis
A Borda count [25, 26] was used to rank the order of questions prioritised in Stage 3, where the Likert rating ‘extremely important’ was given a weighting of 7, ‘important’ a weighting of 6 and so forth, to ‘extremely unimportant’ weighted as 1. Following weighting, the number of times a question was rated as ‘extremely important’ was combined with the number of times it was rated as ‘important’ etc. For example, if 19 people rated a question as ‘extremely important’, eight rated it as ‘important’, and five rated it ‘somewhat important’, then its overall weighted score would be (19 x 7) + (8 x 6) + (5 x 5) = 206. Weighted question totals were subsequently ranked according to median score.
Using the process described above, the top five questions prioritised in each topic area by PPI participants were compared with the rank given to the selected questions by all other participants (designated as non-PPI). The purpose of this comparison was to determine whether substantial differences existed in the type of question that both types of participants prioritised. Such information provides greater insight into the heterogeneity of different stakeholder types. During ranking exercises, participants often rank their most and least favourite choices, based for example on familiarity with concepts, therefore middle rankings may reflect more arbitrary or indifferent choices [27]. As such, the five highest ranked questions are discussed as this number will likely capture the broad range of what participants have actively considered as most important, while allowing sufficient coverage of the overall question set.