In this paper, we describe PPI in the construction of a national, web-based case vignette survey for studying men’s view on their participation in decision-making about PSA screening.
Involving relevant stakeholders, including PPI representatives, in the development of research projects has been repeatedly shown to ensure that, e.g., the survey to be deployed is relevant, functional and acceptable (15, 17). Therefore, we chose to employ a health care user-centered design and an iterative process throughout different phases of the survey development. We require future survey participants to both respond to questions about real life facts as well as hypothetical questions.
The approach of asking participants to familiarize themselves with imagined patient scenarios necessitates thorough exploration into what would be common interpretations of constituents of the vignette variants. In this regard, PPI proved very valuable. Repeated feedback from PPI representatives during stages of the survey development ensured a continuous grounding in users’ understanding and allowed for a gradual improvement of vignette scenarios and questionnaire items. Representatives simultaneously provided the project group with a greater insight into the issue under study and with a better rapport with the community by expressing perspectives that reflected different but first-hand knowledge (30). Unsurprisingly, this was especially the case with the patient panel. By way of a simple example, we could delete considerable amounts of superfluous information that we had believed to be important to a layperson. Likewise, representatives repeatedly identified an unfortunate overtone of draft questionnaire items that could lead study participants to respond unintentionally negatively. As a result, we realized the necessity for reversing the formulation of items derived from previously validated instruments (25, 31). More in-depth analyses, critical reflections, and discussion gradually came up and verified that spending sufficient time is a critical factor in PPI (16, 30). It is crucial to spend sufficient time with PPI members in order to build a good rapport, so that they feel sufficiently safe to challenge potential preconceived assumptions of researchers about the material under development (30).
Overall, through working with PPI representatives, we aimed to increase response rates of our future survey by keeping it as short as possible, adjusting language to meet the user perspective, and by including information about estimated time to complete the survey in the invitation material (32). Besides, PPI served to warrant the survey’s acceptability among potential respondents. Respect for other people's time and well-being itself demands prior discussion with representatives of the target population before launching a large-scale survey. The latter pertains not least to a survey that illustrates scenarios on cancer management with fatal outcomes that may cause emotional distress and where some study participants may be real life cancer patients.
Dependent on the time of involvement, lay people also may help prioritize research topics, thereby offering important input into research plans that sometimes involve very abstract ideas (15, 33, 34). In our project, collaboration with the patient panel naturally provided a format especially suitable for discussing patient relevant research questions. In this regard, it is noteworthy that panel members perceived variability in patient preferences regarding use of medical tests to be a particularly important issue.
Apart from requiring some more time and research team resources, we identified no negative effects from PPI in our research design. It was our impression that following engagement, PPI representatives drawn from the public setting as well as from the patient organization became very committed. In this regard, it is noteworthy that although time spent on participation is important for PPI members regardless of how long it takes, the first group received no means of remuneration (16, 34). Anyway, we considered that participation demanded relatively little time and offering reimbursement of any kind would appear cumbersome and inconvenient. For the patient panel, more time was required, and it felt natural to offer a small gift as well as meals during the workshop.
In research with PPI, apart from explaining its purpose, providing a description of methods used for involvement, and its results, GRIPP 2 principles recommend critical reflection on what could be improved (15). Considerable steps in designing our research project had been taken before PPI was undertaken. Our study was driven by a conceptual interest into how participation of health care users in decisions about PSA may influence their perception of health care. From the beginning, this greatly influenced our study scope and we did not involve patients or the public in these stages of our survey development. By way of an important example, asking for PPI comments on research questions that are largely pre-defined by the project group may seem a little backwards. In part, though being a poor substitute for the patient voice, we attempted to account for this through initially obtaining ‘third party’ feedback and advice during the academic workshop (16).
When involving users in research, it is recommended that diverse groups are reasonably represented (16). We aimed to ensure diversity regarding age and socioeconomic background in the large group of PPI representatives by recruiting men from various settings and our large group PPI composition suggests some heterogeneity was achieved (see table 2). Collaboration, however, necessitates that potential users want to participate. Some men chose not to participate, although they could potentially have provided particularly relevant feedback. Ensuring participants with a broad range of health literacy in a study regarding preferences for involvement in health care decision making might be important, as health literacy might influence their impressions of and suggestions for survey development. In our study, we did not assess PPIs' levels of health literacy. However, we sought to ensure that various educational backgrounds were represented. Even though this was succesful in the large group of PPI representatives, diversity was less in the small patient panel with two members being academics. To our knowledge, no PPI participant had severe, incurable cancer. Involving such men might be difficult, however, lacking their perspective could be problematic, as we have little knowledge about their perception of the fatal scenarios of the survey. We tried to partly account for this by particularly discussing these scenarios with panel members who themselves had gone through the distress associated with cancer, well knowing that they felt they had been successfully treated.
It is possible that more men could have participated in the large PPI group to have two or more members review every one of the 30 vignette variants. Our approach reflects a compromise. Then again, it must be kept in mind that vignette components are identical across scenario combinations. As a result, three participants reviewed each of 10 variations in vignette descriptions of patient participation, and 10 participants reviewed each of three possible outcome types.
Some limitations regarding PPI representatives’ feedback on the hypothetical questionnaire scenarios deserve mention. Comments during survey development indicated that they could identify with the patients in the situations described in vignettes, but some points must be kept in mind. Case vignettes have been previously used in studies of health care decision-making in prostate cancer (21, 35), however, it is important to emphasize that participant responses to vignettes reflect hypothetical judgements and not real-life behavior. Hence, we cannot rule out that responses might have been different in a factual situation. It can be very difficult to relate truly realistically to a critical situation that one has never experienced. In the context of discrete-choice experiments, it has been argued that patients’ preferences for choosing hypothetical scenarios may differ from their preferences for making actual treatment decisions (‘hypothetical bias’) (36). This caveat seems contradicted in a few studies comparing actual choices with stated preferences (36, 37). Again, it should be noted that studies mostly have looked at decision-making from the health provider perspective. Overall, there is a scarcity of research concerning the external validity of methodologies applying hypothetical choices (38). Furthermore, in our survey, respondents are passive witnesses to the course of health care described without any personal influence on the decision-making. Respondents only influence their judgement of health care as expressed in the attached questionnaire. Alternatively, respondents could have been granted the possibility to ‘participate’ in the mock up scenario e.g. at the step of deciding whether to have the PSA test done. Constructing the survey in such a way that it is possible for the respondents to decide whether to have the PSA test done might be an attractive approach in future studies. However, this would require additional steps to ensure the necessary number of respondents for each of the 30 different scenarios and achieve a satisfactory statistical power. Another potential limitation of our study is the failure to include women. As women may have an influence on men's health behaviours and health care choices, it may be important to know what the partners of men think about participation in decision-making. However, including only men in our study reflects a decision to sharply focus on health care users’ perspectives regarding their own healthcare matters.