The results are structured along the seven induced normative evaluation criteria.
The discourse project’s objectives for stakeholder engagement were clear, as set out in the internal documents (see Figure 1). The topic of the discourse approach is novel and has, to date, not been discussed in the German context. A nuanced and comprehensive approach appears inevitable when addressing a new medical technology which will affect not only providers, counselors, consumers but also society as a whole. This is why an iterative approach was chosen with varying opportunities and depths of engagement (see Figure 1) embracing the widest possible inclusion of organizations as well as political entities considered stakeholders on the topic of dementia prediction.
With the methodology described above, the developed normative evaluation criteria and related key concepts and keywords (see Table 1) were formulated to guide the analysis. In the analysis, explicit and implicit references to the normative evaluation criteria were identified in the corpus of material (see Figure 1). These were assessed and are elucidated in the following section.
1) Depth of participation
The first criterion can rather be understood as an overall description of participation in the stakeholder procedure and thus a criterion embracing all following criteria.
One uniting thrust in all material was the raised necessity to deal with the topic of dementia prediction before its clinical implementation, stressing the relevance of the project initiation and recognizing the unresolved issues needing public and scientific attention. Scientific publications called the discourse approach a sophisticated and elaborate procedure which addressed a wide spectrum of participants and of engagement options over an extended period of time.
In a separate step, apart from the stakeholder conference, with the aim of broadening the spectrum and recipients of engagement, a social media strategy[1] was implemented by the project team which also allowed the public to participate on the topic debate. The aim was to create a discursive platform on which questions were posed to Facebook and Twitter users on ethical, social, and legal aspects of dementia prediction. Further, an essential aspect of the discourse was to engage future counseling professions. This was pursued in subsequently conducted seminars and workshops and the sustainable development of teaching materials.
While a wide range of stakeholders was addressed and invited to take part in the discourse and, no organization with a government mandate actively participated in the discourse by means of a position paper. Only at the podium discussion following the stakeholder conference did Lower Saxony’s Minister for Social Affairs, Health and Equality highlight the importance of such a conference to bring everyone to the table and to incorporate different perspectives.
2) Legitimate selection of stakeholders
From the project’s self-evaluation it became clear that the process of identifying potential stakeholders was a highly iterative one and an important step for the entire project. With the phase of thorough research on relevant organizations in Germany and the creation of an extensive list of the initially identified organizations, a first step was initiated to open a wide selection of stakeholders. This process was accompanied by a critical examination of the list by colleagues and acquaintances who are scholars in the field of dementia and could act as multipliers or mediators in the process of identification. In a subsequent step, the list was proposed to the project’s advisory board. We asked the members of the advisory board to examine the list carefully for necessary additions or alterations. In this process the aim was to achieve a balanced representation of fields, political parity, and few exclusive experts who, for example, had previous experience in committee work.
Based on the state of knowledge of previously successfully conducted discourses on bioethical topics and after consultation with the project’s advisory board, the project team decided to activate only those organizations and institutions that claim to represent social interests or have a relevant governmental regulatory mandate to participate. A meaningful participation procedure including individuals as experts would have overstrained the logistical capacities of the project and risked bias of individual interests or ties being put forward in the process that cannot be justified. However, due to pragmatic reasons, we had to expand the selection to some extent, as two organizations suggested individuals who did not act as named representatives of the organization. This led to the creation of different levels of representation in two cases where organizations were less monolithic and rather heterogenous regarding the organization’s structure and thematic perspective.
Further, the project team made clear in its summaries, protocols, and its self-evaluation of the project that the identified organizations should submit a position paper from their organization’s perspective which served as a condition for participation in the stakeholder conference. This aimed at identifying how the respective organization perceived its own stake in the discourse topic and which issues were most relevant from their perspective. Here there was also space for participants to formulate demands and expectations or wishes to the stakeholder conference, the entire discourse or further discourses in the future. Obtaining the position papers required immense communication coordination and perseverance on our side with continuous contact by post, email, and telephone.
After 29 position papers were submitted and carefully examined concerning the content, those organizations were invited to take part in the stakeholder conference who had expressed their willingness to participation and had named a concrete representative. The strategy of early commitment, encouraged by the project’s advisory board, served a form of motivating certain underrepresented fields to feel addressed and motivated to bring their perspective into the discourse.
Overall, the reflection in the self-evaluation of the project showed that, for the phase of stakeholder identification, the securing of transparency of the selection of participants (especially regarding organizations with potential conflicts of interest such as, for example, the pharmaceutical industry or private insurers), the insuring of balance of positions and results, the address of local levels of dialogue, and the sharing and dissemination of results to connect stakeholders were declared aims in the entire discourse procedure. It was set out that any declarations of bias should be required from all participants to avoid potential conflicts of interest. At the beginning of the project, the project team hesitated regarding the decision on whether to invite government organizations. However, due to the fact that our project was funded by the government, this affected our decision to invite governmental bodies. This could be very different for discourses with other sources of funding.
3) Representativeness
From the beginning of the discourse, it seemed inevitable to engage a heterogenous compilation of representatives to be able to formulate accurate recommendations, especially based on the topics addressed and discussed in the position papers as well as in the joint statement. The project team decided not to include individual experts as far as possible to strengthen legitimacy. However, while good representation can help strengthen legitimacy, it does not guarantee it.
In a few position papers, the procedural step of engaging various stakeholders was highlighted to jointly cooperate and develop best practices as well as standards for the field of diagnostics and for the validation of biomarkers. In several positions papers, organizations referred to the multidisciplinarity and interdisciplinary networking in their own organization or to multidisciplinarity being a driving factor in their organization’s policy work. This was linked to their expectations of the working environment and approach of the discourse at hand. Further, organizations described certain attributes in the position papers including their work in a competent, comprehensive, and multidisciplinary manner to the needs of those affected.
Assessment in the position paper, the joint statement, and the commentaries by stakeholders show that the multidisciplinary composition of stakeholders was appreciated and commended. Especially, in the majority of the evaluation forms, the diversity and combination of perspectives as well as the interdisciplinary exchange and approach were perceived very positively. Others highlighted the respect of minority opinions in the process and the opportunity of good networking and exchange across fields. However, in one evaluation form it was mentioned that some representatives of associations ultimately did not act as representatives but acted as individual expert representatives. According to formal documentation including the position papers, the joint statement, project steps, protocols of project meetings, however, only two people actually took part in the stakeholder conference who wanted to be considered “individuals” and not as representatives of the organization they worked for.
Looking more closely at the aspect of representation, there were some perspectives that were missing in the discourse. As reflected in the self-evaluation of the project, the perspectives of health economics and the general practitioners were missing in the first phase of the obtained written statements. During the conference, there was no personal representation of patient advocacy due to last-minute cancellation because of illness. According to the self-evaluation of the project, it would have been desirable for these perspectives to be represented more prominently in the discourse. In the end, it was possible to methodically introduce the perspective of those affected in the further procedure. The detailed position paper of the German Alzheimer’s Society found resonance in the joint statement and subsequently the German Alzheimer’s Society and the German Society for General and Family Medicine expressed their explicit support for the joint statement in the commentary phase. Nevertheless, the inclusion of the health economic perspectives remains an important task for future discourse processes on the issue at hand as the financing of predictive biomarkers was a crucial issue in the joint statement. In the discourse, organizations focusing on the health economic perspective did not show interest in participation.
Further, as mentioned above, none of the organizations invited who had a government mandate actively participated in the discourse. With the exception of the Federal Ministry of Family Affairs, Senior Citizens, Women and Youth which did not actively take part in the stakeholder conference but took a clear stance towards the discourse project. It outlined its positive expectations towards the output of recommendations that should serve as a stepping stone for further measures to be taken. In the phase leading up to the stakeholder conference, the Ministry applauded the project’s accomplishment due to the wide-ranging selection of stakeholders and representation as well as the members of the project’s advisory board. The letter also referred to the ministers working on the German national dementia strategy as one of the stepping stones for dissemination.
As taken from the self-evaluation of the project, the project team was very satisfied with the breadth of interests represented in the discourse process. Also, newspaper reports commented on the heterogenous composition of stakeholders during the conference and as authors of the joint statement. Further, the advisory board in its project commentary valued that the project managed to reach a wide spectrum of actors.
The project design had features of useful redundancy, allowing multiple entry points for organizations to engage in the discourse according to organizations readiness and ability (see Figure 1). With time, the project became more well-known through media coverage and a snowball principle. This had the effect that organizations saw the need to also participate and bring their perspective into the discourse.
4) Process management
The material of the project steps, the protocols of project meetings, and the self-evaluation of the project showed that the time frame of the discourse procedure was optimized with the incorporation of participatory phases; the encouraged position papers led to an early discussion of the organizations with the topic which enabled a nuanced but goal-oriented discussion at the conference itself. The establishment of thematic working groups at the conference helped structure and make the discussion and joint writing possible in the available time frame.
From the project team’s perspective, much effort was invested in setting of deadlines, communication, and sending of reminders to reach project milestones (see Figure 1). This included the initial step of inviting identified stakeholders to take part in the discourse with a position paper and committing to attendance at the stakeholder conference as well as in the editing phases and the commentary phase. A strict timeline and time slots between the different participatory phases made it possible to achieve such a process in the dedicated time frame. Regular meetings of the project team allowed for concise planning and also revision of planning. The process management was made possible with two full project positions over the extent of two years.
Regarding the stage of research, the importance of engaging on a discourse early on for a topic that has not manifested its implications in practice yet, was stressed in the self-evaluation of the project as well as by participating stakeholders in the evaluation forms. As can be taken from the evaluation forms, the majority of stakeholders felt well informed about the project at all times and in all evaluation forms, stakeholders expressed that they had received adequate information in preparation for the stakeholder conference. Timely provision of information was a crucial aspect determining the successful implementation of the discourse.
The explicit comments on a procedural level focused on the difficulty given the complexity of the topic and the wish to have more time to specify statements and definitions of terms. Also, in the commentary by advisory board, the advisory board emphasized that the stakeholder conference and the result of the joint statement highlighted that further discourses are needed to address unresolved issues. Thus, the initiated discourse, can be regarded as a successful starting point for further discourses.
5) Output
References to thematic output
The material, especially the project steps, protocols of project meetings, and the self-evaluation of the project showed that the overarching aim of the stakeholder conference was to achieve a consensus statement. The second aim was to make the landscape of interests and demands transparent and to define the frame of reference, in which the discourse procedure can take place.
From the protocols of project meetings, it became evident that at all times, the project team took queries, criticism, and praise seriously. Regarding the procedure, numerous loops of more clearly defining information leading up to the stakeholder conference were taken to address the issues put forward by participating stakeholders.
The positions papers included a concrete discussion of the critical aspects of dementia prediction from their organization’s perspective which made it possible to efficiently frame the topics to be discussed at the stakeholder conference. First, organizations were encouraged to think about and deal with the topic in advance. Second, this allowed the project team to prepare adequately by condensing the 29 position papers’ content and preparing three thematic working groups. The positions papers were sent to all conference participants in advance to enable a form of exchange before the actual conference.
Regarding the timely addressing of the issue at hand, many position papers addressed topics that needed clarification before the medical technology was actually applied in clinical practice. For the patient-relevant use of biomarkers, it was further put forward by the participating stakeholders in the position papers that the arising ethical aspects and arguments should be revealed transparently and systematically as well as transported to other stakeholders to allow for a comprehensive assessment of the technology. This could also apply to the entire discourse at hand.
As becomes evident from the scope of the position papers, the joint statement and also the commentaries by stakeholders as well as the scientific publications and reports on the project, the discourse has accomplished an analysis of the topic put up for discussion which, however, is not concluded in its entirety. The discourse was able to spark discussion and attention and also to develop crucial areas relevant to health policy, restructuring of counseling, and public perception or dealing with dementia. Certain issues were touched upon but not resolved which concurs with the initially set out aim of the discourse procedure. Numerous topical points were addressed in the joint statement with the need of further exploration and discussion regarding concrete implementation.
Also perceived limitations to what the discourse or at least the stakeholder conference could accomplish were formulated in some of the commentaries by stakeholders, taking up points that should be further discussed in a societal discourse. As can be taken from one interview, sustainable solutions are more likely to emerge from open discourses than from leaving such discussions to individual experts.
References to appreciation and support of the discourse
In some of the positions papers, the author(s)/organization expressed gratitude for being able to take part in the discourse or and in several positions papers the discourse approach with a wide discussion was actively welcomed.
In other position papers, support of dealing with the topic of dementia prediction was expressed and participation of specific organizations was recommended that were found imperative to include such as the German Alzheimer’s Society and the German Ethics Council. Further, some organizations stressed why it was important that they themselves were included in the discourse.
The advisory board in their commentary on the entire discourse procedure expressed praise over the project’s thorough and critical approach. The commentary reflected the interaction and active participation in the project. The advisory board applauded the highly ambitious aim of reaching potentially all relevant stakeholders in Germany with the discourse and added that the success of the discourse was not guaranteed.
Overall, the evaluation forms by stakeholders show that the stakeholder participants were satisfied with the stakeholder conference and the engagement process. Further, in all EFs stakeholders stated that they would take part in such a discourse on medical-ethical topic again.
References to expectations and demands
In the SPs, expectations to the process were raised regarding the need for discussion on certain aspects of prediction that were missing up until the discourse was initiated or needed further attention and dealing with. In some of the SPs, the authors referred, on the one hand, to an inclusive and appreciative approach aimed at catering to the needs of those affected and, on the other hand, to an approach dealing with the topic of dementia prediction as such. This allows to make inferences about the expectation of identifying such attitudes in the discourse process itself.
The Federal Ministry of Family Affairs, Senior Citizens, Women and Youth, as mentioned above, did not actively take part in the discourse but took a stance towards the discourse project and outlined their expectations on the discourse in a letter. The Federal Ministry welcomed the initiated discourse through the German Federal government and described its expectations towards the output of recommendations that should serve as a stepping stone for further measures to be taken and applauded its success due to the wide-ranging selection of stakeholders and the representation as well as the members of the project’s advisory board.
Further, the letter referred to the ministers working on the German national dementia strategy and the possibility of linking outcomes.
In a SR, the project aims were described as to stimulate a social discourse and accompany the process of medical progress. In a few cases, organizations used the format of the SPs to demand certain professions be equipped with more competencies from which people with dementia could potentially profit and also specifically regarding biomarker tests.
Dissemination of output
The evaluation forms by stakeholders state that the majority of stakeholders wanted to introduce certain objectives into the procedure. Further, in the majority of the valuation forms it was stated that the stakeholders were able to gather new information that is relevant for their field of work through the discourse including specific knowledge about the issue at hand, perspectives from other professions, and processes of health policy work. The interest in and perceived importance of moving the discussions of the stakeholder conference forward were strongly expressed in the evaluation forms highlighting ideas for dissemination of the results of the stakeholder discourse.
As can be taken from the self-evaluation of the project, the project team pursued the goal of producing a joint consensus statement with the initiation of the engagement process and specifically with the stakeholder conference. The chosen methodology in thematic working groups during the stakeholder conference took into account the plurality of perspectives. The participants were able to identify and clearly state areas of consent and also of dissent at any time. The commentary phase made it possible for a multitude of organizations to present or highlight points of consent and support or of dissent and controversy on the produced content of the joint statement. Especially controversial points could be presented with different arguments and in their plurality.
The results of the discourse were disseminated at various points in scientific conferences and at public project events such as a press conference and the final conference of the project. The project team always activated press representatives for important steps in the procedure contributing to a flow of dissemination during the entire approach. The project aims and results were taken up by multiple news agencies, organizations, and other institutions. Some of the participating organizations made an effort to present the project on their channels and beyond. Importantly, the project was mentioned in the report of the German National Dementia Strategy referring to the procedure as an important step for a societal discourse on dementia prediction.[2]
The project team reflected early on at the beginning of the project, which kind of follow-up project could be initiated. It became clear from the beginning that the focus should lie on counseling approaches and the development of guidelines.[3]
6) Role definition
The material shows only few references to role expectations of the discourse organizers and moderators. The advisory board made clear in its commentary, which was also described in the project steps, protocols of project meetings, and the self-evaluation of the project, that its primary purpose is to consult the project team ranging from the initial identification and selection of stakeholders to dissemination of the project results.
In the self-evaluation of the project, the project team states its task to transparently steer, moderate, bring together and reflect on the scientifically-based, result-oriented and participatory discourse and, if necessary, to modify or adjust project steps and participatory phases. On the content level, the project team is editorially responsible for obtaining representation of relevant organizations, all project results as well as for the formulation of ethically reflected recommendations for political decision-makers. Project reporting and other forms of publication also are declared as part of the project team’s responsibility. Furthermore, the project team sees its responsibility in initiating follow-up research and supporting the continuation of the discourse (e.g. through dissemination, teaching, input into specialist discourses such as the German National Dementia Strategy).
According to the project steps, protocols of project meetings, and the self-evaluation of the project, the project team also sees it as its task to promote the discourse in the general public. For example, a population survey conducted by the AbbVie Healthcare Monitor placed a number of questions on dementia prediction in its survey. A further measure was social media engagement including Facebook and Twitter, enabling an exchange of discussions on questions the project team provided on the prediction of dementia and current information on the project. The general social media uptake can be considered moderate.
The role of the editing team of the joint statement was a politically sensitive task. The editing team was appointed by the stakeholders present at the conference with the assignment to finalize the joint statement without significant changes to consented essence of the composed statement to that point.
During the planning of and preparing for the discourse, it became an immense issue to reflect and define our own roles researchers and moderators of the discourse. As the project team, we were also accountable to the sponsor of the project which promised an output but at the same time needed to let stakeholders drive the content and consensus procedure.
7) Transparency
Overall, the project team clearly documented all procedural steps. My research interest in evaluation was supportive of this. The underlying premise of the entire project based on disclosure of interest-driven exertion of influence and openness to the discourse as a condition of participation.
Regarding the stakes of participating organizations, the project team asked the authors of the position papers in the first question to make clear how or to which extent their organization dealt with issues of dementia and with this, elucidating their stake in the discourse. Participating organizations specified in which form they approached the topic of dementia and people affected by dementia including relatives and partners.
Further, by inviting organizations to answer questions and send the position papers in advance to the stakeholder conference, the participating organizations dealt with the topics from their organization’s perspective. All position papers addressed issues that were later taken up during the conference. Although some papers may have indulged in the topic in a deeper fashion, all organizations in advance positioned themselves towards the issue put forward in the discourse approach.
Regarding the procedure leading up to the stakeholder conference, it can be taken from the project steps, protocols of project meetings and the self-evaluation of the project that all material and information relevant to participating organizations prior to the stakeholder conference as well as after including all position papers, the joint statement and the commentaries by stakeholders was made publicly available on the project’s website striving for procedural and content-related transparency.
However, in the evaluation forms by stakeholders, it became evident that some information provided in preparation for the stakeholder conference and at the beginning of the conference was missing for some stakeholders. This mainly referred to definitions of terms distinct to the topic of dementia prediction, for example, the term ‘prediction’ or that the target group for the clinical application of blood-based biomarkers included people with early symptoms of cognitive impairment but also asymptomatic people. We as the project team should have been more precise with definitions and should have consulted experts on these in advance.
For the production of a joint statement, the working groups served to produce topical texts, the plenary sessions were used to reach a form of consensus on the results while at the same time identifying and denominating areas of dissent. Directly after the stakeholder conference the content of the joint statement was presented at a public panel discussion at which also Lower Saxony’s Minister for Social Affairs, Health and Equality took part.
However, regarding transparency of the editing procedure of the joint statement, one organization in the commentary phase used its commentary to highly criticize that changes were made to the consensus-version by the editing team that strongly went against this organization’s perspectives. The issue regarding specific formulations in the joint statement was resolved in an additional round of editing and was made transparent in the discourse.
Following the stakeholder conference, all stakeholders were invited by the project team to comment on the joint statement. This commenting phase was opened not only to the stakeholders who participated during the stakeholder conference, but to all organizations originally identified and invited to take part in the discourse. This procedure, supported by the project advisory board, made it possible to present or highlight points of consent and support or of dissent and controversy in retrospect. Especially controversial points could be presented with different arguments and in their plurality.
As can be taken from the self-evaluation of the project, the discourse procedure, which guided heterogeneously acting and thinking stakeholders to agree on a common position, reflects both value-oriented interests (e.g. protective standards) as well as particular interests (e.g. naming responsibilities for counseling processes and further health-policy interests). This can partially be explained by the fact that the discourse included organizations representing particular interests due to the professional background and mission of the organization. None of the organizations participating at the stakeholder conference had a government mandate to mediate interests. For external readers, it must therefore be communicated transparently which stakeholders were involved in the joint statement. Due to the participating stakeholders’ heterogeneity and solution orientation, they were able to find overarching value-oriented statements. From the project team’s perspective, this is an indicator for the qualitative added value of this method, namely to sound out concrete consensus on standards. The procedure including the editing in the aftermath of the stakeholder conference and also the commentary phase, required much coordination, allowed for further consensus and support to be reached.