This paper presents the results of a unique clinical ethics support evaluation study. By implementing structural Ethics Reflection Group (ERG) sessions about the use of coercion at seven Norwegian departments within three different mental health care institutions, we tried to study changes over time. This study has been designed because of recent insights from coercion reduction studies. These studies demonstrated that changing staff’s normative attitudes regarding the use of coercion and reducing a paternalistic department culture is a key for both reducing the use of coercion and for executing coercive measures in a better (…) way. In order to study changes over time, we asked at three subsequent moments how employees thought about: the use of coercion, the competence of the team regarding the handling of coercion, the way they involved patients and families in situations of coercion, the team cooperation, and the way they handled disagreement in their team. In order to do so, we performed a repeated cross-sectional survey at baseline, after 12 and after 24 months of implementing ERG sessions about moral challenges related to coercion (T0-T1-T2). Of the in total 817 respondents only 8% (N=62) responded at all three points in time. In order to use all observations and to estimate the missing values, we used a mixed model analysis which is a well-known statistical procedure [83]. This method enabled us to incorporate all available observations, including those from participants with repeated measures, for whom dependency of these repeated measures is taken into account. However, the results of this study should be interpreted with caution.
After presenting our main results we will give our interpretation of the results. Furthermore, we will briefly summarize and interpret the specific correlation between participating in the ERG sessions and presenting a case at the one hand and the seven parameters at the other hand, including the role of departments and professions there in. Subsequently, some methodological and normative reflections on the results will be briefly discussed. Finally, we will reflect upon the lessons-learned with this research design and the used methodology, ending with some recommendations for future evaluation research regarding outcomes and changes over time due to clinical ethics support.
Main results regarding change over time
Despite the fact that ERG is a complex intervention and that its evaluation is even more complex, and that we had only limited longitudinal data, we were able to find several significant changes over time during the implementation of ERG sessions. In the multivariate analyses, taking all predictors into account, we found that the extent to which all respondents agreed that coercion can be seen as Care and Security decreased over time. This concerns a relatively small decrease of the extent in which respondents agreed that coercion could be seen as protection in dangerous situations (see Textbox 1). We also found that the extent to which respondents reported that they involved patients and families increased over time. For example by discussing with the patient if and how they experienced coercion before and now, or by asking patients what kind of alternatives for coercion employees should try (first) (see Textbox 3). Finally, respondents agreed stronger with items resembling the constructive handling of disagreements in the team (see Textbox 5). For example, they reported that they focused more on different opinions or views on treatment issues rather than on disagreement between persons. They also reported more respect for each other’s viewpoint, even when they disagreed.
Interpretation of main results regarding change over time
These three significant changes during the implementation of ERGs could have been caused by various factors of which we are not aware of. For example, teams and team leaders could have changed and\or severe coercion or team incidents might have an impact on how respondents think about coercion, user involvement and the constructive handling of disagreement. Furthermore, the more general attention for coercion during the implementation of the ERG sessions (e.g. via other projects and educational programs) could have influenced changes in culture and attitudes. Yet, when looking at the specific intervention of implementing ERG sessions on coercion for two years, changes could also be very well caused by the ERG sessions on moral challenges related to coercion. For example, the slight decrease for Care and Security could have been caused by a more nuanced deliberation on the use of coercion with the ERG sessions. Perhaps, due to the ERG facilitator and the specific focus of the conversation method within the ERG sessions, ERG participants wondered more whether the use of coercive measures actually prevents, or in some cases even could contribute to, a dangerous situation. Or, because of the ERG sessions, perhaps respondents learned more about alternative actions for protecting care and security. The increase of User Involvement could have been caused by the fact that due to the ERG sessions, respondents were urged to think more actively on a) the values and norms of patients and families in situations where coercive measures were used, and b) how to take their perspectives into account. Finally, the increase of Constructive Disagreement within the teams can be related to the fact that within ERG sessions both a respectful dialogue and learning from different and opposing viewpoints is at the core of the ethics reflection.
Results and interpretation of the results specifically related to Participation in ERG
For participation in ERG we found one significant change over time within the seven outcome parameters: those who participated in ERG six or more times each year, perceived coercion clearly more as Offending. Repeated ethical reflection groups about the use of coercive measures can have made these respondents extra aware of the potential offending character of coercion and possible alternatives for the use of coercion.
Results and interpretation of the results specifically related to presenting a case in ERG
Those who presented their case in ERG more than 2 times a year, ended up with seeing coercion also as significantly more Offending. Perhaps those who presented their moral challenges regarding the use of coercion within the ERG session experienced already some initial moral concerns which made them present a case within the ERG sessions. And perhaps, during the participation in ERG, they became extra aware of the fact that, and in which way, the use of coercion can be seen as offending. Finally, those who presented their case in ERG more than 2 times a year, ended up with lower scores for User Involvement. Perhaps, due to the ERG sessions, they started to realize that they knew relatively little about how to involve patients and families and what their specific values, norms and perspectives are with respect to the use or the prevention of coercion.
We found more significant changes over time for the other parameters (due to Participation in ERG and Case presentation in ERG) yet they did not remain statistically significant after adjustment for departments and professions within the statistical analyses. This could be an indication for the fact that the initial significant changes over time can be better explained by differences between departments and professions rather than differences in ERG participation or Case presentation itself. Therefore, we will now specifically summarize and interpret the results for the various Departments and Professions.
Results and interpretation of the results for Departments
We found several significant differences among departments when looking at changes over time. We observed both a decrease and an increase of seeing coercion as Offending (for respectively Community Care and Rehabilitation at the one hand and Acute Care of hospital 2 at the other hand). A possible explanation for the decrease can be that some health care professionals in community and rehabilitation care felt they waited too long with not using coercion (e.g. due to the fact that health care law does not allow the use of coercion outside the hospital). Using coercion too late might even cause harm. In contrast, health care professionals working at Acute Care departments might use coercion more often, sometimes even as a routine. So participation in ERG sessions could make them more aware of the offending nature of coercion. Furthermore, respondents from Rehabilitation perceived an increase of Team Coercion Competence regarding the handling of coercion. One explanation could be that since coercion if often not that much used at the Rehabilitation department, the deliberation about specific coercion cases in ERG sessions made them think they became more competent in handling coercion. With respect to User Involvement, Acute Care of hospital 3 demonstrated a decrease while Acute Care of hospital 2 and Specialist Care had an increase. Respondents from Youth Care and Rehabilitation thought their Team Cooperation improved. Finally, Youth Care thought their Constructive Disagreement improved.
Most of the hypotheses mentioned in the Introduction were actually realized for at least some of the seven involved departments. These significant changes and the differences among the various departments can be explained in many ways yet it is difficult to know how plausible the explanations are. Perhaps the departments already had very different points of departure concerning their normative attitudes regarding coercion and user involvement, including different cultures for team cooperation and the handling of disagreement, when entering the study. Furthermore, the various patient categories on each department can explain differences in perception and evaluation of coercion. Also, the amount trainings and courses related to the use of coercion might vary among departments.
Results and interpretation of the results for Professions
We found several significant differences among professions when looking at changes over time. When compared with the group of ‘psychiatrists and related medical professions’, psychologists perceived coercion significantly more as Offending and managers significant less. Overall, when compared with ‘psychiatrists and related medical professions’, it seems that psychologists are more critical about the use of coercion, and managers less critical[1]. This could be explained by the fact that usually ‘psychiatrists and related medical professions’ have the final responsibility for and should decide about the use of coercion; psychologists usually not. Furthermore, psychologists are perhaps trained differently in ways to take care for patients and how to manage conflicts or possible dangerous situations (i.e. more relational, less focused on interventions and use of medicine). Managers are perhaps more distanced from the actual context in which coercion is used.
Psychologists and ‘psychiatrists and related medical professions’ reported they involved patients and families more (often) when compared with what nurses and ‘other professions’ reported. A possible explanation could be that nurses and ‘other professions’ already have more frequent and intensive contact with patients and next of kin than ‘psychiatrists and related medical professions’, hence they did not experience that they involved them more than before.
Finally, the fact that Psychologists were slightly more critical about Team Cooperation and experienced less Constructive Disagreement could perhaps be correlated with the somehow weaker hierarchical position of psychologists when compared with ‘psychiatrists and related medical professions’.
Overall interpretation of changes over time: response shift & normative evaluation
Above we learned that not only studying ERG as intervention and evaluating changes over time are complex matters, we also learned that the interpretation of changes over time in the responses of the respondents can be complex. Changes over time and lack of changes over time could be explained by the fact that the phenomena under study actually changed during the time of this study or by various kinds of ‘response shift’. During the many ERG sessions on coercion, respondents’ idea about what coercion stands for and how one understands for example team disagreement can have changed. To make it even more complex, the way the concepts and the questions within the survey at T0, T1 and T2 are interpreted can also change. ‘Response shift’ was defined by Sprangers and Schwartz [84] as a change in the meaning of self-evaluation of a target construct. Response shift can be caused by: a) a redefinition of the target construct (i.e. reconceptualization of what coercion actually means or how one should interpret ‘Offending’); b) a change in the respondent’s values (i.e. reprioritization of importance of domains substituting the target construct); or c) a change in the respondent’s internal standards of measurement (scale recalibration). There are possibilities to check and calculate whether there is actually a response shift yet give the data set of this study this was not possible here [see 8.5.6 in 85]. Besides response shift explanations, it could also be the fact that specific routines or procedures for how to use coercive measures at the various departments have been changed during the study. This would imply that when asking about coercion two years later, we in fact evaluate another clinical practice of coercion than at the beginning of the study.
Another precaution concerns the way in which one normatively interpret changes over time. This of course applies to drawing normative conclusions based on empirical results in general [86], yet this certainly applies to research in which researchers aim to study both changes in normative attitudes and the normative value of outcomes or impact of ethics support services (e.g. ERG or MCD sessions). Drawing normative conclusions, e.g. whether a specific result or outcome of this study can be interpreted as morally better or as a moral improvement, is a complex matter [35]. For example, given the initial hypotheses of this study, it sounds perhaps plausible that seeing coercion as more offending, after two years of critical reflection on moral challenges regarding coercion, could be seen as a desirable and hence, morally, good result. Yet, after deliberation in ERG, and finding good ways of performing coercion in a more transparent and respectful way, respondents perhaps also realized that coercion can be performed in a less offending way. In order to draw normative conclusion when interpreting the results of this study, one needs complementary qualitative data (e.g. thick descriptions of specific situations in which employees uses coercion & together with respondents carefully studying how to interpret and judge the specific situation). Finally, as mentioned in the Introduction section, one should not automatically conclude that eventually positive outcomes of CES also become the primary goal of or justification for CES. Stimulating ethics reflection by means of implementing ERGs or MCDs has a value in itself. Despite the value and importance of CES evaluation studies in general; participating in ERGs and MCDs should not get instrumentalized as an intervention in which the only aim is to reach specific outcomes because this would threaten the inherent intellectual and normative freedom of ethics reflection.
Relationship with other ERG or MCD impact evaluation studies
As mentioned earlier, this study took place within a much larger study in which also qualitative analyses of transcribed focus groups were organized with some of the respondents at every department, at both T0 and at T2 [55]. In line with some of the significant changes or trends in the findings of this paper, respondents reported at T2 that ERGs increased their awareness of various examples of formal and informal coercion and that they learned to challenge ‘problematic’ concepts, attitudes and practices regarding coercion. This is in line with significant changes in respondents’ normative attitudes towards coercion which is described in this paper. Furthermore, Hem and colleagues mentioned respondents reported that they improved their professional competence and confidence, a greater trust within the team, and more constructive disagreement and room for internal critique (i.e. less judgmental reactions and more reasoned approaches) [55]. This resembles the significant changes in the Constructive Disagreement scale within this paper. Yet, this is not confirmed by changes in the Team Cooperation scale in this paper.
Furthermore, in a recent systematic literature review in which 25 empirical papers on evaluation of ERG or MCD were analysed in order to identify various impacts of ERG, Haan and colleagues found a change in one’s professional opinion or attitude and a more critical attitude towards professionals’ practice. Again, this resembles our findings in which respondents became more aware of and more critical towards the use of coercion. According to Haan et al, most reported changes took place on a personal and inter-professional level [23]. For example, health care professionals felt more feelings of relief, relatedness and confidence. They also mentioned that their understanding of the perspectives of colleagues, one’s own perspective and the moral issue at stake increased. In particular, Haan and colleagues mentioned that several studies found that ERG or MCD reduced conflicts and leads to more solidarity, respect, tolerance, collegial support and cooperation. Again, these findings resemble the changes in Constructive Disagreement which we found in our study. Finally, Haan et al reported about MCD participants being more aware of patients’ and families’ rights in decision-making processes and thinking more about patients’ and families’ perspectives, wishes, and needs. This is in line with the significant increases for User Involvement in this study. Yet, at the same time, Haan et al concluded that empirical evidence of ERG’s or MCD’s concrete impact on the (improvement of the) quality of patient care is limited and is mostly based on self-reports [23]. This clearly sets the agenda for future CES evaluation studies.
Strengths and limitations of the study
There are several strengths and certainly limitations of this study. A unique strength is the fact that this study focuses on changes over time after two years of ERG or MCD at seven different departments within three different hospitals by means of indirect observations of reported answers (i.e. not by asking respondents how they perceived changes over time themselves). Another strength is that this study combines a specific clinically relevant topic (i.e. the use of coercion in mental health care) with more general evaluative measures of CES. The latter are potentially interesting for other CES evaluation studies. Also, the fact that we were able to compare the findings of this cross-sectional survey study with the focus group interview study and the experiences we had when working closely with the hospitals contributed to a better understanding of the quantitative data. The choice for measuring changes over time with respect to normative attitudes, team cooperation and constructive disagreement fits well with the theoretical understanding of ethics support, in particular ERG and MCD. Finally, we learned about how to develop and execute a specific research design and methodology. The latter also applies to the limitations of this study.
The few actual longitudinal data stresses the importance of guiding and monitoring the response rate more intensive in future evaluation studies. The mixed model analyses helped us in this respect yet more longitudinal data is preferable. Furthermore, studying change over time within seven different departments in three different hospitals made it difficult to relate the changes over time to the ERG or MCD sessions themselves. Finally, despite the significant changes over time, the differences between no, little or much ERG participation were generally small in absolute terms. The fact that most respondents only participated in a few ERGs over two years could also play a role here: perhaps more participation in more frequent ERGs will contribute to stronger changes over time. A meagre comfort is perhaps that when measuring changes over time after the implementation of a complex intervention one always is confronted with serious methodological challenges [43]. According to Craig et al [87], a lack of finding effects of a complex intervention may perhaps more reflect implementation and methodological challenges rather than genuine ineffectiveness of the intervention. Yet, this is still something we need to explore. Experiences with and results of these kind of explorative studies on the impact of CES might pave the way to new study designs with control groups and some sort of randomization in combination with qualitative research methods. Learning about the actual contribution of clinical ethics support to a better health care is important; for researchers, for health care professionals, for ethics support staff, and for patients. For, despite the intrinsic value of participating in ethics support activities, clinical ethics support inherently aims, and should aim, at actually improving clinical practices.
Footnote:
[1] This finding is also reported in another paper from the overall PET study. In a national Norwegian survey among various mental health care professions, psychologists were more critical towards the use of coercion than psychiatrists [77].