Control Group Design in a Complex Intervention Study: Challenges, Dilemmas and Possible Solutions

The use of a control group is one of the most critical components of an RCT. The control conditions may change over time and include assessment interviews and standard stroke treatment. Therefore, the control group should be monitored and described in as much detail as the intervention group. It is important to find ways to reduce the risk of study-induced influence on the members of the control group. The aim of this study was to explore the possible influence of the assessment interviews on the adjustment of the members of a control group in an RCT exploring psychosocial well-being following stroke.


Abstract Background
The use of a control group is one of the most critical components of an RCT. The control conditions may change over time and include assessment interviews and standard stroke treatment. Therefore, the control group should be monitored and described in as much detail as the intervention group. It is important to find ways to reduce the risk of study-induced influence on the members of the control group. The aim of this study was to explore the possible influence of the assessment interviews on the adjustment of the members of a control group in an RCT exploring psychosocial well-being following stroke.

Methods
Fifteen participants in the control group of the RCT, six women and nine men, aged 29-88 years, were interviewed in narrative semi-structured interviews. Ricoeur's interpretation theory guided the analysis.

Results
The perceived influence of the assessment interviews on the control group varied considerably. Two different themes with subthemes were identified, describing the influence of the assessment interviews in the control group. Theme one described how participants emphasized the perceived influence of the assessment interviews that served as a safety net, enhanced their awareness and understanding, facilitated their adjustment after stroke, encouraged them to seek support, and allowed them to vent their disappointment of having been allocated to the control group. Theme two described participants' experiences of handling their adjustment process on their own without describing any influence of the assessment interviews on their condition. These participants highlighted mild strokes, spontaneous recovery, setting own goals, support from family and friends and supporting research.

Conclusions
In view of the design challenges in RCTs, it seems important to explore in depth how to design assessment interviews with control group members without introducing risk of bias, and to uphold rigor and stringency in the trials.

Background
The principal reason to use a control group in a randomized controlled trial (RCT) is to eliminate alternative causal explanations, and to reduce the potential influence of extraneous variables and sources of variance which are separate from the effect of the intervention under investigation (1,2).
The challenge is to ensure minimal or no influence on the control group (3).
The use of a control group is one of the most critical components of an RCT (1), and several factors might influence and threaten the RCT's internal validity. The control group members' beliefs and expectations may have a significant influence on the outcome of an intervention (4). Their motivation for participation, treatment preferences and the potential disappointment of being allocated to the control group may result in feelings of resentment and annoyance (5). If participants randomized to a control group are very disappointed, it may raise the question of whether information given before consent has been reasonably balanced (6). Behavior change may be induced when participants are informed about the trial (3), and behavior might also be modified in unanticipated ways during the very act of collecting data (7).
Control conditions in trials seem especially challenging when including participants with cognitive impairments (1). Stroke survivors' capacity to process information and make judgments may be reduced. This may influence their ability to understand information provided, and to make informed decisions. The control conditions in complex intervention studies can be as complex as the intervention being evaluated, and may also change over time (8). Thus, the control group should be monitored and described in the same level of detail as the intervention group, to identify and diminish potential risks of study-induced bias.
The design challenges mentioned above can be illustrated and discussed in relation to a multicenter, randomized, controlled trial that tests a dialogue-based psychosocial intervention aimed at promoting the psychosocial well-being and adjustment of stroke survivors (9). In this study we hypothesized that stroke survivors in the intervention group would experience significantly higher levels of psychosocial well-being and lower levels of depressive symptoms and anxiety (measured by GHQ-28), than stroke survivors in the control group at six and twelve months post-stroke. We also hypothesized that stroke survivors in the intervention group would experience significantly higher levels of sense of coherence (measured by SOC-13) and health-related quality of life (measured by SAQOL-39), than stroke survivors in the control group at six and twelve months post-stroke (9).
Participants were recruited from 11 acute stroke or rehabilitation units in South-Eastern Norway between November 2014 and November 2016. The randomized controlled trial (RCT) included 322 stroke survivors, 166 of whom were randomized to the intervention group and 156 to the control group. The controls received standard stroke treatment. Both in the intervention group and in the control group, data were collected by means of a standardized test battery (Appendix 1) at baseline (T1), at six months post-stroke (T2) and twelve months post-stroke (T3) (9). However, contrary to our hypothesis, no significant differences between the intervention group and the control group were demonstrated on the outcome measures, neither at six months nor at twelve months post-stroke (10,11). Therefore, an in-depth analysis of the control conditions involved might reveal possible factors contributing to understand these non-significant results.
Specially trained nurses and occupational therapists performed the assessment interviews. The data were collected face-to-face, in individual, structured assessment interviews, mainly in the participants' homes. The data collectors underwent training including a technical component, i.e. the use of a web-based questionnaire on iPad, an electronically available test battery, practical information with a written training package combined with individual training, guidance and follow-up when needed. The training included specific instructions and procedures in case participants answered affirmatively about suicidal thoughts. The data collectors were instructed to adhere to the questions of the test battery, and to administer the questions in the designed, standardized order, but they were not instructed to refrain from dialogue with the participants.
A process evaluation alongside the trial was conducted to gain an in-depth understanding of the participants' experiences of being in the control group and of their adjustment process after stroke (9). Substantial psychosocial problems following stroke are common (1,9,(12)(13)(14)(15), and adjustment involves hard psychosocial and physical work. Mental strategies, cognitive adjustment and meaning making are key elements (16), and include plans, tasks, actions and coping efforts initiated by stroke survivors to manage their illness (17). Adjustment is an ongoing, evolving dynamic and cyclical process in which patients evaluate their intra-and inter-personal coping responses, and modify them (17). The control group participants' reflections about their condition may have changed as a result of the interviews and the questions raised. These factors might have affected their adjustment, and thus potentially affected the internal validity of an RCT. Thus, the aim of this study was to explore the possible influence of the assessment interviews on the adjustment of the members of a control group in an RCT exploring psychosocial well-being following stroke.

Participants and recruitment
The participants who were invited had participated in the control group, had sufficient cognitive functioning to provide informed consent and to participate, and they understood and spoke Norwegian. People with moderate to severe dementia, serious physical illness in addition to the stroke, serious mental illness, or severe aphasia were excluded.
A reiterative purposive sampling procedure was applied (18), based on demographic and strokerelated characteristics. Upon completion of their participation in the RCT, twenty-eight members of the control group were invited by letter with a stamped addressed envelope to participate in this qualitative part of the process evaluation. Sixteen participants gave their informed consent by returning the form by post. A lack of response was recorded as a rejection of the offer to participate.
One of the participants who consented was subsequently excluded because his health condition deteriorated. Fifteen people (six women and nine men) participated in this study. In accordance with research ethics regulations in Norway, those who did not respond were not asked about their reasons for refraining from participation. Characteristics of the participants are shown in Table 1.

Interviews
All authors took part in developing the interview guide, and nine of the authors conducted the interviews from July 2016 to June 2017. The interviews were primarily narrative in style to encourage participants to convey their illness experiences (19). The interview guide had another main topic, namely to explore participants' experiences of the assessment interviews in the RCT (Appendix).
Interviews were conducted in a setting chosen by the participant, and lasted from 17 to 76 minutes, (median = 43 minutes). The interviews were tape recorded and transcribed verbatim.

Naïve interpretation
Answering the questions in the test battery in dialogue with dedicated data collectors experienced in stroke may enhance awareness and understanding of one's state of health. For some participants belonging to the control group may provide a safety net and lessen their fear for another stroke. The questions in the test battery may encourage some participants to seek support to improve their condition. If personal benefit from participation is expected, ending up in the control group might cause disappointment. If participants experience spontaneous recovery or control of their rehabilitation process, belonging to the control group would be less likely to influence their adjustment.

Structural analysis
In the structural analysis we identified two main themes: Influence of the assessment interviews, and No influence of the assessment interviews. The main themes with subthemes are presented in Table 2. It is notable that the encounters and the assessment interviews were perceived as "discussions", and thus as an opportunity to converse about important questions: The meetings helped me to understand much more why things work and don't work.
Other statements illustrated that the interview sessions were perceived by the participants as an opportunity to receive emotional support, and to have meaningful dialogues about their condition: I think it's really been a pleasure, that they visit you at home, ask you about your experiences and how you feel. I think that's really nice. Knowing that there is some kind of follow-up.
The questions in the test battery helped this participant to recognize her own post-stroke situation: Encouraged to seek support Some participants expressed surprise at some of the questions in the test battery, especially their thoughts regarding whether life after stroke was worth living. A typical comment among the participants was the assumption that "other people" might react negatively to such questions: Obviously -I know that many people react when there's talk about death …Because you don't feel you're worth anything anymore, or you feel weaker than you thought you were … you make a decision that you don't want to be part of it any longer.
At the same time, these participants tended to maintain that they themselves did not perceive that kind of question as negative.
"But I didn't feel uncomfortable answering the questions. It's possibly because you've experienced so much. Seen some terrible things, life is fragile." One participant perceived such questions as irrelevant and "stupid", and another stated that some questions were intrusive and possibly "dangerous". He expressed concern that in general such kinds of questions could trigger suicidal thoughts among other vulnerable participants, and he asked if the data collectors could involve advisors in case some of the participants had suicidal thoughts.
However, this participant described how the questions about his mental state had encouraged him to consult a psychologist: But [these questions] started thought processes. That was when I started to think that I ought to talk to a psychologist. Because there were some thoughts that weren't positive.
It therefore seemed that these questions in the test battery had encouraged him to seek support.

Sense of disappointment
Some participants expressed a sense of disappointment because they were randomized to the control group. One participant expressed envy of other participants who had been lucky to be in the intervention group.
I feel a bit jealous, but some people got to be in the control group, as well.
Other participants expressed a sense of disappointment about certain aspects of their participation in the control group. One was disappointed because he had expected information from the data collectors about his diagnosis and prognosis: There were some things I was wondering about. I was asking about several things, about my disease and about how long it would take to get well and get back to work.
Participation in the trial seemed to trigger expectations to fulfill unmet information needs.

No Influence Of The Assessment Interviews
Several participants expressed that the assessment interviews did not influence their reflections about their condition, and that their participation did not accommodate any of their unmet needs.
Rather, they felt it was important to participate to support research to the benefit of other stroke survivors.

Spontaneous recovery
Some participants who had experienced a mild stroke and felt that they had recovered spontaneously stated that it was "okay" to participate, and that they subsequently had not reflected at all on the interview sessions. One participant said: Follow-up is okay, but I've felt well all the time, so I cannot say I've had any personal benefit.
Other participants emphasized their feelings of luck and relief at having survived the stroke without major impairments.
I've been lucky, I recovered quickly. I just answered the questions, nothing more, and I haven't thought about it afterwards.
To talk about their recovery and answer the questions in the test battery was not perceived as a burden, and they did not find the questions to be inappropriate or intrusive.
It was perfectly fine. None of the questions were unpleasant.
These participants seemed to have managed the adjustment process on their own, and with their family.

Managing one's own recovery
Other participants highlighted support from family and friends, or their internal coping strategies as their capacity for problem solving and maintaining a proactive approach. One explained the latter point: To me, this study did not make a difference either way. I've drawn up subgoals all along and made a training program. I try to practice things that I know I have problems with.
Participants who highlighted that they did not experience any personal benefit also emphasized the importance of contributing to research for the benefit of other stroke survivors.
Not for me, but I think it might be beneficial for research. They see, OK she's done well, she's in that category. And then you have others, who are not doing well.
The assessment interviews did not seem to reveal any reflections about this participant's condition, or any new thoughts about implications for her life post-stroke.

Critical Interpretation And Discussion
Our study aimed to explore the possible influence of the assessment interviews on the adjustment of the members of a control group in an RCT exploring psychosocial well-being following stroke. The results showed that the controls were divided as to whether the assessment interviews had facilitated their adjustment process. Several participants expressed that the assessment interviews had influenced their reflections about their condition with the potential to facilitate their adjustment process. The assessment interviews might also encourage some participants to seek professional support. Other participants, however, stated that the assessment interviews did not influence their reflections about their condition or their adjustment process.
Ideally, in behavior experiments conducted in stringent and tightly controlled conditions, experimental manipulation would be the only difference between groups formed by random allocation (26). In complex intervention studies, however, performed in participants' natural environment, the design of the control conditions will be less stringent. In the current study, the basis of the RCT design was that responding to the test battery would have no influence on participants' adjustment process.
The test procedure implied that the data collectors were instructed to adhere primarily to the questionnaire. But they were not instructed not to respond to any questions from the participants.
Participants' unmet needs for information and support were demonstrated in this study. In addition, the questions in the test battery might have been perceived as an invitation to reflect about their condition and existential issues. Such factors could blur the difference between the intervention group and the control group. In that sense, the assessment test procedure might threaten the internal validity of the RCT (4), and possibly influence the results.
One striking result of this study was that dialogues of 30-45 minutes with a dedicated professional on three occasions seemed to facilitate these participants' reflections on their condition. Our results suggest that the mere attention, listening and dialogue with a professional might have had an impact on the adjustment process of the controls. However, considering the premises for an experimental trial, no influence on the controls, it is necessary to assess how to counteract such tendencies to influence participants' adjustment process. It is therefore essential to clarify the underlying factors contributing to influence or lack of influence. These underlying factors might be related to characteristics of the participants or of the data collectors, or to contextual factors, as will be discussed in the following.

The participants
Participants' descriptions of their illness experiences and of their adjustment process varied considerably. Most participants had minor to moderate impairments, and one had a moderate to severe stroke, based on the National Institutes of Health Stroke Scale (NIHSS) score of stroke severity (27) (Table 1). For five participants the NIHSS score was unknown. Thus, we cannot conclude that the influence of the assessment interviews corresponded with the degree of stroke severity. The results illustrated participants' drive and struggle to recover and regain their perception of their pre-stroke self, independently of their statements of "influence" or "no influence". In this way, participants' desire and efforts to gain support, information and understanding might be an expression of positive adaptation in terms of their sense of coherence (SOC) (28), or their level of resilience (16,29). In cases when participants described their experiences of inadequate follow-up and shortcomings in hospital and community health care, they also emphasized the influence of the assessment interviews. Thus, participants' needs for support might be related to their personal ability to cope and adjust, to the quality of their family network, and to possible gaps in the quality of health care services.
Some participants might have confused the role of the data collector with the roles of primary health professionals. Our results illustrate that some participants had expectations for the trial that extended beyond the limitations of their roles as control group participants. Considering that some participants' interpreted the assessment interviews as "dialogues" or "discussions", the boundaries between a mere data collector and a health care professional seemed to be blurred. These expectations indicated that they were somewhat confused as to how the study related to the ordinary health care It is important to consider the fact that several of the questions in the test battery invited the participants to reflect on existential issues. Some participants expressed concern that assessment questions about mental problems and suicidal thoughts could cause harm in terms of triggering dangerous thoughts in other participants. Thus, it might be challenging, yet unavoidable, to balance the research ethical principle of doing no harm with the necessity to pose sensitive questions based on the validated instruments.

The data collectors
Nonspecific factors such as "human interaction variables", clinicians' warmth and empathy may have a substantial impact on the outcome of an RCT, as previously shown (4). It might be challenging to balance the participants' needs for information and support with the methodological and research ethical guidelines guiding RCTs, i.e. avoiding influencing the participants' reflections about their condition and their adjustment process. However, in order to avoid bias, it seems important to train and supervise data collectors to balance these seemingly contradictory demands. It should be noted that this might be particularly demanding when collecting face-to-face data from participants who might have cognitive impairment. Considering stroke survivors' medical condition and vulnerability, it might be challenging to take a neutral or distant attitude towards the participant. However, data collectors are confronted with the demand to balance the ethical challenges that might arise when opposing ethical obligations to care and research overlap (40).
Is there any solution to these inevitable and seemingly conflicting obligations? Is it possible to counteract and minimize these tendencies to influence control group participants when conducting assessment interviews? Some authors propose that disclosure of information about the trial should be restricted, and that the participants should be given neutral information (36). If the controls had been unaware of the existence of the study arm, this might have diminished the risk of study-induced behavior change (3). However, there is a fundamental principle and broad agreement among research ethicists that the duty to obtain informed consent follows from the principle of respect for research participants (41,42). Thus, to withhold such vital information would be considered unacceptable from a research ethical point of view (3). However, it is argued that in pragmatic trials with high social value, and with low risk or no risk, a waiver of informed consent should be considered ethically acceptable (43). Another argument is that a waiver of consent might be acceptable in intervention studies, in cases where bias is likely to occur (44).
The Zelen design involving obtaining consent from participants after randomization has been suggested to minimize these kind of threats in RCTs (45). Only those who had been randomized to the experimental group would then be asked to consent to participation in the trial, while the controls would remain uninformed (44)(45)(46). In a modified two-stage consent design, those assigned to the control group would receive the usual care, and they would know that other people received different care, but without knowing what that care entailed (47).
Applying the Zelen design adjusted to our study, the control group participants would neither be informed nor aware of the existence of the intervention. But they would be informed about, and could then consent to participate in, the assessment interviews at the prescribed three points in time, T1, T2 and T3. This approach might be perceived as meaningful by the controls, both as a potential confirmation of their own progression and adjustment, and as an opportunity to contribute to research for the benefit of other stroke survivors. Simultaneously, one would avoid the experiences of disappointment of not being allocated to the intervention group. This might be considered as an ethically sound approach, especially taking into account that this trial does not imply any risk for the participants, neither for those in the intervention group nor for the controls. However, out of respect for research participants' autonomy, obtaining informed consent has been considered as a cornerstone in research ethics (42,48). At the same time, taking into account research participants' vulnerability, the importance of building and maintaining long-term trusting relationships between researchers and participants has been highlighted (49). Thus, it may still be debatable whether, or in which circumstances, a waiver of consent can be ethically justified.
We cannot disregard the possibility that the number of data collectors involved, their diverse professional backgrounds, and possible variation in conduct in the interviews might have resulted in different approaches during the sessions. This might raise questions about possible alternative, or more uniform, approaches in order to achieve no influence. However, for practical reasons, and to obtain valid data from stroke survivors with different stroke-related impairments, we considered a certain number of experienced and trained health professionals as necessary to conduct the assessment interviews.

Contextual factors
We decided to implement the assessment interviews of the RCT in the participants' natural environment, primarily in their private homes. Face-to-face assessment interviews were considered a necessity, as the sample of stroke survivors eligible for trial participation was considered a vulnerable group that might have difficulty in answering the questions without sufficient guidance. When designing this trial, the population of stroke survivors from where we recruited our participants were expected to be elderly, with severe impairments, such as paresis, fatigue and cognitive and language impairment. Thus, we anticipated a low response rate if we collected the data by mail or phone.
However, both the assessment interviews themselves and the setting of the participants' homes had the potential to draw the participants' attention to their condition, and also to influence their helpseeking behavior.
The one and only influence on the controls was meant to be the delivery of standard stroke treatment in terms of primary rehabilitation services.
To protect the control group participants from the influence of face-to-face assessment interviews, either telephone interviews or answering the questionnaires on their own would have been alternatives. However, one should consider the obvious challenges associated with assessments from stroke survivors with varying degrees of impairment. In that case, the planning and implementation of the assessment interviews should have been subject to meticulous attention and preparation.

Methodological considerations
Our sampling procedure was selected to include participants with various socio-demographic and stroke-related characteristics. Participants' disclosure of their experiences of illness and of participating in the RCT revealed rich and nuanced data. However, in view of the small sample size, we cannot conclude that the results were representative of all the controls in this RCT. Considering some of the participants' stroke-related impairments, memory loss might have influenced their perception and judgment of the influence of the assessments. It is still not clear whether or how far the assessments actually had any influence. The interviews were performed in retrospect after the completion of the T3 assessment interviews. The participants might have had difficulty in recalling the assessment interviews and the encounters with the data collectors, which could have influenced the results.
All the researchers were involved in the development of the interview guide and in the analysis process. Nine researchers conducted the qualitative interviews. Six of them had participated as intervention providers, but did not interview any of the participants they had visited and followed up in that capacity. Several other interviewers had participated in the development of the trial, and had worked as project coordinators. Some interviewers had acted as data collectors, although they had not previously collected any data from these participants. Some of the interviewers' extensive knowledge of and involvement in the trial might have strengthened the depth and nuances of the interviews. On the other hand, it is possible that such insight and involvement might represent a disadvantage in terms of not maintaining a sufficiently distant and objective view during the interview sessions and in the analysis. However, having a number of researchers with different levels of involvement in the RCT probably counteracted this potential weakness.

Conclusion
The influence of the assessment interviews on the adjustment process of members of the control group of the RCT varied considerably. The results demonstrate that the assessment interviews had the potential to facilitate participants' adjustment after stroke. However, some participants' statements indicate that they relied on their existing personal capacity to cope and adjust, and that the assessment interviews had no influence on their process of adjustment. The rigor of an RCT may be challenged by the possible influence of individual assessment interviews as described in this study.
Training and supervision of data collectors and alternative approaches to assessment to avoid influence are important factors to consider, in order to reduce the threats to internal validity. Tape recordings of the assessment interviews would have been useful in order to explore a potential therapeutic content in the interactions. written informed consent, also adjusted for patients with aphasia was collected from all participants in the RCT and the process evaluation.

Consent for publication
The written consent form contains information about scientific publication of results from the study in anonymous forms and all control group participants have given their written consent.
has been done in close collaboration between all authors, who also accept direct responsibility for version to be submitted.