3.1. Study implementation
In order to verify the hypotheses, a 360°D process was performed at the University Hospital of RWTH Aachen University in the context of advance directives and living will communication. Here, 360°D Phases 1 through 3 are reported to indicate differences of opinion in stakeholder groups.
3.1.1. Phase 1 – Qualitative phase
Step 1.1. Creation of the monitoring team. The monitoring team (HW, SMJ) was assembled and tasks were distributed.
Step 1.2. Creation of advisory board. An advisory board (CS, RR) was assembled by the monitoring team. During the first two phases of the study, monthly meetings were held to control the study and verify decisions by the monitoring team.
Step 1.3. Problem definition. Little is known about the actual functionality and problems arising in the creation, distribution and interpretation of advance directives. Also, options for the improvement of communications about advance directives between people affected by an emergency situation and helpers are rare. The scientific problem was therefore defined as: “What are the current perceptions AND proposals about exchange of information in non-responsive situations?”
Step 1.4. Definition of stakeholders. Criteria for the stakeholder groups were defined. The monitoring team identified five stakeholder groups:
- Medical doctors, who are assumed to act in the patient’s interest and have a valid medical licence by state law. The level of training (resident, consulting physician) was not defined.
- Qualified nurses or caregivers, who have a close relationship with patients and might are the first responder in cases of emergency; A person who is a qualified nurse by law or in training of his / her qualification (2nd year or higher)
- Patients, who need to formulate their will appropriately and select how it should be transmitted if they are unable to do so themselves; A person who has or had an illness and is or was in medical treatment. The severity of illness or affected body system was not defined, however, only patient groups identified through self-help groups of life-threatening diseases such as myocadiac infarction or stroke were approached.
- Relatives, who eventually have to make medical decisions in the absence of an advance directive or lack of direct patient-doctor communication; A relative is in a familiar connection with a patient that matches the criteria of item 3; a relative doesn’t have to have an advance directive on behalf of the patient.
- Developers, who develop and maintain the system. A person who has been trained or is in training of information technology. The level of training was not defined.
Intersections between “patient” and “relative” or any of the others can occur in specific cases, such as a relative of a person with a severe health condition who is also undergoing medical treatment, or a patient who is also a medical professional. Therefore, each participant was instructed to answer as a member of the most applicable stakeholder group.
Step 1.5. Infrastructure. The contact channel was email, social networks, phone or personal, with a focus on email. The survey infrastructure was primarily online, to allow remote participation and limit interviewer bias.
Step 1.6. Ethical advice. Data security was defined by the data protection officer and confirmed: to allow anonymity the monitoring team agreed not to save the IP-addresses, timestamps, or any personal data of the participants. Ethical approval was obtained from the ethics committee of the Uniklinik RWTH Aachen (EK 363-15).
3.1.2. Phase 2 – Qualitative round
To investigate the difference between the current DT and the novel 360°D, a qualitative survey was performed. Round 0 statements were not set by the monitoring team, but were collected using a qualitative questionnaire to reduce bias.
Step 2.1. Creation of open questions. The monitoring team created open questions on different aspects related to the topic (see Addenda A and B). A scenario-based question form was developed to compensate for informational gaps. Additionally, technical terms were either avoided or were defined in the survey. An online survey was created for three different categories: (i) medical professionals for doctors and nurses, (ii) care recipients for patients and relatives and (iii) developers for developers. The main difference between the questionnaires was formulation and introduction to the topic.
Step 2.2. Test-phase. The questions were tested using semi-structured interviews. The test respondents were three medically qualified individuals (one doctor and two nurses) who were asked to select and then modify the questions for a better outcome.
Step 2.3. Collective definition. In order to find contacts appropriate to the collective, a list of medical institutions was created using the practically oriented stakeholder principle. To create a heterogeneous collective, there was an equal number of institutions with each of the following characteristics: level of care (maximum care institution, specialist care institution, primary healthcare, doctors office, retirement home), institution sponsor (church-related, public, private), speed of emergency reaction (fast, slow), medical area (oncological, internist, geriatric, emergency services), geographical area (rural, urban). Stakeholders were collected associated with the primary contact person (mostly doctors) such as patients of the doctor, the relatives of the primary contact persons patient. Software developers where identified by projects associated with the field of medical informatics. There was no personal relationship between monitoring team and collective.
Step 2.4 Round 0 main poll. The institutions were contacted by phone or email. A follow-up email, including information on the topic and a printable flyer with access to the online surveys (Short-URL, URL and QR-Code), was sent. Information of the project consisted out of the reason why the study is conducted, to establish an emergency data set for will communication, and a hypothetical use case. The contact person in the identified institutions was asked to print out and hand the flyer to: one patient, one doctor, one nurse and one relative of a patient. The survey was closed in January 2016 with the following numbers of answers: 30 in total, of which ten were complete. There were three in each of the medical doctor and patient stakeholder groups, and two in each of the qualified nurse and relative stakeholder groups. The response rate of the first contact by phone was 80% (n= 32), the response rate for completed answers was 25% (n= 10). The stakeholder label was selected by the participant during the survey and saved in the survey database. The average time for the participants to complete the survey was 25.87 minutes. In addition, three developers with experience in medical informatics but not related to the project were surveyed.
Step 2.5 Evaluation of round 0. The database was converted into a single document for each participant so that data analysis could be performed (HW). The documents were named with a database ID and grouped according to the indicated type of stakeholder. Qualitative coding was performed for the full collective and the stakeholder groups with an inductive category development (bottom-up coding) [20]. Two coders performed the analysis independently using two different systems and compared their results (HW, NK). Both the coding and the coding structure were created independently. Rules for creating the categories were defined to evaluate similar content structures. The coding bases were subsequently consented and brought into a joined format.
Inter-rater reliability. Codings were matched and inter-rater reliability was calculated. In total, 238 codes were identified, of which 158 could be matched by their location. Out of the 158 matched codes, 107 (44.95% of total codes, 67.72% of matched codes) were evaluated as semantically similar in a majority-vote scheme, and 132 (55.46% and 83.54% respectively) in a single-vote scheme by three evaluators. In other words, 44.95% of the codes were created at the same position in the raw text, and two out of the three independent evaluators rated the codes given to these positions as semantically similar. Major sources of differences were variability in coded text passages, consistent coding of suggestions that occurred on multiple occasions, and level of abstraction. The coding and coding structure were then unified and agreed upon by the two coders to form a coding base for future use.
3.1.3. Phase 3 – First quantitative round (Traditional Delphi: Round 1)
Step 3.1 Creation of closed questions for Round 1. Based on the coding base, questions were identified and transformed into a quantitative survey (HR) (see Addendum C). The questions were divided into A “simple questions” and B “matrix questions”. Matrix questions are used to weigh different, potentially conflicting aspects against each other. In this case, privacy and data security, and ease of accessibility and change of data was weighted against each other, as they are often mutually exclusive in a system. For example, if personal data is stored encrypted using a password it is less prone to breach of privacy, but also not easily accessible in case of emergency. Especially in these topics the sub-groups did not match in the Phase 1 and 2 questionnaires.
Step 3.2 Test-phase. The questions were tested by scientists associated and non-associated to the 360 degree project. Markers of the test were (1) understanding questions, (2) duration filling out the survey, (3) selection among ordinal and other scales (SMJ/HR/HW).
Step 3.3 Collective selection (2). Sub-groups were contacted on different channels: patients and relative associations via social platforms and support groups, doctors and caregivers via hospitals, IT-developers over direct contact and phone calls. (HR, HW). To access the survey a link for access over the internet was used.
Step 3.4 Round 1 main poll. The questions were converted into a single document for each participant so that data analysis could be performed (HR). The online survey was opened from Febuary 1st 2018 till October 10th in 2018. In total 62 persons participated with 60 persons (11 doctors, 6 caregivers, 9 relatives, 28 patients and 6 IT-developers) completing the questionnaire. The 2 uncompleted persons stopped filling out the form because of connection loss to the data server.
3.3. Hypothesis 1
Several instances were found in which statements were made exclusively or predominantly by just one of the stakeholder groups. The analysis regarding the channels for will communication in an emergency situation (Questions 1 through 4, Addenda A/B) show that the stakeholder groups of patients, relatives and qualified nurses named speech and gesture (i.e. direct contact) as possible options, in 12 statements. In contrast, doctors only once named direct contact with the patient as a possible channel (Table 1).
Indirect contact shows similar characteristics: doctors value relatives as a source of information, and were the only stakeholder group with seven out of a total of eight statements with this coding (Table 2).
General knowledge of advanced directives exists in the doctor, patient, developers and caregiver stakeholder groups, whereas relatives do not mention advance directives. In multiple responses by relatives, the term organ donation card appeared as a named form of will communication, whereas this term was not present in the other groups. Developers did not mention speech, gesture or direct contact at all. In terms of indirect will communication text, patient’s will, organ donor card and advance directives were mentioned.
Any collective made of just one stakeholder group would therefore have missed at least one aspect brought up by another stakeholder group. This proves Hypothesis 1.
3.4. Hypothesis 2
Four Phase 1 and 2, the standard deviation across the population of the occurrences of codings is calculated as a measure of dispersion. It is calculated for each code and for each stakeholder group and the total collective (Fig. 4). Two questions were removed because they were unspecific (Question 7 “Do you have any other aspects to mention?” and Question 8 “What is your opinion of the project?”). The average standard deviation as a measure of dispersion is highest in the total collective. On average, the dispersion is 27.8% higher in the total collective than in the individual stakeholder groups but without a marked effect size.
Similarly, statistical analysis was performed for each question of the Round 1/Phase 3 questionnaire (SMJ, HW). Results are reported as Box-plots (Figure 3). Due to the small sample size and study setup, no inferential statistics (T-Test, etc.) were performed but descriptive statistics are reported. While most sub-groups overlapped in their replies, several questions were answered almost mutually exclusive between sub-groups (Figure 3, Table 3). For example, caregivers and patients were in strong favor of easy access to all data in case of emergency (mean 5,667, Stdev: +/- 0,816 and mean 4,862, Stdev +/- 1,866 respectively), Likert scale, 1 means restrictive data access, 6 easy access to all data), while relatives preferred a more restrictive data access (mean 2,778, stdev +/- 1,093). Since the standard deviation is low in the sub-groups, this question could already be accepted as consented. In comparison, the total distribution across all groups is not consented (mean 4,344, stdev +/- 1,870). Similar effects can be seen in most matrix questions (see Table 3), which supports Hypothesis 2.
3.5. Additional system design considerations
From all the participants (n = 59) in the second round 47,46% (n=28) were patients, 16,95% (n=10) doctors, 15,25% (n=9) relative, 10,17% (n=6) caretakers and 10,17% (n=6) software developers. 26 had created an advance directive. 83% knew about the possibility of an advance directive and donation card, 70% about health care proxy. 44% have an advance directive of their own. Willingness to upload documents online is surprisingly with an average of 94,92% advance directive, 80% emergency card, 78% organ donations, 61,02% vaccination pass and 47,5% personal message. Lowest acceptance has testimonial with 37%.
For practical reasons for how to design a system the collective was questioned. In terms of how to create documents we asked if the participants prefer questionnaire or free text. A majority with 86,44% agreed on questionnaire.
62,71% agreed on a likert scale with 4 items to “agree fully” with “I wish, there would be an easier way than right now to communicate the patient’s interest.
98,31% would create a data set, mostly for themselves (83,05%), but also for their parents (57,63%). Kids and other relatives ranked lowest.
Participants rated, that the data set should be accessible by doctors (100%), the patient himself (76%), relatives (70%) and caretakers (46%). “Everybody” was selected by only 2% of participants. This concludes, that an authorization system is key to a success.
Data changes and management were seen by 89% of patients and 56% of doctors primarily the patients responsibility. 62,71% selected to update the date in 6 months, every year or less often. Time amount per month for content care is mostly selected “a few minutes” (74,48%).