To develop the decision aid, a systematic development process model [23] will be conducted in combination with user-centered design (UCD) [24]. This study will be conducted in three phases. The whole development of decision aid prototype will be presented in phase 1. The user-testing study for revision will be conducted in phase 2 and phase 3, with alpha-testing in a simulated way and beta-testing in the clinical context respectively. This study follows the IPDASi v4.0 [25] to meet the minimum quality criteria at least. The main elements of this development process illustrate in Fig. 1.
Figure 1 Development process of decision aid
A multidisciplinary steering committee to guide the whole development is significant. Therefore, a steering committee was created firstly to ensure a systematic protocol, which consists of two experts in SDM, three in critical care medicine and two patient representatives. Two SDM experts have many years of research experiences in shared decision making and patient decision aids. These critical care experts include an ICU director, a head nurse and a critical care specialist with expertise in RRT decision-making and clinician-family consultation. Our patient representatives are family surrogate volunteers who experienced RRT decision-making in ICU. Steering committee meetings will be hold regularly to supervise the design, content and iterative revision of the decision aid.
Phase 1: Development of decision aid prototype
Scope: literature review
A comprehensive literature review occurs early to scope the purpose of decision aid development. Firstly, extensive literature reviews will be performed to understand the current state of RRT decision-making and identify the evidence-based necessity to develop a DA in ICU. In addition, we will conduct a competitive analysis by summarizing relevant DA resources, which is an important way for digital projects development, to determine the development value rather than duplicating the available contributions. Furthermore, this is a significant step to ensure whether the development project is compatible with our research protocol.
Design: needs assessment
Healthcare professionals’ interviews
Concurrent with the literature review, decisional needs assessment is critical in confirming potential DA use cases as identified in literature. In terms of the complexity (as one of the life-sustaining treatment) of RRT decision-making in ICU, the decision trigger always opts patients’ attending clinician. Therefore, semi-structured interviews were conducted with ICU clinicians to identify main points that a DA might address and opportunities to integrate DA into clinical workflow. We will recruit ICU clinicians who recently (<6 month ago) went through clinician-family consultations about RRT decision-making for patients with AKI. Each interview will typically last 30 and 60 minutes, and aims to elicit professional opinions on key aspects in content, design and distribution of DA. Firstly, clinicians will be asked to indicate what information family surrogates needed to know before joining in such decision-making. Secondly, they will be asked to outline the surrogate decision-making pathway to dig out when and how to refer to a DA is appropriate. Accordingly, individual interviews with ICU clinicians are to explore professional suggestions and reach consensus on their clinical needs for this DA.
All interviews will be recorded and transcribed verbatim. The interviewee’s extra perspectives will be written down by the interviewer, then analyzed alongside with transcripts. Colaizzi seven-step qualitative analysis approach will be used for coding emerging categories and themes in transcripts [26]. Firstly, all transcripts will be independently reviewed and coded by two researchers with expertise in qualitative research methods. Then, final coding will be determined by further comparison and discussion, and disagreement of opinions will be submitted to another qualitative research expert for arbitration. Computer software NVivo version 12 will be used for analysis.
Target users’ interviews
UCD requires a deep understanding of target user’s role and responsibility related to the DA in development. To comply with this requirement, we will conduct semi-structured interviews with family surrogates, the potential target user of the DA, who are informed with and engaged in the currently RRT decision-making for their loved one. For generating a comprehensive concept regarding to surrogate decision-making in RRT, site observations especially when clinician-family encounters happen in ICU routine are needed. Therefore, we will timely focus the ICU clinician-family consultation before face-to-face interviews with surrogates to uncover the opportunity that a DA can fulfill their decisional needs. Opportunity statement exercise[24] will be used, which is a type of design thinking activity used to gather, explore and refine the information needed for decision aid development, bringing together with family surrogates to collect delineating facets of current practice that are not meeting surrogate decisional needs to identify in what way a DA can make measurable impact. Finally, family surrogates will be asked to provide feedback on prepared statements (summary outcome of literature review and healthcare professionals’ interviews), which describe a future decision aid and its general contents, formats and functionalities. Similar to the analysis of interviews among ICU clinicians, all of the audio recordings will be transcribed verbatim and analyzed along with written notes using qualitative analysis approach.
Then, the decisional needs from both ICU clinicians and family surrogates will be translated into a general framework for next-stage detailed evidence synthesis.
Draft: working prototype
Evidence synthesis
The best evidence and other decisional information will be synthesized based on the needs assessments between ICU clinicians and surrogates. Firstly, we will review current guidelines on the management of AKI and guidance of RRT for general aspects of treatment recommendations. Secondly, available treatment-related outcome evidences from systematic reviews and meta-analysis will be systematically searched for generating suitable information for users concerning whether or not to choose RRT. The whole process of evidence synthesis will be as following four steps: 1) identify the key words; 2) systematically searching; 3) screen and extract relevant evidences based on clinical needs, usually about treatment recommendations and prognostic information; 4) sum up evidences into a decision aid script.
Drafting the prototype
According to literature review, key stakeholders’ needs assessment and best available evidences, a paper-format prototype will be drafted under the guidance with several criteria for development of decision aids [27, 28] and assessed following the IPDASi v4.0. In general, the content of the DA will be presented in five aspects: 1) brief instruction of disease and treatment; 2) pros and cons of treatment options; 3) possible decisional outcomes; 4) patient story; 5) value clarification exercise. Once the paper-format prototype has been completed, the steering committee will audit and decide the final content included in the decision aid. Then, the paper-format prototype will be redrafted and redesigned into a working prototype which can be accessed by scanning Quick Response Code (QR code) using smart phone.
Phase 2: Alpha-testing
The aim of this phase is to examine the comprehensibility, acceptability and usability of the DA in a simulated decision-making scenario,[23] for a better understanding of the interaction between decision aids and target users in conducting tasks. This testing will be done in surrogates, requiring interaction with the DA and carrying out think-aloud method. The evaluation objectives include assessment of the functionality and usability of the DA’s overall interface and identifying problems in user interaction.
Participants
A purposive sample of surrogates not involved in development process will be invited into participate. We will recruit family surrogate decision-makers of AKI patients from Southwest hospital in Chongqing, China. The surrogate samples will include those who opted RRT for their loved one, those who considered but declined RRT, and those currently deliberating treatment options in ICU. We expect a sample size of 8-12 participants, to detect up to 80% of the surface level usability problems[29]. However, sample sizes will be determined by data saturation, which is guided by transcripts of users’ interaction with the aid. We will conduct and complete alpha-testing within four weeks.
Data collection
A researcher with expertise in health informatics and usability testing will guide each session in a quiet waiting room. Data will be collected during the whole process of participants’ usage of the DA, including questionnaires, recording of the interaction and semi-structured interviews. Firstly, participants will be required to complete the before-questionnaire related to their demographics, self-perceived on-line information acquiring skills on smart phone and experiences about RRT decision-making, and a knowledge survey [30] about general RRT information. Secondly, participants will be subsequently presented a scenario describing a simulated decision making about RRT for AKI patients in ICU, which is closely matching their history experiences. Then, they will be given an introduction card containing the retrievable address (a QR code scanned using WeChat on smart phone) of the DA, and also a brief task description that participants need to complete the usage of the DA with think-aloud method, which means oral expressing their behaviors and thoughts while using. With the task completed, participants will be required to finish the after-questionnaire, including a knowledge survey like before and an acceptability survey [31]. Finally, participants will be interviewed regarding satisfaction, pros and cons of the DA and the quality of this decision support. We will open screen-recording on the phone to typically record the whole user interaction (all screen activities and voice). Field notes will be written for expanding data sources as further analysis.
Data analysis
In keeping with effective sample size, data analysis will occur in conjunction with data collection. Data analysis will be conducted with a coding scheme developed by usability engineering team from examination of human-computer interactions and relevant cognitive literatures[29]. This scheme consists of eleven categories of usability problems, such as navigation, graphics, layout and so on. It allows application of these available categories in analyzing user interaction with the DA along with think-aloud reports. However, codes that indicate other user problems generated from our testing results will be added appropriately.
The screen recordings will be watched and annotated independently by two researchers. Disagreement will be discussed to reach a consensus. Semi-structured interviews will be analyzed as described in key stakeholders’ interviews in Phase 1. Knowledge surveys will be analyzed by paired t-test. Acceptability surveys will be analyzed by one-sample t-test. SPSS Statistics version 26 will be used for analysis.
Based on these findings, refined details for improvement will be generated. These efforts will be reviewed by steering committee to decide following revision.
Phase 3: Beta-testing
Beta-testing will be conducted to test the feasibility of the DA in ICU clinical context, for considering contextual factors which may influence the effect of the DA in real life. Without developers’ control, this stage is a more stable acceptance testing. A mixed-methods approach will be used to conduct a before/after study in RRT decision making procedure. The objectives of beta-testing are to assess the impact of the DA on family surrogate engagement in RRT decision making and the quality of decision making.
Participants
Both family surrogate decision makers and ICU clinicians will be recruited from ICU clinical workflow in the same hospital as described in alpha-testing. We will recruit participants based on patients. If AKI patients indicated with RRT requirement, we will first recruit clinicians to ensure the following family-clinician consultation for decision making. Then, we will invite each family surrogate to participate in. We will take the criteria, that who is responsible for written informed consent for patients’ medical decision making, as the surrogate standard. Clinicians needed to be with at least three years of ICU clinical experiences, and are responsible for guiding the decision-making process. Clinicians who are reluctant for shared decision making with patients’ family will be excluded. Family surrogates who are not able to use general function of WeChat on a smart phone will be excluded. Either clinician or surrogate is not eligible, patient will be excluded.
A consecutive sample of 15-20 family-clinician dyads will be invited to participate, which is in line with previous study in order to conduct inferential statistics in usability testing[29]. We are proposing to recruit more participants as soon as possible, to generate sufficient feedback for further refinement.
Data collection
Once the family-clinician dyad is available, DA will be introduced to surrogates before the decision-making consultation occurs. After completion of the usage, family surrogates will be invited into a formal consultation with clinicians for their loved one’s treatment decision making. All eligible surrogates need to complete questionnaire survey before and after using the DA, which includes demographics, surrogate decision making self-efficacy scale[32], preparation of decision making scale[33] and decisional conflict scale[34]. After the consultation, family surrogates and clinicians will be interviewed independently by two interviewers respectively, for providing verbal feedback on the process of decision making. Family surrogates will be asked for description of whether or not the DA facilitates engagement in decision making and meets their decisional needs. Clinicians will be required for sharing their experiences in this kind of decision making, and evaluating the decision quality. Interview outlines for surrogates and clinicians will be respectively prepared in advance and approved in steering committee meeting. All interviews will be recorded and last 30~45 minutes. All sessions will be undertaken in a quiet waiting room.
Data analysis
Descriptive analysis will be conducted for reporting baseline data. Paired sample t-test will be performed to assess self-efficacy, preparation and decisional conflict. CIs will be set at 95%, and significant level is 5%. All interviews will be transcribed and coded using NVivo version 12.0, and analyzed in line with the methodology in phase 1. Based on these findings, we will refine the aid if necessary. Finally, the steering committee will confirm the completion of development as a final version waiting for next-stage formal clinical trial.
Patient and public involvement
Two family surrogate decision makers were recruited as patient representatives in the steering committee. These two patient representatives will narrate their decision-making stories, which is anticipated to contribute participants heavily to clarify their loved one’s preferences. Patient representatives will engage in the whole development to focus on the content and design of the aid. We expected it would be user-centered for the presentation of our decision aid in the user’s perception. All of these representatives will not contribute to the recruitment.
Ethics and dissemination
Research ethics approval from Ethics Committee of the First Affiliated Hospital of Army Medical University, PLA (Ref: KY2020104) has been granted. Study statement information will be given to each participant. Written informed consent will be obtained from all participants. Participants will have the right to withdraw at any time of this study. All the important original innovative data will be considered for publication in peer-reviewed journals.