Ethical, Legal and Social Implications of Symptom Checker Applications in Primary Health Care (CHECK.APP): Study Protocol of an Interdisciplinary Mixed-Methods Study


 Background: Symptom checker applications (SCA) are accessible tools that provide early symptom assessment for users. Ethical, legal, and social implications of SCA, their impact on the patient-physician relationship, the health care providers and the health care system have sparsely been examined. This study protocol describes an approach to investigate possible impacts and implications of SCA on different levels of health care provision. It considers the perspectives of users, non-users, general practitioners and (international) health care experts. We aim to assess a comprehensive overview of the use of SCA and reveal problematic issues.Methods: The primary outcomes of this study are empirically informed, multi-perspective recommendations for different stakeholders on ethical, legal, and social implications of SCA. Quantitative and qualitative methods will be used in several consecutive and interconnected parts: Study part 1 comprises a survey that will be analyzed with a logistic regression. It aims to assess the user degree of SCA in Germany as well as predictors for SCA usage. Study part 2 will comprise self-observational diary and user-interviews, which will be analyzed as integrated cases to assess the user perspectives, usage pattern and arising problems. Study part 3 will comprise GP interviews to assess their experiences, perspectives, self-image, and concepts and will be analyzed with the basic procedure by Kruse. Moreover, interviews with health care experts will be conducted in study part 3 and will be analyzed by using the reflexive thematical analysis by Braun and Clark. Discussion: Possible ethical, social and legal implications of a widespread use of SCA affect stakeholders and stakeholder groups on different levels of health care. The proposed methodological approach provides a multi-faceted and diverse empirical basis for a broad discussion on these implications.Trial registration: The study is registered in the German Clinical Trials Register (DRKS) with the ID: DRKS00022465 since the 7th of August 2020.

approach provides a multi-faceted and diverse empirical basis for a broad discussion on these implications.
Trial registration: The study is registered in the German Clinical Trials Register (DRKS) with the ID: DRKS00022465 since the 7 th of August 2020.

Contributions To The Literature
Mixed-methods approach that integrates different data sources, methods and scienti c disciplines in a collaborative research process The number of health-related software in consumer and research-oriented applications is increasing rapidly. Symptom Checker Apps (SCA) are one example for health-related software that could have a major impact on health systems on all levels. SCA process medical symptoms that users enter by applying algorithms and databases with medical information (1). Based on these symptoms, they generate a list of probable causes and suggest medical follow-up actions (e.g., wait at home, see a doctor, go to the emergency room). SCA advertise that they use arti cial intelligence (AI) and big data as basis of their algorithms (1)(2)(3); however, it is rarely speci ed by developers how and to which extent AI is implemented in these applications. Even though there are already strong claims from ethical research that emphasize the importance of criteria like transparency, trustworthiness, agency and responsibility for arti cial intelligence-driven decision support systems like SCA (4-6).
The "google play store" already lists 249 applications (retrieved 24.03.21) for the key words 'symptom checker' (7). At the moment SCA are not hardly regulated, even though they can be classi ed as high risk applications according to the National Institute for Health and Care Excellence (NICE(8)). According to NICE, the use of high risk applications requires the highest evidence level 3b (8).
SCA can be used on any device with internet access and are often free mobile-based applications, thus easily downloadable in app stores or accessible with an internet browser if a suitable technical device and internet access is available to users. Nevertheless, users must be able to handle technical devices and to interpret the SCA's output. This could lead to a disadvantage of speci c population groups e.g. seniors, people with disabilities (9) or people with limited economic resources (10). Some SCA exclude speci c user groups for symptom analysis e.g. pregnant women, children, seniors and patients with speci c comorbidities (11). Although, some users found that SCA are useful tools for self-diagnosis and even reported positive health effects (12). Other users had problems providing and interpreting concrete information on symptom time patterns or severity (13). The output of SCA recommends actions and probable causes for the entered symptoms, which can lead to confusion and negotiations due to mismatch with users' experience or expectation (14). Such negotiations may initiate unnecessary healthcare seeking behavior (15).
In terms of their medical value and validity, commercially available SCA struggle with accurate triage (determining user's medical condition based on their input and recommending the optimal course of action). A review of 23 studies showed that SCA often suggested risk-averse actions (16, 17). SCA's diagnostic and triage accuracy is still limited and was even less reliable in non-urgent scenarios, which are common in primary care (16). A recently published study compared the performance of n = 8 SCA with the performance of telephone consultation of n = 7 GP by using 100 case vignettes. GP outperformed SCA on all assessed outcomes (accuracy, condition suggestion, appropriateness and safety of urgency advice). The comparison was limited to telephone consultations and did not comprise direct patient-physician contact. Some SCA performed over-conservative, simultaneously other applications tended to take advices with risks for users (11). In high performing health care systems, this can cause preventable costs and increase the risk of unnecessary procedures that could lead to avoidable risks for patient's safety (2,18).On the other hand, in structurally weak regions with restricted access to medical care, SCA can provide a rst-line assessment that otherwise would not be available (19).
In summary, potential risks of the use of SCA (exclusion of users, stress, induction of health-seeking behavior) contrast advertised opportunities of SCA like patient empowerment and better health care for underserved regions. There is a substantial gap in the literature concerning the effect of SCA on different health care systems, different levels of health care (micro-, meso-and macro-level (20)) within these systems and on the system's different participants (users, non-users, health care providers). If SCA become more widely used, their ethical, legal and social impact on those levels and participants must be better understood despite complex interactions and methodological challenges.
In this study we aim to clarify ethical, legal and social impacts of SCA on users, non-users, GP, the primary health care systems and their work by means of an independent, empirical, integrated multi-perspective, multi-disciplinary discussion. Due to the lack of systematic research of SCA in primary health care, the recent study uses an explorative hypothesis generating approach in which the above-mentioned discussion is informed by four foci of interest and the study aims as stated in Table 1. Table 1 Foci of the present study Focus 4) Impact of SCA use on health care systems and health care workers − changes that will specially focused on: changes in the patient-physician-relationship, strategies of handling pre-informed patients, changes in the role concept of physicians and requirements of GP, as well as potential psychosocial risks and demands and perceived work stress resulting from these changes.

Study design and course
The study's areas of focus and corresponding study parts will be conducted by multi-disciplinary partners from the areas of social medicine, ethics and medical history, legal studies, general practice, sociology, occupational health medicine and health services research. The German federal ministry of education and research funds the project for three years (Grant No. 01GP1907A/B (21)). Ethical approval for this study was obtained from the ethics committee of the University of Tübingen (ID: 464/2020BO).
The project partners will co-develop study materials, subsequently discuss results and hold method workshops throughout the three years. During the nal study year, a series of workshops will include participants of the preceding study parts. This nal year workshop series is led by a social scientist with comprehensive experience of working in cross-disciplinary research and holding method workshops. The workshops will also serve as internal quality control and monitoring. An advisory board is continually informed about the progress and the results of the study parts. Members of the advisory board will be recruited from different contexts and disciplines. The advisory board will meet annually to give feedback on the research process, preliminary results and the dissemination of the latter.
The present study will be conducted in accordance with the Declaration of Helsinki. Study participants will be informed thoroughly about the study and their rights, and written informed consent will be obtained from all study participants. Other research ethics requirements such as data protection, will be diligently considered.
The general study design and the involved research partners are outlined in Fig. 1  Four main data sources will be considered, each representing a speci c stakeholder of SCA: SCA users, SCA non-users, GP, and health care experts. Data collection and analysis methods will comprise qualitative and quantitative approaches. The different methods are applied to the data sources in four interconnected study parts, each representing one of the four main foci of interest of the study (see Table   1): A literature review (study part 1, lead: IEHM, IFAS), a representative survey of SCA users and SCA nonusers (study part 2, lead: IGP), a SCA user diary-based self-observation combined with individual semistructured interviews (study part 3, lead: IEHM) and lastly, single semi-structured interviews with GPs and health care experts (study part 4, lead: IOSM for GP interviews, lead: IGP for health care experts interviews). For the present study protocol, we will follow the GRAMMS (25) checklist. PRISMA-ScR (22), STROBE (23), SRQR (24) and GRAMMS (25) checklists will be applied during the project process. The PRISMA-ScR, STROBE and SRQR guidelines will be applied on speci c study parts, the GRAMMS (25) guideline will be used in the context of mixed-methods approaches and in the integration of results.
In the following sections, the four study parts and their connections will be described in detail.

Study part 1 -literature review
The rst study part will comprise a comprehensive literature review that will assess the existing research on SCA and their impact on primary care. Aim of this study part is to gain a clearer picture of the state of the art of SCA considering the ethical, legal, social and systemic (e.g. risks and potentials) impacts of SCA. A literature search, oriented on scoping review (ScR) methodology, will be conducted and reported according to the PRISMA-ScR Statement (26). In recent years, the ScR methodology has been adopted and further developed for the eld of bioethics, which is characterized by normative research questions. When analyzing argumentative literature, adjustments need to be made to the "classic" ScR methodology (27)(28)(29). Key terms will be de ned for the search strategy regarding the research questions and databases covering the relevant dimensions (biomedical, ethical, social and legal) are selected (Web of Science, PubMed (Medline), Belit / Ethmed, ProQuest, SowiPort, GESIS, Philpapers, Juris, BeckOnline, etc.). Three researchers will screen the identi ed literature via hand and database search and discard publications not meeting the inclusion criteria. Publications will be analyzed by two authors using the method of qualitative content analysis proposed by Kuckartz (30).
Study part 2 -cross-sectional survey of SCA users and SCA non-users A questionnaire of 15-25 minutes will be used in a case-control design. The questionnaire will comprise different evaluated scales (for further details see appendix 1) and sociodemographic variables as well as speci c questions to the usage of SCA. This study part will be conducted and reported according to the STROBE Statement (23).
Due to the limited amount of literature on SCA users and non-users, pilot interviews with n = 2 SCA user and n = 1 SCA expert will be conducted to ensure a meaningful concept selection for the survey. Simultaneously, concepts will be derived from existing literature that is connected to the use of health applications and could reveal potential characteristics of the user group, such as: eHealth Literacy (31), personality (32,33), hypochondria (34), self-e cacy (35,36) and need for cognition (37). A nity for technology (38), satisfaction with the GP (39) and the overall life satisfaction (40) will also be considered.

Sample and Recruitment
The sampling process will be conducted from November 2020 until May 2021. A case-control design using two strands of recruitment is planned.
An a-priori power analysis using PASS 2020 (v20.0.3) revealed a sample size of N = 375 (β = .8, α = .05, P1 = .2), n user =188, n non−user =188 for an Odds Ratio of 2.5. The targeted OR correspond to small to medium effect of Cohen`s d (41) and was selected, because we consider that this will be an effect size that contributes a meaningful explanation of variance in the logistic regression. As this study has an explorative character, we could not derive theory driven assumptions for a multivariate logistic regression.
Hence, we based the power analysis on univariate logistic regression. We will use univariate logistic regression to identify meaningful predictors for a usage of SCA and will moreover set up a multivariate model that includes all identi ed predictors. Our multivariate model will be a rst proposal and will need further research to con rm the univariate predictors in a multivariate model. The sample will be composed of different recruiting strands to achieve a representative sample.
In the rst strand, German citizens will be contacted via mail to participate in the survey. The intended recipients will be representatively selected by an external partner (T + R Dialog Marketing (Berlin, Germany) and Acxiom (Neu-Isenburg, Germany)). Further participants will be recruited via mailing lists of the University of Tübingen and the University Hospital of Tübingen, social media and via cooperating GP practices. After three months, the representativity of this sample will be checked. If the return rate is too low or certain groups are not su ciently represented, there will be additional recruiting via the proposed channels.
The second strand of the sampling process aims to integrate symptom checker users only. The rst strand will result in approximately n = 20 symptom checker users. To ensure a sample size of n user =188, a targeted recruitment via social media advertisements and the social media channels of the University Hospital of Tübingen will be conducted.
Inclusion criteria in general are the ability to give consent and German language skills of at least B1 of the Common European Framework of References for Languages. Participants of the second strand recruitment can only be included if they have experience with SCA.

Analysis
The level of use for SCA, awareness of SCA, and general interest in SCA will be described using the rst recruitment strand with descriptive statistics.
Following the case-control design, SCA users from the rst and second recruiting strand will be matched with non-users (matched controls) from the rst recruitment strand. Meaningful predictors will be extracted with a logistic regression. A correction for multiple testing will be applied. The recent versions of IBM SPSS (42) and R statistic (43) will be used for the analyses.
Study part 3 -diary-based self-observation combined with semi-structured user interviews Study part three investigates SCA users and their usage patterns and effects of SCA on individuals. A speci c SCA (Ada App) was chosen, since it is considered one of the most prevalent SCA in Germany. Following an online training, participants will engage in a diary-based self-observation. During the observation time of 6 weeks, participants will note their daily (non-) usage of the Ada App. Next, individual semi-structured user interviews are performed with the diary study participants. The interviews allow participants to re ect on values, concepts, and knowledge gaps. This allows a supplemental exploration of experiences recorded in the diaries. The user interviews in this study part will be conducted and reported according to SRQR Statement (24).

Sample and recruitment
For this study part, n = 50 Ada users will be recruited from the SCA user strand of part 2) and, if needed, additionally via social media. Considering a drop-out rate of 30%, a sample size of n = 36 is assumed. Of these participants, n = 20 will be recruited for single semi-structured interviews using maximum variation sampling. Sampling will consider the content of the diary-based self-observation, usage behavior of the app, medical indication and socio-economic factors. Sample size calculations of the interview parts are based on the ve-dimensional model of information power by Malterud et al. (44).
All participants will receive a web-based training on the self-observation period. The diary will be used to document symptoms and events, as well as other expected in uencing factors such as stress or quality of life. Furthermore, it will offer structured questions about the use of SCA, but also enables the participants to write down their own reports or short " eld notes". Participants will thus keep note of and describe their experience and their dealing with action recommendations, appearing problems, emotions etc.
These notes will be used as a basis for the following semi-structured interviews. The interviews will be conducted via video call, audio-recorded and transcribed verbatim by a researcher from the IGP and the IEHM. The users will receive nancial compensation for the interviews as well as for the participation in the member check meeting described below.

Analysis
The diary-based self-observation and the interview transcripts will be analyzed and integrated into cases.
Triangulating the self-observation diary data with interview transcripts provides both prospective (longitudinal) and retrospective (narrative) insights: Quantitative results of the diaries (frequency of use, use of health care, symptoms, etc.) will be considered as prospective observational outcomes. The user diaries give the opportunity to record detailed situational experiences, feelings and thoughts that probably cannot be remembered or re-created during an interview. On the other hand, the interviews will provide indepth reports on values, concepts, gaps in knowledge, etc. that tend to remain invisible in the diaries. The qualitative analysis via Kruse's integrative basic procedure will provide an overview of recurring themes and patterns within each case as well as between cases. At the same time, it will allow a more holistic consideration of the data and analyzation of semantics, grammatical structures, and metaphors to reveal latent meanings and the way users "make sense" of the app, derive meaning and understanding of the recorded events (45).
A member check with participants of the interviews is planned in which results of the cases are presented to study participants to enhance rigor.
The quantitative analysis of the diaries will be performed using a recent version of Microsoft Excel (46), IBM SPSS (42) and R Statistics (43). The qualitative analysis of the study is supported by MaxQDA (47).

Study part 4-semi-structured interviews with GPs and health care experts
The fourth study part investigates possible effects of SCA on health care delivery and health care providers (module (a)) and the health care system (module (b)). As primary care is most affected by patients' usage of SCA, we will interview GPs in module (a). We will gain more insights into patients' usage of SCA and similar applications results in potential psychosocial demands and resources and perceived work-related stress (48), especially regarding workload, work content, work organization and social environment (49). Module (b) aims to deliver a multi-perspective view on possible effects of SCA on the health care system. To ful ll this aim we will conduct interviews with health care experts with different backgrounds to assess the state of the art of SCA in practice from a multi-perspective standpoint. Moreover, we aim to identify relevant aspects of SCA at the moment, gain a clearer picture of possible scopes of SCA in the future and derive quality criteria for SCA. This study part will be conducted and reported according to SRQR Statement (24).

Sample and Recruitment
In module (a), the sample will consist of n = 10 GPs in Germany. We aim to build a heterogeneous sample regarding the GPs (age, gender, race) and their practices (structure and location of the practice, main patient clientele, availability of online services). Sample size calculations of the interview parts in module a) and b) are based on the ve-dimensional model of information power by Malterud et al. (44). An interview guide (50) will be developed, containing questions about pre-informed patients and diagnosis in general and questions about SCA and similar applications at the example of Ada App in particular. It will be developed by the IOSM and receive feedback from the other project partners. The IOSM will apply various forms of sampling such as snowball sampling in the established networks of the IOSM and online research to ensure the stated heterogeneity of the sample. Two researchers of the IOSM will conduct the interviews mostly via video call due to the ongoing pandemic. The interviews are expected to last about 45 minutes and will be audio-recorded.
In module (b), n = 15 international experts on health care systems will be interviewed. Experts will be recruited consecutively and comprise politicians, IT developers for medical software, patient advocates,representatives from jurisdiction, medical associations and health insurances. Moreover, we aim to interview public health experts form different countries with similar health systems (e.g. Canada) and countries with different challenges in health care provision (Uganda, Brasil) to enable a comparison of SCA's relevance in different health care systems and cultural contexts. We assume that 15 interviews will provide su cient information power (44). Each interview guide will be tailored to the respective expert. Possible topics are the implementation of SCA, recent issues with SCA, requirements and how SCA in uence different players in health care. The interview guides will be developed by the team of the IGP with input from the other project partners. The experts will be contacted via already existing research networks of the project partners. One researcher of the IGP will conduct the interviews mostly via video call. The interviews are planned to last about 45-90 minutes and will be audio recorded.
The expert interviews will provide an information background on the status quo of SCA in health care as well as ideas for future developments. This background is important for the discussion of user experiences (Study part 3) and for the patient-physician relationship (integrated study parts 3 and 4). All participants will receive nancial compensation for the interviews as well as for the participation in the member check meeting. Analysis All interviews will be transcribed. In module (a), the interviews will be analyzed with the integrative basis procedure according to Kruse (38). As already stated, this method is useful to understand how interview participants "make sense" of their experiences. This allows insights into tacit changes in self-concepts, implicit values psychosocial demands and resources, and perceived work-related stress as well as an analysis of themes and content provided by the interview partners. The IOSM team will evaluate each interview individually and then compare the interviews to analyze common patterns. To ensure quality control and richness of analysis, each interview will be analyzed by two researchers and preliminary results will be continuously discussed with additional researchers from the joint project. In module (b), the interviews will be analyzed with re exive thematical analysis by Braun and Clark (51) as we aim to collect and structure the overarching themes and their various dimensions. The same measures for quality control will be applied as in module (a). For additional quality control, a member check with interview participants of part 4 is planned. Preliminary results will be presented and participants will be invited to give critical feedback, which will be integrated into the further analysis.

Discussion
The present study presents a protocol for an interdisciplinary, mixed-method multi-phase research program to comprise the impact of SCA on the three levels (micro-, meso-, and macro-level) of the health care system (20).
Digital health care innovations and their impact on health care are one of the major topics of our time. In contrast to applications for specialists, user-accessible applications such as SCA are an unknown variable in the development of health care delivery in the future. The con icting information about ethical, social and legal implications requires more empiric data to inform the debate: Con icting claims as to the ethical and social advantages or disadvantages of SCA characterize the current state of the debate. For example, SCA are both claimed to lead to a better health literacy and user empowerment and, at the same time, to confusion among the users. On the one hand, they are said to undermine trust in the patientphysician-relationship; however, they are said to lead to improved exchanges between the affected persons. Little is known about the psychosocial demands and resources of GPs in this context. SCA are attributed to result in an overuse but also in an underuse of health care resources. Finally, existing health inequalities may be improved or worsened by their impact. Based on a ScR of the ethical, social and legal literature the project will provide evidence, which of these contradicting assumptions is con rmed by the empirical study of the user experiences in the case study at hand. Using SCA as an example of consumeroriented digital innovations, the present study aims to research and integrate questions that are important for the general debate on digital transformation: What is known about the topic? How widespread is the phenomenon? How do users apply the innovation? How does that modulate their behavior and impact their health care usage? How do negotiations with health care providers play out? What legal regulations are necessary? What are the implications for the physicians in particular and the health care system as a whole? However, only collecting data from one stakeholders' perspective without considering possible interactions will generate blind spots. Thus, the main challenge is to consider different stakeholders' perspectives, wants and needs and to engage in a transparent debate on the current dynamic developments.

Strengths and limitations
The project's concept integrates different data sources and methods from the very start. The multidisciplinary, multi-phase design and the methods and skills mix of the study partners create a scenario in which methodological strengths are complimentary and perspectives can be negotiated. For example by limiting recruitment to a single SCA (the Ada App) in the qualitative study parts, we are able to focus the analysis (44). At the same time, through the representative survey, those qualitative results can be put in a broader context, which helps to negotiate implications on the health care system. Another strength is the immanent consideration of the user perspective by combining a survey, user diary and user interviews.
Considering the still widely unregulated implementation processes of digital health care applications, the common goal of a holistic approach seems rewarding and developments at two different velocities (consumer-oriented and governmental/scienti c) are needed. In integrating relevant perspectives and plotting study parts to converge in integrated workshops, we present an approach for integrated research in ethics, social and health sciences. This is possible due to long-standing cooperation between all involved partners and previous positive experiences in common projects.
The study offers a unique opportunity for multi-disciplinary research: It considers different research perspectives and methodologies from ethics, legal, social, health care and medical science and integrates them in a study process. We are con dent that this will lead to new insights for the use of symptom checker applications and digitalization in health care while providing a novel methodological approach for integrated research in health care digitalization.

Abbreviations
If abbreviations are used in the text they should be de ned in the text at rst use, and a list of abbreviations should be provided. Ethics approval and consent to participate Ethical approval for this study was obtained from the ethics committee of the University of Tübingen (ID: 464/2020BO). The present study will be conducted in accordance with the Declaration of Helsinki. Study participants will be informed thoroughly about the study and their rights, and written informed consent will be obtained from all study participants. Other research ethics requirements, such as data protection, will be diligently considered.

Consent for publication
Not applicable.

Availability of data and materials
Not applicable.

Competing interests
The authors declare that they have no competing interests.

Funding
The present study is fully funded by the German Federal Ministry of Education and Research, who had no role in the design of the study and will have no ruling during its execution, or in the analyses of the data, or in the decision to submit any results.