Impact of Artificial Intelligence, With and Without Information, on Pathologists’ Decisions: An Experiment

doi:10.21203/rs.3.rs-614881/v1

Download PDF

Research Article

Impact of Artificial Intelligence, With and Without Information, on Pathologists’ Decisions: An Experiment

https://doi.org/10.21203/rs.3.rs-614881/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

Background: Artificial intelligence (AI) is rapidly gaining attention in medicine and in pathology in particular. While much progress has been made in refining the accuracy of algorithms, thereby increasing their potential use, we need to better understand how these algorithms will be used by pathologists, who will remain for the foreseeable future the decision-makers. The objective of this paper is to determine the propensity of pathologists to rely on AI decision aids and to investigate whether providing information on the algorithm impacts this reliance.

Methods: To test our hypotheses, we conducted an experiment with within-subjects design using an online survey study. 116 respondent pathologists and pathology students participated in the experiment. Each participant was tasked with assessing the Gleason grade for a series of 12 prostate cancer samples under three conditions: without advice, with advice from an AI decision aid, and with advice from an AI decision aid with information provided on the algorithm, namely the algorithm accuracy rate and the algorithm model. Scores were computed by comparing the respondents’ scores with the “true” score at the individual-question level. A mixed effects logistic regression was used to analyze the difference in scores between the different conditions, controlling for the random effects of participants and images and to assess the interactions with Experience, Gender and beliefs towards AI.

Results: Participant responses to the questions with AI decision aids were significantly more accurate than the control condition without aid. However, no significant difference was found when subjects were provided with additional accuracy rate and model information on the AI advice. Moreover, the propensity to rely on AI was found to relate to general beliefs on AI but not with particular assessments of the AI tool offered. Males also performed better in the No-aid condition but not in the AI-aid condition.

Conclusions: AI can significantly influence pathologists and the general beliefs in AI could be major predictors of future reliance on AI by pathologists.

Medical Informatics

artificial intelligence

pathology

trust

reliance

diagnosis

Gleason grade

interpretability

Introduction

Decision support systems have been used for some time now in clinical pathology practice to support image analysis (1) for quantifying tasks such as oestrogen receptors, progesterone receptors (2) and HER2/neu assessments in breast cancer (3). but more sophisticated “artificial intelligence”-driven software tools are still rarely commercially available yet for pathology (4,5).

The limited use of AI in the field of pathology is changing quickly. In the last few years, artificial intelligence has made great strides and multiple papers have tested algorithms that achieved significant performances in accuracy (6). IDx-DR, an AI diagnostic system able to detect signs of retinopathy in retinal images, was approved in 2019 by the Food and Drug Administration (FDA) for use in the USA (7). And eighty percent of pathologists anticipate that AI will be introduced into their pathology laboratories within the coming decade (8).

To turn AI from promising algorithms to a practical reality, pathologists, solution providers and healthcare organizations need to overcome significant implementation challenges (7,9), which have thwarted many promising medical technological innovations in the past (10). Crucially, research within the field needs to establish whether the alleged performance of algorithms will translate into better clinical decisions and better health outcomes for patients and, furthermore, under what conditions these changes would be most effective (7).

Pathologists will play a key role in this translation into health outcomes. Many AI studies compare AI algorithms to pathologists, who are blinded to the algorithm’s results (11) often reaching accuracy rates very similar or better than pathologists. But when these systems will be implemented, pathologists and AI systems are unlikely to be competing or blinded to each other. Pathologists are expected to retain the final say as to whether to rely on algorithms’ conclusions or not (12). As a result, algorithms are likely to be used as tools that augment pathologists’ skills rather than substitute them (7,13), which corresponds to a level of only 3 or 4 on Parasuraman et al’s scale of autonomy of human interaction with automation (10 being a fully autonomous system) (5). For the foreseeable future, the impact of AI on clinical decisions and health outcomes will be mediated by the behavior of pathologists and their reliance on AI expertise.

Reliance on Artificial Intelligence

Experiments suggest that AI does indeed influence pathologists’ decisions and improve clinical outcomes over either pathologists or AI making decisions separately (14). A recent survey suggests that 73.3% of pathologists are interested or excited about integrating AI in their practice (8). However, there is also a societal skepticism towards AI for vital decisions related to healthcare (10,15). Pathologists may distrust AI expert advice and choose not to rely on it, which could slow down the adoption of these tools (10,16,17). Moreover, surveys and opinion papers reflect an abstract opinion towards “AI”, which is often a loaded and fantasized word and it is yet to be determined whether such beliefs translate into corresponding behaviors. Pathologists may express optimism towards the role and impact of AI, but the question remains whether they will rely on it to make decisions that are critical to their work and the welfare of their patients.

The algorithmic nature of the recommendation may influence decision in different ways (18). Disclosure of the artificial nature of a chatbot, for instance, can negatively impact their effectiveness (19). As Feldman et al note: “individuals may be more likely to trust those who are similar to them and nothing seems more different from a human being than an algorithm” (20). Pathologists may thus experience what has been referred to as algorithm aversion and be less likely to rely on an algorithm when making decisions (21), especially when they have expertise (22). Simply referring to these technologies as “artificial intelligence” may also generate misconceptions and pre-existing fears that may impact reliance on the technology Alternatively, people commonly form the opinion that machines never make mistakes (23), which can lead to an uncritical trust in the technology and such complacency may induce excessive reliance on decision support systems (20). Experiments suggest that faulty AI can mislead pathologists (14). Overall, the presence of AI is likely to influence the decision-making process in at least some cases (24) and we posit that:

H1: AI decision aids will influence pathologists’ diagnosis

Information on Artificial Intelligence

Researchers and surveys of decision-makers suggest that even highly accurate algorithms could face adoption challenges if AI remains a black box (25). The opacity of AI tools and their “black box” nature may thus impede adoption (17). Model interpretability means, for the user, the ability to understand how the algorithm works and how it reaches its conclusions (7). Multiple studies have confirmed that transparent automation systems are associated with greater trust (23,26) and lack of interpretability in AI conclusions is a primary concern when implementing AI technology (20). It has been argued that machines should be as comprehensible and transparent as possible (27), to the extent that more simple systems may lead to more trust and reliance than more efficient but more complex systems (26).

Research suggests that information about performance is the key factor in generating trust in machines (28). Specifically, providing confidence level helps users adjust their level of trust towards a decision aid system (23). Disclosing accuracy rate estimates with predictions of AI tools may thus help pathologists build trust in the tools’ results (17). As a result, we posit that:

· H2: Pathologists will rely more on AI if information on the AI algorithm is disclosed

Trust and reliance on AI

Outside of pathology, a mature stream of research has investigated the antecedents of reliance on automated systems, to which artificial intelligence systems belong (23,26). This research highlights that trust in the automated system is a key determinant of reliance and use of the system - “people tend to rely on automation they trust and tend to reject automation they do not” (26). We posit that:

H3: The more pathologists trust an AI system, the more they will rely on it.

In this study, we test these hypotheses and investigate the following research questions: to what extent do pathologists rely on AI? To what extent does information on AI influence reliance on AI by pathologists? To answer these questions, we conducted a survey experiment completed by 116 pathologists who assessed the Gleason grade for a series of prostate images.

Experimental design

The research design was a 1x3 within-subjects online experiment. In consultation with an expert pathologist, the Gleason grading system was identified as an appropriate task for this experiment for the following reasons: 1) a large proportion of pathologists are familiar with it; 2) it is performed relatively quickly; 3) it can lead to important clinical decisions for patients with prostate cancer diagnoses, 4) there is significant ambiguity and heterogeneity in results, with low-moderate agreement scores, ranging from 47%-70% (29), suggesting an opportunity for the implementation of machine learning tools that show comparable accuracy.

Digitized prostate biopsy samples were provided by The Radboud University Medical Center and Karolinska Institute as part of the 2020 Prostate cANcer graDe Assessment (PANDA) Challenge, which made public around 11,000 whole-slide images to explore the potential of automated deep learning systems in pathology (30). The whole slide images (WSI) are colour images and vary between 60,000–100,000 pixels in each dimension. The images were anonymized and uploaded to the cloud using PathcoreFlow™ (31), a web-based commercial image management solution and viewer for digital pathology (see Appendix A and B).

The online experiment was developed via the online survey platform, QualtricsTM. Each participant had to assess the 12 WSI through a secure link to PathcoreFlow, with four images presented into each of the three following conditions:

No-aid: no expert advice provided.

AI-aid, no information (AI-aid): expert advice that respondents were told came from an algorithm, with no further information

AI-aid with information (AI-aid+): expert advice that respondents were told came from an algorithm with accuracy rate and model information (See Appendix A for full description).

For condition C, transparency was operationalized as a brief summary in lay terms of steps typically followed by an algorithm to reach its conclusion (6,32) and revised by a team of researchers to improve readability and accessibility. The algorithm accuracy was expressed relative to typical pathologist agreement from previous studies of deep learning system (DLS) AI for whole-slide Gleason scoring (29,33) (See Appendix C). Participants were instructed to review the prostate sample image and read any additional information before providing their clinical decision.

The AI advice was presented as a result of algorithmic processes; however, these scores were actually based on the predetermined ground truth established from the PANDA challenge by 3 experienced pathologists with a subspecialty in urological pathology (34).

To control for order/carryover effects, conditions were presented in random order, except that the “AI with information” condition was always presented after the “AI, no information” condition to avoid a previous opinion bias. The more ambiguous Gleason Grade (2 and 3) were overrepresented to minimize the agreement with AI by chance (35). The conditions under which each batch of 4 images were presented were randomized so that each image appeared as frequently under each of the three conditions. The order of the items within each batch was also randomized.

After the task, participants were asked questions about the perceived trustworthiness and helpfulness of the AI tool, and general trust towards AI (see appendix D). These questions were answered on a 5-point likert scale. Finally, participants answered socio-demographic questions about age, gender, employment status, workplace, years of experience and specialization and were offered a $20 Amazon gift card. Upon completing the survey, all participants received a debrief form explaining the use of deception and a summary of their answers to allow for constructive reflection.

Participants

Participants were recruited from medical associations and social media groups for pathologists to review a series of biopsy sample images and assess the Gleason Grade for each sample. Eligible participants were practising, student or retired pathologists, with an adequate knowledge to assign a Gleason Grade. All participants provided their informed consent to the researchers and were subsequently provided with a link to practise using the PathcoreFlow™ digital pathology software before starting the survey.

460 people completed the survey. However, 59 were excluded from the analysis for incomplete responses, 101 for repeat email addresses or IP addresses and 45 for potentially using spam email addresses. An additional 139 respondents were excluded for selecting a Gleason Grade without even viewing the image on more than one of the images that had no decision aid associated with it, or for answering in less than 10 seconds, as this was deemed inappropriate by the pathology expert for proper analysis of the image. The final sample was 116 respondents. Information on the sample is provided in Table 1.

Table 1

Sample description (out of 116 participants)
Gender	%	Age	%
Male	72%	20–29	7%
Female	28%	30–39	35%
		40–49	45%
Specialization	%	50–59	9%
General/Clinical Pathology	13%	60–69	3%
Medical Microbiology	40%
Medical Biochemistry	6%	Years of Experience	%
Neuropathology	5%	0–9	25%
Anatomical Pathology	27%	10 to 19	56%
Hematologic Pathology	8%	20–29	16%
Forensic Pathology	4%	30–39	4%
Workplace	%	Location	%
Academic Health Sciences Centre (AHSC)	48%	United States of America	86%
University	8%	Canada	10%
Free-standing Lab/Diagnostic Clinic	5%	Other Countries	3%
Non-AHSC Teaching Hospital	14%
Community Clinic/Health-Centre	3%	Current status	%
Community Hospital	21%	Fellow	12%
Other Hospital	3%	Practising pathologist	66%
		Retired pathologist	9%
		Resident	8%
		Medical student	5%

Data analysis

To test our hypotheses, we performed a mixed-effects logistic regression using the lme4 package in R. Mixed-effects logistic regression is an extension of the generalized logistic model that accounts for fixed and random effects. These models are particularly useful in settings where repeated measurements are made on the same statistical units.

In the first model, the condition (No AI aid, AI-aid and AI-aid+), as our variable of interest, was specified as the fixed effect while participants and images were specified as the random effects. This approach allows us to control for potential biases introduced by individual level variances, such as ability, and image level variances, such as level of difficulty to grade. The dependent variable of the model is the score, computed as “1” if the respondent assigned the “true” Gleason Grade to the image, and as “0” otherwise. It is not essential to our experiment how accurate the “true” score is since our interest is on how influenced by the algorithm respondents are.

In the second model, to identify the factors contributing to the reliance on AI, we added the interaction between the condition and Experience, Gender and the answers to the 5 survey questions.

The average accuracy without aid was 75.3%, against 81.4% for both the AI-aid and the AI-aid + conditions. Timing data indicates that questions in the AI-aid + condition (M_AI−aid+ = 21.25s) required less time to complete than the AI-aid condition (M_AI−aid = 27.08s) and no-aid condition (M_no−aid = 44.2s).

The first mixed effects logistic regression model showed at the 0.05 significance level that AI-aid and the AI-aid + conditions improved score compared to no-aid condition (see Table 2). The results thus suggest, after controlling for image and individual level variations, when AI recommendations are present, pathologists are more likely (92% versus 87%) to provide accurate answers (i.e. align with the AI recommendations), supporting hypothesis 1. Since the AI recommendations are the true Gleason Grades, the coefficients of the conditions from the logistic mixed effect model is the extra effect of the recommendation on accuracy over the respondent’s skill and the image’s complexity, in other words, the reliance on AI recommendations (how much more likely a participant will have the right answer when AI recommendation is presented to the participant, compared to when there is no recommendation). However, no significant difference was found between scores in the AI-aid and AI-aid + conditions, thus providing no support for hypothesis 2.

Table 2

Impact of AI on physician decision making - mixed effect model results.
Condition	Log odds (standard deviation)	p-value	Odds ratio (including intercept)	Accuracy
Intercept (No aid)	1.91 (0.31)	0.000000000676***	6.75	87%
AI aid	0.60 (0.21)	0.00434**	13.30	92%
AI aid+	0.58 (0.21)	0.00595**	12.02	92%

In the second model, we introduced explanatory variables Experience, Gender and the 5 survey questions in Appendix D. Since there were no differences between the AI-aid and the AI-aid + model, we only distinguished between the “AI-aid” (including both AI-aid and AI-aid+) and “No aid” conditions. Table 3 presents the model results.

First, there are three main effects. AI-aid and Gender (Male) are significantly associated with higher scores, while the less pathologists believe that AI can help pathologists to be more efficient (1 means “strongly agree” and 5 means “strongly disagree”), the higher the accuracy of the respondent. Second, the interaction effects provide some more details on these effects. The interaction between Male and AI-aid (-1.35) was significant and almost entirely countered the impact of Male as a standalone factor (1.65). In other words, the higher accuracy of males almost entirely disappeared when AI advice was offered. Likewise, the effect of the two questions Q4 “Artificial Intelligence can help pathologists make better decisions” and Q5 “Artificial Intelligence can help pathologists be more efficient” (Q5) was entirely offset in the AI-aid condition. In other words, participants who agreed with these statements were less accurate in the No-aid condition but scored the same in the AI-aid condition (the main effect of Q4 was insignificant). Q1, Q2 and Q3 had no significant direct or interaction effect.

Finally, the interaction between AI-aid and Experience was positive, the more experienced, the more likely respondents are to rely on the AI advice, but just below significance level (p = 0.064).

Table 3

Results of the mixed effect model including interactions
	Log odds (Standard Deviation)	p-value
(Intercept)	-0.27 (1.18)	0.81919
AI-aid (baseline is “No Aid”	5.24 (1.00)	6.76e-10***
Experience	0.02 (0.03)	0.56451
Male (baseline is “Female”	1.65 (0.55)	0.00287**
Q1. Self-Confidence	-0.39 (0.42)	0.34745
Q2. Advice trustworthiness	-0.51 (0.33)	0.12751
Q3. Advice usefulness	-0.18 (0.32)	0.55908
Q4. AI and better decisions	0.55 (0.36)	0.13063
Q5. AI and efficiency	0.98 (0.38)	0.01034*
Parameter estimates of odds ratio for the interaction effects
AI-aid * Experience	0.05 (0.03)	0.06434.
AI-aid * Male	-1.35 (0.46)	0.0036**
AI-aid * Q1	-0.58 (0.34)	0.09277.
AI-aid * Q2	0.07 (0.27)	0.79939
AI-aid * Q3	-0.29 (0.25)	0.23922
AI-aid * Q4	-0.67 (0.3)	0.02473*
AI-aid * Q5	-0.89 (0.31)	0.00412**
Number of Observations: 1387; *p < 0.001, p < 0.01, *p < 0.05.

This study has two key findings. First, hypothesis H1 is supported. The difference in correct answers between the no aid condition and the two AI conditions suggests a statistically significant influence of AI on pathologists’ decisions when assessing the Gleason grade. Therefore, the presence of an AI recommendation influences pathologists. This finding confirms previous findings on the impact of AI on the decision-making process (14) and suggests that AI influences physicians in making expert decisions. Considering that such AI systems are not widely operational yet to assess the Gleason grade, this suggests a significant openness to relying on AI among pathologists, in line with positive opinions expressed by pathologists (4,8). It may also point to a risk of over-reliance on computer-aided diagnostic systems, which has been identified in radiology (37), of confirmatory bias and of alert fatigue (38).

The second key finding is that there is no difference in reliance on AI between AI without information and AI with information. This result contradicts previous findings that explicit transparency in automated tools is essential to develop trust and reliance (23). We offer two explanations for this. The first explanation is that pathologists may have preconceived opinions on the reliability of AI systems. The reliance on AI would thus depend not so much on the worthiness of the specific AI system but on their general beliefs towards AI. This is confirmed by the partial support of hypothesis H3. Reliance on AI was significantly related to general beliefs in the potential of AI to help pathologists be more efficient and make better decisions (Q4 and Q5) but not to the belief in the trustworthiness and usefulness of the AI advice provided in the experiment (Q2 and Q3). This is a surprising result suggesting that the general beliefs towards AI in general are more decisive than beliefs towards specific AI advice being used. One explanation is that pathologists may find it difficult to evaluate specific algorithms. This echoes studies showing a general positive attitude towards AI but a difficulty in having confidence in its results (39).

The second explanation is that the disclosure of a less-than-perfect accuracy rate (70%), even though it was better than the average accuracy of pathologists, nullified the impact of information on AI reliance. Medical professionals tend to have high expectations of AI accuracy (over 80% of clinicians expect AI systems to be superior to the average performing specialist) (40). There is an expectation that AI will produce perfect answers (21) and people more quickly lose confidence in algorithmic than in human forecasters even after demonstrating that the AI is more accurate on average (21). High profile harmful or inadequate performance have thus produced reactions that hindered the development of AI (36). However, the lack of correlation between Q2 and Q3 and scores again contradict this explanation. This result might also highlight the specificity of pathologists in their decision process.

Finally, the use of the advice may have been used to shorten the decision process rather than to improve its quality by double-checking, as is suggested by timing data, which shows that decisions were made faster in the two AI-aid conditions than in the no-aid condition. This confirms previous findings that algorithms reduce time spent developing a diagnosis (36).

First, despite significant efforts to eliminate suspicious responses, recruitment through social media carries some risk of participation by non-pathologists. However, no respondents had no correct answers out of the four questions in the No aid condition and only 7 out of 116 respondents had only one correct answer. Scoring the Gleason grade is a complex and non obvious task and it is unlikely that a non pathologist would be able to achieve such results. However, the sample may have included respondents from related professions that were not captured in the demographics survey such as urologists or laboratory technicians.

Second, while the within-subject experimental design can help reduce errors associated with individual differences by explicitly controlling for individual variances, and significantly increase the statistical power compared to between subject design, such design may have biased the data by anchoring reliance behaviors in the first AI condition or by activating a fatigue effect when reaching the “AI with information” task. As all participants were first exposed to the “AI with no information” condition before the “AI with information” one, they may have formed and anchored their opinion on whether to rely on the AI advice by the time they reached the “AI with information” questions. Further replications of this experiment may benefit from using a between groups design to isolate each condition and by demanding that participants complete their assessment first before viewing the AI response (21).

Implications

This study provides encouragement to implementers of AI solutions. It suggests that pathologists (and possibly other professionals) are open to using AI recommendations for critical and ambiguous decisions. Future research should investigate further how physicians and other professionals with high responsibility rely on AI recommendations, in particular, improving our understanding of the extent to which reliance on AI decision aid differs from reliance on second opinions from other physicians, the extent to which pathologists overly rely upon AI decision aid and the effects of task ambiguity on AI reliance.

It also suggests that information on the algorithms, at least on such applications, may not be as decisive as previously thought. Some of the most complex and opaque AI systems, such as deep learning ones, may thus not be at a disadvantage. This also suggests that pathologists may not feel confident to assess algorithms. Further research should investigate what factors or information can enable pathologists to confidently assess specific algorithms or the decision. Practitioners and managers may also consider that the decision (and responsibility) to rely on algorithms should be delegated to institutions, which comes with its own set of challenges.

Methodologically, it remains a challenge to recruit doctors as participants in empirical research studies (41) and our approach has been successful in recruiting a statistically significant number of physicians as respondents by enabling respondents from all over the world to participate despite restrictions caused by the COVID-19 global pandemic.

The results of this experiment suggest that pathologists are open to relying on AI recommendations for critical and ambiguous decisions. It also suggests that information on the algorithms, at least on such applications, may not be as decisive as previously thought. AI algorithms need to be seen as tools that realize their value when physicians decide to rely on them. Taking physicians into account and ensuring that they are in a position to make the best decisions, rather than placing them in competition with AI, should be a priority to practitioners, developers, implementers and regulators alike.

Artificial intelligence; FDA:Food and Drug Administration; PANDA:Prostate cANcer graDe Assessment; DLS:Deep Learning System; CMA:The Canadian Medical Association; AHSC:Academic Health Sciences Centres.

Ethics approval and consent to participate

This project has been reviewed and approved by Ryerson’s Research Ethics Board (REB 2020-316). Participants provided their written consent to participate at the onset of completing the survey.

Consent for publication

Not Applicable

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Competing interests

The authors declare that they have no competing interests.

Funding

This project is funded by the Social Sciences and Humanities Research Council (SSHRC) of Canada, which paid the research assistant and the gift cards to respondents.

Authors' contributions

JM supervised the project, performing the majority of the statistical analysis and was a major contributor in writing the manuscript

DR developed the framework for the Qualtrics^TM survey and was also a major contributor in writing the manuscript

BT provided domain expertise to develop realistic pathology cases and helped with the recruitment of pathologist as participants

AK provided the image-viewing interface and prostate image slides as well as expertise in diagnostic AI tools for the experimental design

PN supported the initial development of the experimental methodology

WH supported the statistical analysis of the study results and in formalizing the study design

Acknowledgements

We would like to thank the Social Sciences and Humanities Research Council for funding this project.

1. Niazi MKK, Parwani AV, Gurcan MN. Digital pathology and artificial intelligence. Lancet Oncol. 2019 May 1;20(5):e253–61.

2. Tuominen VJ, Ruotoistenmäki S, Viitanen A, Jumppanen M, Isola J. ImmunoRatio: a publicly available web application for quantitative image analysis of estrogen receptor (ER), progesterone receptor (PR), and Ki-67. Breast Cancer Research. 2010 Jul 27;12(4):R56.

3. Gavrielides MA, Gallas BD, Lenz P, Badano A, Hewitt SM. Observer variability in the interpretation of HER2/neu immunohistochemical expression with unaided and computer-aided digital microscopy. Arch Pathol Lab Med. 2011 Feb;135(2):233–42.

4. Tizhoosh HR, Pantanowitz L. Artificial Intelligence and Digital Pathology: Challenges and Opportunities. J Pathol Inform. 2018 Nov 14;9(1):38.

5. Parasuraman R, Sheridan TB, Wickens CD. A model for types and levels of human interaction with automation. IEEE Trans Syst Man Cybern - Part Syst Hum. 2000 May;30(3):286–97.

6. Chang HY, Jung CK, Woo JI, Lee S, Cho J, Kim SW, et al. Artificial Intelligence in Pathology. J Pathol Transl Med. 2018/12/28 ed. 2019 Jan;53(1):1–12.

7. He J, Baxter SL, Xu J, Xu J, Zhou X, Zhang K. The practical implementation of artificial intelligence technologies in medicine. Nat Med. 2019 Jan;25(1):30–6.

8. Sarwar S, Dent A, Faust K, Richer M, Djuric U, Van Ommeren R, et al. Physician perspectives on integration of artificial intelligence into diagnostic pathology. NPJ Digit Med. 2019 Apr 26;2(1):28–28.

9. Houssami N, Kirkpatrick-Jones G, Noguchi N, Lee CI. Artificial Intelligence (AI) for the early detection of breast cancer: a scoping review to assess AI’s potential in breast screening practice. Expert Rev Med Devices. 2019 May 4;16(5):351–62.

10. Vourgidis I, Mafuma SJ, Wilson P, Carter J, Cosma G. Medical Expert Systems – A Study of Trust and Acceptance by Healthcare Stakeholders. In: Lotfi A, Bouchachia H, Gegov A, Langensiepen C, McGinnity M, editors. Advances in Computational Intelligence Systems. Cham: Springer International Publishing; 2019. p. 108–19. (Advances in Intelligent Systems and Computing).

11. Somashekhar SP, Kumarc R, Rauthan A, Arun KR, Patil P, Ramya YE. Abstract S6-07: Double blinded validation study to assess performance of IBM artificial intelligence platform, Watson for oncology in comparison with Manipal multidisciplinary tumour board – First study of 638 breast cancer cases. Cancer Res. 2017 Feb 15;77(4 Supplement):S6-S6-07.

12. Jha S, Topol EJ. Adapting to Artificial Intelligence: Radiologists and Pathologists as Information Specialists. JAMA. 2016 Dec 13;316(22):2353–4.

13. Holzinger A, Malle B, Kieseberg P, Roth PM, Müller H, Reihs R, et al. Towards the Augmented Pathologist: Challenges of Explainable-AI in Digital Pathology. arXiv:171206657 [cs, stat] [Internet]. 2017 Dec 18 [cited 2021 Feb 22]; Available from: http://arxiv.org/abs/1712.06657

14. Tschandl P. Human-computer collaboration for skin cancer recognition. Nat Med. 2020 Aug 1;26(8):1229–34.

15. Feldman RC, Aldana E, Stein K. Artificial intelligence in the health care space: how we can trust what we cannot know. Stanf Law Policy Rev. 2019;30:399.

16. Parkes A. The effect of individual and task characteristics on decision aid reliance. Behav Inf Technol. 2017 Feb;36(2):165–77.

17. Colling R, Pitman H, Oien K, Rajpoot N, Macklin P, Snead D, et al. Artificial intelligence in digital pathology: a roadmap to routine use in clinical practice. J Pathol. 2019;249(2):143–50.

18. Park SH, Han K. Methodologic Guide for Evaluating Clinical Performance and Effect of Artificial Intelligence Technology for Medical Diagnosis and Prediction. Radiology. 2018 Jan 8;286(3):800–9.

19. Luo X, Tong S, Fang Z, Qu Z. Frontiers: Machines vs. Humans: The Impact of Artificial Intelligence Chatbot Disclosure on Customer Purchases. Mark Sci. 2019 Sep 20;38(6):937–47.

20. Gretton C. Trust and Transparency in Machine Learning-Based Clinical Decision Support. In: Zhou J, Chen F, editors. Human and Machine Learning: Visible, Explainable, Trustworthy and Transparent [Internet]. Cham: Springer International Publishing; 2018. p. 279–92. Available from: https://doi.org/10.1007/978-3-319-90403-0_14

21. Dietvorst BJ. Algorithm aversion: People erroneously avoid algorithms after seeing them err. J Exp Psychol Gen. 2015 Feb 1;144(1):114–26.

22. Logg JM, Minson JA, Moore DA. Algorithm appreciation: People prefer algorithmic to human judgment. Organ Behav Hum Decis Process. 2019;151(Complete):90–103.

23. Hoff KA, Bashir M. Trust in Automation: Integrating Empirical Evidence on Factors That Influence Trust. Hum Factors. 2015 May 1;57(3):407–34.

24. Petitgand C, cile, Motulsky A, Denis J-L, R&#233, Gis C. Investigating the Barriers to Physician Adoption of an Artificial Intelligence- Based Decision Support System in Emergency Care: An Interpretative Qualitative Study. Digital Personalized Health and Medicine. 2020;1001–5.

25. Bera K, Schalper KA, Rimm DL, Velcheti V, Madabhushi A. Artificial intelligence in digital pathology — new tools for diagnosis and precision oncology. Nat Rev Clin Oncol. 2019 Nov;16(11):703–15.

26. Lee JD, See KA. Trust in Automation: Designing for Appropriate Reliance. Hum Factors. 2004 Mar 1;46(1):50–80.

27. Patrzyk PM, Link D, Marewski JN. Human-like machines: Transparency and comprehensibility. Behav Brain Sci [Internet]. 2017 ed [cited 2021 Feb 22];40. Available from: http://www.cambridge.org/core/journals/behavioral-and-brain-sciences/article/humanlike-machines-transparency-and-comprehensibility/5A1BCA43DEE578220D66B3089F8DED46

28. Hancock PA, Billings DR, Schaefer KE, Chen JYC. A Meta-Analysis of Factors Affecting Trust in Human-Robot Interaction. Hum factors. 2011 Oct 1;53(5):517–27.

29. Nagpal K, Foote D, Liu Y, Chen P-HC, Wulczyn E, Tan F, et al. Development and validation of a deep learning algorithm for improving Gleason scoring of prostate cancer. npj Digit Med. 2019 Jun 7;2(1):48.

30. Zenodo [Internet]. [cited 2021 Feb 22]. Available from: https://zenodo.org/record/3715938/preview/ThePANDAchallenge_ProstatecANcergraDeAssessmentusingtheGleasongradingsystem.pdf

31. Pathcore [Internet]. [cited 2021 Apr 28]. Available from: https://www.pathcore.com/

32. Arvaniti E, Fricker KS, Moret M, Rupp N, Hermanns T, Fankhauser C, et al. Automated Gleason grading of prostate cancer tissue microarrays via deep learning. Scientific reports. 2018 Aug;8(1):1–11.

33. Samaratunga H, Delahunt B, Gianduzzo T, Coughlin G, Duffy D, LeFevre I, et al. The prognostic significance of the 2014 International Society of Urological Pathology (ISUP) grading system for prostate cancer. Pathology. 2015 Oct 1;47(6):515–9.

34. Ström P, Kartasalo K, Olsson H, Solorzano L. Artificial intelligence for diagnosis and grading of prostate cancer in biopsies: a population-based, diagnostic study. Lancet Oncol. 2020 Feb 1;21(2):222–32.

35. Epstein JI, Allsbrook WCJ, Amin MB, Egevad LL. The 2005 International Society of Urological Pathology (ISUP) Consensus Conference on Gleason Grading of Prostatic Carcinoma. Am J Surg Pathol. 2005 Sep;29(9):1228–42.

36. Printz C. Artificial intelligence platform for oncology could assist in treatment decisions. Cancer. 2017;123(6):905–905.

37. Lehman CD, Wellman RD, Buist DSM, Kerlikowske K, Tosteson ANA, Miglioretti DL, et al. Diagnostic Accuracy of Digital Screening Mammography With and Without Computer-Aided Detection. JAMA Intern Med. 2015 Nov;175(11):1828–37.

38. Yu K-H, Kohane IS. Framing the challenges of artificial intelligence in medicine. BMJ Qual Saf. 2019 Mar 1;28(3):238–41.

39. Jungmann F, Jorg T, Hahn F, Santos DP dos, Jungmann SM, Düber C, et al. Attitudes Toward Artificial Intelligence Among Radiologists, IT Specialists, and Industry. Academic Radiology [Internet]. 2020; Available from: https://www.sciencedirect.com/science/article/pii/S1076633220302038

40. Scheetz J, Rothschild P, McGuinness M, Hadoux X, Soyer HP, Janda M, et al. A survey of clinicians on the use of artificial intelligence in ophthalmology, dermatology, radiology and radiation oncology. Scientific Reports. 2021 Mar 4;11(1):5193.

41. Verma P, Ford JA, Stuart A, Howe A, Everington S, Steel N. A systematic review of strategies to recruit and retain primary care doctors. BMC Health Services Research. 2016;16.

No competing interests reported.

Download PDF

Version 1

posted

You are reading this latest preprint version

Impact of Artificial Intelligence, With and Without Information, on Pathologists’ Decisions: An Experiment

Status:

Version 1

Abstract

Background

Materials And Methods

Results

Discussion

Limitations

Conclusions

Abbreviations

Declarations

References

Additional Declarations

Status:

Version 1