A whole‐slide imaging based workflow reduces the reading time of pathologists

Even though entirely digitized microscopic tissue sections (whole slide images, WSIs) are increasingly being used in histopathology diagnostics, little data is still available on the effect of this technique on pathologists' reading time. This study aimed to compare the time required to perform the microscopic assessment by pathologists between a conventional workflow (an optical microscope) and digitized WSIs. WSI was used in primary diagnostics at the Laboratory for Pathology Eastern Netherlands for several years (LabPON, Hengelo, The Netherlands). Cases were read either in a traditional workflow, with the pathologist recording the time required for diagnostics and reporting, or entirely digitally. Reading times were extracted from image management system log files, and the digitized workflow was fully integrated into the laboratory information system. The digital workflow saved time in the majority of case categories, with prostate biopsies saving the most (68% time gain). Taking into account case distribution, the digital workflow produced an average gain of 12.3%. Using WSI instead of conventional microscopy significantly reduces pathologists' reading times. Pathologists must work in a fully integrated environment to fully reap the benefits of a digital workflow.


INTRODUCTION
Over the last two decades, the field of pathology has seen the introduction of devices capable of digitizing entire microscope slides at a resolution high enough to be used for primary diagnostics. This technology, known as "whole slide imaging," provides numerous benefits while also posing challenges. Whole slide imaging enables a diagnostic workflow in which physical glass slides are not distributed among pathologists, are not damaged or lost, and do not need to be retrieved from the archive. Instead, digital images are easily distributed, retrieved from digital storage, and offered for consultation to colleagues (even from afar). 1 It greatly facilitates expert consultation and revision in the event of patient referral, potentially increasing the overall diagnostic chain's efficiency and quality. On the other hand, the introduction of whole slide imaging is hampered by the fear of incorrect diagnoses and the high additional costs associated with digitizing microscope slides. 1 Many studies focused on the first issue: can whole slide images (WSI) safely replace traditional optical slide microscopy? The first WSI device was cleared by the US Food and Drug Administration in 2016 to be marketed as an alternative method of slide reading by pathologists. This indicates that there is a substantial body of evidence demonstrating that using WSI for diagnosis is not inferior to using an optical microscope. 2 While this addresses the first issue mentioned above, the widespread implementation of WSI remains a pipe dream, owing primarily to the associated costs. There has been little research into the financial implications of implementing WSI in primary diagnostics. The costs of digitizing the entire pathology workflow are relatively simple to calculate, though some "hidden" costs may be more difficult to assess. Financial gains are more diffuse and difficult to establish. Workflow logistics and pathologists' diagnostic efficiency are two significant sources of potential financial gain. 1 The former has already been addressed, 3 while the current study focuses on the latter: the time it takes the pathologist to sign out a case. Currently, there is very little data on the effect of WSI on pathologists' reading time. Preliminary data published in 2010 indicated a potential efficiency gain of about 14% based on extrapolation of radiology data and limited observations of pathologists. 4 In a study of 510 surgical cases, WSI reading times required 4 s more per case than microscope assessment, 5 but reading time decreased significantly with increased WSI assessment experience. The current study was performed in one of the first pathology laboratories worldwide that have adopted an entirely digitized workflow, offering a unique opportunity to research the influence of this transition on time required for diagnosis making. The log files of the digital image management system were studied and compared with data collected from pathologists using an optical microscope in the same diagnostic setting.

Experimental setup and case inclusion
This study was performed in the Laboratory for Pathology Eastern Netherlands (LabPON, Hengelo, and The Netherlands). During this study, LabPON conducted preliminary readings of cases using WSI and conventional microscopy. The study had two independent arms: digital slide reading and conventional slide reading. This study used consecutive cases offered for diagnostics during the study period, resulting in cases being randomly distributed across the two arms.
The pathologist's time required for the entire microscopic case reading, excluding possible consultation of colleagues, was registered. This study did not involve any patient-related data or influenced the diagnostic process, obviating ethical approval requirements.
Conventional slide reading Figure 1 depicts a detailed breakdown of pathologists' workflow in a routine clinical setting using conventional light microscopy (also known as "conventional" workflow). In summary, a pathologist is given a stack of case folders containing one or more glass slides for each case. The pathologist prioritized the cases based on the date the tissues were received and the complexity of the cases. Following that, diagnostics were performed as usual on a case-by-case basis. The red rectangle in Figure 1 shows the workflow executed for every single case. Data are first retrieved from the lab management system (Poema, Finalist, Groningen, The Netherlands), containing patient data including patient history, clinical question, and information from grossing. Next, slides were microscopically assessed, after which the diagnostic report was produced.
The time required per case was recorded in this study using the standard barcode attached to each slide. This included the pathologist's reading time for all slides in a case (including immunohistochemistry). Before microscopic assessment and after completing a reading session for a specific case, participating pathologists (n = 4) manually scanned the barcodes.
F I G U R E 1 Overview of the diagnostic workflow using a conventional light microscope by the pathologist. The red box indicates the part of the workflow that was analyzed in this study.
All data were digitally recorded, allowing for detailed time-per-case analysis later on. Subanalyses for individual pathologists and case categories were also carried out.

Digital slide reading
Starting in 2015, LabPON gradually adopted an entirely digitized workflow using WSI scanners (IntelliSite Ultra-Fast Scanner, Philips, Best, The Netherlands) and the Philips IntelliSite image management system (IMS). 6 The diagnostic workflow of the pathologist was facilitated by the IntelliSite Pathologist Suite ( Figure 2). Figure 2 shows the case list window (right panel). The case diagnosis window shows scanned slides at high resolution, enabling zooming, creation of annotations and tags. This workflow will be referred to as "digital" workflow. A total of five pathologists participated in this arm of the study, with varying experience in primary digital reading of cases (one pathologist >2 years, one pathologist >1 year, and three pathologists with at least 6 months experience).
The analysis of the time required per case for the digital workflow in this study was based on the IMS log files. The log files contained data on the pathologist's interactions with the IMS and internal machine-generated messages. As these files were not produced for this specific purpose, post-processing of the log file data was required. To separate the data required for this study from the bulk of available data, specific data filters were designed for the present study.

Post-processing of IMS log files
Reading a single case may comprise multiple reading sessions when the case cannot be finished in one session. The duration of different sessions for one case was all taken into account in this study. Opening a reading session for a case is a clearly defined event in the log files. Closing a reading session was only explicitly coded in the log file if the pathologist changed the state of the case to "finished" or "review." However, finishing a reading session that did not end with one of these events is not explicitly coded in the log files. Therefore, the last action of a pathologist before opening the next case was considered to indicate the end of a reading session for a case.
No image was opened in approximately 35% of sessions. In 26% of these, the pathologist only changed the case status to "review" or "completed." The pathologist did not use the case diagnosis window in the remaining 74% of sessions, indicating that no F I G U R E 2 Shows the case list window (right panel). The case diagnosis window shows scanned slides at high resolution, enabling zooming, creation of annotations and tags.
A WSI BASED WORKFLOW REDUCES FOR PATHOLOGIST | 129 digital inspection was performed. As a result, these sessions were ignored in the current study. Sessions in which cases with the status "completed" were reopened were also excluded.
Significant time gaps between successive events in the log file were occasionally observed. Indeed, pathologists may engage in non-diagnostic activities during the diagnostic process as a result of observations of the diagnostic process and as is well known from daily practice (so called non-diagnostic gaps). These gaps could only be identified when WSI from the slide navigation and login and logout. Observations and discussions with pathologists and medical staff revealed that up to 5 min was spent in case of professional discussion of the current case, so gaps up to 5 min were taken into the analysis time. Gaps of >5 min were excluded from analysis time, assuming they were "non-diagnostic." To compensate for the bias introduced by this phenomenon, an upper threshold had to be set to the time accepted between successive events. The threshold was established by inspection of histograms of time gaps for the participating pathologists. Figure N2 shows the number of these gaps and their duration for each pathologist in the digital arm. From this, we can conclude that the gap time is approximately 5 min. After consultation with the participating pathologists, the approximate median time (5 min) was acceptable as a safe upper limit for gap duration to be considered in the analyses. Please note that because in the conventional workflow, the pathologists explicitly indicated the beginning and end of reading sessions, no gap analysis was performed for that study arm.

RESULTS
Over 2 months, 1984 cases were read by four pathologists in the conventional workflow, and 1834 cases were read in the digital workflow by five pathologists (Table 1). Only 15 case categories had 20 or more conventional and digital reading cases and were included in the results to enable calculating statistics with sufficient accuracy. Table 2 shows statistics on reading times using conventional microscopy for the most prevalent case categories (N > 20). The median reading time per case ranged from 1.52 min for gall bladder to 9.73 min for prostate biopsies (prostate care program). Figure 3 shows the reading time distribution in boxplots, per case category (left boxes).

Digital slide reading
To reduce the impact of short periods when a pathologist was engaged in activities other than diagnostics, the duration of gaps in the log files was limited. Figure 3 depicts the distributions of these gaps for each of the five pathologists. A total of 41 gaps (equal to 3.5% of total reading time) exceeded 1 h. Figure 3 depicts gaps <1 h in length. For the various pathologists, we observed a median gap time of about 5 min. This was determined to be an acceptable safe upper limit for gap duration to be considered in the analyses after consultation with the participating pathologists. Because the reading time was determined using log files in digital diagnostics, it was necessary to count the gaps and account for them T A B L E 1 Overview of cases assessed in this diagnostic reading time study in two arms: use of conventional optical microscopy versus digitized whole slide images. when calculating the reading time. This problem was not present in the standard workflow; pathologists explicitly indicated the start and end of reading sessions, and no gap analysis was performed for that study arm. Table 2 displays reading time statistics for the most common case categories (N > 20), ranging from a median of 1.25 min for skin neoplasias to 3.44 min for lung biopsies. Figure 4 depicts the reading time distribution by case category graphically (right boxes). Table 2 compares the median reading time for the digital and traditional workflows for the most common case categories. As can be seen, the digital workflow required less time than the traditional workflow for the majority of common case categories (calculated not for each pathologist but average times). Due to the fact that in the process of digital diagnostics it is possible to view several slides and therefore several slides with different staining together, this fact makes the reading process much easier and reduces the reading time. Prostate biopsies had the greatest time difference (both relative and absolute) (6.64 min/68% time gain for the digital workflow). The total reading time for the conventional and digital workflows was calculated by adding the reading times of all cases in these workflows. The average reading time for 1984 conventionally read cases and 1834 digital cases was calculated separately. The median reading time per case (irrespective of the number of pathologists readings conventionally or digitally) ranged from 1.52 min for gall bladder to 9.73 min for prostate biopsies (prostate care program). This resulted in a relative time gain of 12.3% by digital reading.

DISCUSSION
This study assessed the effect on pathologist reading time when transitioning from a conventional workflow with a traditional optical microscope to an entirely digitized workflow using WSIs. The study was performed in a primary diagnostic setting and comprised 3818 cases total. We found that for most case categories, the digital workflow required less reading time by the pathologist. Taking the case mix into account, an average time gain of 12.3% was observed in favor of the digital workflow.
The effect of a digital workflow on pathologist reading time is not consistent across published studies. One important explanation is that the advantages of a digital workflow are only fully realized in an ideal situation. 7 Previous research found that a digital workflow required more reading time from pathologists. 8,9 A less-than-ideal user interface, combined with a lack of integration of the WSI viewer with the AP-LIS, most likely hampered a smooth operation, resulting in lower efficiency for the digital workflow. Using an improved virtual microscope in a later study, no overall time difference was observed between digital and conventional workflows. 10 Although the digital workflow benefitted from the absence of physical loading and unloading of glass slides (making up 16% of the time on the microscope), this potential time gain was lost again by significantly (even up to 6.6 times) more additional slide views in the digital workflow. The authors hypothesized that the additional slide views might be attributed to unfamiliarity with the relatively new technology. 10 A similar phenomenon was observed in a recent study on 510 cases. 5 The time required for diagnosis in a conventional versus digital workflow for three pathologists was studied in an experimental (i.e., nonclinical) setup. It was concluded that the time difference between both modalities was negligible. However, the three pathologists were inexperienced in the digital workflow, probably resulting in longer reading times. This was evidenced by the fact that during the course of the study, case reading times in the digital pathology workflow significantly decreased. 5 A randomized equivalency and efficiency study aimed to evaluate the equivalency and efficiency of glass slides to WSI reporting in pathology practice workload in clinical settings. The data showed that despite the WSI being no inferior to glass slides, there was a decrease in the efficiency of diagnostic turnaround time. 11 However, the reading time of WSIs and the efficiency of turnaround time can be improved with the entire digitalized workflow and fully integrated laboratory information management F I G U R E 4 Boxplots of the distribution of time needed for diagnoses, per case category for the conventional (left) and digital (right) workflow. Data from the digital workflow are corrected for gaps, as indicated in the text. system, which can be related to the benefits of a digital workflow.
The present study setup comprised an entirely digitized and integrated primary diagnostic workflow, which has been operational for several years. Therefore, the negative biases described above will likely not be present. The digital pathologist workplace was optimized over a more extended period before this study, comprising broadband network connections and big, high-resolution Dell 27 inch screens. Participating pathologists were experienced in digital slide reading. We consider this the optimal setup for this type of study, giving a realistic view of the time required for diagnostic work in both study arms. The potential improvements in pathologist efficiency can only be fully realized in such an optimized setting. A single pathologist's study comparing diagnostic time in a digital versus conventional workflow discovered that digital slide reading saved 5.9% of time (going from 97.8 to 92.05 min per session). Cases were diagnosed using a digital workflow at first, then a traditional workflow 6 months later. This time gain is 52% less than what was discovered in the current study. However, only the slide reviewing time was examined, as opposed to the current study, which included time for case reporting, resulting in the total time required for signing out a case. This process consisted of workflow tasks (e.g., handling individual slides, ordering immunohistochemistry, etc.), slide review, and reporting. Stratman et al. 4 performed a pathologist time and motion study, observing several pathologists during their routine work in a conventional (i.e., non-digital) setting. Time spent on these different subtasks was 36.0%, 34.6%, 13.4%, and 16.0% for, respectively, slide review, reporting, workflow-related tasks, and other tasks (not further specified). They concluded that workflowrelated tasks (13.4%) in particular offered a window of opportunity for time gains by transitioning to a digital workflow, which appears to be consistent with the current study results.
In addition to the time gain found in the current study for reading, significant pathologists' time may be saved in other tasks such as tumor board preparation and consultations. 1 However, this was outside the scope of this study and was dependent on the storage parameters. Furthermore, it is expected that artificial intelligence-based algorithms will significantly reduce pathologists' reading time, for example, by prescreening slides for the presence of tumors. 12 Because the current study was conducted entirely in a primary diagnostic setting, the two workflows contained inherently different cases, with 1984 cases read by four pathologists in the conventional workflow and 1834 cases read by five pathologists in the digital workflow. However, we assume that no bias was introduced because there was no selection bias in including cases in one or the other study arm, the large number of cases in the two arms, and the broadly comparable numbers per case category (see Table 2). Due to the use of separate large datasets, we purposefully chose to use the randomized cases for both conventional and digital pathology to avoid any discussion about washout times, which may vary significantly between individual pathologists. The requirement to extensively post-process the digital log files and filter out gaps that most likely represent non-diagnostic activities is one of the current study's most significant limitations. We chose a safe lower limit of 5 min for the duration of gaps to exclude, assuming that this would not negatively bias the digital workflow study results. Finally, moving to a fully integrated digital diagnostics workflow has the potential to save pathologists over 10% of their reading time. Only a fully integrated setup fully realizes the benefits of such a workflow. This will help with the development of the business case for the significant investments required for a fully digital workflow.

AUTHOR CONTRIBUTIONS
Alexi Baidoshvili, Paul J. van Diest, and Jeroen A.W.M. van der Laak developed the theory and performed and verified the analytical methods, designed the model and computational framework, and supervised the findings of this work. Mariam Khacheishvili oversaw the data collection process and co-wrote the manuscript with Alexi Baidoshvili. All authors worked out technical details and performed the numerical calculations for the proposed research, discussed the results and contributed to the final manuscript, and provided critical feedback and helped shape the research, analysis, and manuscript. The study was carried out by Jeroen A.W.M. van der Laak and Paul J. van Diest.