Health Information from Management to Technology: Development of a Radiology Patient Safety Monitoring System

Background: Medical imaging is an intervention through which patient safety (PS) is of great importance. In monitoring PS, the major challenges include lack of relevant data, proper control, and appropriate feedback in taking the necessary measures. To meet these deciencies, the related data should be investigated precisely and advanced technology should be applied to monitor the quality of imaging. Objective: The purpose of this study was to design and develop a PS monitoring system at the radiology department to mitigate adverse events. Methods: This developmental research was conducted in multiple phases including content determination using Delphi technique, conceptual modeling using Rational Rose software, database designing using SQL Server, user interface (UI) building using Agile software, and system evaluating in three aspects of UI requirements, the accuracy of calculating, and usability. Results: In this study, 110 PS-related important data elements were identied in 14 main groups and 26 PS performance metrics, as the system contents. The ERD and UML diagrams were drawn and the UI was created in three tabs: pre-procedure, intra-procedure, and post-procedure. Finally, the evaluation results proved the technical feasibility and application prospect of the radiology patient safety monitoring system (RPSMS). Finally, the usability of the system was highly rated (76.3 from 100). Conclusion: The RPSMS, offers the possibility to complement the datasheets for gaining a more accurate picture of the PS status and bringing up its aspects, which might otherwise go unnoticed or be underestimated by clinicians.


Introduction
In the healthcare system, the concept of patient safety (PS) should be monitored and audited as a critical component of care quality. The PS concept allows the health care stakeholders to operate without harming individuals, even in complex high-risk environments (1).
An error is de ned as a deviation from the expected norm, regardless of any damage (2). In this regard, medical errors represent unwanted acts, failure to perform the desired action(3), failure to conduct a planned action, mistake in reaching the goal (4), or deviation from the care process that may harm a patient (5).
In health systems, preventable medical errors can have fatal consequences. Advances in health information technology can have the potential to improve the health care delivery process and decrease medical errors (6).
Medical errors are one of the most common causes of death with millions of victims in the world (7). To deal with these errors, we need to learn from them. To hit this target, medical errors should be identi ed, recorded, and investigated accurately by the patient's safety information system (8,9). Such systems can prevent medical errors proactively and are among the most important requirements of implementing PS plans to mitigate medical error risks. They also have the capabilities to identify, analyze, and report the causes of incidents, adverse events, and near-misses (10).
In the radiology department (RD), medical errors cause a great challenge. Considering the complexity of the health care system, Larson et al. posited that maintaining and improving levels of safety depends on developing a system and a culture that can intelligently integrate individuals with technology and processes to create a safer patient care environment (1).
This research aimed to investigate the implementation procedure of a routine PS monitoring approach at an RD using a radiology patient safety monitoring system (RPSMS), which collected data before (pre), during (intra), and after (post) an imaging procedure.

Materials And Methods
In this qualitative and developmental research, a multi-method approach was employed including interviews, structured observation, questionnaires, and workplace walkthroughs. The developmental steps were based on soft system methodology as follows: Step1: Appreciate the problem situation We analyzed the situations and determine the problems. In this regard, routine work activities and all paper artifacts used during medical imaging examinations were observed. During this investigation, we found that there was no systematic and electronic approach to monitor the patient's safety in Iran. This situation analysis posed the following questions: -How can improve Ps in RDs?
-What kind of changes may be systemically desirable and feasible in this situation?
Step 2: Formulating root de nitions -Content selection A researcher carried out direct observation and open-ended interviews with key informants to gather preliminary contextual data about policies, procedures, and normal routines related to documenting and checking safety.
Then, in-depth interviews were conducted with radiologists to determine the effective issues of PS in RDs. Also, these interviews focused on the sequence of tasks, and the information required to performing these tasks.
Next, a literature review (8, was performed to gain a good understanding of the data elements, which contributed to the design of this system as the data content of the system. After the extraction of data elements, again, the in-depth semi-structured interviews were conducted with radiologists to determine the relevant items for the pre, intra, and post procedures. -Delphi technique A panel of experts (n=30 members) composed of radiologists, medical physics, and radiology technicians with more than 5 years of professional experience were asked to con rm these items using the Delphi technique. Furthermore, an open-ended question was added to the questionnaire to seek further potential data elements. In this process, the experts were asked to determine whether an item is necessary or not. Considering that no additional suggestions existed in the rst round, Delphi was terminated at this stage. The experts' opinions' mean scores were calculated for each item; a mean score of higher than 50% showed that the item was acceptable, while a mean score of less than 50% indicated that the item was not acceptable. Based on the nal panel's rating, a list of data elements was developed to be used.

-Patient Safety Indicators
A serious challenge in implementing the quality improvement plans for medical imaging was related to the exact de nition of metrics. In a previous study, we have determined the key performance indicators (KPIs) and related metrics to patient safety in the RD for Iran (47). In another article, we have de ned them and prepared the identi cation table (7). In this study, we have used them.
Step 3: Building conceptual models the system's conceptual model was depicted to determine the relationships among the data elements by drawing ERD (Entity Relationship Diagram) and UML (Uni ed Modeling Language) diagrams (class, activity, sequence) using the Rational Rose software. These graphs were used to represent the entities and their relationships graphically.
Step 4: Converting models to real world The user interface (UI) was built by Agile software concerning three components of front-end, back-end, and reporting. This software was designed based on C # and written using the .Net Framework to use WPF (Windows Presentation Foundation) language tools with the ability to monitor, record, and report the results in charts and tables. The reports are also presented by a combination of C #, HTML, JavaScript, and CSS (Cascading Style Sheets), which is easier and faster than the Crystal Report software in the Windows environment.
For the data acquisition, the limited amount of data such as demographic information, medical record number, physician's name, and ward, the system utilizes from the hospital information system (HIS). The rest of them are captured by the users (dedicated staff) through keyboards instantly. The data can be entered by a radiologist, a radiologist's assistant, a radiology nurse, and an educated radiology technician.
Next, the database of this software was developed by Microsoft SQL Server software (2016), which is used to report the latest version of the browser considering the JavaScript libraries upgrade. Finally, the UI was developed using the Microsoft Visual Studio software.
Step 5: De ning possible changes This system was tested preliminarily using three types of evaluations.
-The user interface veri cation To ensure that all UI requirements were met completely, the system was evaluated by 5 IT experts using the International Standard ISO 9241/10. This is a checklist consisting of 50 criteria in seven main axes analyzed using Excel software. The veri cation was performed by 5 IT experts.

-The metrics calculation validation
To ensure that the PS metrics are calculated accurately, these calculations were validated using the test data manually and machinery. In this order, rst, some of the metrics with test data were calculated manually. Then they were calculated by the RPSMS. Finally, the numbers obtained from both methods were compared. The validation was performed by a researcher of the team.
-The usability evaluation The usability of the RPSMS was measured through the System Usability Scale (SSU), designed by John Brooke in 1986. The SUS consists of a 10-item questionnaire with ve response options ranging from strongly agree to strongly disagree and provides a reliable tool for measuring usability. This type of evaluation was performed by 10 end-users of the system including radiologists.
To calculate the SUS score, the score contributions from each item (ranging from 0 to 4) were summed up. For items 1, 3, 5, 7, and 9 the score contribution included the scale position minus 1. For items 2, 4, 6, 8, and 10, the contribution was 5 minus the scale position. The sum of scores was multiplied by 2.5 to obtain the overall value of SUS. It should be noted that although the SUS scores ranged from 0 to 100, they are not percentages and should be considered only in terms of their percentile ranking.

Results
By reviewing the related studies, 143 important data elements in 14 main groups were extracted. After the rst round of Delphi, 110 data elements were selected by consensus to have good potential for application in RPSMS .These data elements related to pre, intra, and post procedure are presented in Table1.
The conceptual models designed using ERD and UML diagrams are presented in Figures 1-3. The RPSMS was the implemented prototype software tool for monitoring and managing safety in RDs. Its main tabs (including pre, intra, and post procedure) contain a collective set of safety-related data elements associated with KPI metrics. A dropdown list box was considered for checking the data elements for each patient (Figures 4-6).
The system included the following functionalities: 1) application of a username and password to log in, 2) selection of the system language (Persian-English) by the user, 3) edition and modi cation of the users' information, 4) use of reports, back to previous pages, signing out, alert, and help icons for all pages and processes, 5) capability to search, edit, remove, and display all patient's information historically, 6) deactivation of tabs and options to check all data elements accurately, and 7) presentation of the reports using tables and charts in printable pdf and excel formats.
Furthermore, all KPI data could be further ltered using various selection criteria such as modality, dates (day, week, month, or year), as well as examination and encounter type. User selection of these lters or a combination of the lters results in the system dynamic re-visualization of the data in real-time.
Finally, the accuracy of KPI metric calculations was individually tested and veri ed. Later, to determine the appropriateness of capabilities, structure, and visualization of the safety-related KPI metrics were used ISO 9241/10. As shown in Table 2, participants indicated that was user-friendly and could support the main UI requirements. Finally, as presented in Table 3, the participants indicated the usability of RPSMS to be at the desired level (76.3 from 100). After these evaluations, the necessary corrections were made to RPSMS.

Discussion
The quality improvement process is different from the strategies designed to increase pro ts. In medicine, the main goal is to bene t the patients and the healthcare organizations' endeavor is aimed to increase the quality of care along with cost reduction. Therefore, PS is a responsibility of great importance, which is ful lled better using related indicators that improve performance (7,48,49). Safety is optimized in the environment by monitoring the safety indicators. A signi cant relationship was observed between quality improvement and PS performance indicators (50).
To move in this path, proper data elements should be selected initially and then appropriate methods should be employed to collect these data actively to calculate the PS metrics. Therefore, obtaining the required data should be instilled into the culture of healthcare organizations as a part of their daily routine. Furthermore, appropriate mechanisms should be adopted for achieving and controlling them (30, 47, 51-53).
The Joint Commission also emphasized that all healthcare organizations are required to develop a comprehensive system for identifying and mitigating safety risks. In medical imaging processes, the application of a system to assist the staff in monitoring and identifying the potential PS-related events and reporting the adverse events can increase PS (7, 19, 22, 53)(9).
In Reason's Swiss Cheese Model of Medical Error, Brook et al. proposed an error classi cation system for diagnostic radiology that included personnel, communication, cause, and impact characteristics. Their system was focused on discovering the latent system failures to decrease the odds of future errors and diminish their adverse impact (23). Accordingly, for RPSMS, we used all the patient safety-related metrics obtained from Karami's study (47) which covers the items of the cheese model.
The quality value map was drawn by Swensen et al. for radiology that followed the patient's path from the referring physician's o ce to the RD containing the major steps required for ordering, performing, and reporting an examination. This map provides a basis for understanding the opportunities for improvement of the radiologic safety, reliability, quality, and appropriateness of the examinations and interventions (35). We found that it is critical to check safety in different steps. So, we design this system for monitoring and measuring the safety indicators in the pre-procedure (before the patient enters the RD), intra-procedure (when the patient is ready for the procedure in the RD), and post-procedure (after the procedure and before the patient leaves the RD) stages.
Multiple studies showed that different methods and interventions were used to improve the quality and safety of RDs. In these studies, we found the traditional paper-pencil forms were used for PS data collection, which is generally associated with relatively laborious working procedures (especially, regarding data processing and storage), prone to data input errors, and dependent on a lot of human resources. On the other side, Schultz et al. developed a Web-based radiology-speci c event reporting system. During the initial development of this system, it identi ed and addressed the potential safety concerns and shortcomings positively and easily (27). Therefore we decided to develop the PRMS as an e-monitoring tool with pre-de ned requirements to control and prevent adverse events.
In various studies, checklists have been used to control safety, but each focuses on a part of safety and a speci c modality. For example, Koetser et al. developed a speci c radiological PS System (RADPASS) checklist for performing interventional radiology and assessing the effect of this checklist on the health care processes of radiological interventions. Application of this checklist reduced deviations from the optimal process by three quarters and was associated with fewer procedure postponements (54).
In another study, Schultz et al. introduced two means for standardizing the work processes. The rst was a checklist for a time-out routine to comply with the Universal Protocol and help to eliminate errors in interventional procedures. The latter was a owchart applied for the radiologic imaging evaluation of pregnant patients that exempli es lean standardization of a work process to eliminate errors (30).
To design a checklist, Ra ei et al. stated that items of a well-designed checklist should address the underlying failure modes effectively for the adverse events that occur in any particular operational environment. In addition, the checklist should be designed to facilitate reliable execution of the control strategy for the failure modes. In other words, addressing the causes of every conceivable adverse event would result in a long and impractical checklist. Patient safety is better served by allowing local teams to design a checklist from a list of items that match the operational requirements of the working environment and case mix. As a result, a series of potential checklist items should be provided along with their rationale" (55).
In the present study, we designed an electronic checklist and addressed various aspects of safety and also included in this system an item called modality. This checklist may be necessary to complete by two or three persons. As ''Rad Check'' was a two-person veri cation system introduced by Rubio et al. to decrease wrong-patient or wrong-study errors. In the veri cation process, two health care employees read back the patient's name and medical record number with additional verbal con rmation of the required intervention to be conducted. The read-back procedure needed both a patient armband and paper or electronic order (22).
Our system also provides recommendations and alerts to prevent errors, reports adverse events, and presents the results in tables and charts. Kim et al. also set up an automatic real-time patient medical radiation dose management system for all modalities. This system utilized the radiation dose dataarchiving method of standard digital imaging and communications in medicine (DICOM) dose structured report combined with a DICOM modality performed procedure step. The capabilities of this system were to display the graphs for analyzing each patient, equipment setup, operator, and examination (14).
As Kruskal et al. described a quality management system including PS, process improvement, customer service, professional staff assessment, and education for implementing continuous programs to monitor performance, analyze and depict data, implement change, and meet the regulatory requirements in a large academic radiology department (48). The PRMS is designed to monitor, collect, and report the PS data for performance improvement.
This system is currently being pilot tested to determine its usability and feasibility. More advanced features will be added to it based on the pilot usability study. In future studies, PRMS will be tested to determine its effect on the improvement of PS during months and years as mentioned in studies as follow: Gottumukkala et al. developed a checklist-based scoring system to rigorously assess compliance and used a system of video monitoring and feedback to track performance and improve the time-out process in pediatric interventional radiology. This system led to substantial improvements in time-out performance over 3 years and could address the common failure modes (46). And, to enhance PS, Corso

Conclusion
This study provides a picture of how to manage health information to achieve a health information technology that can effectively make changes in routine work. Understanding work ow, information ow, and provider needs can play a critical role in developing strategies to design an effective e-monitoring system.
The RPSMS, offers the possibility to complement the datasheets for gaining a more accurate picture of the PS status and bringing up its aspects, which might otherwise go unnoticed or be underestimated by clinicians. It is noteworthy that the health care stakeholders have to develop a pragmatic implementation strategy for collecting data routinely.
Acceptance and application of this system by staff can pave the way to extend RPSMS data collection from pre/Intra/post procedures. Maybe the employees consider it as an increasing workload. Therefore, they should be educated and justi ed.
De nitely, this system has some limitations, by increasing its use, more feedback will be received from users, which then will be used to improve it.