In this study, 27 participants took part in the interviews which represented 79% of all possible staff. The average length of interviews was 45 minutes. The demographic characteristics of the participants are presented in Table 1. Most of the participants (n = 23, 85%) were female and were represented by doctors (n = 10), pharmacists (n = 7) and nurses (n = 10). The impact of the EMMS on the various elements of the UTAUT have been described in detail below. Interviews generated rich data relating to usability and acceptability of the EMMS. The themes in these categories were mapped to the UTAUT framework (Table 2).
Table 1
Interview themes mapped to the UTAUT framework. EMM = Electronic Medication Management System
Measure | Item | N | % |
Job | Senior Doctor | 7 | 26 |
| Junior Doctor | 3 | 11 |
| Senior Pharmacist | 1 | 0.40 |
| Junior Pharmacist | 6 | 22 |
| Nurse | 10 | 37 |
| Total | 27 | 100 |
Gender | Male | 4 | 15 |
| Female | 23 | 85 |
Age | 24–29 | 4 | 14 |
| 30–34 | 6 | 22 |
| 35–39 | 3 | 11 |
| 40–49 | 8 | 30 |
| 50–59 | 5 | 19 |
| 60+ | 1 | 0.4 |
Years of Experience in healthcare |
| < 10 | 15 | 56 |
| 10–19 | 5 | 19 |
| 20–29 | 4 | 14 |
| 30 or > | 3 | 11 |
Months of experience using OHMIS |
| 1–3 | 2 | 7 |
| 3–6 | 7 | 26 |
| >than 6 | 18 | 67 |
Table 2
Demographic of interview respondent
| Categories |
EMM System Factors | Organisational Factors | Moderator |
Themes mapped to the Unified Theory and Use of Technology | Performance Expectancy | Automation and medication safety Standardising Protocols Communication and documentation | | Expectations based on experience |
Effort Expectancy | Mental Demand Temporal Demand | | Expectations based on experience |
Social Influence | | Hospital’s Social Structure | |
Facilitating Condition | | Training Hardware | |
Table 3 Interview themes mapped to the UTAUT framework. EMM = Electronic Medication Management System
Performance Expectancy:
Overall, all user groups felt that improved automation and standardization introduced by the EMMS improved their overall performance. However, doctors and pharmacists expected a greater impact on their performance. They were disappointed by the limited degree of automation, influenced by pre-conceived ideas as well as experience with other systems.
- Automation And Medication Safety:
Reduced ‘mental energy’ required by the EMMS was described by a registrar involved in transcribing medication charts as well as the automated calculation of Body Surface Area and Area Under the Curve were appreciated across all user groups, as described by doctor 1 (table 3). However, doctors and pharmacists believed the safeguards within the EMMS aren’t adequate to prevent errors or inappropriate prescribing. As described by a pharmacist 4 (table 3).
Table 4
User perceptions of elements of performance expectancy extracted from nurses, doctors and pharmacists’ interviews.
Themes | User group | Illustrative quote |
Performance Expectancy | | |
• Automation and medication safety | Doctor 1 | ‘There’s a certain element of automation of AUCS and BSAS that are done that are handy and dose rounding, it probably has taken some of that load off’. |
| Pharmacist 4 | ‘A person started chemotherapy on the ward today and now they’re day 22 and 23 dexamethasone has ended up for tomorrow…that should not happen, I know like it’s probably a bit of user error and computer error, but stuff like that should be prevented by the system’. |
| Nurse 5 | ’ The provision of the protocols that are listed, and in the plan view there all in sequence so it’s quite easy and allows me to better prepared prior for a patient’s arrival’. |
• Communication and documentation | Pharmacist 1 | ‘It’ll (the system) always says, what the original or protocol dose would be, so you'd know that it’s different so that is good’. |
| Doctor 2 | ‘We needed a refresh of the way we communicated, and I think the EMMS provided a trigger point to do that’. |
| Nurse 2 | ‘We can see the timings like if it’s given by someone, suppose I was looking after the patient and someone comes while I’m on break, so they can easily see which has been given and what time and what they do’. |
Table 5
User perceptions of elements of effort expectancy extracted from nurses, doctors and pharmacists’ interviews
| Theme and illustrative quote |
| Mental Demand (MD) | Temporal Demand (TD) |
Doctors (Review & Prescribe) | Order sets have reduced the MD for simple regimens, however a general lack of system flexibility, which was paradoxically often a symptom of the system attempting to improve safety (eg, by making certain tasks or viewing of screens compulsory or sequential). • Doctor 3: ‘I think it’s quite straightforward, it’s difficult as soon as you need to alter something… because the reality is more patients are coming back for treatment and it’s when you’re having to alter cycle 2 or 3 or 5 and you have to drop this dose, change that, delay the treatment, change treatment. Which is reality for majority of patients at some stage’. The EMMS has introduced new steps that increases the pressure on doctor’s memory. • Doctor 1: ‘It still requires a clinician to remember certain things and check certain things that perhaps goes against of what an intuitive path would be’. | The EMMS has allowed for tracking of previous prescribing and remote access has reduced TD for prescribing simple regimens. • ‘Doctor 4: For prescribing simple regimens and searching through paper for previous treatments’, facilitated by the digital print and remote access.’ Increased time pressure felt when troubleshooting, extended by the inability to individually solve the problem. • Doctor 2: ‘I find it very difficult to troubleshoot if I’m asked to change something by nursing or pharmacy, I often don’t know. I find it difficult to understand what I’m being asked to change’ |
Pharmacists (Review & dispense) | The Increased steps to perform simple tasks are mentally draining. • ‘Pharmacist 5: The number of steps to get one label out is just so much more. It’s just very labour intensive’. | Greater time pressure due to: - More administrative tasks due to changed workflow • Pharmacist 3: If they don’t want something you have to reverse the repeat, re-attach it, go find the file again and like yeah, the amount of time you spend looking for stuff is ridiculous in an EMMS world. • Pharmacist 4: We have to double product assign which against waste of time. - Reliance of doctors to troubleshoot their prescribing issues, confirmed by a doctor’s view that • Doctor 6: I think they frequently seem to prioritise those queries from us just not sure how much of a burden it puts on them’. |
Nurses (Review & administer) | Layout of charted medication as well as ease of access to all parameters required in treatment such as BSA and pathology has reduced the mental; demand required to determine the order of administration of medication. • Nurse 2: ‘I think it’s great in the sense that it’s all there, I can look at the bloods, I can look at the BSA I can look at the medication, it’s listed out like following a recipe’. | Layout and remote access allow you to better prepare for patients and reduces time pressure to complete tasks. • Nurse 2: ‘We couldn’t prepare earlier before. We didn’t have the file in hand and sometimes you don’t have time when the file arrives. But now say the patient was not there yet, I really look at their thing and I can communicate well and I know what exactly I need to do’. |
- Standardizing Protocols And Dosing
Both nurses and doctors appreciated the impact of standardised protocols on their performance. The standardised layout of medication charts improved medication safety and positively impacted on nurses perceived performance as nurses described that there was less chance of chart misinterpretations, as described by nurse 3 (table 3) and a user-friendly layout that supports medication administration, nurse 5 (table 3). Similarly, doctors appreciated that ‘the protocols written in there are set in there …and are established protocols’ (doctor 3). The benefits introduced by an automated dose variance report allowed management to oversee unusual prescribing, as described by doctor 3 (table 3). Collectively standardisation and the ability to track dose variance were perceived as beneficial in improving overall performance.
- Communication & Documentation
Doctors and pharmacists felt that communication and documentation improved to a degree, both due to the system itself as well as the system highlighting gaps in workflows. Pharmacists appreciated the clarity of communication regarding dose reductions as described by pharmacist 1 (table 3). The system also highlighted issues that previously existed, indicated by doctor 2 (table 3). However, there were concerns about the format of the treatment plan documents. The templates were being misused as described by pharmacist s (table 3). Further reflected in the doctor’s perception of them being ‘clunky’. From the nursing perceptive, it greatly improved communication between the multidisciplinary teams as well as amongst each other, as identified by nurse 2 (table 3).
Effort Expectancy:
Effort levels for the various user groups drew on various dimensions of effort such as mental demand and temporal demand. Users described varying impacts of the EMMS on these elements of effort and are summarised in Table 4. For doctors and pharmacists there was consensus that greater effort was required to perform tasks previously described as simple, whilst previously complex tasks had eased.
Facilitating Condition
Training
Doctors and pharmacists expressed varying levels of confidence and competence in using the EMMS. Difficulties were associated with the inability to comfortably use the system when dealing with complex regimens, such as being unable to ‘amend or interpret dose adjustments’. Some users attributed this to the system not being intuitive enough and inadequate training, as described below:
Doctor 5: ‘It should be intuitive enough but it’s not. And to actually pick them up when you only do the clinic once every 2 or 3 months it’s going to be hard’.
Pharmacist 3: ‘I had 3 days to learn everything OHMIS and solve all problems and that wasn’t enough time’.
Concerns were raised by pharmacists that the increased complexity in using the system had shifted their role to ‘EMMS support’ for doctors. Reflected in a doctor’s perception that ’ I do worry that I have done something, like I’ve forgotten 3 out of the 30 steps and that, but usually I have enough faith that the system will hold and a pharmacist will tell me that this is wrong (doctor 5)’. Whilst pharmacists believed they were best positioned to support prescribing, the unintended consequence meant that it was impacting on their limited staffing and workflows.
Hardware:
Doctors and Pharmacists were generally satisfied with the hardware being used to support the EMMS. On the other hand, all nurses expressed concern for the ergonomics of the mobile trolleys that the computers were being supported on, describing them as ‘ergonomically it’s not very well set up. The screens are heavy and always falling forward, and everybody is going to have a bad back and a bad neck (nurse 5)’.
Social Influence
Amongst the user groups there was hierarchical influence on user-groups. Both senior and junior doctors believed their superiors to be supportive of the system. Senior doctors felt the organizational support due to the ‘benefits of having the same EMMS facility wide’. However, some senior doctors ‘would prefer another system’ due to familiarity. Social structure however was not influencing their juniors who described their heads of department as being ‘very enthusiastic’. Similarly, across all nurses there was the perception that their seniors supported the EMMS. On the other hand, junior pharmacists were feeling the ‘anxious or negative environment partially influenced by their seniors’ which they believed was brought about by the seniors being concerned about the increased workload and complexity.
Experience:
This study showed that the perception of the UTAUT constructs were moderated by previous experience in using an EMMS due to its indirect impact on pre-formed expectations (Table 2). Previous experience was found to influence expectations of the system related to performance and effort expectancy and resulted in frustrations with the current system. Previous experience using the EMMS meant that users expected a degree of automation and safeguards not achieved by the current EMMS.
Pharmacists 2: ‘So the system shouldn't be so prone to errors like that. If you’ve selected one, other systems I’ve used will populate for the rest of the cycles, assuming things haven’t changed, for this system to keep allowing each cycle to be different isn’t right’.
Expectations were associated with concerns raised by both pharmacists and doctors related to the increased reliance on a clinician memory rather than automation. There were expectations that a system would prevent ‘duplicated prescribed medications’, include reminders such as ‘standardised vitamin b12 every 3 cycles’ as well as ‘cycle 1 dose reductions carrying over to cycle 2’, rather than putting greater reliance on prescriber’s memory as outlined by the doctors 4 and 5:
Doctor 4: ‘Remembering that cycle 1 dose reductions don’t carry over to cycle 2 and you have to do it manually…I find it dangerous’.
Doctor 5: ‘I don’t understand why things like vitamin b12 every 3 cycles isn’t just integrated into the pemetrexate regimen and why we have to remember, like if it’s a computer system it should be able to do those things automatically’.
Whilst nurses felt that a patient’s previous experience at other hospitals, positively influenced their perception, as described by nurse 5 ’ But I think the patients think the computers are safer…there's been a few patients before we went onto the EMMS who said, well when I went to another hospital, every nurse has got a computer’.
Behavioural Intention:
Overall all user groups were enthusiastic about continuing to use the system in the short-term if they had a choice not to. For nurses this was attributed to the benefits they’re experiencing across all elements of the UTAUT model, whilst for doctors and pharmacist the major draw point was that the system was integrated with the remainder of the hospital’s EMMS. However, in the long-term doctors and pharmacists believed it to be only sustainable if the system were ‘optimised’ and ‘enhanced’ based on their expectations around automation and safety of the EMMS.