Based on the reviewed papers (see Tables 1 and 2), a total of 36 factors were identified. However, based on the frequency referred to and used in the literature as used by [40], a factor that counts as more or equal to three is taken into the factors list, so only 16 factors were selected. Table 3 presents the usability factors that were identified.
Table 3
Frequently Used Factors in REVIEWED STUDIES
Factor | Freq. | Factor | Freq. |
Efficiency* | 22 | Simplicity | 7 |
Effectiveness* | 19 | Features | 6 |
Satisfaction* | 18 | Attractiveness | 6 |
Learnability* | 15 | Safety | 5 |
Accessibility* | 11 | Memorability | 5 |
Error | 11 | Understandability | 4 |
Cognitive Load | 8 | Operability | 4 |
Navigation | 7 | UI aesthetic | 3 |
Table 3 shows that Efficiency, Effectiveness, Satisfaction, Learnability, and Accessibility are the most frequent factors used to evaluate usability, marked with *. For example, the Efficiency factor has been cited 22 times by existing studies. These results are similar to the findings of a previous systematic literature review study about the usability factors used for the evaluation of mobile application [28].
A total of 16 factors to be used in evaluating the application for VI users are thought extremely much for any model development, and therefore simplicity is mandatory [42]. Thus, the most relevant factors will be carefully chosen by considering the requirements and contextual factors agreed in previous studies (i.e., user, technology, task, and environmental aspects). At the same time, some factors were omitted because they are irrelevant to the requirements of visually impaired users and the contextual factors [43].This assures reliable evaluation of mobile e-book applications for VI.
By looking at the factors identified in Table 3, some factors like safety, UI aesthetic, and cognitive load may not be suitable and applicable to evaluate accessible mobile e-book applications for the VI. That is because when considering slow learner users as an application target, such as VI users, safety must not be critical, and for the e-book applications for the visually impaired, safety is not the primary concern because most of the e-books for them are available for free. The cognitive load will also be omitted as VI users may not be able to interact with many interfaces at once and move around with their mobile phones [42]. Moreover, the current study by [31] does not consider cognitive load as a core usability evaluation factor of mobile applications for low vision users because cognitive load must be minimal for any interface application designed for VI users [44]. Moreover, UI aesthetics is omitted since VI users have limited sight, and any type of aesthetic may hinder the user from using the application easily as the interface must be designed carefully, and any clutter should be avoided [3].
Furthermore, attractiveness is renamed UI aesthetic in ISO/IEC 25010:2011 (Quality Requirements and Evaluation of Software Products) and as used in [7] study. Therefore, it will also be omitted. The second step is to identify usability factors with the same meaning but different terms and combine them into one term. In applying the Coursaris and Kim procedures, efficiency is the degree to which the product enables tasks to be carried out rapidly, effectively, and economically or reduces performance [28]. It also attempts to represent the “simplicity” usability factor because the simpler the application uses, the less recurrence is spent on a given task [40]. Moreover, efficiency is frequently identified as a measurement of error-free application [28]. Thus, efficiency is needed to ensure a simple yet reliable application is to be used by them. Likewise, it reflects the support provided by the applications, such as help and voice feedback which is presented by the features [45]. Therefore, simplicity, error, and features can be absorbed with the efficiency of the usability factor, as shown in Table 4.
Effectiveness is known to measure the operability of the application in that it provides the users with necessary functionalities that help them perform tasks correctly [23]. Thus, effectiveness is a synonym for operability which is omitted and combined into effectiveness. Considering the theoretical history of HCI and the characteristics of e-book applications, navigation/navigability is a vital usability measurement of effectiveness [46]. When a user opens an application, navigating will be the first thing they must do. Interface navigation, or a visual outline of what the reader is expected to find, helps the readers see easily what they are expected to click on or where they can discover what they are looking for [47]. Therefore, navigation is about whether users can move around in the application efficiently [7]. A good e-book application has to offer the reader quick and easy ways to find what they are looking for and allow them to use the e-book optimally.
On the other hand, in this proposed model, learnability is also selected, which illustrates how simple it is for casual users to learn a system [48]. Learnability is defined twofold. First-time and overtime learnability can be measured by controlled observation and survey [28]. Learning is necessary to assess the level of successful learning among VI people, who are typically slow learners. Understandability also tests the degree of user understanding about the use of the application and enables faster learning [29]. Therefore, understandability can be merged with learnability. In addition, memorability is often seen as part of learnability because the memorable behaviour carried out in the application interface is related to the application interface learning [46]. Memorability tests how easy it is to recall an application or product after a long time-lapse between visits which is similar to learnability over time [28]. Memorability is thus merged with learnability.
Accessibility is the fifth factor selected. This factor is considered a significant measuring element when the application is centred on disabled people since the accessible standard is different from normal people. In addition to the W3C recommendation, a comprehensive measurement was developed to evaluate web pages. Consequently, the study proposed the initial evaluation usability factors for evaluating the accessible mobile e-book applications to ensure that VI users can access the application. Figure 1 shows the usability factors proposed for this evaluation. Tables 4 and 5 show the merged and omitted usability factors together with their definitions.
Table 4
Selected Usability Factors and Definitions
Factors Merged | Selected Factor | Definition |
Simplicity Features Error | Efficiency | Efficiency is the degree to which the accessible mobile e-book application for visually impaired enables tasks to be carried out rapidly [21, 30] |
Operability Navigability | Effectiveness | Effectiveness is the capability of a user to accomplish a task within a specified context [42]. It is also defined as the accuracy and completeness with which users achieve specified goals [21]. |
Attractiveness | Satisfaction | Satisfaction is the perceived level of comfort and pleasantness afforded to the user when using an accessible mobile e-book application for visually impaired, or a user’s perceived level of fulfilment of his expectations and needs [26, 28]. |
Understandability Memorability | Learnability | Learnability measures are used to assess the degree to which an accessible mobile e-book application can be used by visually impaired to achieve specified goals of learning with effectiveness, efficiency, freedom from risk and satisfaction in a specified context of use [24]. |
Nil | Accessibility | Accessibility measures are used to evaluate the ability to use the accessible mobile e-book applications to achieve a specified goal in a specified context of use by visually impaired [24]. |
Table 5
Omitted Usability Factors And Their Supporting Definitions
No. | Omitted Factors | Definitions |
1 | Safety | It is defined as the degree to which risk/damage derived from the use of the software can be avoided [23]. |
2 | UI aesthetic | The degree to which a user interface enables pleasing and satisfying interaction for the user [22] |
3 | Cognitive Load | Refers to the amount of cognitive processing required by the user to use the application [42]. |
4 | Attractiveness | Measure the capability of the system to be attractive to the user (Coursaris & Kim, 2011). |
The final step in developing the proposed factors is the verification phase. The verification form was sent by email to forty experts and only eleven experts responded. Using the five Likert scale, respondents indicate their agreement (or disagreement) with items in the questionnaire [49]. The results were analysed by descriptive statistics (i.e., mean value). To reach a consensus on any nominal scale, a cut-off level of two-thirds (i.e., minimum mean 3.33) of agreement (for positive or negative answers) is required [50]. The results show a very good agreement and acceptance among the experts on the proposed factors. All the factors get a high average of at least four out of five, as presented in Table 6. Hence, all the proposed factors are accepted by the experts.
Table 6
Scores for individual experts
Expert | Efficiency | Effectiveness | Satisfaction | Learnability | Accessibility |
1 | 4 | 4 | 4 | 4 | 4 |
2 | 4 | 4 | 4 | 5 | 3 |
3 | 5 | 5 | 5 | 5 | 5 |
4 | 5 | 5 | 5 | 5 | 4 |
5 | 5 | 5 | 5 | 5 | 5 |
6 | 4 | 4 | 5 | 4 | 4 |
7 | 4 | 5 | 5 | 5 | 5 |
8 | 5 | 5 | 5 | 5 | 3 |
9 | 2 | 3 | 4 | 4 | 1 |
10 | 5 | 5 | 5 | 5 | 5 |
11 | 5 | 5 | 5 | 2 | 5 |
Mean | 4.36 | 4.55 | 4.73 | 4.45 | 4.00 |