The success of e-learning in high-income countries is anchored on a combination of models such as Delone and McLean’s information systems success model, user satisfaction models and technology acceptance models [53, 54]. Certain challenges were reported regarding the model of Delone and Mclean (2003) due to complexity for implementors [55]. Several revisions were made to the model with evident integration challenges. The technology acceptance models have been widely used to evaluate e-learning success. However, criticism has been directed towards the application of these models and attempts to expand these models leading to theoretical challenges and confusion [56]. User satisfaction models, focusing on learner satisfaction have been extensively used in various disciplines, including health sciences [57]. The standards developed in this study differ extensively from those readily utilized in high-income countries. Despite this, the authors’ specific focus was the LMIC context, which differs immensely in terms of curriculum, availability of resources, contextual culture, infrastructure, educators, learners, and technical support. A plethora of models are available for evaluating e-learning quality in high-income countries with comprehensive criteria that are successful in that context. The standards developed in high-income countries, are applicable in contextually similar settings and have been challenging to apply in LMICs [58]. This is the quintessential focus behind the development of these standards namely to provide quality standards for evaluating undergraduate nursing programmes in LMICs and involvement of experts within LMICs.
The urgency to develop standards for e-learning in LMICs is anchored in its rapid expansion, especially post-pandemic, and the need to maintain quality in the education of nurses who are critical determinants of health outcomes in communities and the health of the patients they serve [59]. Currently, there are no standards for evaluating the e-learning quality in LMICs. The central focus in developing such standards was to bridge a gap and develop a holistic evaluation of e-learning, as opposed to a fragmented approach model [36]. The focus was on highlighting the challenges regarding e-learning in LMICs, the available opportunities, and best practices across various LMIC settings. These perspectives shaped the foundation of the study and grounded the process of developing e-learning standards in LMICs. This approach was pragmatic in developing the quality of e-learning, especially in LMICs which are beset by a host of challenges. It offers a systematic perspective for university boards of management approach regarding the needs of e-learning rather than a fragmented perspective which obscures the development of e-learning [60, 61]. Widyanti et al. (2020) found that most university management structures realized the value of e-learning educators when the COVID-19 pandemic led to the global closure of universities [62]. This led to a higher mental workload in e-learning compared to face-to-face learning, coupled with the weight of designing content, teaching, evaluating learners, and infrastructural challenges [61]. The ensuing pandemonium led to some universities opting to close indefinitely, while those that managed to remain open had difficulty manoeuvring the e-learning landscape with no standards to guide the transition process [63]. The timing of these standards is perfectly interwoven with the desire of most universities to actualise e-learning as a mode of study. It is quickly becoming and reality in the health science departments and the quality of the programmes is critical.
The current study reports the findings of a modified Delphi on standards for evaluating the quality of undergraduate e-learning programmes. The study accentuates expert opinion and consensus on the standards in LMICs and highlights various spheres of concern to experts across the different criteria. Shifting learning practices and the advent of e-learning and virtual learning are shaping contemporary approaches to understanding the curriculum [64]. The curricula in use were developed in previous years and were subject-centred. While it was quintessential at the time, the shift to learner-centred approaches in e-learning demands a change in the approach to curriculum design [65]. The two interacting constructs of societal needs and learners are critical factors in shaping a curriculum. The former is the foundation of the traditional curriculum, while the latter shapes the e-learning curriculum [64]. Integrating technology into the curriculum, blending thinking, innovation, and ICT skills is inevitable in engaging the millennial learner who is central to the organisation of learning whilst the educator supports learning. The use of a traditional face-to-face curriculum in e-learning has therefore been a huge impediment to the growth of e-learning, and undermines the progress made toward improving e-learning quality in LMICs [66]. A curriculum specific to e-learning is a starting point for quality undergraduate e-learning programmes, as the curriculum would provide all stakeholders with a measurable structural plan for the programme’s accountability and social responsibility, especially in LMIC settings [67, 64]. Furthermore, the novelty, clarity and comprehensiveness of these standards will enable the development of an e-learning curriculum. We argue that with the shifting dynamics of the learners, content, teaching approaches and demands of modern-day education, a rigid traditional face-to-face curriculum would be impractical in e-learning.
The second standard regarding the proficiency of educators in is a common but often overlooked concept in e-learning. The following aspects are important: the attitude of the educator; learners who consider educators as role models, and the educators’ attitude toward e-learning that rigorously affects the learners’ attitude [67, 57]. Experts agree that the increased workload associated with e-learning, class size issues, teaching resource availability, mental workload and educator motivation are critical factors in educators’ perceptions of e-learning [68, 69]. Almahasees et al. (2021) identified differences in training and e-learning delivery between high-income and low-income countries. The lack of teaching resources available to educators, reduced knowledge of ICT and the challenges of class control emerged as areas of agreement among most experts [70]. This has been a problem in the development of e-learning in low-income countries [71, 68, 26]. The educators’ motivation for the use of e-learning, teaching styles, their perception of e-learning and control of technology will shape the behaviour of learners and significantly improve learner satisfaction [72, 73]. The experts agreed on the use of e-learning to advance the continuing professional development (CPD) of educators to ensure a commitment to continuously improve their knowledge and skills. This is supported by Mlambo et al. (2021) who found that the accessibility and availability of CPD through e-learning for educators greatly improved their competence in the use of e-learning tools [74]. The role played by educators in shaping learners’ perceptions cannot be overestimated, especially in e-learning.
The third standard, the relationship between learner proficiency and attitude and the use of e-learning and consequent success in e-learning is significant and leads to the satisfaction of learners [75, 53]. Perceived satisfaction of learners regarding the use of e-learning is an essential measure of the success of e-learning and will lead to the optimal use of the system [76]. Usefulness is a key determinant of the success of e-learning among learners, because if they perceive e-learning to be useful, it will increase to the likelihood of their use of e-learning and significantly influence satisfaction [76, 77]. The ability of learners to objectively evaluate teaching and learning was an area of concern as most of the learners were not actively involved in the evaluation of e-learning programmes compared with learner challenges limiting the objective evaluation of e-learning. However, this contrasts with Al-Fraihat et al. (2021); Penna and Stara (2007), who identified that learner-centred evaluation is essential to the quality of e-learning [53, 78]. Experts agreed that learners should be prepared in advance for e-learning and notified on how to navigate the course and its expectations. The developed standards focus on the learner preparedness, equipment, and evaluation of e-learning to ensure satisfaction and actualise the maximum benefits. Furthermore, sufficient, concise, and clear information contributes immensely to satisfaction.
Fourthly, the myriad of infrastructural e-learning challenges faced by universities in LMICs requires contextually sound standards and comprehensive evaluation, although progress in this regard is evident [79]. Policy issues surrounding e-learning infrastructure were an area of contention as most of the experts agreed that operationalising policies in e-learning is challenging in LMICs. Commitment by key stakeholders is often tenuous. This view was similarly shared by Kibuku et al. (2020) and Mbugua (2014), who identified that policy gaps have hampered higher education in the Kenyan context [68, 80]. The impetus to develop these standards with a focus on policy is to ensure that stakeholder commitment is captured from the start when developing e-learning programmes. The availability of infrastructure to support e-learning was described as challenging for most LMICs, with funding and systemic challenges overshadowing meaningful progress to improve e-learning. Despite the promising internet penetration in LMICs, especially in Africa, the concentration is mostly in urban areas. In addition, the cost of internet access is prohibitive [79]. The learning management systems (LMS) in most developed countries are customised to suit their specific needs, but in LMICs, reliance on open source LMS limits the ability to customisable options [82, 83]. The ease of use, system quality and satisfaction of learning management systems (LMS) by educators and learners are critical in improving e-learning [83, 82]. We argue for the commitment of governments and university management boards to strengthen the resolve toward improving e-learning which has enormous potential to scale up health workers.
Fifthly, support systems for educators and learners were an area of concern. In most LMICs, there’s is not adequate e-learning support for educators and learners [84]. Wang et al. (2018) identified that support for e-learning resources is an area of interest in e-learning [85]. Experts agreed that e-learning support is a critical and essential standard for evaluating e-learning, and its accessibility is crucial for learners’ satisfaction. Experts agree that e-learning support is essential to the operation of e-learning. There is a need to improve internet bandwidth, although better bandwidth doesn’t necessarily mean that learners and educators will use this for learning purposes. The need for control of content and site restriction is crucial in ensuring appropriate utilisation. According to the findings of Chanboualapha and Islam (2012), who compared the relationship between internet usage and learning and found that the internet has a positive relationship with learners’ knowledge [86]. The absence of social media in their study can be attributed when the study was conducted. However, Waweru (2018) found that learners spend most of their time on social media, during the time which they could be studying, leading to poor academic performance academically [87]. The site access restriction should be so that learning is not impaired. These are some of the areas in which the panel of experts expressed concern, and it was worth noting that the changes to the standards improved the quality and applications of LMICs.
Lastly, these standards will be critical for evaluating the quality of undergraduate e-learning nursing programmes. They will also assist in shaping the intricate capacity to develop local solutions to local problems in LMICs [23]. Based on the iterative process, the experts reached consensus on six broad categories: curriculum planning, proficiency of the educator, learner proficiency and attitude, infrastructure for learning, support, and evaluation, with 105 criteria for evaluating the quality of e-learning in undergraduate nursing programmes in LMICs. These standards will provide schools of nursing seeking to evaluate the quality of e-learning with a guide to practical evaluation. The standards are comprehensive and broader covering wider range of criteria than those available in developed countries. They align with nursing education at a degree level. Consequently, educators and learners will have better learning experiences with e-learning leading to quality in e-learning in nursing education hence improving patient outcomes. The feasibility, usability, and practicability of the standards' are crucial in ensuring they are easily usable and practical for the context-specific challenges in LMICs. The complete agreement (100%) by the experts (100%) on the final standards’ feasibility, usability, and practicability is critical in applying these standards in LMICs. The crux of this study is standards for quality evaluation. The feasibility, usability, and practicality of these standards have a high propensity to improve the quality of e-learning holistically in LMICs.
4.1 Limitations
Despite numerous efforts to recruit more experts from more LMICs, this did not materialise. This hesitation to volunteer may be due to busy work schedules and unavailability and can be explained by the critical roles of most of the experts identified in nursing education and e-learning. A further limitation was that the experts drawn from different LMICs had to be proficient speak in English.