Included studies, objectives, types of platform, evaluation tools, outcome measures, key findings and recommendations
In all of the studies, the e-platform users’ perceptions and experiences were assessed. The objectives were to evaluate, assess, or discuss either the efficacy and impact or the development of an online platform or programme. The types of platforms that were assessed involved two platforms, one website, and four nursing programmes or courses. One study was a systematic review on a variety of programmes. The evaluation tools were survey questions (Likert scale as well as open ended questions), SDLNRE questionnaire, narratives from focus group discussions and individual face-to-face or telephone interviews, or log records analysis. Outcome measures varied from student engagement, achievement of learning outcomes, nursing skills performance, relevance of contents to website usability, technology, and course usefulness. The objectives, types of platforms or programmes, evaluation tools and outcome measures are presented in Table 3, including the key findings along with the major recommendations.
Table 3
Included studies, objectives, types of platform, evaluation tools, outcome measures, key findings and recommendations
Included studies | Objectives | Types of platform or programme | Evaluation tools | Outcome measures | Key findings and recommendations |
Davidson, SJ. et al. (2016) | Describe the development and evaluation of game-based learning to better prepare nursing students to engage in EBP | Quest-based learning platform (3D Game Lab) | Survey with 12 questions 1–5 pt. Likert scale, platform analytics, thematic analysis of students’ comments | Student satisfaction, student engagement, student achievement of learning outcomes | Provide timely and individualized feedback, create student choice in selection of learning quests, create customization of learning, provide visible milestones, level learning activities to build on each other |
Du, S. et al. (2013) | Examine the efficacy of web-based distance education for nursing students and employed nurses | Websites containing distance education as experimental teaching strategies | Web-based distance nursing education compared to traditional teaching or blank control | Knowledge acquisition, nursing skill performance, satisfaction, self-efficacy, cognition, affection, interaction | Develop course information, a curriculum map, teaching materials, communication tools, formative and summative assessments, student management tools, links to databases, teleconferencing, case-based learning. Enhance flexibility, learner independence, time efficiency, content in text/video/audio formats, interactive designs. Minimize hardware and software problems, inadequate information literacy, dropout levels, loss of social process, inadequate time/skills to develop materials, expenses involved, lack of research-produced proof |
Gagnon, J. et al. (2015) | Evaluate online self-learning modules on critical appraisal skills to promote the use of research in clinical practice | Online self-learning course (InfoCritique Program) | Two questionnaires (pre and post course) (SDLRNE and knowledge questionnaire), one satisfaction questionnaire | Self-directed learning readiness (SDLR), satisfaction, knowledge acquisition, instructions, design, effectiveness, usability, course usefulness | Content satisfaction involves difficulty, number of examples, language of materials, complexity of content, number of modules addressing literature searches, experimental designs, and statistics. Participation satisfaction involves motivation, commitment. Platform satisfaction involves user-friendliness, rapid load, feedback, interaction, structure, easy navigation. Prevent lack of space for interaction, enhance transfer of knowledge into practice and to colleagues |
Karaman, S. et al. (2014) | Evaluate the online continuing education program from the perspective of nurses | Online degree in nursing (HELITAM: first online bachelor nursing program in Turkey) | Survey with 1–5 pt. Likert scale, open-ended questions | Program and course structure, course materials, technology, support services, assessment | Monitor programme and course structure, including quality of design of asynchronous online education, quality of effective support, quality of technological infrastructure, and quality of teaching |
Kowitlawakul, Y. et al. (2017) | Discuss the development and piloting process, including the variety of evaluation perspectives | e-learning research project module using interactive multimedia | Two semi-structured focus group sessions, two individual interviews | Usefulness, engagement, organization, relevance of content | Focus on impact of the e-learning module on students’ performance and students’ learning outcomes |
League, K. et al (2012) | Describe the development and impact of a Web-based tool to improve nurses’ access to evidence | Website Launched 2009 | Survey with 1–4 pt. Likert scale (pretest 2008 and posttest 2010) | Perceived access to EBP, perceived ability to use EBP | Support increasing use of EBP, improve perception of supervisors’ support for EBP, allocate time for staff to use evidence at work |
McIntyre, M. et al. (2013) | Discuss how peer dynamics influence student learning in an online environment | Two online nursing programs | Individual face-to-face or telephone interviews (N = 30), one focus group interview (eight participants), fieldnotes | Issues related to time, demands of online participation, experiences of conflict, development of skills | Consider pros and cons of flexible access to education, consider differences in expectations and limits of electronic media, online skills development should also address clinical practice, take into account the specific nature of online environments |
Seixas, CA. et al. (2016) | Assess the usability of a virtual learning environment for nursing students | Distance education platform (Moodle) | Questionnaire, log records analysis | Ergonomic and interface usability, country of origin, experience in distance learning | Consider variables such as country of origin, previous experience in distance education |
The key findings and recommendations generated a list of e-platform requirements that could be classified into nine categories: aim and users, platform structure, educational content and materials, knowledge acquisition and achievement of learning outcomes, communication, interaction and feedback, nursing skills, usability, management, and infrastructural and technological support.
Aim and users
All of the platforms and the content that was included addressed nursing students or clinical nurses with the aim to encourage nurses to use evidence in their studies or in clinical practice. These platforms, therefore, addressed a variety of users and levels varying from undergraduates, graduates, clinical nurses, or combinations of these students. On some platforms that were studied, different target groups participated simultaneously. The study of McIntyre addressed undergraduate as well as graduate nurses [29]. Students’ levels are master’s degree (MSN), bachelor’s or baccalaureate’s degree (BSN), or the associate or diploma degree (AND) level [28]. None of the studies in the review involved PhD students. In Kowitlawakul’s study, two faculty staff members also were included in the study [27].
Some platforms explicitly addressed clinical nurses for whom different descriptions were used such as clinical nurses [25], registered nurses [26], staff nurses [28], or employed students [24].
It is important to consider whether the platform is accessible to only registered members for which users should login or whether open access is allowed. In some of the studies, access was limited to registered members of a specific learning community. In League’s study, the website was only available to employees within the organization [28] whereas, in others, there was open access. In all of the nine studies of Du’s systematic review [24], users had open access.
Most of the platforms in the studies allowed one language which could be the users’ own language or English. The use of one language – and, if so, which language - or the use of multiple languages and the consequences for the aims of the platform should be considered before developing the platform [25].
Platform structure
Different types of platform structure were determined in the studies. There were three platforms, one of which was a website, and there were four online nursing programmes. In League’s study, the website was called ‘Evidence Based Practice and Nursing Research’. In Davidson’s study [23], the platform was a game-based platform. The Distance Education Platform (DEP) in Seixas’ study [30] was Moodle, which was also referred to as a Virtual Learning Environment (VLE). All of the platforms or websites were structured in a way to contain distance learning courses or modules, for example, in a series of nursing learning modules called Mooshak [24].
Some of the platforms in the studies of the review included additional structures that enabled content and tools such as forum discussions, webinars about nursing topics, podcasts [29], or teleconferencing [24]. The content can be in text as well as in audio and video formats [24]. These formats may also include audio-visual presentations as well as graphics and animations, usually with multimedia and interactive elements [26]. In game-based learning, visible milestones and badges play a significant role [23].
The structure of the platform should allow access to electronic databases as is mentioned in all of the studies.
Educational content and materials
Educational content includes providing direct access to practice-based and evidence-based guidelines [28]. Three studies explicitly explain what educational content concerning nursing research skills is included [23, 27, 28]. This includes expertise in basic research principles and expertise concerning research paradigms such as quantitative and qualitative methods [27] and the major characteristics, strengths, and limitations of quantitative and qualitative research designs [23]. In Gagnon’s study [25], expertise in additional research designs is also included. Research skills include developing search strategies that are quest-based in order to formulate clinical questions [23] and, in particular, using the PICO (= Patient, Intervention, Comparison, Outcome) structure [23, 28]. Subsequent skills include performing these literature search strategies using a variety of library databases [23, 25]. The literature search begins with having access to library sources and search engines such as Google Scholar or Medline [28]. All of the studies mention that there should be access to and the skills to use electronic databases with scientific nursing publications. After performing the search strategy, additional skills concern the critical appraisal of research reports [23].
Additional research skills that are addressed in the content of learning modules are data analysis, for example, statistics [25] and academic writing [29].
Knowledge acquisition and achievement of learning outcomes
Knowledge acquisition outcomes were defined in terms of improving research skills or improving the use of EBP. In League’s study [28], learning outcomes were the perceived access to EBP and the perceived barriers, ability, or confidence to use it. A significant improvement in perceived confidence and ability to use evidence and an increase in awareness and interest in EBP was identified [28]. Davidson’s study [23] demonstrated the need for guaranteeing progression by increasing levels of learning activities.
Progression also depends on the quality of the teaching, which is vital [26] even though the type of learning is self-directed [25]. Therefore, requirements of the e-platform may involve that distance learning courses should include clear learning goals and learning outcomes to support knowledge acquisition by self-directed learning. Consequently, courses should provide explicit instructions, examples, and assignments.
The specific characteristics of e-learning or online learning were considered thoroughly in all of the studies. Specific consideration should be given to assessments for measuring progression in learning. Specifically in self-directed learning, it is important to distinguish between formative and summative assessments [24]. In testing the achievement of learning outcomes, priority was on formative assessing which corresponds with the typical features of self-directed learning [25]. However, caution must be taken when prioritizing formative assessing because of the value of summative assessments using online exams required for grading.
Communication, interaction, and feedback
A basic characteristic of learning with a distance education platform is that it should be interactive, and the way a platform is designed is explicitly referred to as an interactive design [24]. All modules or courses that are included in the platform should be interactive [27].
The interactive character of the platform includes interaction between peer students, teachers, and content, which must be maintained at all times [29]. Communication and interaction with instructors is essential and lack of interaction was mentioned as one of the primary weaknesses of online education [26]. In addition, the role of administrators is explicitly indicated [29].
Interaction is aimed at learning outcomes using feedback from tutors and peers. It is mentioned that feedback should be individual-based and prompt or timely [23]. The study of McIntyre [29] was aimed at discussing how peer dynamics influence student learning in an online environment. One disadvantage of online learning might be the loss of the social process of learning, as indicated in Du [24].
Although learning is realized by distance self-directed learning, which is flexible, feedback and interaction are essential. The lack of opportunity for interaction with tutors or other participants is a barrier, according to Gagnon [25]. In McIntyre’s study [29], it is suggested that peers from one geographical area could be connected. Interaction can be realized with a-synchronous as well as synchronous tools, and both tools must be available [29]. It is recommended to anticipate adequate balancing between real-life and online communication. Therefore, it is suggested that the e-platform includes materials for use in classroom settings as well.
Nursing skills
Apart from learning outcomes concerning nursing research, outcomes also include clinical nursing skills performance which was one of the outcomes in Du’s study [24]. All of the authors agree that the acquisition of research skills should enhance the transfer to clinical practice and, according to League [28], should also increase the access and use of evidence in their clinical nursing profession. Knowledge acquisition is aimed at the transfer of knowledge, that is, the e-platform should enhance nursing skills performance to put acquired knowledge into practice [25]. A further requirement of knowledge transfer could be that knowledge should be disseminated among colleagues [25]. It was suggested that it is important to allocate – extra – time for staff to realize knowledge transfer at work [28].
Usability
In all of the studies, usability was an important feature that involves user friendliness, enhancing student motivation, and engagement [23]. Usability requirements such as easy site navigation and rapid load [25] as well as attractive design and content of the platform were proposed by all of the platform users. In addition, ergonomic usability is needed [30].
Another feature of usability is accessibility which may be open, closed, or a mixture of open and closed for certain modules. Usability of the platform should enhance and support flexibility which is the key of distance learning and involves flexibility in time, place, and learning activities [23]. Flexibility for accessing education encourages students to work across time zones [29].
Usability also concerns platform structure and information to users thus requires a clear platform and course information, for example, by curriculum mapping [24] and useful instructions [25]. Information including nursing information and platform contact information should be easy to locate.
Users should be willing to engage and commit themselves to work on it which is connected to the level of independence of the user, the quality of instructions, and the system feedback [30]. One important barrier is inadequate information literacy [24]. Earlier experience in distance education in users is important, however, sometimes this earlier experience is insufficient [30].
Management
A platform should include management tools for developing and maintaining structures and materials. It must be easy to further develop the platform and to add information. Management requirements involve monitoring tools for student tracking [24] to check the number of logins or the number of users and login security. Some platforms contain a combination of systems, for example, the platform Moodle also includes a teaching and learning management system (LMS) and a course management system (CMS) to facilitate educators in developing online courses or online support [30]. Educators should resist the convenience of pre-established packages [29] as, apparently, the development of new educational materials is highly valuated compared to the use of pre-established packages.
Infrastructural and technological support
All of the studies demonstrate that, as a primary condition for developing an online platform, access to the Internet should not be restricted in terms of time and place. In addition, there should be no restrictions in costs when access to the internet is concerned [24]. The quality of the effective technological support that is provided is essential [26].
Requirements concerning infrastructural and technological support include that the e-platform can be used from different devices such as a laptop, tablet, or smartphone. In Karaman’s study [26], platform users mentioned perceiving problems reaching the call centre. Hardware and software problems often occur [24]. Further requirements to be considered are the unlimited availability of technological support by a support center or helpdesk, at any time, from any place, and at no cost. Testing by information technology personnel is required to check for viruses in the system [27]. All of the studies showed that the technological infrastructure should enable unrestricted access to links to other websites or electronic databases.
Requirements to be included in the Delphi study
Most requirements that were identified from the literature review and presented above, were included in the Delphi study. All of the requirements, which were categorized into nine categories, could be incorporated into four groups of statements for the Delphi. The group content-related requirements encompasses ‘aim and users’, ‘educational content and materials’, ‘nursing skills’ and ‘platform structure’. The group functional requirements contains ‘knowledge acquisition and achievement of learning outcomes’, ‘communication, interaction and feedback’, and ‘management’. The group usability requirements covers the category ‘usability’. The group technological requirements includes ‘infrastructural and technological support’.
Results Delphi study
Experts’ characteristics
As presented in Table 4, in the first round, 59 respondents completed the online questionnaire including 48 from Kazakhstan (81%) and 11 from European countries (19%). Almost half of the group of experts are employed at a university (44%) and 46% in a healthcare organization. The respondents’ expertise is in nursing practice (46%), nursing education (32%), nursing research (24%) and development of e-platforms (5%). In the second round, 17 experts completed the questionnaire: 11 from Kazakhstan (65%) and six from European countries (35%). In contrast to the first round, most respondents in the second round work at a university.
Table 4: Experts’ characteristics
Characteristics | First round (N = 59) | Second round (N = 17) |
n | % | n | % |
Country | | | | |
• Kazakhstan | 48 | 81% | 11 | 65% |
• Europe | 11 | 19% | 6 | 35% |
Work1 | | | | |
• University | 26 | 44% | 13 | 76% |
• Health care organization | 27 | 46% | 2 | |
• Other organization | 6 | 10% | 3 | |
• missing value | | | 1 | 6% |
Expertise1 | | | | |
• nursing practice | 27 | 46% | 1 | 6% |
• nursing education | 19 | 32% | 8 | 47% |
• nursing research | 14 | 24% | 6 | 35% |
• Development of e-platform | 3 | 5% | 3 | 18% |
• missing value | 1 | 2% | 1 | 6% |
1Respondents were able to select more than one option
Essential and useful requirements
Requirements rated as ‘essential’ with a cvr of 0.29 and above are presented in Table 5. Requirements rated as ‘useful but not essential’ with a cvr below 0.29 are presented in Table 6.
Table 5
GROUP Category | It is ESSENTIAL that the e-platform | cvr first round (n = 59) | agreement second round (n = 17)1 |
CONTENT | | | |
Aim and users | enhances | | |
1. to put nursing knowledge into practice | 0.83 | 93.75% |
2. nursing skills performance | 0.69 | 76.47% |
3. increasing usability of evidence in clinical nursing professions | 0.47 | 87.50% |
4. transfer of knowledge to colleagues | 0.46 | 81.25% |
5. nursing management and leadership | **2 | 81.25% |
has the following characteristics | | |
6. Information is available in Russian and English and Kazakh | 0.63 | 100.00% |
includes for educational purposes | | |
7. materials for different target groups (bachelor, master) | 0.66 | 100.00% |
Educational content and materials | includes educational materials about | | |
8. evidence based nursing | 0.90 | 94.12% |
9. evidence based nursing clinical guidelines for use in practice | 0.82 | 100.00% |
10. nursing research skills | 0.55 | 100.00% |
11. nursing leadership | 0.42 | 93.75% |
includes information concerning nursing research skills such as | | |
12. To apply evidence in the clinical practice | 0.64 | 93.33% |
13. Basic research principles | 0.61 | 87.50% |
14. Skills to use electronic databases (information systems in nursing) | 0.61 | 87.50% |
15. Literature searches | 0.44 | 86.67% |
16. Qualitative designs | 0.36 | 80.00% |
17. Formulate quests/ research questions e.g. in PICO format | 0.35 | 86.67% |
18. Quantitative designs (including surveys and cross-sectional designs) | 0.32 | 82.35% |
Nursing skills | includes information concerning nursing topics such as | | |
19. Patient safety (such as hygiene, prevention of adverse events) | 0.73 | 100.00% |
20. Physiological-complex (such as care for cancer; optimize neurological function) | 0.56 | 93.75% |
21. Family (such as child baring care and care to support families) | 0.52 | 93.33% |
22. Health promotion and prevention of health risks | 0.45 | 93.75% |
23. Behavioural (such as dementia and depression) | 0.39 | 93.33% |
24. Health System | 0.39 | 85.71% |
25. Nursing documentation (development and practical use) | ** | 93.33% |
26. Quality management (tools, techniques for improving nursing practice) | ** | 93.33% |
27. Standard operating procedures (SOP) and complex routine operations | ** | 93.33% |
28. Laws and regulations for nursing practice | ** | 93.33% |
Platform structure | includes | | |
29. an agenda/calendar with relevant nursing research and nursing educational activities in Kazakhstan and beyond | 0.34 | missing value |
FUNCTIONAL | | | |
Communication interaction and feedback | includes functionalities for | | |
30. online collaboration | 0.30 | 81.25% |
includes distance learning courses which | | |
31. provide timely feedback from tutors and peers | 0.62 | 88.24% |
Knowledge acquisition and achievement of learning outcomes | includes for educational purposes | | |
32. distance learning courses | 0.47 | 100.00% |
33. instruction films | 0.41 | 93.75% |
includes distance learning courses which | | |
34. include clear learning goals and learning outcomes | 0.86 | 87.50% |
35. have a clear structure and a course map | 0.83 | 100.00% |
36. include instructions, examples and assignments | 0.73 | 100.00% |
37. support knowledge acquisition by enabling self-directed learning | 0.61 | 100.00% |
38. include summative assessment with certification | 0.49 | 87.50% |
39. include formative assessment to assess knowledge acquisition | 0.40 | 81.25% |
Management | includes tools | | |
40. to secure personal data and administration of courses | 0.44 | 93.75% |
41. to manage user management accounts | 0.40 | 68.75% |
42. for content management so it is easy to add and change information | 0.29 | 82.35% |
TECHNICAL | | | |
Infrastructural and technological support | should technically support | | |
43. use on a computer, tablet, and mobile phone. | 0.83 | 100.00% |
44. guaranteed technological support (not restricted to time, place, or cost) | 0.53 | 87.50% |
45. use in different browsers | 0.51 | 93.75% |
USABILITY | | | |
Usability | has the following characteristics | | |
46. well designed and attractive | 0.82 | 100.00% |
47. open and free access to information | 0.80 | 100.00% |
48. clear and simple site navigation | 0.75 | 100.00% |
49. text on each page is not too much and easy to read | 0.62 | 81.25% |
50. contact information is easy to be found | 0.58 | 100.00% |
51. login for educational and collaboration purposes | 0.44 | 93.75% |
52. clear site structure | ** | 100.00% |
1 Due to the fact that some questions were not answered by one or two persons, the percentages can differ because percentages are calculated by the number of respondents per question. |
2 ** added in the second round based on answers on the open questions in the first round. |
Table 6
Useful but not essential requirements
GROUP Category | It is USEFUL BUT NOT ESSENTIAL that the e-platform | cvr first round (n = 59) | agreement second round (n = 17)1 |
CONTENT | | | |
Platform structure | includes | | |
1. webinars about nursing topics | 0.28 | 75.00% |
2. a blog about several nursing topics, updated once a month | 0.22 | 82.35% |
3. links to electronic databases with scientific nursing publications | 0.14 | 86.67% |
4. forum discussions for nurses on nursing topics | 0.07 | 93.75% |
5. links to international websites with additional nursing guidelines | 0.07 | 73.33% |
6. an overview of important stakeholders for nursing in Kazakhstan such as universities with bachelor and/or master programmes in nursing, university medical hospitals | 0.00 | 81.25% |
7. an overview of national and international institutions working together with the CNE | -0.09 | 87.50% |
Nursing skills | include information concerning nursing topics such as | | |
8. Physiological-basic (such as physical activity, nutrition support) | 0.19 | 53.33% |
Educational content and materials | includes information concerning nursing research skills such as | | |
9. Analysis, such as statistics or qualitative analysis | 0.16 | 73.33% |
10. Critical appraisal of scientific literature | 0.04 | 80.00% |
11. Academic writing | -0.05 | 81.25% |
FUNCTIONAL | | | |
Communication interaction and feedback | includes functionalities for | | |
12. online discussions | 0.11 | 80.00% |
includes distance learning courses which | | |
13. enable communication and interaction with tutors and peer students | 0.05 | 88.24% |
Knowledge acquisition and achievement of learning outcomes | includes for educational purposes | | |
14. educational materials for use in classroom settings to download | 0.28 | 81.25% |
Management | includes tools | | |
15. to report functions with the option to monitor the number of users, popularity of the different items, and improve the site content | 0.00 | 81.25% |
1 Due to some missing values percentages may deviate |
Four groups of requirements
The four groups including the essential and useful requirements are presented below.
Content related requirements
Aim and users
Regarding the aim of the e-platform, it is considered to be essential that the e-platform enhances nursing skills performance, the transfer of knowledge into practice and to colleagues, and increasing usability of evidence in clinical nursing. Resulting from the open questioning, ‘enhancing nursing management and leadership’ was added as being an essential requirement, which was confirmed in the second round. Concerning users, it is vitally important that information is available in Russian and English as well as in Kazakh. For educational purposes, it is absolutely necessary that the e-platform includes materials for different target groups such as bachelor’s and master’s students. No requirements concerning aim and users were rated as useful but not essential.
Educational content and materials
It is considered essential that the e-platform includes educational materials about evidence-based nursing, nursing clinical guidelines, nursing research skills, and nursing leadership. It is also very important to include information about nursing research skills such as the application of evidence in clinical practice, basic research principles, skills to use electronic databases, literature searches, qualitative and quantitative designs, and the formulation of research questions.
Advanced research skills such as statistics or qualitative analysis, critical appraisal of scientific literature, and academic writing were rated as useful but not essential.
Nursing skills
Concerning the nursing topics obtained from the seven NIC domains, six of these were rated as essential. These are: patient safety, physiological-complex, family, health promotion, behavioral, and health system. The domain physiological-basic was rated as useful but not essential, although no agreement was achieved in the second round.
With open questioning, experts added statements concerning nursing documentation, quality management, standard operating procedures, and laws and regulations. These additional requirements were rated as essential in the second round, which was confirmed by high levels of agreement.
Platform structure
It is indispensable that the e-platform includes an agenda/calendar with relevant nursing research and educational activities in Kazakhstan and beyond.
Other requirements related to platform structure are useful but not essential. These are - with decreasing cvr values - webinars about nursing topics, a blog about several nursing topics, links to electronic databases with scientific nursing publications, forum discussions on nursing topics, links to international websites with additional nursing guidelines, an overview of important stakeholders for nursing in Kazakhstan such as universities with bachelor or master programmes in nursing or university medical hospitals, and an overview of national and international institutions working together with the CNE.
Functional requirements
Communication, interaction and feedback
Online collaboration was rated as an essential requirement for the e-platform whereas online discussions are perceived useful but not essential. It is very important that each distance learning course provides timely feedback from tutors and peers, and it is considered useful that distance learning courses enable communication and interaction with tutors and peer students.
Knowledge acquisition and achievement of learning outcomes
For educational purposes, it is essential that the e-platform includes distance learning courses and instruction films. Educational materials to download for use in classroom settings are perceived useful but not essential. For knowledge acquisition and achievement of learning outcomes, it is essential that distance learning courses include clear learning goals and learning outcomes, have a clear structure and a course map, include instructions, examples, and assignments, support knowledge acquisition by enabling self-directed learning, include summative assessment with certification, and include a formative assessment to assess knowledge acquisition.
Management
It is an essential requirement that the e-platform includes tools to secure personal data and administration of courses, to manage user management accounts, and to include tools for content management to add and change information. It is useful but not essential to include tools to report and monitor the number of users and popularity of the different items of the e-platform.
Technical requirements
Infrastructural and technological support
All technical requirements are rated as essential. With high levels of agreement, it is perceived essential that e-platform technically supports the use of a computer, tablet, and mobile phone, use in different browsers, and that technological support (not restricted to time, place, or cost) is guaranteed.
Usability requirements
Usability
All usability requirements are perceived as essential. The usability characteristics refer to proper and attractive design, open and free access to information, clear and simple site navigation, text on each page is not too much and easy to read, contact information is easily found, login for educational and collaboration purposes, and a clear site structure.