This descriptive study was conducted in three stages at Tabriz University of Medical Sciences in Tabriz City of Iran, during 2019.
First of all we used the ADDIE model for designing the course(11). The first stage was “Analysis” which was Goal-Setting Stage and included the determining the course topics and contents. In this phase, Based on a review of literatures, the syllabus of the established courses in other countries were identified and a list of topics was drawn up. Then, two sessions of focus group discussion were held with the faculty members of the department of medical library and information science to identify the eligible syllabuses for course. Then a joint focus group discussion session was held with the expert panel (seven evidence-based medicine teachers including: three medical information specialists, two officials at the evidence-based medical research center, and an epidemiologist and a medical education specialist). The opinions of group members were obtained on the topics, contents and the eligibility of trainers.
10 syllauses were determined based on the focous group deiscussionincluding: Introduction to the evidence-based medicine, concepts and general principles of evidence-based librarianship, structured questions and their types, levels of evidence, mind map and critical thinking, systematic searching and retrieving evidence, critical appraisal of evidence, and Introduction to systematic reviews and meta-analyses. Then the syllabuses were developed in 5 madules based on the five stages of evidence-based performance, including 1) asking an answerable question, 2) acquiring evidence, 3) critical appraisal of the evidence, 4) applying the evidence, and 5) assessing the performance. In addition to these courses, some topics related to librarianship and medical sciences were designed in different scenarios appropriate for the topic as the tasks. The designed scenarios in librarianship and medical sciences were based on daily problems and questions encountered by librarians in libraries. In designing the scenarios, an attempt was made to consider relevant, engaging, challenging, and educational information so that librarians could gain experience in solving each of the issues with evidence-based problem-solving perspectives. Finally, for this course, the course plan was developed, and a study guideline was designed for the learners including the types of content, the objectives of the course, and learners’ and professors’ responsibilities.
The content were delivered in different formats including :text, audio, and multimedia.
A session was held with the officials of the virtual education center and the required domain was allocated. With the approval of the virtual unit authority, some topics that were presented by professors outside the country and could not be presented synchronously due to time difference were presented in an asynchronous manner.
Then the implementation stage was performed. The course continued for a duration of six months (24 weeks) in the learning management system of Tabriz University of Medical Sciences (http://lms.tbzmed.ac.ir). Virtual teaching was offered based on the type of courses, supervised by the department of Medical Library and Information Sciences. To facilitate interaction between learners and trainers, a social chat group called EBIMP was created on the WhatsApp social network (https://chat.whatsapp.com). Participation in the educational course (connection to the system, steps of logging into the virtual education system, receiving and studying the educational content) was designed step by step along with images and explanations, and sent to individual learners/participants by the research assistant via WhatsApp or email. Before the onset of the course, the participants had to fill in the online pre-test questionnaire within the pre-specified time (45 minutes) before receiving a username and password. The learners could log into this system via their individual username and passwords 24 hours a day at desired times (synchronous and asynchronous). The educational content was gradually presented to the learners on a weekly basis over 24 weeks. The librarians could talk to professors on the forum presented on the virtual education platform, if necessary. The participants had a week to solve the problems and send the homework to receive feedback. In this educational course, an attempt was made to present important and up-to-date knowledge and topics related to evidence-based performance stages, along with case scenarios, to the librarians in order to guide and motivate them to further explore knowledge with a critical thinking approach. In this process, learners have an opportunity to follow a complete evidence-based process virtually and be actively engaged with critical thinking. In some group tasks, the learners chose one person as the head of the group, who had to share the duties among the members and upload the final report on the forum. Each group criticized the tasks of other groups, and the groups’ criticism was also uploaded to the forum. The professors were facilitators in all steps of doing the homework and team projects. An online session was held after completion of the project, and the heads of groups presented the group projects. At the end of the session, the learners’ criticism was evaluated, and the professor summed up the session. This learning method was chosen for enhancing the skills and increasing learner-learner and learner-professor interactions. One week after the end of the intervention, the final examination and the course evaluation were performed.
The final stage wasthe course evaluation, which was done based on the two initial levels of Kirkpatrick’s model, i.e. I) reaction: satisfaction and II) learning: formative and summative assessments. Formative assessment was done by assignments and feedback and the summative was done through final project protocoldesigning. A satisfaction questionnaire with three domains of course design, support services, and promotion of personal capability along with three open-ended questions about the strengths and weaknesses of the course and suggestions, was designed.The data were analyzed in SPSS 19 using descriptive statistics (mean and standard deviation).
Face validity was assessed qualitatively and quantitatively. In the quantitative method, to examine the content validity index (CVI), the method of Waltz and Bausell(12) was adopted, and the judgment of each member of the expert panel about each item was checked by three criteria of simplicity, clarity, and relevance on a four-point Likert scale. The minimum acceptable value for CVI, based on the expert panel size (n = 8) was 0.79, and if the CVI of an item was < 0.79, it would be modified based on experts’ opinion.
To determine the content validity ratio (CVR), the judgment of each member of the expert panel about each item was evaluated on a three-point scale: 1 = It is essential, 2 = It is useful but not essential, 3 = It is not essential. Based on Lawshe’s table and the number of evaluators (n = 8), items whose CVR was > 0.75 were deemed essential(9).
To examine face validity, the participants were asked to check the degree of importance of each item on a five-point Likert scale from 1 (not important at all) to five (very important). Only items with scores > 1.5 were deemed as having acceptable face validity(13).
The reliability of the instrument was checked with Cohen's kappa.Seven faculty members of the panel of focus group scored the questionnaire items from 0 to 100. The mean agreement for each item was calculated, and the mean score of the items determined the questionnaire’s final reliability(14).