Designing and Implemenitin of an Evidence-Based Information Management and Practice Course for Librarians Based on ADDIE Model

Background: Evidence-based practice is competency required by librarians to provide quality services in today’s rapidly growing world of information. This study aimed to instructional desianing and implementing of an evidence-based information management and practice course based on ADDIE Model. Methods: This action research was conducted based on ADDIE Model in 2019 in Tabriz University of Medical Sciences. The Analysis,Design, Development, Implementation and Evaluation was performed accordingly. The course was offered virtually through the Learning Mangement System of one of the top ranked universities in Iran. course evalouation was done based on Kirkpatrick’s model. A Likert-based satisfaction evaluation questionnaire was developed, which included three open-ended questions about strengths and weaknesses of the course and an open-end question for recommendations. After psychometric analysis, the questionnaires were distributed among the participants. The data were analyzed in SPSS 19 and reported by descriptive statistics (mean and SD). Results: After three sessions of focus group discussion, 10 lessons was determined inclouding: evidence-based practice in clinical medicine, critical thinking and mind map, searching and retrieving evidence, systematic reviews, critical appraisal of evidence, systematic search, levels of evidence, systematic questions, and meta-analysis. The syllabuses was categorized in 5 modules based on 5 steps of evidence-based practice guidline.The overall satisfaction about the training was reported excellent (83.52 ± 16.27), and addressed the highest satisfaction with the support services (94.52 ± 29.68). Conclusions: This syllabus is appropriate for short course education of evidence-based approach in librarianship as a branch of medical librarianship and information sciences. It also is suitable for professional continuing education of librarians. made to present important and up-to-date knowledge and topics related to evidence-based performance stages, along with case scenarios, to the librarians in order to guide and motivate them to further explore knowledge with a critical thinking approach. In this process, learners have an opportunity to follow a complete evidence-based process virtually and be actively engaged with critical thinking. In some group tasks, the learners chose one person as the head of the group, who had to share the duties among the members and upload the nal report on the forum. Each group criticized the tasks of other groups, and the groups’ criticism was also uploaded to the forum. The professors were facilitators in all steps of doing the homework and team projects. An online session was held after completion of the project, and the heads of groups presented the group projects. At the end of the session, the learners’ criticism was evaluated, and the professor summed up the session. This learning method was chosen for enhancing the skills and increasing learner-learner and learner-professor interactions. One week after the end of the intervention, the nal examination and the course evaluation were performed.


Background
The evidence-based approach uses a combination of the latest and most reliable research ndings with professional experiences and customer preferences to promote the quality of the provided services(1). This approach is recommended for professions that need strategic decision-making to reach certainty against uncertainty, or performances that seek to reduce error in decision-making and service provision (2). Evidence-based practice is a professional competency required by librarians to provide accurate services in the rapidly-growing world of information (1,3,4). The previous researches indicate that among from the 7 competencies developed by Medical Library Association (MLA), the last that is perception of scienti c research methods, having ability of critically apprising, and screening the evidences has been considered less than others. While librarians with biomedical and health science background shows more capabilities for these competency due to their previous knowledge and workplace experiences (5).The development of suitable content for the acquisition of these capabilities and presenting this content through an appropriate platform are the requirements of promoting this professional competency in medical librarians(6, 7). Its signi cance becomes more evident when we consider the role of clinical librarians in providing valid and up-to-date medical information to the healthcare team and physicians. When discussing the design and presentation of the educational content of evidence-based practice, Belk et al. suggest that the educational content of evidence-based performance should include a review of the evidence-based approach, the steps of implementing this approach, presenting new theories, and using evidence in practice through quality improvement projects or research(8). Examining the content of 85 articles involving evidence-based performance, educational intervention and evaluating these studies in terms of covering the ve evidence-based stages, showed that the focus has been only on the teaching of some stages of evidence-based performance (mostly the critical appraisal of evidence), and high-quality tools were used for measuring the outcomes (2). The study suggested that educational packages and tools should consider all the practical stages of evidence-based performance to necessarily improve the evidence-based performance education. To achieve these competencies an appropriate curriculum should be developed by quali ed individuals or bodies. Education of professional competencies for library science should be developed throughout the educational journey of librarianship. Continuing educational opportunities for this profession also has to be created by library science departments to meet the expectations of professional competency(9).
The platform for education is another factor affecting the effectiveness of the course. Studies show that integrating evidence-based practice education in an active and technology-based educational environment, compared to traditional teaching, improves social intersections between learners, cooperative learning, and learner-centered learning (10). These will, in turn, promote re ection, sharing various points of views, content cohesion, and learner's understanding of content usage. In web-based education, variety of content formats, including text, text and picture, slides, slides and text, videos, podcasts, and interactive content can be used for promotion of quality of learning (6).
With respect to peoples' busy lifestyle, and the progress loof society in the digital age, web-based education is a good option for distance learning especially among librarians. Moreover, since the members of a speci c population, such as librarians working in universities of medical sciences, are often geographically distant from one another, in-person participation in educational courses may be di cult. Thus, what helps the effectiveness of this education to reduce limitations on physical attendance in classroom, direct contact with the teacher, promoting interactions for effective learning in this course, and ensuring learner satisfaction.
At present, in the syllabuses of Medical Librarianship and Information Sciences, the evidence-based practice has been introduced as one of pre-requisites for the clinical librarianship lesson in the B.A. It is expected to introduce lesson or capacity to learn prior to BA course when a lesson is introduced as prerequisite to another lesson in the program. The syllabuses is not responsive enough, if the prerequisite workshop for Evidence-Based-Medicine will be presented to students before entering in the course or during the course before introducing in the lesson? Who will be responsible to present/deliver this workshop? And when it will be delivered? Another ambiguity is that if it is essential to learn about evidence based approach, systematic review and question formulation by PICO for clinical librarianship why is not de ned as one independent lesson or why not combined with the clinical librarianship lesson?
How the competency for evidence based practice will be assessed and recognized when there is no systematically de ned education for that in the syllabuses? Furthermore, learners do not acquire capabilities, especially practical ones, with only two theoretic credits of clinical librarianship and prerequisite workshop for evidence based medicine, and thus most of graduates have di culties in their jobs. In addition , those who have graduated before the last revision of BA curriculum in 2017,have not passed anyformal education for evidence based practice. Meanwhile, due to the daily expansion of evidence-based performance-related topics in the medical community, the design and updating of medical resources and databases with an evidence-based medicine outlook requires the adjustment and cooperation of medical librarians with this trend. Therefore, it is essential to design an educational course that provides the competencies necessary for service provision in libraries or required by an employee trained to be a clinical librarian. With respect to the analysis of the learners' occupational responsibilities, this course can be effective for training medical librarians based on an evidence-based performance view, leading to accurate and practical service provision by librarians. Thus, this study aimed at designing content suitable for an evidence-based librarianship course and presenting it through web-based platforms so that librarians would acquire this ability in practice.

Methods
This descriptive study was conducted in three stages at Tabriz University of Medical Sciences in Tabriz City of Iran, during 2019.
First of all we used the ADDIE model for designing the course(11). The rst stage was "Analysis" which was Goal-Setting Stage and included the determining the course topics and contents. In this phase, Based on a review of literatures, the syllabus of the established courses in other countries were identi ed and a list of topics was drawn up. Then, two sessions of focus group discussion were held with the faculty members of the department of medical library and information science to identify the eligible syllabuses for course. Then a joint focus group discussion session was held with the expert panel (seven evidence-based medicine teachers including: three medical information specialists, two o cials at the evidence-based medical research center, and an epidemiologist and a medical education specialist). The opinions of group members were obtained on the topics, contents and the eligibility of trainers.
10 syllauses were determined based on the focous group deiscussionincluding: Introduction to the evidence-based medicine, concepts and general principles of evidence-based librarianship, structured questions and their types, levels of evidence, mind map and critical thinking, systematic searching and retrieving evidence, critical appraisal of evidence, and Introduction to systematic reviews and metaanalyses. Then the syllabuses were developed in 5 madules based on the ve stages of evidence-based performance, including 1) asking an answerable question, 2) acquiring evidence, 3) critical appraisal of the evidence, 4) applying the evidence, and 5) assessing the performance. In addition to these courses, some topics related to librarianship and medical sciences were designed in different scenarios appropriate for the topic as the tasks. The designed scenarios in librarianship and medical sciences were based on daily problems and questions encountered by librarians in libraries. In designing the scenarios, an attempt was made to consider relevant, engaging, challenging, and educational information so that librarians could gain experience in solving each of the issues with evidence-based problemsolving perspectives. Finally, for this course, the course plan was developed, and a study guideline was designed for the learners including the types of content, the objectives of the course, and learners' and professors' responsibilities.
The content were delivered in different formats including :text, audio, and multimedia.
A session was held with the o cials of the virtual education center and the required domain was allocated. With the approval of the virtual unit authority, some topics that were presented by professors outside the country and could not be presented synchronously due to time difference were presented in an asynchronous manner.
Then the implementation stage was performed. The course continued for a duration of six months (24 weeks) in the learning management system of Tabriz University of Medical Sciences (http://lms.tbzmed.ac.ir). Virtual teaching was offered based on the type of courses, supervised by the department of Medical Library and Information Sciences. To facilitate interaction between learners and trainers, a social chat group called EBIMP was created on the WhatsApp social network (https://chat.whatsapp.com). Participation in the educational course (connection to the system, steps of logging into the virtual education system, receiving and studying the educational content) was designed step by step along with images and explanations, and sent to individual learners/participants by the research assistant via WhatsApp or email. Before the onset of the course, the participants had to ll in the online pre-test questionnaire within the pre-speci ed time (45 minutes) before receiving a username and password. The learners could log into this system via their individual username and passwords 24 hours a day at desired times (synchronous and asynchronous). The educational content was gradually presented to the learners on a weekly basis over 24 weeks. The librarians could talk to professors on the forum presented on the virtual education platform, if necessary. The participants had a week to solve the problems and send the homework to receive feedback. In this educational course, an attempt was made to present important and up-to-date knowledge and topics related to evidence-based performance stages, along with case scenarios, to the librarians in order to guide and motivate them to further explore knowledge with a critical thinking approach. In this process, learners have an opportunity to follow a complete evidence-based process virtually and be actively engaged with critical thinking. In some group tasks, the learners chose one person as the head of the group, who had to share the duties among the members and upload the nal report on the forum. Each group criticized the tasks of other groups, and the groups' criticism was also uploaded to the forum. The professors were facilitators in all steps of doing the homework and team projects. An online session was held after completion of the project, and the heads of groups presented the group projects. At the end of the session, the learners' criticism was evaluated, and the professor summed up the session. This learning method was chosen for enhancing the skills and increasing learner-learner and learner-professor interactions. One week after the end of the intervention, the nal examination and the course evaluation were performed.
The nal stage wasthe course evaluation, which was done based on the two initial levels of Kirkpatrick's model, i.e. I) reaction: satisfaction and II) learning: formative and summative assessments. Formative assessment was done by assignments and feedback and the summative was done through nal project protocoldesigning. A satisfaction questionnaire with three domains of course design, support services, and promotion of personal capability along with three open-ended questions about the strengths and weaknesses of the course and suggestions, was designed.The data were analyzed in SPSS 19 using descriptive statistics (mean and standard deviation).
Face validity was assessed qualitatively and quantitatively. In the quantitative method, to examine the content validity index (CVI), the method of Waltz and Bausell(12) was adopted, and the judgment of each member of the expert panel about each item was checked by three criteria of simplicity, clarity, and relevance on a four-point Likert scale. The minimum acceptable value for CVI, based on the expert panel size (n = 8) was 0.79, and if the CVI of an item was < 0.79, it would be modi ed based on experts' opinion.
To determine the content validity ratio (CVR), the judgment of each member of the expert panel about each item was evaluated on a three-point scale: 1 = It is essential, 2 = It is useful but not essential, 3 = It is not essential. Based on Lawshe's table and the number of evaluators (n = 8), items whose CVR was > 0.75 were deemed essential(9).
To examine face validity, the participants were asked to check the degree of importance of each item on a ve-point Likert scale from 1 (not important at all) to ve (very important). Only items with scores > 1.5 were deemed as having acceptable face validity (13).
The reliability of the instrument was checked with Cohen's kappa.Seven faculty members of the panel of focus group scored the questionnaire items from 0 to 100. The mean agreement for each item was calculated, and the mean score of the items determined the questionnaire's nal reliability (14).

Results
Design: Topics of the course and the number of sessions About 20 topics were identi ed through literature retrieved from PubMed and Google Scholar and gray literature via website of educational institutions. During two sessions of focus group discussion, some of the topics were merged and some modi ed nally 10 modules were selected as syllabuses for the course. The 10 syllabuses was devided into 6 modules. The rst module was the introduction and the next 5 modules were as same as the 5 evidence based steps. Table 1 shows the topics offered through course.
It is necessary to explain that the steps of evidence based performance in information science is different in practice from the evidence based medicine. Therefore, in the applying step a protocol for performing systematic review/meta-analysis was de ned. In the assessment step the protocol had to assessed by the learners using standard checklists.

Implementation
In this course, 10 lessons, 15 electronic contents, and 10 tasks were uploaded. Figure 1 presents the activities of the participants.
All of the participants downloaded/viewed the contents but few of them engaged in the applying and assessing steps by designing a systematic review protocol and assessing the protocol by the standard guidelines.

Evacuation
We assessed the satisfaction of participants to nd out the quality of implementation of the course. The tool showed high content validity (CVI = 0.88, CVR = 0.85).And reliable based on Cohen's kappa was 0.89 . Therefore, the face and content validity and the reliability of the questionnaire were con rmed.
Subsequently, the questionnaires were completed by the learners. Thirty (out of 70) participants lled in the questionnaires. The demographic information of the participants is given in Table2. Men (11) 25-30 (17) 35-40 (10) 41-45 (3) 1-5 (14) 5-10 (13) 10-15 (5) Participants satisfaction level is given in Table 3. The overall satisfaction of the participants was excellent (83.52 ± 16.27), and the highest level of satisfaction was related to support services (94.52 ± 29.68).  Table 4 presents the most signi cant strengths and weaknesses of the course from the participants' perspective and their recommendations for improving the course. Overall, responsiveness to questions and ambiguities in the virtual course was good.
The professor did a good job in designing the topics and contents.
The professors had good responsiveness and behavior.
All those interested in the course can take part in this course which overcomes time and place limitations.
Audio or video les were used.
The scenarios were tangible and practical.

Weaknesses
Few slides contained video and audio.
Less content was uploaded at the beginning of the course, but a large volume of content was uploaded towards the end of the course. Therefore, it was impossible to use the content well due to the learners' numerous administrative and personal tasks at the end of the semester.
The duration of the course was too long, but the number of topics was very limited. An intensive course with a sequence of adequate topics could be presented so that the topics would remain coherent in learners' minds.
The beginning of the course coincided with the new year holidays, followed by Tehran International Book Fair. This should have been taken into account.
The noti cation for exams was not e cient, and the topics were presented as slides and headings.
The questions at the end of some lessons were di cult.
The initial lessons were not accessible for several participants at the beginning of the course Fewer video les were provided compared to audio les The user interface of the website was not good.

Recommendations for improving the course
It is good to recruit foreign professors, but it is better to add Persian subtitles.
A semi-virtual course is more effective than a fully virtual one.
There should be more online than o ine classes.
Learner-learner and learner-professor interaction should be boosted.
To promote the quality of the contents of each lesson and provide a better understanding of the topics, the professor should provide his/her teaching in the form of a recorded video with explanations.
Complementary references should be introduced for better learning.
Scenarios should be published as teaching material for all.
The signi cance of international interactions in education and research is undeniable. Therefore, planning and policy-making for improving the learners' English language skills are necessary.

Discussion
An attempt was made to design an evidence-based course to respond to the professional needs of learners and promote librarians' decision-making skills to meet the information needs of the library users. Based on the survey, the learners' satisfaction level was very good, and they found the course effective in promoting their personal capabilities. This nding is in line with the study by Hsieh and Chen (15) who examined the effect of a virtual educational intervention on evidence-based performance for nurses in Taiwan. In their study, the participants' scores in the domains of knowledge and self-e cacy were signi cantly increased over the course of the study, showing remarkable progress in the nurses' knowledge, skills, and self-e cacy (15). In the study by Kagan, it is found that evidence-based educational intervention was effective in improving the nurses' critical thinking and decision-making skills(16). In implementing this educational course, we recruited distinguished Iranian and foreign professors and pioneers of evidence-based medicine and performance in librarianship and information sciences, which was a strength of this program. This approach to program implementation enhances the interaction between those who work in the domain of evidence-based librarianship and information sciences with their peers and scholars in Iran and elsewhere, and provides an opportunity for interdisciplinary interaction for these disciplines (17). In education based on simulated environments, the use of scenarios is very practical(18). One of the attributes of the evidence-based approach is placing the person in possible future situations in which one has to make important decisions with few errors. Therefore, to increase the learners' decision-making and critical thinking power, scenarios are written to place them in possible and simulated conditions (19). Scenarios are adjusted based on the possible incidents and events that may affect the formation of a topic or event in the future (13). This stimulates energy, movement, and change throughout the scenario, and is a way to start scenario-based critical thinking. We used the scenario model to assess the learners' level of critical thinking in nding, appraising, and using evidence. As positive feedback given by the learners in open-ended questions, they stated that "the scenarios were practical" and "the scenarios should be published to be used by everyone." The ndings showed that scenario writing is an effective way for promoting evidence-based learning. As emphasized by Chermack, scenario writing is a tool for altering and improving people's perception for decision-making, and their capability of monitoring uncertain situations (20). In this course, we also used a virtual education platform that facilitated the interaction of participants from different cities and the recruitment of professors outside the university and the country. Studies show that integrating evidencebased performance education in an active and technology-based educational environment, compared to traditional teaching, improves social intersections between learners, cooperative learning, and learnercentered learning. These will, in turn, promote re ection, sharing various points of views, content cohesion, and learner's understanding of the uses of content (10). According to Brânzei et al., the use of active learning strategies is essential to increasing the acquisition of research topics. They believe that evidence-based performance requires educational strategies that reinforce interaction and research (21,22). One of the main suggestions of the learners was to hold the course in a hybrid way (inperson/virtual), which can be taken into account in designing future courses. In the study by Es jani, the use of hybrid education for the course of "integration of technology with the curriculum" improved learners' satisfaction (23). Despite the advantages of virtual education, the use of this platform alone reduces learner-learner interaction and limits their social skills, e.g., the sharing of feelings, knowledge, and experience, and a sense of social belonging in learning (24, 25)(18, 19). Hybrid education can help reduce these challenges and enhance the e ciency of the teaching-learning process. If it is impossible to hold in-person group sessions, especially due to geographical distance, the facilities of the virtual space can be exploited. By forming virtual groups and online video sessions, live interaction and discussion can be possible. These sessions can also be used to share the learners' experiences to promote their knowledge.
In the long run, by implementing these educational courses, resolving their weaknesses, and improving their strengths, they can be personalized for different target groups in order to teach and promote evidence-based performance. Moreover, these courses can be implemented in the form of in-service training to improve the learners' motivation for participation and exploit the administrative potentials of other organizations. With the experience of these virtual courses, more specialized courses such as teaching advanced search in evidence-based databases, a systematic review training course for librarians, and other specialized courses related to evidence-based performance for medical librarians can be planned and implemented. One of the limitations of this educational program was the learners' weakness in using English-language contents which necessitated the addition of Persian subtitles to multimedia contents. Moreover, no interaction was observed between the learners and international professors in the WhatsApp group.

Conclusions
One of the main responsibilities of educational centers is to provide education based on the abilities and professional responsibilities of the learners. To keep the curriculum size in check, educational courses can be presented as continuing professional education based on the learners' professional needs and with respect to the changes in their needs. The evidence-based librarianship course has received little attention in the curriculum. Librarians, especially those working in organizations a liated with the health system, should be empowered to respond to the current professional needs. There is a dire need for the use of new and diverse methods and platforms for librarianship education. It seems that scenario writing models and scenario-based problem-solving greatly improves critical thinking, learning, and evidence-based performance in librarians. The use of virtual education platforms for this empowerment enables the librarians from different regions to take part in these courses and bene t from the knowledge of Iranian and foreign professors.

Practical results
Evidence-based performance promotes the librarians' critical thinking. This can be facilitated by the o cial teaching of evidence-based performance in librarianship as a branch of medical librarianship and information sciences, its short-term presentation, and including it in continuing education courses for librarians.