The results of the short-term and long-term data are presented according to the Kirkpatrick model (Figure 1). The short-term outcomes included participants’ reactions and learning and were measured through pre and post knowledge assessments, course evaluations administered following each course, and reflective commentaries administered following completion of all eight courses. The long term-outcomes, which include behavioral changes and results at the organizational level, were measured through semi-structured interviews and organizational surveys, conducted 6-months and 7-months post completion of all courses, respectively. The data obtained from semi-structured interviews and organizational surveys was triangulated and merged in the analysis.
[Insert Figure 1]
3.1 Short Term Outcomes
3.1.1 Knowledge Assessment
A significant increase in the test scores from pre to post training was noted in all courses. A statistically significant increase in the test scores from pre to post training was noted for all courses except Course 2, although post training scores were still higher than the pre training scores for Course 2. Table 4 outlines learners’ pre and post test scores for each course.
[Insert Table 4]
3.1.2 Course Evaluation Survey
Results from course evaluations obtained from learners at the end of each course indicate that the training program was well received, and that satisfaction was high.
Regarding the method of course delivery, most learners agreed or strongly agreed that the course objectives were fulfilled (80.3%) and were clearly stated and presented (83.2%). Most learners also agreed or strongly agreed that the time allocated for the course (77.1%) as well as the pace of the course (77.2%) was appropriate. Additionally, most learners reported that the content was relevant to their needs (74.5%) and that they would take another course through this delivery method (87.3%).
Evaluations of course content demonstrated that the majority of learners agreed or strongly agreed that the course contents were organized and easy to follow (79.8%) and that they obtained useful competences (79.9%). Similarly, a high percentage of learners agreed or strongly agreed that their knowledge and competence in the subject matter increased (86.3%). Most learners reported being satisfied with the overall quality of the courses (78.1%) and would recommend it to others (79.8%). Additionally, many learners agreed or strongly agreed that they would participate in future courses by GHI (91.3%).
Feedback regarding instructors’ performance indicated that most learners agreed or strongly agreed that the instructor was well-prepared and knowledgeable about the topic (86.6%). Most learners also agreed or strongly agreed that the instructor was able to communicate the course’s information clearly and effectively (82.5%) and was able to stimulate students’ interest in the subject (77.1%). Moreover, most learners were satisfied with the overall performance of instructors (83.3%).
Table 5 reports the percentage of participants in agreement (i.e., Agree and Strongly Agree) with each component of the course evaluation; data is also stratified by learning modality.
[Insert Table 5]
3.1.2.1 Course Evaluation (open ended responses)
Course Opinions and Expectations
Short-term course evaluations indicate that the course content was positively assessed. Courses were considered organized, comprehensive, and well-explained. Inclusion of practical assignments and case studies was appraised. Courses exposed learners to new concepts and allowed learners to acquire new knowledge and skills pertinent to learners’ job duties and responsibilities. Courses delivered via online learning modality were positively evaluated in terms of course design, layout, flexibility in time management, and the opportunity to facilitate their learning experience through posting discussions on an interactive platform. Overall, the courses met the learners’ expectations.
However, limitations were also mentioned. Learners noted that excessive theoretical information was presented, some material was repeated across several courses and thus became redundant, some courses were too fast-paced, and some assignments along with their instructions were not clear. Additionally, learners indicated that assignments were expected to be more comprehensive and sought for more enriching feedback on assignments and quizzes. Learners also noted that engagement was limited due to lack of direct communication and discussion with the instructor.
Suggestions for Improvement
Suggested improvements were related to reducing content overload, minimizing content redundancy, and providing more time to cover the content. Suggestions to incorporate interactive material, more visuals such as videos and audio recordings as well as more practical examples and case studies was noted. Some learners called for revision and clarification of assignments to make them more representative of the courses, and to shorten quizzes and modify certain question items.
3.2. Long-Term Outcomes
Data from semi-structured interviews was collected from 8 participants who completed the diploma. Four organizational surveys were collected from colleagues of learners, and the results obtained from semi-structured interviews and organizational surveys were triangulated. The results are presented in Table 6, under the following two categories: Strengths, and Barriers and Challenges.
[Insert Table 6]
3.2.1 Strengths
Five themes emerged under the strengths category after analyzing semi-structured interviews: (1) Online Learning Modality, (2) Significance of Diploma, (3) Course Content, (4) Instructors, (5) Transfer of Learning into Performance, (6) Personal Development.
Theme 1: Online Learning Modality
The online learning modality reportedly allowed for better time management, planning, and organization, providing learners with the added flexibility to manage their work and courses simultaneously. Some respondents expressed that this modality provided a pleasant and beneficial experience. Learners also noted that most learning objectives and expectations from the courses were met through the online modality. Importantly, some learners reported that this was their first time attending an online training and were now encouraged to attend similar engagements in the future.
A.F.5: Having the diploma online helped me a lot from all aspects, to be honest I was working in Beirut then I moved to Zahle, and at the beginning I was accessing all my courses while in Zahle, so I was thinking that if this diploma required physical presence, I wouldn’t have had the opportunity of exposure to the diploma, so it was way easier for us since we already had work.
Additionally, most learners reported that the online modality facilitated access to courses at one’s own convenience, providing additional flexibility to learners. It was noted that the ease of access to comprehensive information allowed learners to obtain a library of knowledge that can be referred to at any point. Several respondents highlighted the importance of this modality in enhancing their learning process as it allowed learners to self-pace their learning while giving them enough freedom to delve deeper into the course content. Some respondents also appreciated the fact that this modality allowed the provision of reliable information in a time-efficient manner, supplemented by proper guidance from instructors. Another respondent notably commented that the flexibility with timing allowed the inclusion of diverse learners from different organizations and different countries.
K.Q.1: I got the eight courses, which are very very useful and essential for any project manager who is in the humanitarian setting; all in one place. You have, it is like you are building your own library for the humanitarian setting; you learn through it, but also you have your own library to go back to. It is a reference whenever needed, but this is an amazing learning process to me.
K.H.8: When someone provides his personal experience and guides me and facilitates the presentation of information and provides books that targets the topic I would like to know more about, then there is a clear difference between this and just taking information from google or YouTube, where information may not be credible and you need to make sure that the information is correct. It is not the same at all, because when the course is delivered in this format, you are confident with the information you are receiving and there are instructors that guide you and support you, instead of wasting time to check the references and make sure the information is credible.
One respondent suggested that the online learning modality did not negatively affect the quality of training in comparison to face-to-face learning and expressed that this modality of learning was more convenient for her since she is not comfortable in a class setting.
N.B.6: Being online, and not having to do it at the classroom or telling them to come, firstly, encompasses more students around the world which I think the diversity especially in these interactive assignments really helps, especially when you have students from different countries and different organizations coming, so this is a big strength for the online modality.
S.K.2: Personally, I’m very comfortable with online learning, I don’t like the class setting, so this was very convenient for me.
Moreover, some learners commented on the suitability of the online platform, describing it as user-friendly, well organized, and easy to navigate. Learners also mentioned that the variation in pedagogy, such as through videos and interactive blocks, rendered the courses more engaging and facilitated the comprehension of information.
KQ1: the online platform was easy to follow but also the way the layout was really nice and gives you where you really are where you are at, at anytime which is really good.
Another learner indicated that online delivery of the HLD allowed them to simultaneously work and study, despite residing in a conflict-affected setting where such a combination is difficult. The diploma was also considered as an added value in an area of instability.
K.H.8: You know the war in Yemen impacted the quality of the education and caused it to be unstable. Also, to be able to study and work at the same time was seen as something very hard, so the online course has become the best thing for employees who work and study at the same time. Even if I was enrolled in community medicine, honestly my education at this point was not stable, so for this reason the online course really benefited me.
Theme 2: Significance of Diploma
The importance of the HLD revolves around its regional significance, local development, online delivery, and being tailored to the context of humanitarian issues in the MENA region. Several learners commented that the availability of a diploma focused on the humanitarian field as being rare and not easily available in the region. Most learners perceived the diploma to be unique, and very much relevant, timely, and of interest to the population. One learner additionally expressed that the humanitarian topics covered by the diploma were needed at the global level.
K.Q.1: I rarely very rarely come across this kind of diploma especially say in middle east countries there is no such institutes on the ground that teach this type of teaching like humanitarian settings; you hardly come across any institutes or universities that teach you this. So this kind of diploma is unique by itself it is a strength, and I am sure if many people that knew about it, they all will take it because it is needed.
N.B.6: The biggest strength of this diploma is the content itself. The idea of the Humanitarian Leadership Diploma because maybe I am not in the field, but I have never seen a course like this before or an outlet of knowledge like this before. So the biggest strength is the content and the type of knowledge that we’re gonna give.
A.F.5: The strength of this diploma is that it is in a high demand by the organizations world-wide. World-wide organizations want to be involved in such a humanitarian sector. So, people will be really interested to be registered in such a diploma.
Several learners expressed that the implementation of the diploma by a reputable and well-known institute was regarded as an added value to their resume.
K.Q.1: having this course coming from a very well-known university which is AUB is a very strong point
S.K.2: I would place it proudly on my CV, Global Health Institute AUB.
One learner also expressed how the diploma was valuable when compared to other universities, specifically with regards to using a unique online platform and continuing to implement courses during the COVID-19 pandemic.
N.B.6: You were already at the forefront compared to other universities in implementing online platforms during the Covid period.
Theme 3: Course Content
Contents of the diploma were evaluated positively by most learners. Most learners reported that the content was appealing, complementary, beneficial, and covered essential information in humanitarian settings. Several learners noted the compatibility of the content with their jobs. Furthermore, the comprehensive nature of the courses allowed learners to gain a rich learning experience and obtain useful resources.
K.Q.1: Full of good materials that were gathered in one area; it really helped me a lot. The whole training gets really complementing each other and I think that’s what a professional project manager a humanitarian project manager will need, so it’s the content really well taught throughout and that's all what a project manager in a humanitarian setting will need.
N.B.6: This course is more than enough information that you are ever gonna need.
A.F.5: The topics that the diploma was covering are very interesting like project cycle management, monitoring and evaluation, budgeting and financials, disaster response and the sectors of intervention. These four main courses were very appealing to anyone who wanted to enroll in these domains and enhance their knowledge.
Additionally, most learners reported that the courses provided basic information that was essential for leadership positions, even for those not working in humanitarian settings. Others mentioned that some of the content was novel to them and thus the HLD exposed them to relevant information pertinent to the humanitarian sector despite having previously attended several trainings in the humanitarian field.
S.K.2: We need to have all the basic information from the course subjects, and we were able to reach this point.
K.Q.1: My capability to learn new skills, there were some areas in the course that were totally new for me; helped me in developing my skills by reading and studying through.
Moreover, most interviewed learners indicated that the course content was well-prepared, well-structured, and thoroughly explained. Most learners also reported that the course objectives were clearly stated and matched course contents. This enhanced learners’ comprehension of major aspects of humanitarian settings as well as improved their ability to respond to their job requirements.
A.J.3: The content was prepared really well. It was highly structured; no steps in presentation of the courses were skipped over. Everything was explained in detail, step by step. All the objectives were very clear, and I could say that the whole diploma was complete.
Two learners specifically noted that the inclusion of interactive features into the course content like interactive blocks including images, videos, flashcards, and multimedia blocks rendered the course content to be sufficiently detailed and increased learners’ engagement.
S.K.2: The courses were presented in detail; when you flip each square, you find a definition behind it. You would not feel that any of the information is missing.
N.B.6: Honestly, I found the lectures posted online to be very informative and very interactive with the inclusion of videos and examples … Those were very engaging.
Some learners also appreciated the availability of supplementary material like PDF links and book references. The presence of these resources enriched their learning experience.
A.J.3: Overall I would say that it was fair, some courses had lots of interest in contents. Most courses had really important references. Diverse book references were presented in the interpersonal communication course and other courses; the provided references were very important.
K.H.8: There were subjects that I wanted to learn more about which advocated for me to read more about them, especially that I received PDF links, so I tried to go over those links, read the shared PDF books and followed up with the courses.
One learner noted the credibility of the content presented in the courses. She indicated that the course content reflected her work-related practices, experiences, and duties.
K.H.8: I began gaining information from my practical experience at work in 2016, but when I entered the online courses I realized that this obtained information is correct and originated from specified principles and guidelines; the diploma confirmed my information.
One learner highlighted that the course content seemed to have been informed from the instructors’ personal experience and expertise and did not include any copied or plagiarized material. The originality of the content presented, along with the information directly obtained from experts in the field accentuated the diploma’s distinctiveness and increased the learner’s engagement.
N.B.6: It could show the professors themselves wrote the lectures, and the material was not copy-pasted. On the contrary, it was evident how the teachers placed their full efforts into the subject matter, and you could see that some of the teachers at least drew from personal experience and wrote them in the lectures, and this was all very engaging.
N.B.6: Like what I really enjoyed about the course was despite the presence of a reference for it, the lectures themselves didn’t make me feel like I was studying from a book, it made me feel like I was studying from a person who actually knew in the field.
Additional importance was also attributed to the assignments. Many learners described the assignments as suitable, practical, and fostering greater understanding of course material.
A.J.3: Some assignments were really nice, what else, like through this assignment which is similar to an exercise, you really challenge yourself and it really takes time and effort, but it does lead to greater understanding, so it was well crafted and suitable for the level of the learner.
A.F.5: A strong point of the exercises at the beginning is that we were working on case study proposals. We used to work with case study proposals and use them as examples. This was a strong point because we were really practicing everything that we were learning.
Several learners also commented on the practical use of the assignments in their workplace roles and responsibilities. Learners specifically mentioned that the assignments enhanced their practical experience, accentuated their discussions and interactions with colleagues, and improved their understanding of colleagues’ roles within their organization.
A.W.4: I tried to use the assignment in my project with the modules that I was undertaking in order to present both the assignments for the training program and at the same time to present them for my proposal at work. So while doing the assignments, I was at the same time using them in my project in order to incorporate improvement plans, since I am using the same information from the training.
A.J.3: The nice thing is that when I had an assignment, I used to consult my colleagues in the organization since I work in an NGO. This allowed the sharing of experience, and allowed me to get more familiar with the work duties and work experiences of my organization colleagues. This also helped me to compare my experience with the textbooks from which I was learning from; this comparison was really good.
One learner also highlighted how the inclusion of quizzes at the end of each course module allowed learners to self-evaluate their learning and comprehension level.
KH8: There was a quiz in the verbal non-verbal communication course. I really liked this idea because it allowed me to be assured that I mentioned everything and that I understood the main idea; it also showed whether I understood the questions that I had to answer. The quizzes served as small evaluation tools for us to know what we understood and what we didn’t. The quizzes also allowed for revision of the correct and incorrect answers; so this was really beneficial.
Theme 4: Instructors
Overall, it seems that the performance of instructors in this diploma was well received. Instructors were described as highly knowledgeable and cooperative. They reportedly met high professional standards and conveyed knowledge in a simple and effective manner. One learner reported enhancement of their own abilities specifically due to the instructor's elaborate knowledge and expertise in humanitarian work. An appreciation was expressed for instructors who showed an effort to include personal experiences. According to learners, the inclusion of personal experience rendered the courses more engaging. Instructors also provided constructive feedback in a timely manner and were noted to be ready and available to answer all questions via email. In addition, having a variety of specialized instructors was considered one of the strengths of the diploma.
S.K.2: The variety of instructors also was a strength, and how much they were available and ready to answer any questions and provide their emails.
A.J.3: The instructors are really professional and knowledgeable in this area.
K.H.8: I also liked the idea of receiving feedback from instructors whenever we sent them the assigned tasks. I liked that they provided grading and feedback and what should the answer for each question be.
Theme 5: Transfer of Learning into Performance
Most learners reported acquiring and/or enhancing their knowledge and skills through the HLD. The information provided through the courses was reportedly beneficial in complementing previously acquired knowledge and provided appropriate guidance for carrying out humanitarian work, particularly in areas of conflict.
K.Q.1: There were some areas in the course that were totally new for me; helped me in developing my skills by reading and studying through. It is like for instance I was not knowing very well about the differences in the international humanitarian law and the international human rights law and which one applies where and when, and this is a new skill.
Learners reportedly gained new skills that influenced their practice, such as team management skills, budgeting, negotiation, and communication skills, all of which allowed them to expand their scope of work, increase their involvement in tasks, and improve their overall performance.
M.Z.7: Adding the management aspect to the things helped a lot; it helped me to understand more what is happening in the project. Previously for example I was involved in writing proposals but I was involved in the technical parts only. I didn’t go, for example, in the monitoring plan or in, let’s say budgeting, and these two things were part of the diploma, and after that I started to be more involved; I started to suggest things and see things, and this proves valuable with my technical background.
A.J.3: In the M&E course, I used to work on basic reporting which is to report my activities, but now I am doing reports for the whole team. I know that M&E is a cornerstone, and this I was able to acquire from the course, and I learned also how it plays an important role if you want to do a proposal, let's say for a project, so you need to have your budget drafted and your M&E log frame drafted.
In addition to reporting that the training familiarized them with the roles of their coworkers, learners noted that this training allowed them to have a holistic understanding of organizational structures and processes in the humanitarian field. This, in addition to the acquired knowledge, allowed some of them to engage and coordinate more effectively with coworkers. For instance, one learner reported an improvement in internal coordination with his coworkers after gaining a better understanding of the process of logistics implementation. Other learners also reported implementing new skills such as self-evaluation through setting indicators to monitor work performance, becoming independent learners, and increasing their capacity and output to deliver online sessions to other learners.
A.W.4: I am now more goal oriented and I am able to put more indicators so I can trace my performance or the activity that I’m doing, so from the start I can think in a more strategic and systematic manner instead of being somewhat random so when I’m doing any activity or anything else, I am focused on the end-goal and I can do evaluation routinely and clearly from the start.
Importantly, one female learner reflected on a positive experience following completion of the diploma whereby she encountered decreased gender-bias and increased acceptance in her work setting. The diploma had enhanced her skills which translated into being better accepted in a gender-biased work setting.
K.H.8: Honestly, I benefited from the diploma in my workplace. I believe that the male’s thinking that he remains better in a managerial position in comparison to a competent female manager has changed. The acceptance of me as a competent manager and a knowledgeable woman rich with information has increased.
Although this was not commonly reported, the training did allow few learners, mainly coordinators and those in managerial positions, to develop new policies and to participate in the strategic planning of their organization. One learner reported that the training enabled him to co-establish a humanitarian non-governmental organization.
M.Z.7: I became more involved in the development section. My organization is representative of local NGOs in the health core group and in the production core group, so we are involved in the development of almost all plans that I have in Lebanon. Since the finishing of the diploma, there has been this Lastrapil which was a framework for Beirut blast and the Covid 19, there was also LCRP Lebanese crisis response plan, where we build on the previous years in order to extend it for one more year and now we are in the process in initiating another crisis response plan, so this is like a strategic plan on a national level, and not only on the level of the organization.
K.Q.1: I re-established a new organization as an international organization, so I am one of the co-founders, and its headquarter is in Germany. But I believe this course enabled me to be able and to be in a position that I could establish this organization; it is a humanitarian non-governmental organization.
This was supported by data collected from learners’ colleagues through organizational surveys. The senior project officer of K.Q.1 noted the following about the employee’s performance: “The strong point was developing all the main subjects and SOPs of the work according to the humanitarian principles and guides that he got from the training”.
The supervisor of another learner highlighted the improvement that was noticed in his employee’s performance: “He has improved in establishing relevant indicators and knows exactly where and when to be used, that has impacted well the improvement of his department general performance”.
He also added “A.W.4 has prepared a proposal for a quality improvement plan but the lack of required authority would be the biggest barrier to implement his innovative thoughts”.
Theme 6: Personal Development
Many learners highlighted the role of the HLD in furthering their personal and professional development. Some learners reported increased self-confidence and others noted improved capability and increased readiness to work on humanitarian projects following the courses.
K.Q.1: This knowledge I gained and the experiences through the courses made me more confident that I am more familiar with humanitarian work now.
N.B.6: So maybe I would no longer be a volunteer, probably I will be doing tasks with more or probably will be doing tasks or recommending or criticizing not in the sense of criticizing but giving constructive criticism to the organizers because of the knowledge I have.
Several learners also reported that their acquired knowledge allowed for career advancement in the humanitarian field. One learner also noted the significance of the diploma in his personal life, particularly because a degree in humanitarian work is well regarded given the extensive humanitarian presence in his country.
A.J.3: My knowledge increased, and this helped me evolve in my work, because I started as a PSS officer and then became the senior, and the thing that helped me most was the fact that I learned at Global Health Institute how things should go, so I paved my way with the help of this program.
K.Q.1: The diploma even helped me in my own personal life; having a degree, it is something really advanced, especially in a field that is humanitarian and the country I am living in, where there are many humanitarian workers and NGOs.
One learner emphasized that the diploma encouraged him to complete further certifications in the field of humanitarian affairs. Another learner explained that the diploma allowed him to acquire in-depth knowledge of a field he was unable to learn about previously due to it being slightly different from his original academic specialty.
M.Z.7: okay so the training exposed me to these topics that I wasn’t aware of before and it showed me the importance of having the leadership aspect, the management aspect in my work and based on the… like what we learned in the diploma was the basics, I decided to go further into that and proceed with an MBA.
K.H.8: Either way I was a doctor. I was initially a gynecologist and then I took community medicine. I wanted to develop myself; look, because the family medicine that I studied, I think it does not include the same content and the context differs from one country to another. So, I really wanted to learn in a deeper and more comprehensive way outside my context. So the diploma was really beneficial from this point.
Additionally, one learner shared his aspirations of being an asset to his organization following the extensive knowledge he acquired from the HLD.
N.B.6: Okay so now probably the next time I go to volunteer I would probably be more aware in for example the different sections, the different characters, the different tasks, the different roles that have been mentioned in my course and that I can probably visualize over there. Probably as a volunteer next time I will be more of an asset to the organization because of the knowledge I have.
3.3.2 Barriers and Challenges
Three themes emerged under the category of weaknesses and challenges: (1) Barriers to Applying Change in Behavior and Performance, (2) Engagement and Interaction, (3) Pedagogical Approach.
Theme 1: Barriers to Applying Changes in Behavior and Performance
The most notable challenge faced by learners who were interviewed was associated with translating the acquired information into organizational changes. This was especially true for learners who are not coordinators or do not hold senior positions. Some noted that they were unable to apply their newly learned skills within their work due to lack of practice, being newly employed, or contextual factors. Others attributed the absence of knowledge transfer to time constraints or to the impact of COVID-19 precautions on restricting the application of various work-related duties during this period. Few participants noted challenges in initiating major changes in organizational policies or in implementing new strategic plans. Learners commented that such changes could not be carried out at the individual level alone; rather, they require structured changes with the involvement of coworkers at different levels of the organization. One respondent reported that the preexisting functional policies in their organizations do not allow for the implementation of new ones.
K.H.8: I faced several situations at work that made me benefit from the experience of online learning and the online courses, from which the changes in policy. However, the change was not complete as we are still at the first steps of change. As you may know from your work environment, you usually put down your notes, discuss them, then they are agreed upon, then you discuss them on a higher level and so on.
K.Q.1: I was with another organization, but they phased out from the country since June, had two months sitting and resting, then I joined another organization and you know this pandemic came in and it did not allow, it did not allow to fully apply the knowledge I gained on the programs I was doing.
A.J.3: I don’t have expertise in the area of strategic development, so to be honest, no, I did not implement new strategic plans, and like I said before the projects are very clear from the start and we operate by them.
Theme 2: Engagement and Interaction
Despite the advantages offered by an online learning modality, one limitation reported by most learners who were interviewed was the minimal engagement and interaction. Specifically, asynchronous courses were perceived to be less interesting than others.
A.J.3: At certain times you are not engaged, there is no engagement. This lack of engagement will make you bored. So, if you have this course, it is a dry material. If we are also accessing two or three chapters each week, we are just reading them, if there is no interaction it is going to be very dry.
It was additionally reported that the online learning modality limited the exchange of experience and information between people from different backgrounds. As such, learners noted low levels of interaction between learners. Some respondents expressed the need to have at least one session in which all learners are present, as this would enhance not only participation but also the sharing of expertise.
M.Z.7: It just needed more encouraging the exchange between the participants more because usually when I attend any session by when I attend any training session, any workshop, one of the most important parts for me personally is the exchange between the participants because participants have different backgrounds. They have different experiences; they would share lessons learned and best practices with each other, and maybe this was what was a bit lacking in the diploma.
Moreover, learners mentioned that the limited ability to interact with instructors and ask questions directly hindered the clarification of some course content and obstructed the learning process. One learner suggested interacting with instructors through posting questions and answers on the modules or through weekly live sessions as it would have added value to the training. It was also noted that the learning process was hindered by minimal engagement, especially for courses that deliver practical content rather than theoretical.
A.J.3: For example, for the personal communication skills, this is a skill to practice you have to exercise, use your verbal and non-verbal skills, this thing was being applied, you have to use eye contact language and all this stuff; you only know them theoretically, you get me, and same applies for negotiation, okay you can write...but the engagement is lacking.
Theme 3: Pedagogical Approach
Although the course content was well received by most learners, a few shortcomings were indicated. Several learners mentioned that the diploma did not sustain the same quality throughout its implementation, noting that the quality seemed to gradually decrease towards the end of the diploma. This was due to an increased focus on presenting theoretical content, without enough provision of practical assignments and exercises.
A.F.5: The first three four courses were very interesting. A high involvement from the participants was observed; we had a lot of projects to work on, but the involvement decreased throughout the year. In my opinion, the level of the diploma deteriorated a lot, and it didn’t maintain its initial level of quality. With each course, the quality level of the diploma was declining, specifically our level of participation and involvement with other attendees. Knowledge wise, the diploma became solely theoretical at the end. We almost stopped doing course homework and exercises. So, by the end of the courses, we didn’t feel ourselves really benefiting from the course by the end of the courses. The modality of the exam, the course became more theoretical, very very theoretical.
Another concern was reported by a learner who expressed feeling overwhelmed by the volume of material presented behind the interactive blocks as it affected the learner's attention span. Another learner highlighted the presence of repetitive and overlapping information across the courses which negatively impacted the learner’s enthusiasm to proceed with the studying process.
S.K.2: A weakness of the diploma is that when you flip the square, 10 pages of information appear. But what is usually required to appear is a sentence or two, so that you will be able to understand the information. But when a whole big paragraph of 10 sentences is presented, you forget what you started to read from the start. This huge presentation of information causes distraction.
N.B.6: if you are taking six to seven courses and for a whole week you are learning the same information, you will get bored if you read for six to seven weeks the same information; maybe changing the flow of the course, given that the courses are sequential one after the other, there should be a better way for the teacher to communicate the content and avoid the overlapping of the information. I felt there was a lot of overlapping information, and this made me feel mad at times because I had to read the same information. Given that we are encompassed with the course for a whole one year and a half, the overlap of information I think is the most important thing that you need to take into consideration because it really gets repetitive.
Several learners perceived the content of the diploma to be insufficient for those with a vast background in the humanitarian field. Conversely, some learners reported that there was minimal accommodation within the course assignments to those with limited humanitarian background, stating that assignments seemed to target individuals who were already employed at an NGO.
K.H.8: Yes the basics of the material were present, but what I wanted is to learn in more depth because I already had the knowledge concerning certain topics.
S.K.2: The point is that each one of us was coming from a different background and I heard that a lot of others were not satisfied with the courses’ content because they thought that the content will be more sophisticated rather than being mostly basic.
S.K.2: The diploma was addressing individuals from different backgrounds, so it was not hard for us to follow-up with the content. However, the assignments implemented across the courses asked us to provide examples based on the NGO that we were working at; like for instance in the Finance course, we had to provide examples of budgeting numbers, and I, however, was not subordinated to any NGO, so it was hard for me to complete the assignment. Not all of us are affiliated with an NGO and some of us were taking the course just to be able to work in an NGO sometime in the future.
Few learners provided negative feedback for the time allocated for course delivery. Several learners reported that the videos were quite long and could have been more direct and concise while others required additional time to cover the material adequately.
A.W.4: There were times when information was discussed through very long videos which could have been shorter and more direct; especially that exposure to the videos for a prolonged time causes boredom.
K.Q.1: Yeah, the content as a whole of 8 courses is sufficient and I think it is very useful, but shortening some videos, shortening some materials; I kind of felt some of the facilitators they put all the materials they came across that is related to the subject that is not necessary need to be there.
M.Z.7: Like I would say, some of the courses were covered very fast for the timing that was assigned to them. Some of the courses were a bit too long or their material was too huge for the time allocated to cover them. Some of the courses needed additional time.
One learner indicated that presenting the course content over a longer duration would have given them more time to manage and plan for the course. This same learner also suggested for the diploma to be delivered by expert professionals who can teach the course in the Arabic language.
K.Q.1: The content of the course was not appearing in the platform on say enough time for the student to plan; it is like my studies tomorrow and the course content could have appeared today which may be ruining my other plans especially it is online course so usually most of the people are working either full-time or part-time. So, these are to say maybe weaknesses.
K.Q.1: Some of the interviews could have been made with more professionals in their sectors and even some native speakers; yes, we are all not native here, but still for an academic study you might seek for native speakers so that it is more beneficial.
Several concerns were reported regarding the assignments and quizzes. One learner noted that towards the end of the diploma, less focus and importance was directed towards assignments, quizzes, and grades. Another concern highlighted the lack of feedback by instructors on assignments and quizzes.
A.F.5: By the end of the courses, we lost interest; the exam modality changed, and less importance was attributed to the exams. Also, less focus was ascribed to grades.
K.H.8: When the instructors used to administer quizzes for us, we would send them back the quizzes. However, we would not know if our answers were right or wrong. We didn’t receive any reviews. We would study but by the end of the day not know if our responses were correct or wrong. So how would I learn? Shall I learn only from reading the course?
Some learners were also concerned with the change in format of assignments and quizzes from written responses to multiple choice questions. A preference for written rather than multiple choice questions was noted as they allowed learners to solidify their newly acquired knowledge.
S.K.2: Like I already told you, the assignments could have been mixed and not stuck to one specified mode. It would have been better to include a mixture of multiple-choice questions and written response questions. During the first few days of the diploma, we started off with written response questions, but by the end it totally switched to multiple choice questions. It would have been better to include a mixture of the two, and not start in a way and end it in another way. When the courses required written response questions, I used to feel that I have work to do, and it would feel different than when being enrolled in other online courses like Coursera.
N.B.6: I think one of the courses that I enjoyed the most was the negotiation in the hostage prices course mainly because the assignments weren’t multiple choice questions. The teacher actually asked us to write a paragraph and try to explain something according to a given format or framework. So, I believe these types of assignments are way better than multiple choice questions. I think there could have been a better way to evaluate the student rather than just using multiple choice questions.
One learner noted inconsistencies between the course content and some assignment questions indicating that the knowledge assessment and evaluation quizzes were not representative of the course lectures and material. They recommended improving the course evaluation and knowledge assessment measures. Another learner mentioned that quiz questions were incoherent, and objectives were not clearly communicated.
N.B.6: The quizzes in my opinion were not really representative of the lectures given that there was a lot of knowledge in the courses, and there were like 10 questions among which 6 were representative of the content, but the other 4 questions were not representative over the content. So, I would say maybe if you try to improve the evaluation of the students for the course that would probably be the biggest thing to improve on, and I perceived this to be the biggest weakness.
K.H.8: I would read and understand the course, but when I reach the questions, I honestly don’t understand what is needed from the question, and what the objective of the question is. I didn’t honestly understand what was required from the question. I couldn’t detect any relationship between the questions and what I learned. I used to feel that the questions were vague.
One learner additionally reported that the administration of quizzes at the end of the course, rather than during, hindered adequate self-evaluation. The same learner provided negative feedback on the length of the quizzes and assignment.
K.H.8: The instructors were administering tests at the end of each module, so the information was accumulating by the end, and I was not able to be assured throughout the courses whether I really understood the material or not. Another thing that I disliked about the course is the big number of the questions; there were courses when the number of the questions reached to 40-50, so it was really hectic.