Reach
A total of 11 schools with 1170 students participated in the study. Of those students, 605 (i.e., those in the experimental group) filled out the process evaluation questionnaire. Of these, 66.2% was female and 50.8% were enrolled in a study in the area of health care and well-being. They were on average 18.05 years old. Thirty teachers kept track of a logbook during and after the implementation of the program, leading to a total of 72 logbooks being returned. Ten teachers were interviewed after completing the implementation of the program. The interviewed teachers were from eight different schools from different geographical regions. Three were male, seven were female, and their age ranged from 23 to 62. Their teaching experience varied from 1 year to 30 years. As not all teachers made use of the right link (i.e. to allow tracing back to class-level), results on reach were only derived from returned forms and interviews and not based on Google Analytics.
Dose delivered
Logbooks from teachers showed that the video fragments and group discussions were delivered by 98.6% of them (Table 2). Deliverance of the second lesson was a little lower, with 87.7% of the teachers letting their students work with the quizzes on the website. Finally, 55.1% let the students practice with a paper-based registration form in lesson 2b, while 39.6% used the digital form, and 88.5% provided the students with information about the registration form. In total, 96.2% of the teachers delivered at least one element of lesson 2b. Teachers were divided on whether paper-based or digital materials work better. One teacher commented that everything an IVE-student does, is online nowadays. On the other hand, teachers mentioned that students benefit from physical materials as well:
“And in the end I printed the registration form, because that often helps, that they then have something tangible”- Teacher, 38, female
Dose received
Active participation
Teachers were divided about the active participation of their students. In some classes, students were very enthusiastic, had vivid discussions and asked a lot of questions:
“They were all very interested and then of course there is a group that says ‘We are not allowed to donate because of our religion’. But then they really discuss it respectfully and that leads to very nice conversations (…) So you have really nice questions and discussions.” – Teacher, 45, female
“The atmosphere in the class was very quiet and attentive” – Student, 16, female, level 2
In other classes, it was harder to motivate students, as they were not convinced of the relevance of (education about) the topic or simply not interested. Some teachers suggest that this depends on the study discipline. Students enrolled in a health care program were more interested in the topic than some other classes:
“Well, what should be noted; these are technical guys, so they are – they just want to know how something works, that’s it. They are not going to philosophize for a long time about feelings and things like that. I can imagine that this is different in other groups” – Teacher, 52, female
Some teachers said personal relevance was very important to motivate their students as well. They suggested that this could be embedded more in the program, but also personal stories from teachers or students led to more active participation.
“And I have to say, I have a friend, for example, who has a new kidney, so I also told them some of my own stories and they listened carefully to that. So that then makes it easier for me, and well, the involvements of students as well I think, when you have your own stories” – Teacher, 38, female
Appreciation of the program (components)
In the interviews, teachers were in general enthusiastic about the program. They saw the relevance of teaching about organ donation, especially for the age group (around 18 years old) they work with. Moreover, the program was easy to use for them as teachers, but also easy to understand for the students. Further, they liked the variety of components and the fact that the program was ready-to-use. Students rated the program as a whole with a 7 out of 10, which is considered a very adequate mark in the Netherlands. They also appreciated the variety of components:
“I liked that there was a lot of variety and that they asked about your opinion a lot and that it makes you think about it” – Student, 16, female, level 4
Lesson 1: Teachers were most positive about lesson 1, scoring it a 7.5 out of 10. Also students were most enthusiastic about this lesson (Table 3). Teachers said the video fragments were appealing and they liked that they involved different perspectives on organ donation (patient, surviving relative, etc). Using videos works well with their students and the actors and stories were easy to relate to:
“The videos worked well. I remember them leaving an impression. Especially one, I remember, that they, I believe it was an interview at a cemetery, or a grave, that someone said “Well, this is a nice way to start a lesson”. But then they were like: “But why? Why is this on the screen?” So I immediately had their attention. And it made use of three people with different perspectives. (…) That was just very good and worked fine” – Teacher, 38, female
Students were in general positive about the videos and liked the discussions followed by these.
“I liked that different people were interviewed, to make us hear different stories” – Student, 17, male, level 4
However, they also commented on the quality of the videos, the presenter and editing of the videos:
“I found repeating the text with white words on the screen a little childish” – Student, 21, male, level 4
Lesson 2a: The second lesson (2a) was rated with a 6.3 by teachers and received critical comments during the interviews. Students were moderately positive about this component (Table 3). The main problem that teachers described was that students did not read the questions or answers properly and therefore finished within a few minutes. Moreover, some teachers found it difficult that they had no idea what their students were doing, which made it hard for them to follow up on it or answer questions. Finally, they believed the entire second lesson was too theoretical and could benefit from some more interaction.
“Only the quiz didn’t reach its full potential, I wrote that down a couple of times. (…) I have no idea what they do with the questions. So you can’t really follow up on that. And for them it sometimes was unclear what the results actually meant. And they passed through it quickly. It took them less than 10 minutes per quiz.” – Teacher, 23, female
Lesson 2b: Lesson 2b was rated with a 6.9 by the teachers, and least appreciated by the students (Table 3). Most teachers did see the added value of paying attention to the registration form, but found it hard to motivate the students. They mentioned that students sometimes felt that they already knew how it worked or did not understand why they had to actually fill it out, as it was not for real. Further, as it was the last component, at the end of a lesson, students were not able to maintain their focus. A few teachers shared examples of things that went wrong when filling out the form, emphasizing the importance of the lesson:
“There was even a boy who said he filled out the form on paper at home and that his father did a final check and that he then appeared to have made a mistake. So that was very nice. (…) Because the question ‘which organs do you want to donate or not’, well, he mixed those up” – Teacher, 52, female
“We have that registration form – for level 2 students, that’s a really good exercise, as they don’t even know their social security number of even their zip code so to speak, really, sometimes they are so ignorant.” – Teacher, 26, female
Fidelity
Compliance to manual
All teachers that were interviews expressed being happy with the manual. It gave them clear guidance, some knowledge about the topic and some said it made them more confident. Most said they took the time beforehand to read it thoroughly. Moreover, the frequently asked questions that were added in the manual were appreciated.
“I found the guidelines, the manual that was added, very nice. Also with all information, a few of the most frequently asked questions. A few times I thought, ‘oh I have some additional questions’, but then I didn’t e-mail the contact persons that were in there, but I could have. But no, that was very nice. Also the extra bits of information. Yes, that gave sufficient guidance.”- Teacher, 29, female
Even though all teachers made use of the manual, not all components were implemented as intended. Teachers made on-the-spot adaptations to make it fit their group or schedule better. Some teachers fit all information in 1 lesson and some spread it out over 3 lessons. Especially differences in the level of education of the students seemed to be an important reason to make adaptations. For the higher educated classes within IVE, teachers sometimes added extra components themselves (e.g. a more elaborate discussion based on statements they found online or asking students to write a summary of what they learned), while for the lower educated classes, they made the lessons shorter (e.g. by watching less videos or only explaining the registration form instead of making them practice with it).
“Yes, it’s too many, 4 [videos] is really too many. The attention span of IVE-students is not that big, so at some point, when you also want to do those discussions afterwards, (…) so I intentionally shortened these because otherwise I would’ve lost them at the third video already. So the first 3 are great, the one with the doctor as well, but that one is just, it also takes like 5 minutes or so, that one is too long. At least, for my classes, they are too long” – Teacher, 29, female
“The lessons could have been shorter, because at some point I didn’t have any concentration left, while the topic was very interesting”- Student, 19, female, level 3
Completeness of deliverance of components
Also from the information from the logbooks and Google Analytics it became apparent that not all components were delivered in a complete manner (Table 4). According to the logbooks, teachers spent 50 minutes on the first lesson, which was exactly as intended. However, Google Analytics showed that the average time spent on the page with the video fragments was slightly under 20 minutes (Table 5). This might mean that teachers spent more time on introducing the topic and the group discussion and thus chose not to show all videos. This also was confirmed by some teachers in the interviews:
“I did watch a few of the videos, but not all of them. So I gave a few as an example, and then of course especially the ones where youngsters were involved, because that fits their experience world.”- Teacher 37, male
Teachers said lesson 2a took 24 minutes on average, but the Google Analytics data showed that the time actually spent on the quizzes was much lower. The first quiz took students 2:19 minutes to complete, while the second quiz only took them 1:05 minutes to finish. This suggests that students did not read the questions properly, did not think about their answers and/or did not read the tailored feedback. Teachers mentioned this in the interviews as well:
“Then I think they didn’t really read what the outcome was, only the score they got and that’s it. Students don’t read any more these days, it’s really useless” – Teacher, 28, female
Teachers indicated they spent on average 17 minutes on lesson 2b. As this lesson is not necessarily done on the web page, it is difficult to confirm this. The way in which lesson 2b was delivered differs between teachers, as some used the paper-based version, others the digital version and others only showed it to them, without making them do the exercise.
The number of page views gives an indication of whether components were delivered in a plenary or individual session (Table 5). It seems like most teachers delivered the video fragments and registration form in a plenary session, while the quizzes were done individually.
Context
Compatibility with existing curriculum
Teachers agree that the program fits very well within the course Citizenship and then especially within the Vitality dimension, but also other options were suggested, such as linking it to their age (what changes in your life when you turn 18?) or the political dimension (as in 2018 a law change on organ donation was accepted). Many teachers mentioned that Citizenship does not have a lot of requirements, which can both be a positive and negative thing. It means that there is quite some room for topics of choice, but on the other hand, there are so many topics to choose from, that organ donation is not always a priority:
“At the moment we have – here I go again with my schedule – but we limited ourselves with the dimension of Vitality, because of time constraints. So we discuss alcohol, and that is – well, I teach technical boys who on top of that want to become car mechanics, so that is an important topic. (...) And another topic we chose is STDs and, well, I always try to find some time for a healthy diet, but then often my time is up”- Teacher, 52, female
Some teachers found it very easy to find time for two lessons, while others mentioned that they have very limited time for Citizenship in general and therefore two hours was too much. This probably highly depends on the school and curriculum. Most teachers said that if they know about the program in advance, it is not too difficult to find time for it.
At the moment, there is no program or information about organ donation in the regular Citizenship methods, but some teachers already spent time on it in previous years, as they find it an important topic. In those cases they usually shared a personal story with the students or invited a guest speaker.
“It’s not a topic that is usually part of Citizenship so to say, it’s not part of the requirements within the domain, but yes, I always discuss it. So I think it’s definitely useful to do something with that” – Teacher, 45, female
Teachers’ capacity to implement the program
Teachers generally see themselves as being capable to implement the program. They said the program was easy to use and the instructions were clear. However, as organ donation is not their expertise, they also encountered some challenges. Teachers sometimes got questions from students they did not know the answer to. They handled these questions very differently:
“It [eligibility for donation] depends on age and some other factors. And then they were quite interested, like, where can I find that? I said, well I think you should just Google it. I mean, you can just find that on the Internet” – Teacher, 62, male
“And I just said in the beginning, like “Guys, I did go through a couple of frequently asked questions beforehand, but I also don’t know everything, so maybe I’ll look something up” – Teacher, 52, female
Teachers were also aware that organ donation is a sensitive topic to discuss. Most teachers did not necessarily see this as a problem, but all dealt with it in their own way. Some teachers found it difficult to get a feeling of how far they could push and talk about taboos. Others said that the sensitivity of the topic led to very open discussions:
“Well, I know that one girl knows she needs a donor lung to prevent her from dying young. But we’ve talked about that in the past as well. So, it is tough, but doable. I am at least - but that also depends on the teacher I think, how he/she deals with that - I am just very relaxed and open about it and I’m also very young. So I’m very close to them. So that makes a difference. So you will get those questions or comments any way, but if you open up and are vulnerable, students will do the same” – Teacher, 28, female
Support by the schools’ board or colleagues
As mentioned before, Citizenship is a course in which teachers have a lot of freedom to teach about topics they find important. Implementing it on a school level might therefore be challenging. However, some teachers commented that they think it would be of added value to implement the program on a school level.
“Yes, well actually, I don’t know how much power you have (…) But I think that if you want it to be very powerful, it is utopian what I’m going to say now, but that it will become an obligatory component of the curriculum. (…) So the government needs to invest in that, in education, like hey, there’s one period in their entire school career, that when they’re 18, we are going to spend time on that.” – Teacher, 37, male
No other teachers mentioned involvement of the school board, but sometimes discussed it among colleagues or in Citizenship meetings.
Reasons for fidelity and dose
Facilitators to implementation
If both students and teachers saw the relevance of learning/teaching about organ donation, this facilitated the implementation. There were several ways in which the program became more relevant. For instance, one teacher chose to deliver the program around Christmas and another teacher suggested that it might be good to link it to the yearly national Donor week. Further, around the time of implementation, an upcoming new donor law got a lot of media attention:
“For me, it was a godsend at that time. Because it was very topical then. So I thought like, how am I going to address that in my Citizenship classes? Because, coincidentally, we were also just dealing with Vital Citizenship, so yes, that fits fine of course. (…) So in that sense: timing was perfect” – Teacher, 38, female
Another factor that helped in making it relevant for the students is their age. During the year after they turn 18, they receive a letter in which they are asked to make a choice regarding organ donation.
“Because it’s of course also the age that fits this very well. So there are 16, 17, 18 year-olds in the first year, that’s where I implemented it, and yes, for 16-year-olds it might not concern them yet, but then there’s also 18-year-olds there, so you can ask “who is already 18 here? Do you remember getting that letter?” And to a 16 or 17-year-old you could say: “Well, now you know what to expect when you turn 18”. So that’s a nice preparation, fits their age as well. So also without topicality, you could make it relevant to address” – Teacher, 38, female
Other facilitating factors for implementation were that the program was easy to use for both students and teachers and that effects were observable. They mentioned that they had interesting discussions about it, students clearly had more knowledge after, that they sometimes said they talked about home or even registered:
“Yes, some of them changed their opinions. In the beginning they often said “oh I don’t want that”, and now they started thinking “Oh well, maybe I do”. Like: “I don’t know yet”, but it’s not a harsh no at least. So that’s really nice to see, that they start thinking critically. (…) They at least became aware of the fact that they have a choice. And that they have to think about it themselves, if you want to influence the outcome. That’s one thing. And secondly, that they started thinking critically. So they might not have an answer yet, but are dealing with it” – Teacher, 38, female
“I learned a lot and now I know more about donor registration and what happens after that with your organs and body” – Student, 17, female, level 4
Barriers to implementation
One of the most important barriers to implementation, as mentioned before, was the fact that IVE-students do not read materials that well. Some teachers said that they just forget and immediately want to do things instead of reading about it. Others are just discouraged when seeing a lot of text. Sometimes this is because of more serious problems such as low-literacy:
“In level 2, and especially in level 1 but we don’t even teach them, there are many people with low-literacy” – Teacher, 52, female
Teachers therefore recommend to stick to the content of the quizzes, but then delivering it in a more interactive way. This could also help overcoming another barrier, which is the low attention span of students in IVE (see: Fidelity: compliance to manual).
Two practical issues were mentioned by teachers, that made the implementation more difficult. One teacher mentioned she had a large group, which made it hard to involve everyone, while another teacher mentioned that she struggled with absenteeism of students during the first lesson. These are issues that cannot be prevented, but suggestions on how to deal with this could be added to the manual.
Finally, different cultural or religious backgrounds were mentioned as being either a facilitator or a barrier. One teacher said that having more students with a migration background made things a little bit more challenging, while others said that the mixed backgrounds of their students facilitated lively discussions:
“I have to say that I found it very interesting because our school is very mixed, so we have around, well I don’t know, maybe 40 to 50 different cultures here. So that leads to very nice discussions” – Teacher, 45, female
Future use
All teachers were positive towards future use, sometimes even actively asking questions about this in the end of the interviews themselves. They seem willing to make time available for this topic in the future. Some of them want to follow up in the same classes as well. There was also a specific request by some teachers to make a follow-up lesson to be able to repeat some things and elaborate in the same classes some time later.
“Look, this time I had a gap to fill, but I think that I’ll make time for this in the future” – Teacher, 29, female
They believed that especially colleagues teaching Citizenship would be interested in using the program as well. However, they also mentioned that you probably highly depend on the teacher that you encounter. Some teachers prefer to spend time on other topics, or might just not be enthusiastic about organ donation themselves.
“I think colleagues in Citizenship would [be interested in using the program], yes. However, it does depend on the teacher. Because I’m a little bit more involved in those things, that I find these things important and others have their focus on other topics. That’s a thing with this subject [Citizenship]. That it’s very dependent on the teacher” – Teacher, 23, female
One teacher (with a background in health care) was afraid that the program might be difficult to teach for people without a background in health care, but other teachers did not conform that. One other teacher recommended to send reminder e-mails, as teachers tend to run from pillar to post.
Recommendations
Both teachers and students gave recommendations for further improvement of the program. Teachers mainly emphasized the need of different versions or options for the different educational levels within IVE. They argued that level 2 classes cannot be compared to level 4 classes in terms of active participation, attention span, reading and knowledge.
“I personally solemnly believe in different programs, with the same subjects and the same goals for both groups. Because for level 2, these kind of topics don’t concern them yet, unless they have personal experiences with it” – Teacher, 52, female
In practice, this could mean that students in level 2 classes need some more (inter)active methods and something that makes it personally relevant for them. For level 4 classes this could mean that there are optional assignments or more in depth information for those who are interested. Teachers commented that the program was not challenging enough for the higher achieving classes or students:
“Maybe it would be nice – because I talked about that with colleagues recently, we are working with differentiation in our classes, that especially those lessons with the quizzes and information that many students were like “I already knew that”, that you might be able to add a bonus component or so. (…) Many students said at the beginning of the program “Yes, I have an opinion and I can support that because it is what it is”. So it would be a challenge to pose some questions about that. That would be nice” – Teacher, 29, female
This also became clear from the comments that students added to their evaluation form. For some students the program was elaborate enough (or even too long). On the other hand, many students commented that they would have liked even more information:
“I’d like more information about the process of organ donation, how it actually works” – Student, 21, female, level 4
Students also commented that the program was very serious and sometimes even boring.
“Maybe it can be delivered in a more playful way. It was very formal and serious” – Student, 24, female, level 3
Teachers gave some suggestions to make the program more attractive for students. One teacher suggested involving a celebrity in the videos, who spoke about organ donation on a television show earlier.
Finally, some students found the program too much pro-donation. They liked having different perspectives on the topic, but suggested that the perspective of someone who is against it or decided not to become a donor could be added.
“The videos and registration form make you feel bad if you don’t want to be an organ donor” – Student, 16, female, level 4
“The only comment some students had was that they felt it was a little bit forced on them. That’s what one or two students said. But I said “Any way you slice it, they ask you to think about it and about the choice you’ll make” – Teacher, 37, male