Overview of results
Participants spent between 4 and 15 minutes, with an average of 8 minutes, using the app to inform themselves about the study and give their consent. With the exception of one participant, who encountered a technical problem in the identification procedure, all could successfully follow the steps to give consent for the study. Nearly all participants mentioned they found the app well-designed and for most parts user friendly. Some participants pointed out that two pages where not user friendly due to buttons on unexpected places and entry fields that were not displayed properly. The look and feel of the app, i.e. including page layout and use of colour, was said to be clear and appealing.
Themes
After repeated reading of the interviews, the following themes were considered to be the most important by the participants: (1) time required to give consent; (2) opinions on the app’s content and preferences for types of information sources; (3) opinions on the identification procedure. A fourth theme (4) on participants’ ability to recall study information; was included to better understand how the app performed as an informative tool.
1. Time required to give consent
Participants collected information in various ways to gain knowledge about the Sarphati Cohort. The majority of participants used the animated video as their sole source of information. Others, in addition to the film, read some of the background information text items. Two participants exclusively used texts to inform themselves about the study. Six participants stopped halfway through the video, some of whom, before continuing, took a brief look at the supplementary text items. They commented that a 3-minute video was too long and that it should either be shortened or divided into smaller parts. They felt that some of the information provided, in particular on data protection, was too detailed and suggested making those parts optional. Some argued their rationale for skipping information was that if they had made the effort to download the app, it likely would have meant they were already aware of the study and would have the intention to participate.
I stopped the video. It's nice that the app doesn't contain too much information. The information is easy to read. A video of 3 minutes is too long. You decide to participate beforehand. So what is the purpose of the video? I'd say make it shorter. I already make the decision before I download the app (Participant One).
Normally I would skip such a video, especially if it takes 3 minutes. I would directly click on “go to the login screen” … (Interviewer: How long should the video be?) Well, about one minute. I do understand that all that information does not fit in a minute. There is a whole section about privacy which is very good. But I think, if I want to know exactly how that is organized, I'll look it up. I just assume that that's okay (Participant Eleven).
However, others felt that the video had a suitable length. One participant mentioned that the video could be extended to include more information, which was currently only available in the supplementary texts.
Fine, not too long. Nice voice-over. The video is short enough. I would stop it if it would take any longer, 10 minutes would be far too long. For such a study I would watch the complete 3 minutes, this is important after all (Participant Eighteen).
The video was a bit short, it could have been a little longer. There could have been more in it. I miss a number of things that are part of the text but are not so clear in the video. I think the video should be 4 or 5 minutes (Participant Twenty-nine).
2. Opinions on the app’s content and preferences for types of information sources
Participants expressed their satisfaction with the fact that they were given the choice to receive the information in a video or in writing. The participants who watched the animated video said the information was conveyed in a clear and understandable manner. Some said that they remembered the information about the study better through the images in the film. The cartoonist style of the video was most often found to be attractive and fitting to the message. Participants in particular responded well to the narrator of the video, who, according to them, spoke clearly and explained the study at the right pace. A few participants felt that the visual effects in the video were at times too chaotic, which distracted them from the spoken information.
The video explains a lot, so I didn't feel the need to read much more... But I think it's good that you get information in both ways (Participant Six).
The video is clear. I didn't really look at the text much. I'm not very good at reading. The video works much better for me than reading a piece (Participant Fifteen).
The style of the video is nice. Children with all those different colours, really fits Amsterdam. It is also nice for children themselves later on: the researcher with the coat is recognizable, children who are sad and children who are happy… well done! (Participant Sixteen)
Nice clear voice. I do think the video is quite busy. You look here and there and then something pops up again: it was visually busy. I think if you have a clear voice trying to explain things then maybe it should go by topic…. Of course it's about one topic, but that you don't have that many of those pop up things (Participant Twenty-three).
The few participants who read the script version of the video noted that the text as a whole was too long and contained sentences that were too lengthy. They also mentioned that this text was a bit boring. More positive responses were given to the supplementary text blocks with detailed background information. These were seen as understandable texts and the length was considered more suitable.
On the script version of the video:
Yes, it was clear. … I was in doubt… the sentences were long, stretched long across the screen (Participant Seventeen).
There was also a lot of text. In the end I thought I should have watched the video instead... (On the supplementary text blocks) This is a bit nicer and shorter. Then you are also able to choose. And such a summary with four dots. Great, yes, I like that. That's what I would expect if I clicked text (Participant Eleven).
3. Opinions on the identification procedure
All but one of the participants were able to complete the 2 step identification procedure by a log-in with the provided usernames and passwords of the test cases and subsequently filling out the date of birth of these children. Due to a technical issue that delayed data retrieval in the identification process, several participants asked if something had gone wrong because they couldn't proceed right away. The first time this occurred the observers were unfamiliar with this issue and aborted the test. As for later cases, the participants said they missed a notification that data retrieval was still in progress.
In the following interview, participants gave mixed reactions to the identification procedure. Although all said to be familiar with DigiD as an identification tool for health insurance or tax return purposes, many mentioned they were surprised to see that they needed DigiD to participate in research. A few mentioned they had some hesitation in using their personal DigiD for such a purpose. In their view, this contradicted the fact that we use their children’s data anonymously for research. One participant said that the mandatory DigiD registration was a reason for him to refrain from participating. Conversely, there were also participants who saw the use of DigiD as something positive. For them, DigiD felt like a trusted, confidential and secure way to give consent.
The notion that it's all so private, that notion fades a bit. Things are well recorded after all... If you are going to log in with your DigiD, how private is the data? (Participant Five).
Yes, that is normal these days, isn't it? the data is quite personal. I would not just log in with some code. Better to use DigiD. DigiD is really confidential. Everyone needs a DigiD (Participant Six).
When inquiring whether participants would be able to log in with their own personal username and password, more than half of them answered that they knew this by heart. Others said they didn't know their username and password, but kept them at home.
In general, I always forget my password. I never remember it. I don't know how that works for other people. Coincidentally, I know my DigiD at the moment. But if I don't know it at this moment, it does not have my priority to look it up or request a new password. I don't know how that usually goes… (Participant Twenty-eight).
No, I do not know my password by heart. I have it at home, in a folder. I do not think people just know this by heart. Where do you use DigiD for? Only for important stuff. Things that you do at home (Participant Fifteen).
4. Participants’ ability to recall study information
Most participants said they found themselves sufficiently informed by the app to be able to make a choice about their participation. The participants who felt they were not well enough informed said they missed information on what types of data were used and for which specific research questions we would use this. When we asked them to tell us something about the study, nearly all participants remembered that the cohort is a study about the health of zero till 18-year-old children that grow up in Amsterdam. Some participants, however, had some misconceptions. For instance, one parent thought the study was conducted to give parents personal advice about the healthy upbringing of their child. When asked what participants should do now that they had given consent, most had understood that they would receive health questionnaires as part of the study. Yet, nearly a quarter of the participants did not recall this aspect. Considering the types of care data that are collected for the study, the majority of participants could correctly name them. Others gave both correct and incorrect examples of data. A few mentioned they had not seen this information or gave an incorrect answer based on a false assumption. About data collection, almost half of the participants remembered that the health data is first encrypted, after which it is securely stored to be used anonymously for research. A third of the participants could only name one aspect about the data collection. A couple of participants could not recall any details about the data collection.