Qualitative evaluation of recruiting materials and online survey
Study population. Fourteen participants took part in three focus groups (Table 2), two participants dropped-out due to personal reasons that were not related with the study. About equal numbers of women and men participated. Almost all of them were employed and had a university degree. One participant did not speak German as her native language.
Table 2 Sociodemographic characteristics of focus group participants
Baseline characteristics
|
0n
|
00%
|
N
|
14
|
100
|
Gender
|
|
|
Female
|
08
|
057
|
Male
|
06
|
043
|
Diverse
|
00
|
000
|
Year of birth
|
|
|
1995 - 2003
|
00
|
000
|
1996 - 1980
|
10
|
071
|
1981 - 1960
|
00
|
000
|
1961 and older
|
04
|
029
|
Employment
|
|
|
Employed
|
10
|
072
|
In education
|
02
|
014
|
Unemployed
|
02
|
014
|
Retired
|
00
|
000
|
Other
|
00
|
000
|
Not specified
|
00
|
000
|
Education
|
|
|
No degree
|
00
|
000
|
Lower certificate
|
00
|
000
|
Intermediate certificate
|
01
|
007
|
Complete apprenticeship
|
01
|
007
|
High school diploma
|
02
|
014
|
University degree
|
10
|
072
|
Not specified
|
00
|
000
|
Note. Focus group participants were divided into three groups.
Evaluation of recruiting materials. A leaflet in size of the vaccination certificate was rated most positively and practical as compared to posters and flyers. Fig. 1 shows two (out of five) different leaflets that were demonstrated to the participants and the final adjusted leaflet for the primary care practices.
In general, headlines were misleading because we only referred to COVID-19 vaccination but not the control vaccinations. Participants wanted headings to be more reader friendly, e.g. larger font. They were confused by the study acronym, CoVaKo, and preferred not to mention it in the title. Some participants also preferred the masculine form of language compared to gender-neutral language that uses the asterisk (e.g. Teilnehmer instead of Teilnehmer*innen; eng. participants). They also favoured the wording vaccine side effects instead of vaccine reaction or vaccine damage since it is most commonly known and/or perceived as more objective.
Study information should be listed as bullet points one below the other as opposed to alternating between left and right columns. Some participants preferred one image over two images on a page. Participants appreciated the information on data protection, but found it too long. Some would want to know what will happen to the results and how data will be analysed. The options to register and access more information on the study via QR-Code or URL were evaluated practical by most participants. However, older participants stated they would not know how to use the QR-Code.
Final versions of the posters, leaflets and flyers were created based on the feedback. We opted for two different versions: One version for vaccination centres that pointed to COVID-19 vaccination only and one version for primary care practices that targeted a variety of vaccinations. We included most of the feedback but decided to still use gender-neutral language. COVID-19 is now written in lowercase and placed at the bottom of the list to avoid too much focus on that vaccination. Information on data protection is now shortened with reference to a more detailed version on our study website. The adjusted version was used for the study test-roll-out.
Evaluation of online survey. Parts of the study information and registration form proved to be incomprehensible. More specific information was requested regarding procedure and participating time, e.g. survey completion time in minutes. When asked about the vaccination received, they tended to report all the vaccinations received in the past (Fig. 2). The response options "Several/None" have been separated as this simplified the response decision. Originally, participants should only be recruited in the last two weeks after vaccination. As it later proved more practical to include participants at a later time as well, the time reference in the question about vaccination was adjusted accordingly.
Some buttons of the survey were in English (e.g. next page, submit) due to the software we used; here the participants expressed the wish that they were in German. When filling out the survey, it became apparent that not all termination criteria were correctly stored; no other technical problems occurred. We incorporated the majority of the feedback. We were not yet able to implement the German system language.
Also, before the test-roll-out we adapted the study procedure strategy. Initially, participants should register within 14 days after their first dose in order to collect data at all survey times. In the test-roll-out participants could register up to 124 days after their first or single dose, in order to receive at least one follow-up survey and to not lose too much information due to the vaccination pace.
Quantitative evaluation of the survey and feasibility
Study population. In 31 days, 2199 participants registered and received either a COVID-19 vaccination (99 %) or at least one of the control vaccinations (1 %) in the last 124 days. Few participants were born before 1951 (7 %) and 10.1 % were from medium sized towns. The distribution between rural areas, small towns and cities was balanced. The majority was employed (69 %) and had a university degree (48 %).
Most participants received BNT162b2 (BioNTech/Pfizer) or ChadOx1 (AstraZeneca) as first COVID-19 vaccination (60 %, 28 %). For the second COVID-19 vaccination, BNT162b2 (BioNTech/Pfizer) and mRNA-1273 (Moderna) were administered most (68 %, 21 %). In five cases, the second dose was cancelled. Named reasons were pregnancy, SARS-CoV-2-infection after first dose, intolerance of first dose or illness at date of second dose. Information on the second dose is missing in 446 cases due to missing information from the follow-up surveys. The mean body mass index was 26.2 (SD = 5.8). Morbidity assessed with the mSCQ-D was in mean 1.6 (SD = 2.4) (Table 3).
Table 3 Sociodemographic characteristics, vaccination types, comorbidities and completion rate for each survey
Baseline characteristics
|
n
|
%
|
N
|
2199
|
100
|
Gender
|
|
|
Female
|
1233
|
56
|
Male
|
0965
|
43
|
Diverse
|
0001
|
01
|
|
|
|
Year of birth
|
|
|
1991-2003 (approx. 18-30 years)
|
0419
|
19
|
1971-1990 (approx. 31-50 years)
|
0772
|
35
|
1951-1970 (approx. 51-70 years)
|
0859
|
39
|
1933-1950 (approx. 70-88 years)
|
0149
|
07
|
|
|
|
Residence
|
|
|
Rural area1
|
0618
|
28
|
Small town2
|
0642
|
29
|
Medium-sized town3
|
0223
|
10
|
City4
|
0716
|
33
|
|
|
|
Employment
|
|
|
Employed
|
1525
|
69
|
In education
|
0228
|
10
|
Unemployed
|
0050
|
02
|
Retired
|
0336
|
15
|
Other
|
0040
|
02
|
Not specified
|
0020
|
01
|
|
|
|
Education
|
|
|
No degree
|
0002
|
01
|
Lower certificate
|
0055
|
02
|
Intermediate certificate
|
0273
|
12
|
Complete apprenticeship
|
0321
|
15
|
High school diploma
|
0459
|
21
|
University degree
|
1047
|
48
|
Not specified
|
0042
|
02
|
|
|
|
Vaccination
|
|
|
Control group
|
0024
|
01
|
Covid-19
|
2175
|
99
|
First vaccination COVID (n = 2175)
|
|
|
BNT162b2 (BioNTech/Pfizer)
|
1313
|
60
|
mRNA-1273 (Moderna)
|
0253
|
12
|
ChadOx1 (AstraZeneca)
|
0609
|
28
|
Ad26.COV2.S (Johnson & Johnson)
|
0000
|
00
|
|
|
|
Second vaccination COVID (n = 1753)
|
|
|
BNT162b2 (BioNTech/Pfizer)
|
1200
|
68
|
mRNA-1273 (Moderna)
|
0375
|
21
|
ChadOx1 (AstraZeneca)
|
0141
|
08
|
Ad26.COV2.S (Johnson & Johnson)
|
0000
|
00
|
Cancelled
|
0005
|
01
|
Not planned
|
0032
|
02
|
NA
|
0422
|
|
|
|
|
Health status
|
|
|
No Pre-existing diseases
|
0832
|
38
|
Allergies
|
0598
|
27
|
Hypertension
|
0408
|
19
|
Backpain
|
0331
|
15
|
Lung Disease
|
0187
|
9
|
Rheumatism/Autoimmune disease
|
0172
|
8
|
Depression
|
0153
|
7
|
Osteoarthritis
|
0142
|
6
|
Gastrointestinal disease
|
0135
|
6
|
Heart disease
|
0117
|
5
|
Diabetes
|
0094
|
4
|
Cancer
|
0072
|
3
|
Coagulation Problems
|
0044
|
2
|
Kidney Disease
|
0040
|
2
|
Liver Disease
|
0034
|
2
|
Anemia
|
0028
|
1
|
|
Mean
|
SD
|
mSCQ-D (n = 2199)5
|
0001.6
|
02.4
|
BMI (n = 2176)6
|
0026.2
|
05.8
|
Note. 1 Rural area = < 5.000. 2 Small town = 5.000 to approx. 20.000. 3 Medium-sized town = 20.000 to approx. 100.000. 4 City = 100.000 or more inhabitants. 5 mSCQ-D = Modified German version of the Self-Administered Comorbidity Questionnaire. 6 BMI = Body Mass Index.
Most participants (58 %) registered in the week of their first dose or single dose, respectively. In the sixth week after first/single dose registered nine percent and in the ninth week twenty percent.
The estimated recruitment rate in the test-roll-out was 7 % (2199 participants with 303050 distributed leaflets). Overall response rate of the surveys was very high with 84 %.
Evaluation. About 92 % of 1993 participants became aware of the study at a vaccination centre. The others heard about it at a physician’s practice, their friends/family or on the internet (3 %, 3 % and 1 %, respectively). About 10 % did not fill out an evaluation form.
Most participants (strongly) agreed that the registration process was well structured, easy to understand that all relevant information was asked and all questions were important. Regarding the study process, most participants (strongly) agreed that the registration process was easy, the completion time was reasonable and the technical framework was manageable (Fig. 3).
Note. If a participant filled out several evaluation forms, first answer was considered. Counts of observations in the bars are reported if n was > 50.
Optional feedback in the comment section. Suggestions in terms of organisation, technical implementation and content were made. After the registration, participants expected to receive an invitation link or a confirmation email immediately or shortly after their vaccination. For confidentiality and technical reasons, we did not send confirmation emails, but an information was added during the feasibility study at the end of the registration. The Redcap system language being English resulted in some participants getting an automatic and poor German translation of the survey in certain Internet browsers, although the survey was developed in German. A software update after the test-roll-out could solve this problem.
Concerning the content, some participants wished for a more differentiated list of symptoms and to specify the duration and onset of their symptoms. The item asking for personal perception of the link between vaccination and the complaints was difficult for some. Few participants suggested to add the answer option "I don't know".
As some participants shared an email address or registered twice, we implemented a disclaimer at the beginning of each survey with their personal information on type and date of vaccine, year of birth and gender. With this information participants could also give us feedback if their entered information was incorrect and/or could register again.
Qualitative Evaluation of the feasibility
Study population. Eleven primary care practices and two large-scale vaccination centres, were recruited to distribute leaflets after vaccinations. In total three interviews with GPs and two interviews with vaccination centres staff were performed.
Evaluation. A total of 1600 leaflets were sent out to primary care practices and 28750 leaflets to the vaccination centres. The leaflets were distributed at different times, e.g. directly after the vaccination by physicians or in the waiting area. Participating partners said that the study is easy to integrate into the regular workflow. All preferred the small leaflets over flyers and posters. Posters were not desired due to lack of space, especially from GPs. One GP reported a greater interest when leaflets were distributed by physicians instead of medical assistants. Physicians reported mostly positive reactions from vaccinated persons. Some reported little additional work due to questions from patients, but overall the questions did not intervene with their tight schedule. According to participants, almost all leaflets were distributed. Physicians suggested to recruit more GPs through mailing lists and by offering compensation.
Nine GP practices and both vaccination centres said they were willing to participate in the main project.