Going web or staying paper? The use of web-surveys among older people

doi:10.21203/rs.3.rs-21136/v2

Download PDF

Research article

Going web or staying paper? The use of web-surveys among older people

https://doi.org/10.21203/rs.3.rs-21136/v2

This work is licensed under a CC BY 4.0 License

Journal Publication

published 08 Oct, 2020

Read the published version in BMC Medical Research Methodology →

You are reading this older preprint version

Read the latest preprint version →

Background: Many surveys today use a web-survey as an alternative to more traditional survey modes. Yet, web-surveys targeting older individuals are still uncommon for various reasons. However, with younger cohorts approaching older age, the potentials for web-surveys among older people might be improved. In this study, we investigated response patterns in a web-survey targeting older adults and the potential importance of offering a paper questionnaire as an alternative to the web questionnaire.

Methods: We analyzed data from three waves of a web-based retirement study, in which a paper questionnaire was offered as an alternative to the web questionnaire in the last reminder. We mapped the response patterns, compared web- and paper respondents and compared different key outcomes resulting from the sample with and without the paper respondents, both at baseline and after two follow-ups.

Results: Paper-respondents, that is, those that did not answer until they got a paper questionnaire with the last reminder, were more likely to be women, low educated, fully retired, non-married and reported higher depression and lower self-rated health, compared with web-respondents. The association between retirement status and depression was only present among web-respondents. The differences between web and paper respondents were stronger in the longitudinal sample (after two follow-ups) than at baseline.

Conclusions: We conclude that a web survey might be a feasible and good alternative in surveys targeting people in the retirement age range. However, without offering a paper questionnaire, a small but important group will likely be missing with potential biased estimates as the result.

Health Economics & Outcomes Research

Survey mode

older adults

web-survey

non-response

generalizability

retirement

sociodemographic differences.

Already 20 years ago, it was stated that web-surveys represent a double-edged sword for the survey industry (1), acknowledging the fact that the use of web-surveys may convey both benefits and limitations. For example, a web-based survey offer a cheap and convenient mode of data collection. They are fast, easy to conduct and allow respondents to complete the questionnaire whenever and wherever they prefer with the use of different platforms, such as computer or mobile devices. On the other hand, web-surveys require that people in the target group have access to internet as well as being willing to answer a web-survey (2, 3). In addition, response-rates have been found to be consistently lower in web-survey compared with other survey modes (4) and web-surveys have been found to have lower survey representativeness compared with other single mode surveys (5).

In this paper, we are interested in the potential use of web-surveys among older people. Our rationale is that internet use are more common in cohorts approaching old age today, than in older cohorts (6). So far, there is limited research on the use of web-based surveys targeting older people and its potentials.

Traditionally, survey data has been collected by interviews (face-to-face or telephone) or by paper questionnaires, usually sent by post. New technologies, such as internet and common access and use of computers and smart phones, provide us with new and efficient possibilities to collect survey data. During the last decades, there has been a dramatic increase in the use of web surveys, used either as an alternative or as a complement to a postal questionnaire or as a stand-alone tool, where a web questionnaire is the only alternative (2, 7-9).

At the same time as this new technology develops and makes it easier and cheaper to conduct a survey, survey research is suffering from decreasing response rates; a trend that developed for several decades (10,11). Much effort is usually required to reach an acceptable response rate and to achieve a study sample that is generalizable to the population it is supposed to represent (8, 12). As such, researchers face the challenges of choosing the most efficient survey strategy to the most reasonable cost.

The best choice of survey mode (interview, paper or web) depends on the purpose of the study, the target population, and financial resources. The main advantages with a web survey are the low cost and potential quick access to data (2, 3). The obvious drawback is that not all people have access to or are familiar with the use of internet. In general, young people are more frequent users than older people are. Hence, a web questionnaire might be more efficient than a paper questionnaire in younger age groups, due to a more widespread use of and access to internet, whilst it might be more challenging among older persons (1, 3, 7, 13).

However, the use of internet is increasing also among older people. According to Statistic Sweden, 94 percent of people 55-64 years old in Sweden reported in 2018 that they have access to Internet in their home. Corresponding numbers for people 65-74 and 75-85 years old are 86 and 68 percent, respectively. In the age group 55-64 years old, 85 percent report that they use Internet more or less every day. Among people 65-74 and 75-85 years old, corresponding numbers are 69 and 42 percent (6). Hence, it might be possible to use a web survey also among older people, at least among the young olds.

Internet use is also associated with several other sociodemographic factors, besides age. Less Internet use has been associated with lower education and female gender (9, 14), as well as lower level of resources, such as income, education and social contacts among older people (15). Research also found that non-use of Internet is associated with lower education, unemployment, disability, and social isolation and that these associations has become stronger over time. Accordingly, the group of people that not use Internet has become a more vulnerable group over time. In addition, today, non-users often report non-interest as the reason for the non-use, not lack of access to Internet (16). It is most likely that this group of non-users would be missing in a web-survey if no other mode options were offered.

Population-based research are currently shifting towards the use of more web-surveys. The expectation is that previous problems with under-coverage in web-surveys will decrease when Internet use is becoming more spread across all societal groups (17). Studies show that a web-survey is likely to generate similar response rates as a paper-survey among younger age groups (3, 13). However, few studies have so far investigated the potentials of using the web as a survey mode among older people. There are evidence supporting a mixed-mode approach (web and paper) in surveys of older people, although most older adults still seem to prefer a paper questionnaire (18). There is also mixed evidence whether web-respondents differ substantially from paper-respondents (19) or not (18).

With this paper, we aim to increase the knowledge about the potentials of using web-surveys among older people. We did this by investigating response patterns and outcomes in an already existing web-based longitudinal retirement study, in which a paper questionnaire was offered as an alternative to the web-questionnaire in the last reminder. Our specific research questions were a) if response patterns (web or paper) differed by sociodemographic factors, self-rated health and psychological outcomes, and b) if results from the study would have varied systematically depending on survey design (i.e., whether or not paper was offered as an alternative to web or not).

Data material

We used three waves of data from the Health, Ageing and Retirement Transitions in Sweden study (HEARTS), a survey based on a nationally representative sample of the Swedish population 60-66 years old at baseline 2015, with yearly follow-ups (20). The sample was generated as a probability sample from the National Register on the Total Population, covering all inhabitants registered in Sweden, by Statistic Sweden. Data is primarily collected using a web-questionnaire, but a paper-questionnaire is offered as a choice in the final reminder letter. The questionnaire contains questions largely focusing on various aspects of health and well-being and in relation to retirement, besides questions about sociodemographic factors. The time needed to respond to all questions varies considerably between individuals, but is typically in the range of 50-90 minutes, and no incentives is used.

At baseline, an invitation letter to the study was distributed by postal mail, including a link to the web-questionnaire. Two reminders were sent by post, in which the last reminder also included the full paper-questionnaire (Figure 1). For follow-ups, the first invitation, as well as the first reminder, was sent by email to those who previously give their email address. Thereafter, a postal invitation was sent to all people that did not responded to the email invitations, as well as two postal reminders in which the last reminder includes the full paper-questionnaire, in line with the data collection strategy at baseline. Finally, a last postal reminder, including a thank you for participating in previous waves, was sent. The present study is based on data from the three first waves of the HEARTS study.

Background Measures

Sex and age was registered at the sampling procedure. Education was measured by self-reported highest level of education (coded into; primary or below, secondary or tertiary education). We also used self-reported country of birth (Sweden or outside Sweden), self-reported marital status (married/partner, unmarried, divorced/separated, widow/widower), and self-reported retirement status (not retired, retired and working-consider myself a worker, retired and working-consider myself a retiree, fully retired).

Outcome Measures

To evaluate the effect of survey design, we chose three different key outcome measures; depression measured by CES-D scale, (21), life-satisfaction, measured by Diener’s 5 item scale (22), and self-rated health, measured by the question “How is your general health”. The answers “Very bad”, “Bad” or “Poor” was coded into poor self-rated health, and “Fair”, “Good” or “Very good” into not poor self-rated health.

Sample groups

To answer the research questions we created two sets of sample groups:

1a) Baseline web-sample (those who answered by web at baseline)

1b) Baseline paper-sample (those answering by paper at baseline)

2a) Longitudinal web-sample (people that answered by web in all three waves),

2b) Longitudinal paper-sample (those answering by paper in all three waves).

The differences between the first and the second set of sample groups is that the longitudinal sample groups (2a and 2b) are restricted to people who answered all three waves with the same survey mode. Hence, the longitudinal sample groups used in this study do not include people with any non-response, although non-respondents, except for the baseline non-responders, are invited to participate in subsequent waves. Neither is people who changed interview mode between the waves included in the longitudinal sub-samples. The main rationale behind the restricted longitudinal sample groups was that we were interested in how the group of people that is most likely to be included in a longitudinal web-survey, without a paper-alternative, differed from those that was positive to participate in the survey, but constantly preferred a paper questionnaire.

Analyses

First, we created a flow chart of the response patterns, to examine how people moved between web-response, paper-response, and non-response across the three waves. Second, we compared the sample groups 1a and 1b, as well as 2a and 2b, regarding background factors and the three specific outcome indicators (i.e. depression, life satisfaction, and self-rated health). In the analyses, we used the Chi2-test or t-test, depending on outcome measure.

Finally, we compared the association between retirement status and the three outcome measures in the different sample groups, separately and together, to analyze the effect of offering the paper questionnaire as an alternative. The underlying assumption is that the alternative to a paper response would have been a complete non-response. We restricted the sample to those who were either “not retired” or “fully retired”, and excluded those with less clear retirement status, who stated that they were “retired and working-consider myself a worker” (n=443) or “retired and working-consider myself a retiree” (n=260). The results for continuous outcomes are based on linear regressions and presented as unstandardized beta-coefficients (β). Results for binary outcomes were calculated by logistic regressions but presented as Average Marginal Effects (AMEs), due to the problem of comparing odds ratios over different models based on different groups (23). The AME gives the predicted absolute differences in proportion from the reference category, given the same value in all other variables included in the model. All models were calculated crude as well as adjusted for sex, age, education, and civil status. Analyses were performed using STATA 14.

Out of the total sample (n=14990), 39.4% (n=5913) answered the questionnaire at baseline, 27.1% (n=4067) by web and 12.3% (n=1845) by paper (Figure 2). Among this baseline response group, 78.7% also answered the 1^st follow-up and 73.1% the 2^nd follow-up. In total, 40 baseline responders died between baseline and 2^nd follow-up (17 before 1^st follow-up and 23 between 1^st and 2^nd follow-up).

The response patterns illustrated in Figure 2 reveal four main findings. First, a majority (42.4%) of the respondents answered by web at all three waves. Second, a majority of the web-respondents answered by web also in subsequent wave (74.9% and 80.4% for the1^st and 2^nd follow-up respectively). In contrast, people that did not respond until they got a paper questionnaire, i.e. paper-respondents, were less stable in their preference over waves, that is, they were more evenly distributed between web and paper response in subsequent wave. In total, 6.2% of the respondents answered by paper at all three waves. Third, paper-respondents were twice as likely to be non-respondent in the subsequent wave compared to web-respondents (30.9% vs 16.6%; p<0.001 between baseline and 1^st follow-up and 24.0% vs 12.3%; p<0.001 between 1^st and 2^nd follow-up). Fourth, 30.3% of the non-responders at 1^st follow-up did a re-entry into the study at the 2^nd follow-up.

Table 1. Sociodemographic differences (measured at baseline) between web and paper respondents.

Characteristics measured at baseline	HEARTS baseline web sample (n=4068)	HEARTS baseline paper sample (n=1845)	P-value	HEARTS longitudinal web sample (n=2510)	HEARTS longitudinal paper sample (n=369)	P-value
Characteristics measured at baseline	HEARTS baseline web sample (n=4068)	HEARTS baseline paper sample (n=1845)	P-value	HEARTS longitudinal web sample (n=2510)	HEARTS longitudinal paper sample (n=369)	P-value

Age
Mean age	63.1	63.2	0.056	63.1	63.5	<0.001
Sex
Women	51.4	59.2	<0.001	50.6	63.5	<0.001
Education
Primary or below	12.1	23.5		10.5	28.0
Secondary	32.7	37.2		31.9	40.4
Tertiary	55.3	39.4	<0.001	57.7	31.6	<0.001
Marital status
Married/partner	75.1	68.4	<0.001	77.4	57.5	<0.001
Country of birth
Other than Sweden	10.7	13.2	0.008	9.1	10.8	<0.290
Retirement status
Fully retired	20.5	25.3	<0.001	20.9	32.9	<0.001

Note: Age is presented as mean values with p-values from t-tests. All other variables are presented as proportions with p-values from Chi2-tests.

We found significant socio-demographic differences between web and paper respondents (Table 1). Compared with web-respondents, paper-respondents at baseline were more likely to be women (59.2% vs 51.4%; p<0.001), low educated (23.5% vs 12.1% with primary education and 39.4% vs 55.3% with tertiary education; p<0.001), born outside Sweden (13.2% vs 10.7%; p=0.008), fully retired (25.3% vs 20.5%; p<0.001), and less likely to be married (68.4% vs 75.1%; p<0.001).

These differences were compounded in the longitudinal sample groups. That is, when comparing those who answered by web across the three waves with those answering by paper in all three waves, the differences between web and paper response groups were more pronounced for all socio-demographic factors, such as 32.9% vs 20.9% (p<0.001) fully retired among the longitudinal paper-sample compared with the longitudinal web-sample. The only exception was country of birth. The proportion of people born outside Sweden was lower in the longitudinal sample compared with the baseline sample, both among web-respondents and paper-respondents but the significant differences between web and paper respondents that were present at baseline disappeared in the longitudinal sample. Age seems to have minor impact on response mode, although the mean age among paper-respondents was slightly higher than among web-respondents in the longitudinal sample groups.

Table 2. Differences in depression, life satisfaction and poor self-rated health among web and paper respondents (measured at baseline for baseline web and paper sample and at 2^nd follow-up for the longitudinal web and paper sample).

Outcomes measured at baseline and at 2^nd follow-up	HEARTS baseline web sample (n=4068)	HEARTS baseline paper sample (n=1845)	P-value	HEARTS longitudinal web sample (n=2510)	HEARTS longitudinal paper sample (n=369)	P-value
Outcomes measured at baseline and at 2^nd follow-up	HEARTS baseline web sample (n=4068)	HEARTS baseline paper sample (n=1845)	P-value	HEARTS longitudinal web sample (n=2510)	HEARTS longitudinal paper sample (n=369)	P-value

Depression scale (0-30p)	4.0 (3.9;4.2)	4.4 (4.2;4.6)	0.003	3.5 (3.4;3.7)	5.0 (4.5;5.5)	<0.001

Life satisfaction scale(7-35p)	24.2 (24.0;24.4)	24.2 (23.8;24.5)	0.781	24.8 (24.6;25.1)	23.3 (22.5;24.2)	<0.001

Poor self-rated health	9.9	15.7	<0.001	8.5	19.3	<0.001

Note: Estimates of Depression and Life satisfaction are presented as mean values with 95% CI and p-values from t-tests. Poor self-rated health are presented as proportions with p-values from Chi2-tests.

Paper-respondents reported higher mean level of depression (4.4 vs 4.0; p=0.003) as well as lower self-rated health (15.7% vs 9.9% with poor self-rated health; p<0.001) at baseline (Table 2). These differences were also compounded in the longitudinal sample groups (5.0 vs 3.5 for mean value of depression; p<0.001 and 19.3% vs 8.5% for proportion of poor self-rated health; p<0.001). No significant differences in life satisfaction was found between web and paper respondents at baseline, but paper respondents reported slightly lower mean level of life satisfaction than web respondents in the longitudinal sample groups (23.3 vs 24.8; p<0.001).

Table 3. Depression, Life Satisfaction and Poor Self-rated health among retired people compared with non-retired people, by sample group.

HEARTS

baseline

web sample (n=3187)

HEARTS

baseline

paper sample (n=1438)

HEARTS

baseline

paper & web

sample (n=4625)

HEARTS longitudinal

web sample (n=1846)

HEARTS longitudinal

paper sample (n=272)

HEARTS longitudinal

paper & web sample (n=2118)

β^a

P-value

Depression scale (0-30p)

Crude model

-0.89

<0.001

-0.22

0.378

-0.65

<0.001

-0.87

<0.001

0.02

0.980

-0.70

<0.001

Adjusted model^c

-0.36

0.080

-0.17

0.608

-0.19

0.272

-0.81

0.001

-0.34

0.741

-0.76

0.001

Life satisfaction scale (7-35p)

Crude model

1.76

<0.001

1.74

<0.001

1.75

<0.001

2.04

<0.001

2.45

0.014

2.01

<0.001

Adjusted model^c

0.90

0.007

1.58

0.004

1.13

<0.001

2.03

<0.001

2.64

0.109

2.10

<0.001

AME^b

P-value

AME

P-value

AME

P-value

AME

P-value

AME

P-value

AME

P-value

Poor Self rated health

Crude model

-0.86

0.477

1.39

0.497

0.20

0.854

-3.86

0.003

0.10

0.983

-2.92

0.025

Adjusted model^c

1.70

0.305

5.01

0.090

2.93

0.047

-3.68

0.045

-8.39

0.338

-3.93

0.038

^aUnstandardized beta coefficient.

^bAverage Marginal Effect, interpreted as the estimated absolute differences in proportion with Poor Self-rated Health among retired compared with non-retired people.

^c Adjusted for sex, age, education and civil status

Finally, we found that the association between retirement status and depression, as well as self-rated health, differed by sub-sample. Retired people reported lower level of depression at baseline (β= -0.65; p<0.001) compared with non-retired people (Third column in Table 3). However, dividing the sample by response group (first and second column) revealed that the association between retirement status and depression was only present among web-respondents (β= -0.89; p<0.001 vs β= -0.22; p=0.378 in the paper sample). We found the same pattern in the longitudinal sample; retired people reported lower level of depression at 2^nd follow-up (β= -0.70; p<0.001), but only among people in the web-sample (β= -0.87; p<0.001 vs β= 0.02; p=0.980 in the paper sample). Retired people also reported better self-rated health (lower proportion of poor self-rated health) compared with non-retired people in the longitudinal sample, an association that we only found among web-respondents (AME=-3.86; p=0.003 vs AME=0.10; p=0.983 in the paper sample). No significant differences in self-rated health by retirement status was observed at baseline. Retired people reported better life satisfaction compared with non-retired, both among web respondents (β= 1.76; p<0.001 at baseline and β= 2.04; p<0.001 at the 2^nd follow-up) and paper respondents (β= 1.74; p<0.001 at baseline and β= 2.45; p=0.014 at the 2^nd follow-up). However, when adjusting the models for age, sex, education and civil status, no clear differences emerged between web and paper respondents regarding the association between retirement status and depression and self-rated health.

In this study, we investigated response patterns in the Swedish HEARTS study on retirement transition, in which a paper questionnaire was offered as an alternative to the web questionnaire in the last reminder.

Our results can be generalized into three main findings. First, most respondents answered by web (69%; n=4067 at baseline) and this was a rather stable group who continued to respond by web in subsequent waves. Paper-respondents on the other hand, that is, those who did not respond until they got a paper questionnaire with the last reminder, were fewer (31%; n=1845 at baseline) and were less stable in subsequent waves, with higher probability of non-response and changing response mode.

Second, compared with those who answered by web, paper-respondents were more likely to be women, low educated, non-married and fully retired and reported more depression and poor self-rated health. In addition, the associations between retirement status and depression and to some extent poor self-rated health were stronger among web-respondents than among paper-respondents, although this result must be interpreted with caution, since some of these estimates changed substantially when adjusting for confounders.

Third, the differences between web and paper respondents were more pronounced in the longitudinal sample, compared with the baseline sample; the differences between web and paper respondents increased when taking into account the longitudinal response patterns, that is, restricting the analyses to those that answered by paper all three waves and those who responded by web across all three waves.

Our results are in line with previous studies of surveys of older persons, were women and non-married (19), low educated (18, 19) and non-working people (18) were found to be less likely to answer a web-questionnaire. Our finding that people answering by web had better subjective health are both supported (19) and non-supported (18) by previous studies. However, different measures of health were used in the compared studies.

Our finding that the response rate in subsequent wave was higher among web-respondents than among paper-respondents are also in line with previous literature. It has been shown that the differences in response rate between paper and web surveys is lower among panel members than among one-time respondents (8). This suggest that given survey response by web one time, the likelihood of response to the next wave of a web-survey are higher than in a new sample were people are contacted for the first time. On the other hand, it should be mentioned that in the HEARTS study, the paper option was only offered in the last reminder. Hence, the paper-respondents in HEARTS are not comparable with paper respondents from a survey with a paper option in the first invitation. It is likely that some people from the web-sample would have preferred the paper version if they had the choice, without being less likely to participate in the subsequent wave. These results imply that the group of respondents that did not answer until they got a paper questionnaire also are the people that are most likely to not participate in a survey. Finally, it should also be mentioned that those respondents that once answered by paper might be less likely to answer by web in subsequent waves, as they know about the coming paper option.

From a previous study, we know that the attrition in the HEARTS study is associated with personality; people with higher scores on extraversion and neuroticism, and lower scores on agreeableness, were more likely to drop out (24). Results from the present study adds to that knowledge by showing that those who did not answer until they got a paper questionnaire, that is, the paper respondents, also were more likely to attrite from the study. Finally, our analyses also showed that differences between the longitudinal sample groups (web vs paper respondents) were greater than between web and paper respondents at baseline. This finding demonstrates that without the option of a paper questionnaire, the response group in HEARTS would have been even more selected over time if not a paper questionnaire would have been offered.

Implications of the Chosen Survey Design

It is not possible to estimate what the response rate in the HEARTS study would have been if data had been gathered using another survey mode. Previous studies show that web surveys in general produce approximately 10-11% lower response rate than other survey modes, such as paper and telephone (8, 25), except among students, were the results are more mixed. A recent meta-analyses, including over 100 experiments, confirm these results and show a 12 % response rate difference between web-surveys and other modes (Daikeler et al 2020). In one study among students, paper and web yielded the same response rate (3), but in another study the highest response rate was reached when both paper and web was offered (26). Further, in an experimental study of a highly internet-literate population, the offer of both web and paper did not improve response rate compared to only paper. Nevertheless, offering paper at a later stage, as an alternative to web, improved the response rate and was equivalent to the use of paper as the only alternative (27). Previous research also show that the number of reminders seems to be less efficient in web surveys than in other modes, such as paper (8, 13). Taken together, this implies that it is likely that the non-response rate in HEARTS would have been higher if not a paper questionnaire was offered as a response option, even if more reminders would have been used.

The major problem with non-response, in addition to the decreased statistical power, is largely related to the risk that the non-response is occurring non-random. Web and paper respondents differed significantly from each other in the HEARTS study, not only in sociodemographic factors, but also in self-reported health and certain psychological outcomes, both in levels and regarding the association with retirement status. In addition, these differences was compounded in the longitudinal sample. That is, the differences between web and paper respondents was more substantial in the longitudinal sample (i.e. among those who either answered by web or by paper all three waves) compared with all those answering at baseline. Hence, we also conclude that without offering a paper questionnaire as an alternative, a small but important group would have been missing in subsequent waves in the HEARTS study.

The next question is whether the quality of the data in HEARTS depended on choice of survey design. We know that survey mode matter for the results and that it can be problematic to change survey mode across waves (28, 29). However, we also know that some of the differences between survey modes can be explained by changes in wording, structure and visual effect used in the different survey modes and it is therefore recommended to use as similar questions as possible when using multi-mode surveys (29). In the HEARTS study, wording and structure were as identical as possible in the paper and web questionnaire. In addition, the paper and web questionnaire were self-administered, which implies smaller differences than if one of the modes were self-administrated questionnaires and one was conducted by interview (28). Further, in a more recent paper, web, paper and telephone mode yielded similar results regarding political opinion and issues (30).

Limitations

An important limitation in this study is that there is no gold standard to compare our results with, that is, we do not know the real population values for most of the studied variables. However, based on the differences we found between web and paper respondents and that we assume that the offered paper questionnaire contributed with data from a group that otherwise would have been missing, we believe that offering the paper questionnaire generated results closer to the true population values.

Although the HEARTs study is a survey comprising questions of relevance for the target population and therefore might motivate participation, the questionnaire is extensive and rather time-consuming. It is most likely that there is differences between people who felt motivated enough to answer the questionnaire and those who did not. Respondents in HEARTs are for example more educated compared with the general population (20). It might be that the differences between web and paper respondents, as well as the response patterns, would have been different in a less extensive and time-consuming survey.

The present study indicate that a web-survey might be a good and feasible alternative in studying older adults in the pre- and post-retirement ages, i.e. in their 60’s and early 70’s. However, without offering a paper-questionnaire as an alternative to solely a web-questionnaire, a small but important subgroup will be missing which most likely would produce more biased estimates. Our results indicated that without a paper alternative, people with low education, women, fully retired and non-married people would have been underrepresented in the HEARTs study. This would also have resulted in an underestimation of the prevalence of depression and poor self-rated health, whereas the association between retirement and depression would have been overestimated. Notably, we found that the differences between web and paper respondents increased in the longitudinal sample, that is, after two follow-ups, which implies that the potential bias from not providing a paper questionnaire as an alternative would have been even greater in analyses using longitudinal data.

HEARTS: the Health, Ageing and Retirement Transitions in Sweden study; AME: Average Marginal Effects.

Ethics approval and consent to participate

Ethical approval for the HEARTS study was granted from the regional ethical approval board of Gothenburg at Gothenburg University (Dnr: 970-14). All subjects gave written informed consent in accordance with the Declaration of Helsinki

Consent for publication

Not applicable

Availability of data and materials

The data that support the findings of this study are available from Gothenburg University but restrictions apply to the availability of these data, which were used under license for the current study, and so are not publicly available. Data are however available from the authors upon reasonable request and with permission of Gothenburg University.

Competing interests

The authors declare that they have no competing interests.

Funding

This work was supported by the Swedish Research Council for Health, Working Life and Welfare with grant numbers 2013-2291 for data collection of HEARTS and 2017-01456 for salary for the main author SK.

Authors' contributions

SK were main responsible for the paper and contributed by designing the study, analyzing the data and drafted the first manuscript. MK, BJ and ML contributed to the study design and were involved in revising the manuscript critically for important intellectual content. All authors read and approved the final manuscript.

Acknowledgements

Not applicable

Couper MP. Web surveys: A review of issues and approaches. The Public Opinion Quarterly. 2000;64(4):464-94.
Couper MP, Miller PV. Web Survey Methods: Introduction. Public Opinion Quarterly. 2008;72(5):831-5.
Kaplowitz MD, Hadlock TD, Levine R. A Comparison of Web and Mail Survey Response Rates. Public Opinion Quarterly. 2004;68(1):94-101.
Daikeler J. Bošnjak M. Lozar Manfreda, K. Web versus other survey modes: an updated and extended meta-analysis comparing response rates. Journal of Survey Statistics and Methodology. 2020;8(3): 513-539.
Carina C, Bosnjak M. "Is there an association between survey characteristics and representativeness? A meta-analysis." Survey Research Methods. 2018;12(1).
Statistic Sweden. http://www.statistikdatabasen.scb.se, visited 2019-01-09
Dillman DA, Smyth JD. Design effects in the transition to web-based surveys. American journal of preventive medicine. 2007;32(5):S90-S6.
Manfreda KL, Berzelak J, Vehovar V, Bosnjak M, Haas I. Web surveys versus other survey modes: A meta-analysis comparing response rates. International Journal of Market Research. 2008;50(1):79-104.
Tijdens K, Steinmetz S. Is the web a promising tool for data collection in developing countries? An analysis of the sample bias of 10 web and face-to-face surveys from Africa, Asia, and South America. International Journal of Social Research Methodology. 2016;19(4):461-79.
Galea S, Tracy M. Participation rates in epidemiologic studies. Annals of epidemiology. 2007;17(9):643-53.
Meyer BD, Mok WK, Sullivan JX. Household surveys in crisis. Journal of Economic Perspectives. 2015;29(4):199-226.
Brick JM. Unit Nonresponse and Weighting Adjustments: A Critical Review. 2013;29(3):329.
Shih T-H, Fan X. Comparing Response Rates from Web and Mail Surveys: A Meta-Analysis. Field Methods. 2008;20(3):249-71.
van Deursen AJ, Helsper EJ. A nuanced understanding of Internet use and non-use among the elderly. European Journal of Communication. 2015;30(2):171-87.
Olsson T, Samuelsson U, Viscovi D. Resources and repertoires: Elderly online practices. European journal of communication. 2019;34(1):38-56.
Helsper EJ, Reisdorf BC. The emergence of a “digital underclass” in Great Britain and Sweden: Changing reasons for digital exclusion. New media & society. 2017;19(8):1253-70.
Bethlehem J. Selection bias in web surveys. International Statistical Review. 2010;78(2):161-88.
de Bernardo DH, Curtis A. Using online and paper surveys: the effectiveness of mixed-mode methodology for populations over 50. Research on Aging. 2013;35(2):220-40.
Bech M, Kristensen MB. Differential response rates in postal and Web-based surveys in older respondents. Survey Research Methods. 2009;3(1):1-6.
Lindwall M, Berg AI, Bjälkebring P, Buratti S, Hansson I, Hassing L, et al. Psychological health in the retirement transition: Rationale and first findings in the Health, Ageing and Retirement Transitions in Sweden (HEARTS) study. Frontiers in psychology. 2017;8:1634.
Radloff LS. The CES-D scale: a self-report depression scale for research in the general population. Applied psychological measurement. 1977;1(3):385-401.
Diener E, Emmons RA, Larsen RJ, Griffin S. The satisfaction with life scale. Journal of personality assessment. 1985;49(1):71-5.
Mood C. Logistic regression: Why we cannot do what we think we can do, and what we can do about it. European sociological review. 2010;26(1):67-82.
Hansson I, Berg AI, Thorvaldsson V. Can personality predict longitudinal study attrition? Evidence from a population-based sample of older adults. Journal of Research in Personality. 2018;77:133-6.
Fan W, Yan Z. Factors affecting response rates of the web survey: A systematic review. Computers in human behavior. 2010;26(2):132-9.
Sax LJ, Gilmartin SK, Bryant AN. Assessing response rates and nonresponse bias in web and paper surveys. Research in higher education. 2003;44(4):409-32.
Millar MM, Dillman DA. Improving response to web and mixed-mode surveys. Public opinion quarterly. 2011;75(2):249-69.
Bowling A. Mode of questionnaire administration can have serious effects on data quality. Journal of public health. 2005;27(3):281-91.
Dillman DA, Christian LM. Survey mode as a source of instability in responses across surveys. Field methods. 2005;17(1):30-52.
Ansolabehere S, Schaffner BF. Does survey mode still matter? Findings from a 2010 multi-mode comparison. Political Analysis. 2014;22(3):285-303.

Download PDF

Journal Publication

published 08 Oct, 2020

Read the published version in BMC Medical Research Methodology →

Editorial decision: Major revision
01 Jul, 2020
Review #1 received at journal
26 Jun, 2020
Review #2 received at journal
22 Jun, 2020
Reviewer #2 agreed at journal
19 Jun, 2020
Reviewers invited by journal
18 Jun, 2020
Reviewer #1 agreed at journal
18 Jun, 2020
Editor assigned by journal
15 Jun, 2020
Submission checks completed at journal
14 Jun, 2020
Editor invited by journal
14 Jun, 2020

You are reading this older preprint version

Read the latest preprint version →

Going web or staying paper? The use of web-surveys among older people

Status:

Journal Publication

Version 2

Abstract

Figures

Background

Methods

Results

Discussion

Conclusion

Abbreviations

Declarations

References

Status:

Journal Publication

Version 2