Survey researchers have retained interest in the continuum of resistance model for 80 years, despite conflicting evidence of its validity. Today, with response rates generally declining,(2, 31) finding effective ways to assess nonresponse bias is as important as ever.
We applied the continuum of resistance model to a large online public health survey, comparing respondents who completed the questionnaire within the first 7 days (wave 1), those who completed it after 8 to 15 days and one reminder (wave 2) and those who completed it after 16 or more days and two reminders (wave 3). For demographic variables, we identified differences between waves that were consistent with previous literature.(1-10, 12-15) However, any differences in health outcomes and behaviours were small between waves and were unlikely to be useful in identifying nonresponse bias. Overall, females and older people were more likely to respond to the questionnaire than males and younger people were. This was most pronounced among wave 1 and 2 respondents, whereas wave 3 more closely resembled the invited sample, containing a higher proportion of males and younger people. However, it is important to note that for sex and age, the composition of wave 3 more closely resembled waves 1 and 2 than it did the non-respondents.
For education, our findings were similar. There was a slight trend towards wave 1 respondents being more highly educated than those in wave 2 and 3. However, the difference between respondents and non-respondents is likely to be much larger than the small differences between response waves. Although we lacked direct information on the education level of non-respondents, data from Statistics Norway show that 35% of Hordaland county residents have tertiary education, and that 24% have only completed junior high school.(56) These proportions differ markedly from our results (52% and 13%, respectively), suggesting that non-respondents had far lower levels of education than respondents did.
Based on the continuum of resistance model, we expected that late respondents would display an overall pattern of poorer health across health outcomes. This has been found in a number of recent studies. For example, compared to early respondents, late respondents have been found to have a 21% to 68% higher prevalence of monthly binge drinking,(3, 40, 42, 43) a 30% higher prevalence of current smokers,(57) and a 50% higher prevalence of people who complete less than 30min per day of physical activity.(40) We aligned our outcome definitions to facilitate comparisons with these studies, but did not find the same results. There was no difference between waves in the prevalence of monthly binge drinking or physical inactivity, and for current smoking, the difference in prevalence between waves 1 and 3 was only 1.4 percentage points. Our findings were similar for other health outcomes; in some cases, there were statistically significant but very small prevalence differences between waves, and in others there were none.
Our findings are supported by a recent comparison of early and late respondents to a national online health survey in the Netherlands.(44) In that study, only small differences in health-related outcomes were identified between response waves, despite substantial differences in socio-demographic variables between waves. Further, when analyses were adjusted for sociodemographic variables, the differences in health-related outcomes all but disappeared. In our results, there was little change when age and sex were adjusted for.
There are several potential explanations for why we did not find evidence to support a continuum of resistance in our data. Indeed, it is possible that the health status of respondents and non-respondents is very similar in our population. We believe this is unlikely, particularly considering the findings of Knudsen et al., who, in 2008, reported a substantially higher prevalence of mental and somatic health disorders among non-respondents to a health survey conducted in Hordaland county.(9) Our definition of late respondents differs from some recent studies demonstrating a continuum-of-resistance, which have used more reminders,(43) longer follow-up periods,(3, 12, 40-42) and/or alternative methods such as telephone calls to contact slow respondents.(3, 41, 42) To our knowledge, this is the only continuum-of-resistance study besides those of Kypri et al.(40, 41) and Klingwort (44) to collect data using purely digital means. It is possible that the barriers to questionnaire completion differ between postal, telephone and internet/smartphone surveys, and that the data collection method has consequences on any eventual continuum of resistance.
This study has several limitations that we were unable to account for, and that may have affected our findings. First, we had no information about the health status of non-respondents, but rather we assumed that there were differences based on previous research. Future studies linking survey data with other sources, such as national registers, are necessary to gain more information on the health status of non-respondents. Additionally, to be eligible for inclusion in the survey, people had to have their digital contact information registered with the Norwegian authorities. This introduces a selection bias that is particularly pronounced among older people.(45) It is therefore likely that the health status of the survey sample is more homogeneous than it is in the general population.