Quality Improvement Questionnaires – Nursing Homes (QIQ-NH): validation of questionnaires for measuring quality of care in nursing homes from various perspectives

This study aims to describe the validation and optimization of a new instrument specically designed to measure and improve the quality of care in nursing homes; the Quality Improvement Questionnaires for Nursing Homes (QIQ-NH). This instrument comprises several questionnaires on the perceived quality of care for various perspectives (e.g. clients, family and professional caregivers) and covers eight themes of the national quality framework for nursing home care in the Netherlands. (QIQ-NH) appears to be a promising and valid instrument for measuring and improving the quality of nursing home care. This comprehensive instrument is designed for measuring, improving and monitoring the quality of care in nursing homes from various perspectives, by covering the national quality themes and the views of all stakeholders involved. This study shows that the QIQ-NH has overall good psychometric quality, with an acceptable to good construct, criterion and content validity. Nonetheless, cognitive testing showed several problems in answering the questionnaires and future research should prove whether the optimization resulted in better, more valid and reliable questionnaires. Results also show that residents, family caregivers and professional caregivers are likely to differ in their values and evaluations of the quality of care in nursing homes. Future research


Page 4/21
Theme: Description: 1 Person-centered care How the nursing home places residents in the centre of what they do.

Living and wellbeing
How nursing homes identify, support and promote the quality of live and well-being for residents and their informal caregivers.
3 Safety How nursing homes guarantee basic safety for residents by following professional standards and guidelines, and through preventing and minimizing harm and learning from safety incidents. 4 Learning and improving quality How nursing homes dynamically learn and improve to provide optimal care for residents, using best available evidence and information. 5 Leadership, governance and management The arrangements put in place by the nursing home for accountability, decisionmaking, risk management and meeting its strategic, legal and nancial obligations. 6 Responsive workforce Adequate and su cient staff with the necessary numbers, skills and abilities to respond to the needs of residents.

Use of resources
Using resources effectively and e ciently to deliver best achievable outcomes for the money and resources used. 8 Use of information Actively using information as a resource for planning, delivering, monitoring, managing and improving services, and transparency of quality information for clients, family and society.
*Source: Zorginstituut Nederland, 2017 [2] Each theme provides speci c standards in order to guarantee and improve the quality and safety of services for people living in nursing homes. Four themes relate to the content or quality and safety of care (themes [1][2][3][4], and four themes relate to the preconditions for quality, such as management, resources, responsive workforce and information (themes [5][6][7][8]. Learning and improving quality (theme 4) is a central theme of the quality framework. The nursing homes have the responsibility to continuously re ect and improve quality of care and measure and evaluate client experiences at least once a year in order to transparently report and work on personalized care and the well-being of their residents.
Measuring quality of care in nursing homes Many quantitative instruments have been developed to register and monitor the quality of care and quality of life in nursing homes. A quick review of reviews highlights at least 75 internationally used instruments for measuring quality of life or satisfaction of nursing home residents as indicators of the quality of nursing home care [9]. Most of these instruments measure quality of life, either focussing on speci c domains of functioning (e.g. psychological well-being or cognitive functioning) or speci c conditions such as dementia. For example, widely used instruments are the dementia-speci c observation instruments QUALIDEM (quality of life instrument for persons with dementia) and PGC-ARS (Philadelphia Geriatric Centre Affect Rating Scale). Roughly half of the internationally used instruments are also available or translated into Dutch, however, only a few are actually used in the Netherlands [9].
Just like other industrialized countries, the Netherlands has a long tradition of measuring the quality of care in nursing home care in a quantitative way. For example, with risk assessments for the national Prevalence Measurement of Care Problems in Care Homes [10], or by measuring client experiences in long-term care facilities with the Consumer Quality Index (CQ-index) [4]. The CQ-index for nursing homes is an instrument partly based on the American CAHPS® Nursing Home Survey [11]. Over the last decade, however, there has been a new trend in the measurement of quality of care in Dutch nursing homes. A lot of qualitative and narrative methods have been developed in order to more fully capture the construct of experienced quality of care, and to obtain more concrete clues for quality improvement. This resulted in a large number and wide variety of instruments and methods to assess and improve the quality of care in nursing homes in the Netherlands, including about 80% qualitative or mixed methods [9]. According to a national inventory in 2018, about 130 different instruments are currently being used in the Netherlands [12]. The growing awareness that quality is not just static but continuously needs discourse, maintenance and involvement of all stakeholders, has led to a wide variety of predominantly qualitative instruments. Furthermore, more dynamic instruments have been developed to invite relevant stakeholders involved to join in the measurement and improvement of quality. One of these new and frequently used instruments is the 'Quality Improvement Questionnaires -Nursing Homes' (QIQ-NH), which is the subject of this study.
Quality Improvement Questionnaires -Nursing Homes (QIQ-NH) The QIQ-NH consists of several questionnaires for measuring the experiences with quality of care in nursing homes from different perspectives (e.g. residents, family and professional caregivers), with questions covering all eight themes of the national quality framework in the Netherlands (see Box 1). This instrument has been developed in 2015-2017 by Vilans, the national Centre of Expertise for Long-term Care in the Netherlands, and has not been published before. The instrument is also called the 'Verbetermeting 360', in Dutch, and is part of a total method for quality management called the 'Kwaliteit Verbetercyclus'.
Considerations in developing this instrument were to design a relevant, comprehensive and useful instrument for providing real-time 360 degrees feedback to staff (i.e. care professionals and managers) in order to continuously monitor, evaluate and improve the quality of nursing home care in line with the PDCA-cycle (Plan, Do, Check, Act) for quality control of Deming [13]. With real-time feedback, the instrument enables management and care teams to select possibilities or areas for improvement, to integrate the various perspectives of different parties involved, and to implement and monitor the effects of quality improvement strategies in nursing homes.
The questionnaires have been developed for various stakeholders or different subgroups (e.g. managers and volunteers), of which residents, family and professional caregivers are the main categories of respondents. All stakeholders involved in nursing home care, including residents themselves, are invited to complete a digital questionnaire. The results are accessible in real-time via an online dashboard. Care professionals, teams and the management can subsequently use the feedback information to improve care and may conduct follow-up measurements in order to monitor and evaluate the implementation of their quality improvement efforts.

Aim of the study
This study aims to validate and optimize three QIQ-NH questionnaires that measure quality of nursing home care from three perspectives (i.e. residents, family and professional caregivers), by answering the following two research questions: 1. What is the construct, criterion and content validity of the QIQ-NH questionnaires?
2. How could the questionnaires be further optimized?

Method
This study describes the validation and optimization of the Quality Improvement Questionnaires for Nursing Homes (QIQ-NH) for three subgroups: residents, family and professional caregivers. The questionnaires were extensively validated and optimized in a multistage approach, including both quantitative and qualitative testing, with psychometric testing and cognitive interviewing respectively. See Supplementary le 1 for the items of the questionnaires in the English language.
Quality Improvement Questionnaires for Nursing Homes (QIQ-NH) The QIQ-NH questionnaires address six to eight themes of the national quality framework for nursing homes in The Netherlands. The themes and number of items addressed in each questionnaire depend on the applicability and the ability of each subgroup to re ect on a certain topic. Questions are formulated as propositions and answering scales range from 1 (completely disagree) to 5 (completely agree). See Table 1 for an overview of domains and number of items, and Supplementary le 1 for a more detailed description of the content of the three questionnaires. The questionnaires are not under license but part of the 'Kwaliteit Verbetercyclus' (in Dutch) and can be requested by contacting Vilans (third author). The Net Promoter Score (NPS) is also part of the QIQ-NH questionnaires. The NPS re ects the likelihood of residents, family and professional caregivers to recommend the nursing home to others (family, friends or colleagues), and is used as a global indicator for the quality of care in nursing homes in the Netherlands [2]. The NPS-question was formulated as 'Would you recommend our nursing home to family, friends or colleagues?, with an answering scale from 0 to 10. Scores are categorized into promoters (9 or 10), passives (7 or 8), and detractors (0 to 6) [14]. To calculate the NPS, the percentage of detractors is subtracted from the percentage of respondents who are promoters: NPS = % Promoters (scores 9 or 10) -% Detractors (scores 0 to 6), resulting in a score between -100 and +100. Answers were also dichotomized into being or not being a promotor ('yes': NPS=9-10; 'no': NPS=0-8).

Data collection and study population
Data was collected with QIQ-NH questionnaires between September 2017 and June 2018, at the start of a quality improvement program in six nursing homes in the Netherlands. The questionnaires were distributed during a three-week period to convenience samples of residents, family caregivers and professional caregivers. Inclusion criteria were: living in a nursing home (residents), or providing informal or formal care to residents (family caregivers and professional caregivers, respectively). The questionnaires were distributed by e-mail to family caregivers and professional caregivers, and residents were asked to complete the questionnaire on an electronic device (i.e. laptop, notebook or tablet). At least 1,325 persons were enrolled: 458 residents, 64 family caregivers, and 803 professional caregivers received the digital questionnaire.

Analyses
Analysis of the data collected with the QIQ-NH questionnaires was performed with the statistical software of Stata 15 [15]. Subsequently, scale analyses (i.e. factor and reliability analyses, correlations) and regression analyses were used to assess the construct and criterion validity of the three questionnaires. Item-analyses were performed to assess the measurement properties of each item and to select problematic items for cognitive testing.
Construct validity: factor structure, scale reliability and inter-scale correlations Principal Component Analysis with oblimin rotation was conducted. The factor structure of each domain was con rmed with an Eigenvalue of >1, a Kaiser-Meyer-Oklin value of >0.60 and when Bartlett's test of sphericity reached signi cance (p<0.05). Factor loadings of items should exceed the threshold of >.40. To assess the reliability or internal consistency of the scales found, Cronbach's alpha was calculated for all factors (domains or scales) of each questionnaire. In classical test theory, a Cronbach's alpha of 0.7 or higher is indicated as cut-off for a reliable scale [16], but 0.6 is generally recommended as a cut-off value in exploratory analyses [17]. Inter-scale correlations were assessed to determine whether the scales of each questionnaire represent unique and independent constructs, with Pearson correlations being preferably between 0.3 and 0.7 [18]. Content validity: item analysis and cognitive testing The content validity of the questionnaires was tested with item-analyses and cognitive interviews.

Item analyses and item selection for cognitive testing
Item-and scale analyses were determined in order to select those questionnaire items that required further testing and optimization with cognitive interviews. These analyses respectively focused on item non-response (% missings), skewness (extreme distribution of answers), overlap (inter-item correlations), and the contribution of an item to a scale.
The following criteria were used to select problematic items for cognitive testing:

Cognitive interviews
Cognitive interviews were used to test and re ne the problematic questionnaire items, in order to optimize the content validity of the QIQ-NH questionnaires. Cognitive interviewing is a valid and frequently-used technique in the development and testing of questionnaires [19]. It can be used to assess the participants' comprehension of questions and to identify confusing or unclear wordings, in order to further improve the questionnaires. The interviews were conducted in July 2018, after a period of data collection, and focused on a selection of problematic items of the QIQ-NH questionnaires. Participants were recruited by four nursing homes, representing the three subgroups (i.e. residents, family caregivers, professionals), and they were interviewed in two rounds.
Two researchers (MT and JM) conducted the interviews by using a combination of two interviewing techniques: 1) the think-aloud technique, and 2) the verbal probing technique [20]. Participants were asked to read out loud each question and to describe their thoughts while answering the question. In addition, probes were used to structure the interview and to gain more information on the understanding of the questions.
Examples of probes were: 'Can you repeat the question in your own words?' or 'How did you get to this answer?'. All cognitive interviews were audio-recorded, transcribed verbatim, and content analyzed. The interviewers (MT and JM) coded problems independently by using the coding system of Willis [20]. This system distinguishes seven categories of questionnaire problems, see Box 2. For each problematic item, the most prominent underlying problem was coded, and sometimes multiple codes were used for one item. All data was entered in a spreadsheet to get an overview of problems per item of each questionnaire; including both qualitative (i.e. verbatim quotations, problem codes, and researchers' comments) and quantitative data (i.e. number of problems and respondents). The QIQ-NH questionnaires were optimized, based on the results of the interviews, suggestions of participants and deliberations of the research team. After the rst interview round, problem codes were compared and discussed between the two interviewers (MT and JM). Subsequently, questions were adjusted and re-tested in a second interview round, with other participants, to see if these questions indeed had improved. If necessary, items were further optimized after the second interview round. All modi cations were discussed and nalized after consensus was reached in the research team (MT, JM and BvdB).

Informed consent
The nursing homes gave their consent to use the anonymized data for our research purposes. All participants of the cognitive interviews provided a written informed consent, after they received written and verbal information about the cognitive testing. The study did not fall under the remit of the Dutch Medical Research Involving Human Subjects act, and ethical approval was not necessary.

Results
Out of the 1,325 persons who were enrolled for the study, a total of 1,055 participants completed the questionnaires (response rate 80%), including 359 residents, 48 family caregivers and 648 professional caregivers.

Construct and criterion validity of the QIQ-NH questionnaires
Factor structure and scale reliability

Content validity and questionnaire optimization Item analyses and item selection for cognitive testing
Results of the construct validation analyses and the item non-response analyses were used to select items for further testing with cognitive interviews. Seven out of 23 items (30%) of the residents questionnaire, 11 out of 24 items (46%) of the family questionnaire, and 12 out of the 54 items (22%) for professional caregivers had more than 10% missing values. In addition, six items (in themes 1, 2 and 6) of the questionnaire for family caregivers showed item overlap (Pearson correlation: 0.70 to 0.78). Nevertheless, no item skewness was present in all three questionnaires and the questionnaires for residents and professional caregivers showed no excessive item overlap. Eventually, 11 items (48% of 23 items) of the QIQ-NH for residents, 17 items (71% of 24 items) of the questionnaire for family caregivers and 18 items (33% of 54 items) of the questionnaire for professional caregivers met one or more criteria for further testing, and these items were selected for the cognitive interviews.

Participants and interviews in two rounds
A total of 20 participants (7 residents, 5 family caregivers, 8 professional caregivers) were interviewed in two rounds; 11 persons in the rst round and nine in the second round. The mean duration of interviews was 26 minutes (range 11-48 minutes). The mean age of participants was 84 years (SD=4.9) for residents, 63 years (SD=6.3) for family caregivers, and 41 years (SD=11.8) for professional caregivers. Most participants were female (65%) and the educational level varied from low (25%) to high (35%) (see Supplementary le 2: Table  4.1).  In the second round (n=9), after re ning problematic questions, the total number of problems declined to 32 problems: 14 problems in 4 residents, 5 problems in two family caregivers, and 13 problems in three professionals. The mean number of problems per question per participant in the second round was 0.32 for residents, 0.15 for family caregivers, and 0.24 for professional caregivers respectively. The remaining problems concerned clarity (18), knowledge (7), assumption (1) and some other problems (6). Other problems were for example ambiguous items actually containing two questions, or items with abstract or di cult words that needed more explanation (e.g. by adding examples) or rephrasing (with concrete or simple words).  Total number of problems 14 5 13 Mean number of problems per item per respondent 0.32 0.15 0.24 * categorized according to the scoring system of Willis (1999) [20] Questionnaire optimization Based on the feedback from the cognitive interviews, more than half of the total number of items of the QIQ-NH were adapted; 15 of the 23 items (65%) for residents, 18 of the 24 items (75%) for family caregivers, and 21 of the 54 items (39%) for the professional caregivers (see Table 4). In addition, one item on Safety

Discussion
This study aimed to validate and optimize the QIQ-NH, a Dutch instrument to measure the quality of care in nursing homes from different perspectives, i.e., residents, family caregivers, and professional caregivers. The instrument assesses experiences with the quality of care in nursing homes for eight themes of the national quality framework [2]. The QIQ-NH appears to be a solid, comprehensive and promising instrument with an acceptable to good construct, criterion, and content validity. Nonetheless, about half of the items of the three questionnaires needed some adaptations and some items have been deleted after cognitive testing, particularly concerning the questionnaire for family caregivers. This optimization is likely to have yielded an even better, valid and reliable instrument for measuring, improving and monitoring the quality of nursing home care.
Acceptable to good psychometric properties Testing the three questionnaires showed robust scales and independent or unique constructs, with Cronbach's alphas varying between .69 and .88 (except for Safety in the questionnaire for family caregivers, which is no longer a scale as an item has been deleted), and Pearson inter-scale correlations between .30 and .70 (except for three domains in the questionnaire for family caregivers showing some overlap, with correlations of .76 and .78). Furthermore, each scale of the questionnaires was signi cantly related to the generic quality measure, the Net Promoter Score, proving criterion validity. With respect to the content validity, however, about half of the items (11 to 18 items per questionnaire) were further tested, because of excessive missing values or redundancy due to item overlap. In the cognitive interviews, these problematic items particularly showed clarity and knowledge problems, and consequently have been adapted.
The psychometric analysis of the QIQ-NH showed good internal consistency or reliability for almost all scales (Cronbach's alpha >0.7). The internal consistency of two scales was somewhat lower, i.e. safety in the QIQ-NH for family caregivers (α=0.64) and living and well-being in the questionnaire for residents (α=0.69), but the Cronbach's alphas were still acceptable [16][17].
The psychometric properties of the questionnaire for family caregivers were the least optimal and this questionnaire also needed most adaptations because of item-overlap, many missing values and various problems shown by the cognitive interviews. Family caregivers reported the most 'knowledge problems' during the cognitive interviews, compared to the other two groups, which relate to di culties in answering a question due to a lack of information [20]. A possible explanation for this could be that family caregivers are less involved in the daily care for their relatives, and that they might be relatively unaware of the speci c care process and activities in the nursing home. Consequently, family caregivers are likely to have most di culty in rating the quality of nursing home care and it is hard to capture the experiences or perspective of family caregivers well. Nevertheless, the cognitive interviews offered many clues for item optimization and seemed to have resulted in a more valid and feasible questionnaire since the second interview round showed far less problems.
Monitoring and improving quality from different perspectives In the past years, many instruments have been developed to assess quality of care in nursing homes in the Netherlands. However, most instruments are not validated and either focus on speci c quality domains or on speci c groups of residents [9]. The QIQ-NH is a relatively new instrument for evaluating the quality of care comprehensively and multi-dimensionally, from different perspectives. It enables the monitoring and evaluation of the various domains of quality of care in nursing homes, for all themes of the national quality framework in the Netherlands, and for all types of residents. It can be applied in an everyday care setting in order to continuously monitor and improve the care in nursing homes, based on real-time feedback. The instrument is increasingly being used for nationwide quality evaluations or 'scans' in nursing homes (in Dutch: 'Scan Kwaliteitskader Verpleeghuiszorg') [21]. Recent national data in particular shows potential areas of quality improvement concerning the preconditions 'Staff composition' and 'Leadership', as well as for the themes 'Safety' and 'Learning and improving' in nursing home care [21].
The results of our study con rm the capacity of the instrument to show potential areas for quality improvement. Nonetheless, residents, family caregivers and professional caregivers might differ in their evaluations and opinions regarding 'good quality of care'. Therefore, it is important to be aware of different perspectives or preferences and to take into account the different views and opinions in the management and improvement of care. It is also important to carefully interpret the questionnaires' outcomes, as participants might overrate or underrate the quality of care. For example, residents might overrate the quality of care due to dependence or factors such as not feeling safe nor being able to speak freely [22]. Additional conversation and in-depth interviews with the different stakeholders could help to determine how to further improve the quality of nursing home care.
Various stakeholders rate quality differently The results of our study show that residents, family caregivers and professional caregivers are generally likely to differ in their evaluation of the quality of care. Interestingly, a varying impact of the eight themes was found in predicting the overall perceived quality of care for the three perspectives as measured by the NPS (re ecting the likelihood of residents, family and professional caregivers to recommend the nursing home to others i.e. family, friends or colleagues). Residents rated person-centered care as the most important theme for a good quality of care, whereas safety was most predictive in family caregivers and leadership and a responsive workforce were the most important predictors of quality of care. These ndings do not only imply that various stakeholders might perceive or interpret quality differently, but also suggest that the experiences or assumptions underlying the overall perceived quality of care actually differ between subgroups. In other words, the concept quality of care is likely to be de ned differently by various stakeholders. Future studies with the QIQ-NH could reveal whether the perceived quality of care in nursing homes and the key areas for quality improvement actually differ between subgroups.
Literature con rms the different views of various stakeholders on the quality of nursing home care. For example, results of a Dutch nationwide improvement program for the elderly showed that older adults evaluated the quality of care less positively than professionals, particularly in evaluating the personcenteredness of care [23]. This difference particularly concerned person-centered care, which seems to be of high priority for elderly when receiving care. We also found the QIQ-NH theme person-centered care to be the most important predictor of the perceived quality of care in residents. In contrast, professionals rated preconditions such as having a responsive workforce and leadership, governance and management as the most important predictors of quality of care. Other research shows that factors related to the work environment, such as team climate and collaboration between team members, contribute to the quality of care in nursing homes [24][25]. Thus, in order to improve care from a residential and professional perspective, it seems crucial to focus both on person-centered care and on preconditions of care, as these factors have been proven to be strongly related to the overall quality of care in nursing homes.
For family caregivers, the QIQ-NH theme safety was most important in predicting good quality of care. This nding is in line with the policy of the health inspectorate in the Netherlands to warrant safety as a basic prerequisite for good quality of care in nursing homes. Previous research also showed that family caregivers particularly emphasize the need to improve the safety of the living environment [4]. However, it is important to notice that the sample size of family caregivers who lled in the questionnaire was relatively small (n=48).
Also, previous research indicated that assessments of nursing home care by family caregivers should be interpreted with caution as their judgement may be in uenced by factors that are not related to the caregiving situation, i.e., a general opinion on nursing homes or emotional feelings such as guilt or sadness because of their loved ones living in nursing homes [26][27]. Therefore, the perspective of family caregiver on the quality of care in nursing homes must be interpreted carefully and is also a subject for further study.

Study limitations
The present study has some limitations. Firstly, the participating nursing homes also took part in a quality improvement program in the Netherlands, which means that these nursing homes had distinct quality issues.
Secondly, the participants who lled in the questionnaire were not randomly selected and a selection bias might have been introduced by professionals who invited residents to participate and who distributed the questionnaires among family caregivers. In addition, since the background and response characteristics of participants were not registered, we were not able to describe the response and demographic features of participants, nor able to verify the representativeness of the data. Thus, the study setting is not likely to be representative for all nursing homes in the Netherlands and the respondents might have under-or overrated the quality of care in the nursing homes. However, irrespective of the level of quality in the participating nursing homes, this study presents important information on the underlying mechanisms of quality of care that can be used to understand and improve quality of care in Dutch nursing homes nationwide.

Future research
We expect the optimized instrument to show even more relevant, valid and reliable quality scores in future measurements. Nonetheless, future research with the QIQ-NH is necessary to determine whether the psychometric quality of the optimized and nal versions of the questionnaires has actually been improved.
These future studies should ensure more representative study settings and participants, with acceptable sample sizes for all subgroups, and could also con rm the wider usability and feasibility of the instrument. Finally, future research is needed to actually measure and improve the quality of care in many nursing homes, for example with comparative research or nationwide benchmark studies, and to further investigate the differences in views and ratings of various stakeholders in order to improve the quality of care from various perspectives.

Conclusions
In conclusion, the Quality Improvement Questionnaires for Nursing Homes (QIQ-NH) appears to be a promising and valid instrument for measuring and improving the quality of nursing home care. This comprehensive instrument is designed for measuring, improving and monitoring the quality of care in nursing homes from various perspectives, by covering the national quality themes and the views of all stakeholders involved. This study shows that the QIQ-NH has overall good psychometric quality, with an acceptable to good construct, criterion and content validity. Nonetheless, cognitive testing showed several problems in answering the questionnaires and future research should prove whether the optimization resulted in better, more valid and reliable questionnaires. Results also show that residents, family caregivers and professional caregivers are likely to differ in their values and evaluations of the quality of care in nursing homes. Future research should actually demonstrate the improved psychometric properties of the optimized questionnaires and the wider usability and feasibility of the QIQ-NH for implementing and monitoring the effects of quality improvement strategies in nursing homes.
Abbreviations QIQ-NH: Quality Improvement Questionnaires for Nursing Homes Declarations Ethics approval and consent to participate: All data are collected and handled in accordance with the relevant privacy protection guidelines. According to Dutch law, the study did not fall under the remit of the Medical Research Involving Human Subjects policy and no approval by a medical ethical committee was necessary.
Verbal consent was obtained from respondents who completed a questionnaire anonymously, and their response expressed their willingness to participate. Written consent was obtained from participants to the cognitive interviews.
Consent for publication: All participants to the cognitive interviews completed a consent form, stating that they were well-informed about the interviews and that they agreed upon the publication of anonymized data.
Availability of data and materials: The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.
Competing interests: Not applicable.
Funding: This research was funded by Vilans, Netherlands Centre of Expertise for Long-term Care, Utrecht, The Netherlands. Vilans provided data for the study but had no interference in the design and analysis of the data.
The third author (BvdB) is a senior advisor of Vilans and she was involved because of her speci c knowledge on the application of the instrument, her responsibility for data recruitment and contacts with the participating nursing homes.
Authors' contributions: MT and JM conceived and designed the study. BvdB was responsible for data collection with the QIQ-NH and she arranged the dataset and the recruitment of subjects for the cognitive interviews via the participating nursing homes. MT and JM analyzed and interpreted the quantitative data, and also conducted and analyzed the cognitive interviews. All authors were involved in the interpretation of data of the cognitive interviews, in order to reach consensus for optimizing the instrument. MT and JM drafted the manuscript and all authors were responsible for writing the manuscript and the nal approval of the version to be published.