Development and validation of a new questionnaire appropriate to assess respiratory symptoms in current and former smokers without chronic obstructive pulmonary disease: The Respiratory Symptom Questionnaire (RSQ)


 Background: Smokers who switch completely to electronic nicotine delivery systems (ENDS) may experience reductions in respiratory symptoms (e.g., cough, wheezing). Assessing such symptoms among smokers without respiratory disease using existing respiratory symptom questionnaires is challenging, as these questionnaires are intended for patient populations, such as those with chronic obstructive pulmonary disease (COPD). The current study aimed to develop a respiratory symptom questionnaire appropriate for current and former smokers including those without COPD. Methods: This research included 3 phases. First, a draft respiratory symptom questionnaire (RSQ) was generated with input from subject matter experts. Second, the RSQ was refined through waves of cognitive debriefing interviews (n=49) with adult current smokers (Smokers) and former smokers who switched completely to ENDS for 6+ months (Switchers). Finally, Smokers (n = 202), Switchers (n=208), and Former Smokers (no tobacco use in 6+ months; smoked 10+ years; n=200) completed the RSQ for purposes of the quantitative psychometric evaluation. Results: The first two phases produced a 5-item RSQ that captured relevant symptoms and was interpreted as intended. During Phase 3 the RSQ was evaluated within the context of the graded response model (GRM). GRM assumptions of ordinality of response categories and unidimensionality were met. Test-retest reliability over a 1-week interval was good, and results provided support for convergent validity. Significant differences between Smokers’ and Former Smokers’ RSQ scores provide initial support for the RSQ’s ability to detect change in respiratory symptoms over time. In exploratory analyses, Switchers’ RSQ scores were significantly lower than Smokers’ scores, but not significantly different from Former Smokers’. Conclusion: Findings from this rigorous instrument development and validation study provide strong support for the reliability and validity of the RSQ as a measure of respiratory symptoms in adult current and former smokers, including those who may be at risk for developing COPD. The finding that RSQ scores among former smokers who had switched completely to ENDS were significantly lower than current smokers’ scores and similar to the scores of former smokers who had quit suggests that switching from cigarettes to ENDS may improve respiratory health.


Background
Cigarette smoke contains thousands of chemicals that are inhaled and deposited throughout the lungs, causing in ammation of the airways, impairing ciliary clearance and leading to oxidative injury [1][2][3].
Numerous validated self-report measures for assessing respiratory symptoms have been published.
However, these instruments were primarily developed and validated speci cally for use in patient populations with COPD, a severe in ammatory lung disease that causes air ow blockage and breathing-related problems. However, symptoms appear prior to formal disease diagnosis [1][2][3], making these measures less appropriate for smokers who may not (yet) have been diagnosed with a respiratory condition, but may exhibit some respiratory symptoms [4].
Questionnaires developed for diagnosed patients may not be appropriate for smokers without pulmonary disease. Many of these questionnaires include items that represent severe disease states that may or may not be applicable to smokers without clinical respiratory disease, e.g., "I am not at all con dent leaving my home because of my lung condition" (COPD Assessment Test [CAT] [5]), "I feel that I am not in control of my chest problems" (St. George's Respiratory Symptom Questionnaire [SGRQ][6]), "In the past 2 weeks, my cough has interrupted conversation or telephone calls" (Leicester Cough Questionnaire [LCQ] [7]), "on average, during the past week, how often did you feel: depressed (down) because of your breathing problems?" (Clinical COPD Questionnaire [CCQ]) [8]. When a respiratory symptom questionnaire's items are too "di cult" (i.e., re ect symptoms of severe pulmonary disease) for the population being assessed, this leads to reduced measurement precision due to a poor match between items and persons and can also result in a " oor effect," which inhibits measurement of true reduction in respiratory symptoms over time. Furthermore, scoring and interpretation of existing measures that were developed and validated speci cally for diseased populations cannot be assumed to extend to nondiseased populations. Thus, there is a need to develop and validate a respiratory symptom questionnaire that is appropriate for use with adult current and former smokers without pulmonary disease who may be experiencing mild to moderate pulmonary symptoms.
Electronic cigarettes (e-cigarettes) or electronic nicotine delivery systems (ENDS) do not burn tobacco, potentially reducing users' exposure to toxicants and carcinogens compared to combustible cigarettes [9].
The literature is mixed regarding the respiratory effects of ENDS: some reviews have concluded that switching from cigarette smoking to ENDS is associated with reduced levels of self-reported respiratory symptoms [10,11], whereas others have concluded that the existing evidence is not su cient to determine whether the adverse respiratory effects of ENDS are less than those of combustible cigarettes [12]. Contributing to the di culty in addressing this research question is the lack of an appropriate selfreport measure of respiratory symptoms. Importantly, a valid self-report measure of respiratory symptoms appropriate for smokers -including those without diagnosed respiratory disease -is needed in order to determine whether switching completely from combustible cigarettes to ENDS leads to a reduction in respiratory symptoms. Therefore, the primary objective of this multi-phase research was to develop and validate a respiratory symptom questionnaire that is appropriate for a general population of adult current and former smokers, including those without COPD. Questionnaire development involved three phases: (1) drafting a preliminary instrument; (2) re ning the draft instrument through multiple waves of semi-structured cognitive debrie ng interviews; and (3) quantitatively evaluating the psychometric functioning of the instrument. Instrument development and validation included anticipated end-users of the questionnaire, that is, adult current cigarette smokers ("Smokers") and adult former smokers who switched completely from cigarettes to ENDS ("Switchers") without COPD to ensure that the resulting questionnaire would be valid for use with these populations. This research also included formers smokers who had not used any tobacco product in 6+ months ("Former Smokers") as well as participants who reported a diagnosis of COPD or another respiratory symptom-related disease (e.g., asthma) for purposes of the psychometric evaluation.
Separate from the study objective of measure development and validation, these data also provided an opportunity for additional exploratory analyses examining how respiratory symptoms reported by

Methods
The study protocol and consent forms were approved by the Sterling Institutional Review Board (IRB ID: 8613). Development of the initial questionnaire (Phase 1) and cognitive testing methodology and results (Phase 2) are provided in Additional File 1. A summary of the Quantitative Psychometric Evaluation methods is provided here, and details are provided in Additional File 1.
The RSQ was administered as part of an online survey to Smokers, Switchers, and Former Smokers ("Time 1 Survey"). Approximately 1 week later, participants who completed the survey were re-contacted and asked to complete the RSQ a second time ("Time 2 Survey") for purposes of evaluating test-retest reliability. The nal study sample included N=610 participants, including n=202 Smokers, n=208 Switchers, and n=200 Former Smokers; 128 of these participants comprised the retest sample for evaluating test-retest reliability.

Analytic Plan
A 2-parameter IRT model appropriate for polytomous items, the graded response model (GRM) [26], was t in Mplus. The GRM's assumption of ordinality of response categories was empirically evaluated by tting a generalized partial credit model (GPCM) [34,35]. To evaluate the GRM's assumption of unidimensionality, a Monte Carlo simulation (parallel analysis) with 10,000 randomly generated datasets was conducted using a principal components analysis (PCA) extraction method. Test information and reliability were evaluated within the context of the GRM. Stability of the RSQ was evaluated by calculating an absolute intraclass correlation coe cient (ICC) between RSQ scores from the Time 1 and Time 2 Surveys among participants who completed the RSQ at both time points. As a proxy for ability to detect change, differences between Smokers and Former Smokers' scores were evaluated via an independent samples t-test. Convergent validity was evaluated by calculating a Spearman correlation between RSQ scores and self-reported global health status, and by evaluating RSQ scores by diagnostic status via independent samples t-tests. Similarly, using independent samples t-tests, exploratory analyses evaluated differences between Switchers' RSQ scores compared to Smokers' and Former Smokers' scores.

Results
The draft RSQ, generated from Phases 1 and 2 and subject to quantitative psychometric evaluation in Phase 3, is presented in Table 1. The RSQ is written below an 8 th grade reading level (Flesch-Kincaid Grade Level = 7.7). Observed category averages and category thresholds derived from the GPCM were ordered as expected (Additional File 1), providing empirical evidence that it requires a higher frequency of respiratory symptoms in order to endorse a more severe response option (e.g., "Every day" vs. "Most days").

Internal Structure
Results from the parallel analysis revealed one signi cant factor (eigenvalue = 3.14), providing support for unidimensionality of the RSQ (Additional File 1). This factor accounted for 62.7% of the variance.
Graded Response Model (GRM) After con rming unidimensionality and ordinality of response options, the GRM was t in Mplus using the weighted least-squares estimator and theta parameterization. Fit statistics for the initial GRM indicated poor t of the data to the model (chi-square = 285.086, df =5, p < .001; CFI = 0.963; RMSEA = 0.303; SRMR = 0.085). Due to the conceptual similarity between items, in conjunction with large observed modi cation indices, the model was re-run allowing for two correlated errors ("Morning Cough" and "Cough Frequently," and "Shortness of Breath" and "Easily Winded"). This model exhibited acceptable t (chi-square = 3.056, df = 3, p= 0.383; CFI = 1.000; RMSEA = 0.006; SRMR = 0.005).
The RSQ items' discrimination parameters were approximately 1 or higher, suggesting that the items were effectively differentiating between respondents with different levels of respiratory symptoms. Item 5 ("Wheezing") had the highest discrimination value (3.18), indicating that this item contributed the most in estimating respondent's total RSQ scores. The RSQ items' discrimination parameters are presented in Table 2, along with the di culty parameters generated from the GRM.  Table 1 for item content.

Test Information and Reliability
The TIF produced from the GRM (Figure 1) indicates that the RSQ most precisely estimates respiratory symptoms from thetas of 0.2 to 2.4. Converting the TIF to a reliability function (Figure 2), we observe that the RSQ exhibits a reliability of 0.80 or higher from thetas of -0.40 to 3.00.
Next, within the context of a classical test theory framework (1) test-retest reliability of the RSQ scores, (2) differences in RSQ scores between study groups as a proxy for the RSQ's ability to detect change, and (3) convergent validity were evaluated. Given the high correlation between the GRM-derived scoring and the raw scores (r =.94) and the complexities associated with using scoring from a 2-parameter IRT model, it is recommended that researchers use the raw RSQ item scores to calculate a composite (mean). Therefore, raw scores were used for the remaining analyses.
Test-Retest Reliability (Stability) Test-retest reliability among participants who did not report being sick with a cold or u-like symptoms at either timepoint (n=128 of the 145 participants in the retest sample) was good (absolute ICC = .89).

Exploratory Analysis: Association Between RSQ and COPD
Results from the linear regression with self-reported COPD and non-COPD diagnoses as the predictors (i.e., asthma, allergies, congestive heart failure, obesity) revealed that these diagnoses accounted for 29.2% of the variance in RSQ scores. After controlling for all other respiratory symptom-related diagnoses, COPD diagnosis was signi cantly related to RSQ scores (p <.001) and explained 20.6% of the variance. Controlling for age and years smoked did not have a material impact on results with a small increase in the overall variance accounted for by the model (31.1%) and COPD diagnosis was still signi cantly related to RSQ scores (p <.001) and explained 17.0% of the variance.
Finally, to directly evaluate the relationship of RSQ scores to self-reported COPD diagnosis, a logistic regression was run with COPD diagnosis as the outcome and RSQ scores as univariate predictors. The analysis showed that with every 1-unit increase in mean RSQ scores, the odds of COPD were 4.72 times greater (95% CI = 3.47 -6.40, p <.001).

Administration and Scoring
See Additional File 1.

Discussion
Accurately evaluating respiratory symptoms is likely to be useful in many contexts, and is particularly important for assessing the health consequences of tobacco product use and informing smoking cessation, as a reduction in respiratory symptoms has been shown to be associated with increased motivation to quit smoking [39][40][41][42]. Ameliorating respiratory symptoms may be a particularly motivating driver for smokers to stop smoking as respiratory symptoms may be experienced as tangible and relatable and re ect more immediate consequences of smoking. However, research evaluating respiratory symptoms among current and former smokers without diagnosed clinical pulmonary disease has been hindered due to measurement challenges, i.e., lack of a self-report questionnaire valid for assessing respiratory symptoms among non-diseased populations. Consequently, the RSQ lls an important gap in the existing toolkit of respiratory symptom patient reported outcome measures as appropriate for use with adult current and former smokers (Switchers) without a formal diagnosis of COPD.
RSQ development and validation was a rigorous multi-phase process conducted in accordance with best practices. The questionnaire was quantitatively evaluated through a psychometric evaluation leveraging both modern and classical test theory approaches. One bene t of leveraging IRT was that it allowed for evaluation of the RSQ's ability to precisely estimate respiratory symptoms among smokers with varying levels of symptomatology (i.e., estimate the amount of information that the RSQ provides at different thetas). The RSQ was found to have the highest measurement precision (information) and reliability when estimating respiratory symptoms for smokers with average to above average (0 to 3 SD above the mean) respiratory symptoms; importantly, these likely represent the range of thetas that are of interest when assessing respiratory symptoms among smokers with and without COPD. While the RSQ may less precisely estimate respiratory symptoms for smokers with very minimal respiratory symptomology, high measurement precision in this subpopulation is unlikely of particular importance.
Aside from the 1-week test-retest interval, the current study was not longitudinal in nature and did not permit evaluation of intra-individual change in respiratory symptoms as smokers abstained from smoking or switched completely from combustible cigarettes to ENDS over time. However, as research has established a decline in respiratory symptoms following abstention from smoking [38], observed differences between Smokers' and Former Smokers' RSQ scores provide initial support for the RSQ's ability to detect change [1]. Furthermore, exploratory analyses revealed that former smokers who switched to ENDS reported signi cantly lower respiratory symptoms than current smokers. Moreover, Switchers' level of self-reported respiratory symptoms was similar to symptoms of former smokers no longer using any tobacco products (Former Smokers). These results provide support for the potential of switching to ENDS to mitigate smokers' respiratory symptoms.
Exploratory regression analyses revealed that COPD diagnosis was signi cantly related to RSQ scores above and beyond the other respiratory symptom-related diagnoses (asthma, allergies, congestive heart failure, obesity), suggesting that the symptoms assessed have some speci city for COPD. Further, each 1unit increase in RSQ scores was associated with a nearly 5 times greater odds of COPD. While the RSQ is not intended to function as a screener or diagnostic tool for COPD, results from these analyses suggest that respiratory symptoms captured by the RSQ may be related to eventual pulmonary disease.
Importantly, the RSQ was developed in a way that supports "bring your own device" (BYOD) application. For example, a single item was administered on each screen to prevent the need for scrolling and the item stem was administered on each screen along with the item to remind participants of the timeframe of reference. Allowing for electronic administration on various devices may be a useful feature of the RSQ as self-report questionnaires may be increasingly administered across various electronic platforms.
Despite using soft target quotas to recruit a demographically diverse sample of adults for Phase 2, in the rst two waves of cognitive testing there were very few participants who identi ed as Hispanic/Latino, reported low educational attainment, and/or who met criteria for low health literacy. This was recti ed by conducting a third wave of interviews exclusively with participants representing these groups to verify that the questionnaire was understood and not differentially interpreted. In Phase 3, the sample was older than anticipated, and differences were observed between study groups with respect to age and years smoked. Although this nding is not particularly surprising (e.g., that on average, Switchers appeared somewhat younger than Smokers and Former Smokers), differences between groups on certain demographic or tobacco use variables can make it more di cult to draw conclusions, even when statistically controlling for these variables. Another limitation of the current study is that it relied exclusively on self-report; tobacco use was not biochemically veri ed, and diagnostic status was not veri ed through medical chart review. Finally, as this study was not longitudinal in nature it was not possible to draw causal inferences between tobacco product use behavior and change in respiratory symptoms over time. Future longitudinal research should evaluate intra-individual change in smokers' respiratory symptoms following complete switching to ENDS using the RSQ and in comparison to smokers who continue smoking.
[1] Of note, this between-group analysis could also provide support for known-groups validity of the RSQ.

Conclusions
The RSQ appears to be a valid and reliable self-report questionnaire for assessing respiratory symptoms related to pulmonary disease in populations that include smokers and former smokers who have not been diagnosed with COPD. The RSQ is understood and correctly interpreted by diverse samples of respondents with respect to age, ethnicity, and health literacy status. Results from a rigorous psychometric evaluation provide evidence of unidimensionality, high levels of measurement precision across a range of relevant thetas, convergent validity, and stability over a 1-week interval. Comparisons between groups of respondents differing in smoking history provide initial support for the RSQ's ability to detect change in respiratory symptoms over time following change in smoking status. Researchers should feel con dent using the RSQ to evaluate respiratory symptomology among adult current and former smokers, including those without diagnosed pulmonary disease.   Test information function (TIF) of the RSQ generated from the GRM This gure depicts the amount of information (precision) that the RSQ provides across different levels of respiratory symptoms (thetas, with mean = 0 and standard deviation =1).