Instruments to Measure E-Cigarette Related Constructs: A Systematic Review

doi:10.21203/rs.3.rs-496491/v1

Download PDF

Research Article

Instruments to Measure E-Cigarette Related Constructs: A Systematic Review

https://doi.org/10.21203/rs.3.rs-496491/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

Background Electronic cigarettes (e-cigarettes) are relatively new tobacco products that are attracting public attention due to their unique features, especially their many flavor options and their potential as an alternative to cigarettes. However, uncertainties remain regarding the determinants and consequences of e-cigarette use because current research on e-cigarettes is made more difficult due to the lack of valid and reliable instruments that measure e-cigarette related constructs. This systematic review therefore seeks to identify the instruments proposed by researchers in the field that are designed to assess various aspects of e-cigarette use or its related constructs and analyze the evidence presented regarding the psychometric properties of the identified instruments.

Methods This systematic review utilized six search engines: PubMed, Medline, CINAHL, PsycINFO, Web of Science, and EMBASE, to identify articles published in the peer-reviewed journals from inception to June 2018 that contained development or validation processes for these instruments.

Results Eighteen articles describing the development or validation of 20 unique instruments were identified. Beliefs, perception, motive, e-cigarette use, and dependence, were the most commonly assessed e-cigarette related constructs. The included studies reported either construct or criterion validity, with 14 studies reporting both. Most studies did not report the content validity; for reliability, most reported internal consistencies using Cronbach’s alpha, with 15 instruments reporting Cronbach’s alpha > 0.70 for the scale or its subscales.

Conclusions Twenty instruments with a reported development or validation process to measure e-cigarette related constructs are currently available for practitioners and researchers.

Health Economics & Outcomes Research

Health Policy

instrument

electronic cigarettes

survey

psychometrics

Electronic cigarette (e-cigarette) use is emerging as a major item on the public health agenda, attracting both greater attention from researchers, and intense scrutiny from the popular media. A significant increase in the prevalence of e-cigarette use among US adults has been reported since 2010 [1–3]; in 2019, 4.5% of adults in the US self-reported using e-cigarettes, of whom 36.9% identified as dual users.⁴ Among youth, steep rises in nicotine vaping and e-cigarette product use have resulted in an overall increase in the use of tobacco products. In 2020, 19.5% of high school students and 4.7% of middle school students used electronic nicotine delivery systems (ENDS) [4–5].

Developed to closely approximate the sensory experience of smoking combustible cigarettes, e-cigarettes produce an aerosol by heating a liquid containing a solvent (generally vegetable glycerin, propylene glycol, or a mixture of the two), one or more flavorings, and nicotine, although nicotine can be omitted if the user prefers [6]. E-cigarettes have gained considerable popularity among both youth and adults in recent years in spite of the dearth of research into the devices’ safety, effects, and efficacy [7]. Hence, while the research regarding the potential health effects of e-cigarette use is still in its infancy, researchers are beginning to try to understand people’s perceptions, reasons, and behaviors in order to better understand their use of e-cigarettes.

Despite reports that e-cigarettes emit substantially lower levels of carcinogens and thus represent a safer alternative to combustible cigarettes [8], young people who use e-cigarettes have shown increased risk of trying combustible cigarettes [9]. Moreover, with hundreds of e-cigarette brands already on the market, vaping products are evolving rapidly in terms of their mechanisms, engineering, design, and usability, all of which are aimed at boosting their appeal for curious youngsters and thus posing an additional concern as sales of these products continue to rise.

As research in this area increases, it is vital that studies that focus on e-cigarette use are able to utilize reliable and valid e-cigarette use measures when assessing their results. Major gaps remain in our knowledge of the effects and potential hazards posed by e-cigarettes that require extensive research, particularly when it comes to exploring major factors associated with e-cigarette use such as the motivators influencing the decision to use e-cigarettes and the consequences of e-cigarette use. However, there are some unique challenges for those developing new instruments to measure these constructs. E-cigarettes are relatively new and rapidly evolving products and thus, there is significant variability in the products currently on the market, including refillable options as well as pens, pods, and other configurations; the different patterns of e-cigarette use include experimentation, regular use, and dual use. However, presently, there is limited information regarding the validity of the various instruments developed to examine the multi-faceted issues involved and a clear need to evaluate the measurement properties of each of these instruments.

To date, there have been no systematic evaluations of the available evidence supporting the measurement properties of these validated instruments for e-cigarette use. Hence, the purpose of this systematic review is to review and synthesize validated survey instruments in the literature that are specifically designed to explore e-cigarette related constructs. In this context, e-cigarette-related constructs are defined as constructs chosen by researchers to explore and analyze the mechanisms of e-cigarette use or the phenomena associated with its use in survey studies. For example, constructs such as motivation, dependence, perceived harms and benefits, and dependency of e-cigarette use are often explored in e-cigarette survey research, thus, these are typical examples that we would expect to include as e-cigarette-related constructs. Specifically, this study aims to provide an overview of existing instruments developed for measuring e-cigarette-related constructs including the development or validation process and psychometric properties, thus bridging a serious gap in e-cigarette survey research. Our ultimate goal is to assist both researchers in the field and clinicians to make informed choices when selecting an appropriate instrument for the measurement of e-cigarette use.

This systematic review was registered with the PROSPERO international prospective register of systematic reviews and was conducted following the guidelines laid out in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) [10].

Search Strategy

A systematic search was conducted using six electronic databases: PubMed, Medline, Cumulative Index of Nursing and Allied Health Literature (CINAHL), PsycINFO, Web of Science, and Excerpta Medica dataBASE (EMBASE) from inception to June 2018. In addition, the works cited in the reviews and the references in the retrieved articles were screened. To broaden our search results, we entered our search terms using two categories, namely e-cigarette and instrument. The following search keywords or Medical Subject Headings (MeSH) were thus used: vaping device or vape or electronic cigarettes or e-cigarettes or e-liquid or electronic nicotine delivery systems [mesh] AND psychometrics [mesh] or questionnaires or surveys or surveys and questionnaires [mesh]. It is important to note that we did not specify search terms that would limit the constructs related to e-cigarette use. For example, we did not utilize terms such as motivation, belief, symptom, perceived harms or benefits, consequences, and behavior even though we were aware of instruments that measured some of these constructs as they are often used in e-cigarette survey research. By adopting this approach, we were able to explore the extent of the constructs that are assessed by validated instruments.

A filter was applied to retrieve articles published in English, but no language restriction was applied for the instruments. In addition, a snowballing technique was used to suggest additional searches: if the article referred to earlier articles that described the process of development or validation of the instruments, we also retrieved those articles and checked their eligibility for inclusion in this review.

Inclusion and Exclusion Criteria

Predetermined inclusion criteria were applied to select relevant studies, which included (1) Articles reporting the development and/or validation process for survey instruments designed to measure electronic cigarette related constructs (e.g., motivation, dependence, perceived harms and benefits, consequences, and behavior); (2) Full text articles published in peer-reviewed research journals; and (3) Articles published in English, where the instruments were translated into English for the purpose of the analysis. Likewise, the study specified the following exclusion criteria: (1) Studies that used non-empirical methods; (2) Studies that did not use specified measures; (3) Single-item instruments; (4) Survey instruments that did not report a validation or development process; and (5) Instrument designed for use in laboratory settings.

Selection Process

Applying the aforementioned inclusion and exclusion criteria, two authors screened relevant titles and abstracts independently. Studies that met the criteria were accessed and independently reviewed multiple times by all the authors and discrepancies were reconciled through consensus discussions.

Data Extraction

To provide an overview of the instruments and the psychometric properties of each, the following coding schemes were used: (1) Basic information on the instruments, including the name of the instrument, the name of the first author, the constructs that the instrument is designed to assess, the country where the study took place, the theoretical background of the instrument, the mode of administration, the completion time, and the response options (Table 1); and (2) The psychometric properties of the instruments, including the constructs, sub-constructs, various types of reliability reported (e.g., internal consistency, test-retest reliability) and validity (construct, content, and criterion validity) tested (Table 2).

Search Results

In total, the search yielded 1,158 articles. After two researchers had independently reviewed the titles and abstracts of all articles, 63 were selected to undergo a full text examination, after which the same two researchers independently reviewed their full texts. Of the 63 articles, 41 were excluded as they did not report specific information about the development or validation process utilized. After the full text review based on the eligibility criteria, 18 studies were found to be suitable for inclusion in the current study (see Fig. 1 PRISMA flow chart). One study tested the validity of three instruments that measure the same construct, namely e-cigarette dependence [11]; one study reported the validity of both the long and short versions of the instrument, and revised the youth e-cigarette outcome expectancies respectively [12], and two studies conducted a validity test on the same instrument [11–13]. Thus, a total of 20 instruments from 18 different studies were analyzed for the current study.

Overview of Instruments

Tables 1 and 2 provide an overview of the characteristics of the instruments presented in the included articles. This section presents an overview of the settings of these studies, the age ranges of their participants, the theoretical frameworks utilized, the modes of administration and durations of the tests, the number of test items, and the response options.

Country and Language

Of the 20 instruments, 75% (15/20) studies tested validation or reliability test in the US; the others were conducted in Canada, Australia, and Hungary. All but one instrument was developed in English. The exception was an instrument originally developed in Hungarian and later translated into English [14]. Although the items of this instrument are available in English, the instrument itself has not been validated using a cross-sectional translational process.

Participants

Regarding the ages of the participants, 80.0% (16/20) of the instruments were designed for use with participants aged 18 years or above; the remaining 22.2% (4/20) were for younger participants who were under 18 years of age. Among the instruments validated for participants aged 18 or above, three instruments were specifically targeted at young adults (18 to 25 years old or college students). One instrument was validated based on its use with hospitalized patients [15].

Administration, Number of Items, and Responses

There were some variability and ambiguity with regard to the modes of administration of the instruments. Although the majority of the tests were administered “online” (70%, 14/20), 30% (6/20) of the studies did not report the mode of administration. The vast majority of the studies (85%, 17/20) did not report the completion time for their instruments; the remaining 15% (3/20) specified either the completion time of the relevant instrument (n = 1) or the completion time of the study (n = 2). However, the number of items in each instrument can provide a rough estimate of the completion time required. There was a considerable variability with respect to the number of items in the instruments, which ranged from 2 to 55 with a mean of 15.81 and a median of 10. Finally, the majority of the instruments (65%,13/20) utilized Likert-type response options, varying from 1–2 to 1–10, a further 20.0% (4/20) did not report the response options, 10% (2/20) had mixed response options, and 5% (1/20) had True/False response options.

Theoretical Background

The majority of the studies (72.2%,13/18) did not describe the theoretical background of their instrumentation. The other 27.8% (5/18) did present the theoretical framework underpinning their instruments by including a discussion of the relevant motivation theories, theories of planned behaviors, social learning theory, and/or expectancy theory.

Constructs

The 20 instruments identified a total of 14 different constructs reflecting the multiple constructs explored in e-cigarette survey research (Table 1). General beliefs and perceptions were identified as the most commonly explored construct, with individual studies specifically including outcome expectancies [14], sensory vaping expectancies [16], and the risks and benefits of e-cigarettes [18–19].

Twelve instruments sought to assess beliefs and perceptions about e-cigarettes, specifically comparing the beliefs or perceptions to the beliefs or perceptions about cigarette smoking using constructs such as comparative beliefs of e-cigarette use [20], e-cigarette expectancies compared to cigarette smoking [21], and perceived harms compared with cigarettes [22]. One instrument was designed to assess the perceived harms and social norms of both e-cigarette and smokeless tobacco in a single instrument [23], while another was specifically developed to assess the expectancies of simultaneous e-cigarette and alcohol use [21].

Motivation for e-cigarette experimentation and susceptibility to future use were identified as constructs that were explored by four instruments. These instruments assess motivation or likelihood of using e-cigarettes specifically among non-cigarettes users [14, 24, 25]. Among these instruments, one assessed susceptibility to four different tobacco products, namely e-cigarettes, cigarettes, cigars, and hookahs [25]. Looking at e-cigarette use exclusively, one instrument assessed habitual e-cigarette use [26], but no studies that specifically reported the development or validation of instruments assessing e-cigarette use were identified.

The next most commonly assessed constructs were e-cigarette craving and e-cigarette dependence. One instrument assessed e-cigarette craving based on three sub-constructs, namely desire, intention, and positive outcome.²⁷ For smoking dependence, four instruments were identified [11, 13, 28]. Of these, three instruments had a one single construct, but one instrument (e-WISDM) has 37 items consisting of 11 sub-constructs: affiliative attachment, affective enhancement, automaticity, loss of control, cognitive enhancement, craving, cue exposure, social/environmental goals, taste, tolerance, and weight control.

In terms of the number of sub-constructs within each instrument, most had several sub-constructs (range = 1 to 11; Mean = 3.29, Mode = 1). Interestingly, six instruments had no sub-constructs and only a single domain, either e-cigarette dependence [28], susceptibility to future e-cigarette use [11, 13, 24], or habitual e-cigarette use [26].

Psychometric Properties of Instruments

In the current study, we examined the psychometric properties of the various instruments included in the 20 relevant instruments identified (Tables 1 & 3).

Reliability

Internal consistency based on Cronbach’s alpha was the only reliability test used in the identified studies. None reported an item analysis or the test and retest reliability. Among the identified studies, most instruments (95%, 19/20) reported internal consistencies, and one study did not report reliability (Chaffee et al., 2015). Among the instruments that reported internal consistencies, most reported values of Cronbach’s alpha ≥ .70 (15/20), although two borderline values of .67 [15] and .68 [14] were found. Two studies reported Cronbach’s alpha < .67 [11, 28]. Most studies did report domain-specific Cronbach’s alpha scores.

Validity

Content validity was reported in only one instrument. This study also reported the process used for the participant interviews as part of the process of the instrument development 11–13].

Construct validity was reported in 95% (19/20) of the all instruments included. Most studies tested the dimensionality to support construct validity using either confirmatory factor analysis (CFA) or exploratory factor analysis (EFA). Only seven instruments were tested by both EFA and CFA or principal factor analysis (PFA) and CFA as analytic methods [12, 16, 20, 21, 26, 27].

In addition, construct validity was supported by testing the correlation with existing e-cigarette use-related measures, such as dependence measures [11, 13, 15, 18, 26, 27]. Two studies reported testing the convergence and divergence validity [12, 27]. A number of studies reported measurement invariance in testing the instrument [17, 26], as well as construct validity in order to test differences in e-cigarette use status [18, 19].

Criterion validity was reported by 85% (17/20) of the included articles. Most studies reported either the concurrent validity or predictive validity. For concurrent validity, the associations with existing measures such as e-cigarette use or e-cigarette experimentation were tested and reported acceptable criterion validity [12, 14, 15, 16, 17, 19, 20, 22, 23]. For predictive validity, four instruments were tested in two studies to explore whether the constructs of the instrument measures would predict positive future e-cigarette use [11, 24].

This paper is the first systematic review of existing instruments on e-cigarette related constructs. A total of 18 studies were identified that focus on the development or validation of 20 instruments. This study provides an overview of these instruments as well as development process, theoretical framework, target population, and psychometric properties. This review can serve as a useful guide for healthcare professionals and researchers seeking to conduct assessments or conduct research into the phenomenon of e-cigarette use.

In this review, we identified several e-cigarette related constructs in existing instruments. Beliefs or perceptions about e-cigarettes were considered the most commonly studied determinants of current e-cigarette use based on the validated instruments. To explore beliefs or perceptions, constructs including outcome expectancies, sensory expectancies, and perceived risks and benefits were explored. These constructs were supported by motivation theories, social learning theory, and the theory of planned behavior. In addition, beliefs or perceptions about the relative merits of e-cigarettes and smoking conventional cigarettes were another commonly explored construct. In terms of the motivations for e-cigarette experimentation and susceptibility to future use were constructs explored, although habitual e-cigarette use was the only construct used to assess current e-cigarette use. The consequences or symptoms related to e-cigarette use were also explored with the constructs of e-cigarette craving and dependence. Only a few studies included in this review provided a theoretical background of the instrumentation and did not clearly present the conceptual framework or definition. This may be related to potential issues of clarity of the constructs that each instrument measures. The constructs that each instrument intends to measure may not be specific enough without a theoretical guide [29]. In addition, most studies only described the validation process but did not provide detailed steps of the development process of the instruments, which also limits the clarify of the constructs that each instrument intended to measure.

Regarding reliability, most studies reported acceptable internal consistencies. However, reliability was supported by only one type of reliability and tested the internal consistencies with a single method. This can be a potential threat of internal consistency [30]. It is suggested that three broad types of reliability need to be assured, including (1) reliability from administering parallel forms of instruments (alternate-form coefficients), (2) reliability from administering the same instrument on separate times (test-retest), and (3) reliability based on total scores or subsets of items within a single test (internal consistency coefficient) [31]. It is important to test multiple types of reliability by multiple methods, such as “test-retest” or “item-analysis”. In this way, any systematic error or variations of instruments can be prevented and the generalizability of the use of instruments can be improved.

For validity, only a limited number of studies conducted both EFA and CFA during their analysis, which again limits the construct validity, and few tested either content validity or criterion validity. Part of the reason why content validity was not tested in most studies may be related to the historical aspects of the development process commonly used for e-cigarette-related measures, most of which are based on existing instruments originally developed to assess cigarette smoking related constructs. As these measures have already been extensively validated, the various authors have simply adapted these for e-cigarette specific constructs, not considering this to be a necessary step in the development process. However, it is actually important to capture the unique aspects of e-cigarettes, which are in many ways very different from cigarettes [29]. Moreover, in terms of the criterion validity, although most studies did provide a test of the criterion validity, only a limited number also tested the predictive validity. This seriously limits the validity of the majority of the existing e-cigarette related instruments [32].

Limitations

As always with studies of this nature, there is a risk that relevant articles may be missed even though we have used a range of different techniques to systematically search for articles; there is also the potential for errors to occur in the review and coding process. To minimize these errors and ensure the reliability of the coding process, two researchers independently coded the articles, and three researchers double checked the accuracy of the coding multiple times. Where discrepancies were identified, three authors reviewed the articles together and came to a consensus. Further review from other researchers would have been considered to deal with any unresolved issues had any such occurred, but this was not found to be necessary.

Moreover, there is also a possibility that not all studies were able to report the full details of their instrumentation or validation process due to limited space in peer-reviewed journals. It is thus possible that the authors were not able to gather sufficient information for each measure in this review from the published reports.

Recommendations for Practice

Twenty unique instruments assessing the constructs related to e-cigarette use in population studies were identified in this study. Our findings suggest that practitioners should first consider choosing instruments based on the constructs that they are most interested in, depending on the purpose of the assessment. For example, if practitioners are interested in the reasons for e-cigarette use, it would be most appropriate to select an instrument that assesses various types of beliefs or perceptions, while to assess non-users’ motivation or susceptibility to future use, they can choose from three different measures: motivations for e-cigarette experimentation [14]; a susceptibility scale [24], and susceptibility to four product classes (e-cigarettes, cigars, hookah and cigarettes) [25]. If they are interested in current habitual use, one instrument has been specifically developed to study this, the Self-Report Habit Index (SRHI) [26], and if the practitioners need to assess the symptoms of current e-cigarette users, a number of measures are available to assess craving or dependence, namely the Questionnaire of Vaping Craving (QVC) [27], the Penn State Electronic Cigarette Dependence Index (PS-ECDI) [11, 13], the E-cigarette Fagerström Test of Cigarette Dependence (e-FTCD) [11], the E-cigarette Wisconsin Inventory of Smoking Dependence Motives (e-WISDM) [11], and the Fagerström Test for Nicotine Dependence applied to Vaping (FTND-V) [28].

After narrowing down the broad categories of constructs depending on the purpose of the assessment, practitioners should consider the age of their target population and make sure that the instrument has been validated for this user group. Among those instruments, those that report a value of Cronbach’s alpha higher than .70 and support both construct and criterion validity should be preferred, although the number of items should be considered to determine the feasibility of their use in clinical settings.

Recommendations for Future Research

Future work on survey instruments measuring e-cigarette-related constructs that take into account content validity and criterion validity will be necessary if we are to establish a stronger evidence base for e-cigarette research. Currently, most studies report only construct validity, with few also reporting the content validity or predictive validity. It is important that multiple types of reliability in addition to Cronbach’s alpha alone need to be explored and supported when a new instrument is developed. In addition, for the instruments reported low reliability coefficients, item modification is needed to ensure a desirable internal consistency [29, 31].

There is a critical need to develop a reliable and valid instrument with which to assess e-cigarette-related constructs of diverse populations. Currently, only a limited number of instruments assessing e-cigarette related constructs have been validated for adolescent populations that have been specifically designed to assess the perceived risks and benefits of e-cigarettes, their perceived harms compared with cigarettes, and susceptibility to future use. However, among the available instruments for this age group, only one study was reported as having an acceptable internal consistency. Given the dramatic increase in the prevalence of e-cigarette use, there is clearly a need to develop and validate instruments targeted specifically at adolescents and young adults, particularly given that these are the people most likely to be using e-cigarettes. The availability of such an instrument will enhance the rigor of research on e-cigarettes and help us understand the rapid growth in the popularity of e-cigarettes among this population. Furthermore, instruments need to be validated in diverse clinical settings, and there is also a need for validated universal instruments that can be administered across age groups to help us understand the impact of differences in the various associated factors, the characteristics of the different types of e-cigarettes, and the symptoms across both clinical and non-clinical groups.

Providing details of the instrument administration is also important for researchers and practitioners. Currently, only a few articles provide basic information on the characteristics of their instruments, such as the mode and response type. It is important for researchers to report detailed information, such as whether the instrument is provided online or on paper, the various response options, and the number of items to make the instruments more suitable for use by practitioners and other researchers alike.

This systematic review provides a critical appraisal and repository of the instruments measuring e-cigarette-related constructs in the current literature. It serves as a user-friendly guide to help researchers select the most appropriate instrument to suit their needs based on the constructs, target population, psychometric properties, and number of items, all of which can help develop a more accurate understanding of e-cigarette related phenomena for practitioners. For future studies, researchers need to expand the validation of the existing instruments to include more diverse populations, and develop new instruments that are specific to the unique aspects of e-cigarettes. The development of instruments capable of assessing different aspects of e-cigarette use with a strong theoretical background and validation process will be essential to support efforts to develop effective e-cigarette use prevention and cessation programs.

E-cigarettes: Electronic cigarettes

CINAHL: Cumulative Index of Nursing and Allied Health Literature

EMBASE: Excerpta Medica dataBASE

ENDS: electronic nicotine delivery systems

PRISAMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses

MeSH: Medical Subject Headings

CFA: Confirmatory Factor Analysis

EFA: Exploratory factor analysis

PFA: principal factor analysis

SRHI: Self-Report Habit Index

QVC: Questionnaire of Vaping Craving

e-FTCD: E-cigarette Fagerström Test of Cigarette Dependence

e-WISDM: E-cigarette Wisconsin Inventory of Smoking Dependence Motives

FTND-V: Fagerström Test for Nicotine Dependence applied to Vaping

Ethics approval and consent to participate: Not applicable

Consent for publication: Not applicable

Availability of data and materials: The data that support the fin available used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Competing interests: The authors declare that they have no competing interests.

Funding: This study did not receive any grant from funding agencies in the public, commercial, or non-for-profit sectors.

Authors’ contributions: EP developed the systematic review protocol, conducted the review and wrote the first draft of the manuscript. MG and TC contributed to search articles, coding, analysis, interpretation, and writing. MG, CL, and YC contributed to the analysis and editing. All authors read and approved the final manuscript.

Acknowledgements: None

McMillen RC, Gottlieb MA, Shaefer RM, Winickoff JP, Klein JD. Trends in electronic cigarette use among U.S. adults: use is increasing in both smokers and nonsmokers. Nicotine & Tobacco Research. 2015;17: 1195–202.
Schoenborn CA, Clarke TC. QuickStats: percentage of adults who ever used an e-cigarette and percentage who currently use e-cigarettes, by age group—National Health Interview Survey, United States, 2016. MMWR Morb Mortal Wkly Rep. 2017;66(33):892.
Schoenborn CA, Gindi RM. Electronic cigarette use among adults: United States, 2014. NCHS Data Brief. 2015;(217):1–8.
Centers for Disease Control and Prevention (CDC). About electronic cigarettes (E-Cigarettes). https://www.cdc.gov/tobacco/basic_information/e-cigarettes/about-e-cigarettes.html. Accessed March 12, 2021.
Cullen KA, Ambrose BK, Gentzke AS, Apelberg BJ, Jamal A, King BA. Notes from the field: increase in use of electronic cigarettes and any tobacco product among middle and high school students — United States, 2011–2018. MMWR Morbid Mortal Wkly Rep. 2018;67(45):1276–1277.
Dinakar C, O’Connor GT. The health effects of electronic cigarettes. N Engl J Med. 2016;375(14):1372–81.
Etter JF, Bullen C. Electronic cigarettes: users profile, utilization, satisfaction and perceived efficacy. Addiction. 2011;106(11):2017–28.
Goniewicz ML, Knysak J, Gawron M, Kosmider L, Sobczak A, Kurek J, Prokopowicz A, Jablonska-Czapla M, Rosik-Dulewska C, Havel C, Jacob P. Levels of selected carcinogens and toxicants in vapour from electronic cigarettes. Tobacco Control. 2014;23(2):133–9.
Stratton K, Kwan LY, Eaton DL. Public health consequences of e-cigarettes: consensus study report. National Academies Press. http://www.cascobaycan.org/cbcwp/wp-content/uploads/2018/02/Public-Health-Consequences-of-E-Cigarettes.pdf. Published 2018. Accessed February 1, 2021.
Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4(1):1–9.
Piper ME, Baker TB, Benowitz NL, Smith SS, Jorenby DE. E-cigarette dependence measures in dual users: reliability and relations with dependence criteria and e-cigarette cessation. Nicotine & Tobacco Research. 2020;22(5):756–763. doi:10.1093/ntr/ntz040
Pokhrel P, Lam TH, Pagano I, Kawamoto CT, Herzog TA. Young adult e-cigarette use outcome expectancies: validity of a revised scale and a short scale. Addictive Behaviors. 2018;78:193–199. doi:10.1016/j.addbeh.2017.11.019
Foulds J, Veldheer S, Yingst J, et al. Development of a questionnaire for assessing dependence on electronic cigarettes among a large sample of ex-smoking e-cigarette users. Nicotine & Tobacco Research. 2015;17(2):186–192. doi:10.1093/ntr/ntu204
Pénzes M, Foley KL, Balázs P, Urbán R. Intention to experiment with e-cigarettes in a cross-sectional survey of undergraduate university students in Hungary. Substance Use & Misuse. 2016;51(9):1083–1092.
Hendricks PS, Cases MG, Thorne CB, et al. Hospitalized smokers’ expectancies for electronic cigarettes versus tobacco cigarettes. Addictive Behaviors. 2015;41:106–111. doi:10.1016/j.addbeh.2014.09.031
Morean ME, Krishnan-Sarin S, Sussman S, et al. Development and psychometric validation of a novel measure of sensory expectancies associated with e-cigarette use. Addictive Behaviors. 2019;91:208–215. doi:10.1016/j.addbeh.2018.08.031
Morean ME, L’Insalata A. The Short Form Vaping Consequences Questionnaire: psychometric properties of a measure of vaping expectancies for use with adult e-cigarette users. Nicotine & Tobacco Research. 2017;19(2):215–221. doi:10.1093/ntr/ntw205
Copeland AL, Peltier MR, Waldo K. Perceived risk and benefits of e-cigarette use among college students. Addictive Behaviors. 2017; 71:31–37. doi:10.1016/j.addbeh.2017.02.005
Chaffee BW, Gansky SA, Halpern-Felsher B, Couch ET, Essex G, Walsh MM. Conditional risk assessment of adolescents’ electronic cigarette perceptions. American Journal of Health Behavior. 2015;39(3):421–432. doi:10.5993/AJHB.39.3.14
Hershberger AR, Karyadi KA, VanderVeen JD, Cyders MA. Beliefs about the direct comparison of e-cigarettes and cigarettes. Substance Use & Misuse. 2017;52(8):982–991. doi:10.1080/10826084.2016.1268628
Hershberger AR, Karyadi KA, VanderVeen JD, Cyders MA. Combined expectancies of alcohol and e-cigarette use relate to higher alcohol use. Addictive Behaviors. 2016;52:13–21. doi:10.1016/j.addbeh.2015.08.005
Persoskie A, O’Brien EK, Nguyen AB, Tworek C. Measuring youth beliefs about the harms of e-cigarettes and smokeless tobacco compared to cigarettes. Addictive Behaviors. 2017;70:7–13. doi:10.1016/j.addbeh.2017.01.033
Waters EA, Mueller-Luckey G, Levault K, Jenkins WD. Perceived harms and social norms in the use of electronic cigarettes and smokeless tobacco. Journal of Health Communication. 2017;22(6):497–505. doi:10.1080/10810730.2017.1311972
Cole AG, Kennedy RD, Chaurasia A, Leatherdale ST. Exploring the predictive validity of the susceptibility to smoking construct for tobacco cigarettes, alternative tobacco products, and e-cigarettes. Nicotine & Tobacco Research. 2019;21(3):323–330. doi:10.1093/ntr/ntx265
Carey FR, Wilkinson AV, Harrell MB, Cohn EA, Perry CL. Measurement and predictive value of susceptibility to cigarettes, e-cigarettes, cigars, and hookah among Texas adolescents. Addictive Behaviors Reports. 2018;8:95–101. doi:10.1016/j.abrep.2018.08.005
Morean ME, DeMartini KS, Foster D, et al. The Self-Report Habit Index: Assessing habitual marijuana, alcohol, e-cigarette, and cigarette use. Drug & Alcohol Dependence. 2018;186:207–214. doi:10.1016/j.drugalcdep.2018.01.014
Dowd AN, Motschman CA, Tiffany ST. Development and Validation of the Questionnaire of Vaping Craving. Nicotine & Tobacco Research. 2019;21(1):63–70. doi:10.1093/ntr/nty046
Browne M, Todd DG. Then and now: Consumption and dependence in e-cigarette users who formerly smoked cigarettes. Addictive Behaviors. 2018;76:113–121. doi:10.1016/j.addbeh.2017.07.034
DeVellis, Robert F. Scale Development: Theory and Applications. Thousand Oaks, Calif: SAGE, 20
Nunnally JC, Bernstein IH. Psychometric Theory. 3rd ed. New York: McGraw-Hill; 1994
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education (Eds.). (2014). Standards for educational and psychological testing. American Educational Research Association.
Boateng GO, Neilands TB, Frongillo EA, Melgar-Quiñonez HR Young SL. Best practices for developing and validating scales for health, social, and behavioral research: a primer. Front. Public Health 2018; 6:149

Table 1

Overview of Instruments

Broad categories of constructs	Constructs	Theory	Instruments	Reference articles	Target age	Reliability (Cronbach’s alpha>0.70)	Validity			# of items
Broad categories of constructs	Constructs	Theory	Instruments	Reference articles	Target age	Reliability (Cronbach’s alpha>0.70)	Content	Construct	Criterion	# of items
Beliefs/ perception	Outcome expectancies	Motivation theories	Revised youth e-cigarette outcome expectancies	Pokhrel et al., 2018	18–25	O	-	O	O	43
	Outcome expectancies	Motivation theories	Revised youth EC outcome expectancies (short)	Pokhrel et al., 2018	18–25	O	-	O	O	12
	Sensory vaping expectancies	-	Sensory E-cigarette Expectancies Scale (SEES)	Morean et al., 2019	≥18	O	-	O	O	9
	Sensory vaping expectancies	Social learning theory	Short Form Vaping Consequences Questionnaire (S-VCQ)	Morean & L’Insalata, 2017	≥18	O	-	O	O	21
	Perceived risk and Benefits of e-cigarettes	-	Perceived Risk and Benefits of E-cigarette use (RABE)	Copeland et al., 2017	≥18	O	-	O	-	30
	Perceived risk and Benefits of e-cigarettes	-	Conditional Risk Assessment of Electronic Cigarette Perceptions	Chaffee et al., 2015	13–18	-	-	O	O	19
	Comparative beliefs of e-cig use and cigarette smoking	Theory of planned behavior (TPB)	Comparing E-Cigarette and Cigarettes Questionnaire (CEAC)	Hershberger et al., 2017	≥18	O	-	O	O	10
	E-cigarette expectancies compared to cigarette smoking	-	E-cigarette-specific Brief Smoking Consequences Questionnaire-Adult (BSCQ-A)	Hendricks et al., 2015	≥19	O (>.67)	-	O	O	25
	Perceived harms compared with cigarettes	-	Direct and indirect measures of perceived harm of e-cigarettes and smokeless tobacco compared with smokeless tobacco	Persoskie et al., 2017	12–17	-	-	-	O	2
	Perceived harms and social norms in the use of e-cigarettes and smokeless tobacco	TPB & integrated model of behavior change	Perceived harms and social norms in the use of electronic cigarettes	Waters et al., 2017	≥18	O	-	O	O	15
	Expectancies of combined e-cigarette and alcohol use	-	Nicotine and Other Substance Interaction Expectancy Questionnaire E-cig Revised version (NOSIE-ER)	Hershberger et al., 2016	≥21	O	-	O	O	8
Motive	Motivations for e-cigarette experimentation	-	Motivations for e-cigarette experimentation**	Penzes et al., 2016	≥18 (non-users; young adults)	O (>.68)	-	O	O	27
Use	Susceptibility to future use	-	Susceptibility scale	Cole et al., 2019	14–17	-	-	-	O	3
	Susceptibility to future use	-	Susceptibility to four product classes (e-cigarettes, cigars, hookah and cigarettes)	Carey et al., 2018	10–18	O	-	O	O	3
	Habitual e-cigarette use	-	Self-report Habit Index (SRHI)	Morean et al., 2018	≥18	O	-	O	O	12
Symptoms	E-cigarette craving	-	Questionnaire of Vaping Craving (QVC)	Dowd et al., 2019	≥18	O	-	O	-	10
	E-cigarette dependence	-	Penn State Electronic Cigarette Dependence Index (PS-ECDI).	Piper et al., 2019; Foulds et al., 2015	≥18	O	O	O	O	10
		-	E-cigarette Fagerström Test of Cigarette Dependence (e-FTCD)	Piper et al., 2019	≥18	-	-	O	-	6
		-	E-cigarette Wisconsin Inventory of Smoking Dependence Motives (e-WISDM)	Piper et al., 2019	≥18	O	-	O	O	37
		-	Fagerström Test for Nicotine Dependence applied to Vaping (FTND-V)	Browne & Todd, 2018	≥17	-	-	O	-	9
Notes. O= indicates the studies addressed reported reliability or validity. Abbreviations. TPB=theory of planned behavior

Table 2

Basic Information about the Instruments.

Instrument

Author & year

Country

Age range

Theory

Mode of administration

Completion

time

Final number

of items

Response

options

Sensory E-cigarette Expectancies Scale (SEES)

Morean et al., 2019

USA

≥18

Online

NR*

1-5 Likert

E-cigarette Fagerström Test of Cigarette Dependence (e-FTCD)

Piper et al., 2019

USA

≥18

E-cigarette Wisconsin Inventory of Smoking Dependence Motives (e-WISDM)

Piper et al., 2019

USA

≥18

Penn State Electronic Cigarette Dependence Index (PS-ECDI).

Piper et al., 2019; Foulds et al., 2015

USA

≥18

Online

Mixed

Questionnaire of Vaping Craving (QVC)

Dowd et al., 2019

USA

≥18

Online

>10 min.

1-7 Likert

Susceptibility to future use

Cole et al., 2019

Canada

14–17**

Revised youth EC outcome expectancies

Pokhrel et al., 2018

USA

18–25

Motivation Theories

1-10 Likert

Revised youth EC outcome expectancies (Short version)

Pokhrel et al., 2018

USA

18–25

Motivation Theories

1-10 Likert

Self-report habit index (SRHI e-cigarette)

Morean et al., 2018

USA

≥18

Online

1-2 Likert

Susceptibility to four product classes (EC, cigars, hookah and cigarettes)

Carey et al., 2018

USA

10–18

Online

NR*

1-4 Likert

Fagerström Test for Nicotine Dependence applied to Vaping

(FTND-V)

Browne & Todd, 2018

Australia

≥17

Online

Mixed

Comparing EC and Cigarettes Questionnaire(CEAC)

Hershberger et al., 2017

USA

≥18

Theory of Planned Behavior

Online

1-5 Likert

Perceived harms and social norms in the use of e-cigarettes and smokeless tobacco

Waters et al., 2017

USA

≥18

Theory of planned behavior & integrated model of behavior

change

Online

1-9 Likert

Perceived harm of EC and smokeless tobacco with cigarettes

Persoskie et al., 2017

USA

12–17

1-4 Likert

Risk and Benefits of E-cigarettes (RABE)

Copeland et al., 2017

USA

≥18

Online

1-7 Likert

Short Form Vaping Consequences Questionnaire (S-VCQ)

Morean & L’Insalata, 2017

USA

≥18

Social learning theory

Online

Motivations of intention to try EC among non-EC users

Penzes et al., 2016

Hungary

≥18

Online

1-4 Likert

Nicotine and Other Substance

Interaction Expectancies-E-cig Revised version (NOSIE-ER)

Hershberger et al.,

2016

USA

≥21

Online

True/False

e-cigarette-specific Brief Smoking Consequences Questionnaire (e-cigarette-specific BSCQ-A)

Hendricks et al., 2015

USA

≥19

Expectancy theory

Online

0-9 Likert

Conditional Risk Assessment

Chaffee et al., 2015

USA

13–18

Social learning theory & health belief model

Online

Notes. *Total study time are reported but instrument-specific time is not reported. ** Grade 9 – 12 is reported;

Abbreviation: NR = Not reported; USA=United States of America

Table 3

Psychometric Properties of the Included Instruments.

Author & year

Name of instrument

Construct

Number of

sub constructs/sub domains

Reliability

Validity

Internal consistency (subscales)

Test-retest

reliability

Construct validity

Content validity

Criterion validity

Morean et al., 2019

Sensory E-cigarette Expectancies Scale (SEES)

Sensory vaping expectancies

3 (taste/smell, pleasure/satisfaction, vapor cloud

Production)

Subscales: .85 - .90

Test Dimensionality (EFA & CFA);

Measurement invariance;

Convergence with dependence

Concurrent validity (associated with vaping frequency and habitual e-cigarette use)

Piper et al., 2019

E-cigarette Fagerström Test of Cigarette Dependence (e-FTCD)

E-cigarette dependence measure

1 (dependence)

.51

Test Dimensionality (CFA); Correlation with e-cigarette use measures and; e-cigarette dependence measures

Predictive validity (associated with e-cigarette use)

Piper et al., 2019

; Foulds et al., 2015

E-cigarette Wisconsin Inventory of Smoking Dependence Motives (e-WISDM)

E-cigarette dependence measure

11 (affiliative attachment, affective enhancement, automaticity, loss of control, cognitive enhancement, craving, cue exposure, social/environmental goals, taste, tolerance, weight control)

.80

Test Dimensionality (CFA); Correlation with e-cigarette use measures and; e-cigarette dependence measures

Predictive validity (associated with e-cigarette use)

Penn State Electronic Cigarette Dependence Index (PS-ECDI).

E-cigarette dependence measure

1 (dependence)

.74

Test Dimensionality (CFA); Correlation with e-cigarette use measures and; e-cigarette dependence measures; Test differences by nicotine concentration of products used

Participant Interviews

Predictive validity (associated with e-cigarette use)

Dowd et al., 2019

Questionnaire of Vaping Craving (QVC)

Craving for e-cigarette measure

3 (desire, intention, positive outcome)

.96

Test Dimensionality (PFA & CFA); Correlation with e-cigarette use, and cigarette use measures; Convergent and discriminant validity

Cole et al., 2019

Susceptibility scale

Susceptibility to future tobacco products or e-cigarettes use

1 (susceptibility)

Predictive validity (associated with e-cigarette use)

Pokhrel et al., 2018

Revised youth e-cigarette outcome expectancies

E-cigarette use outcome expectancies

8 (social enhancement, affect regulation, positive sensory experience, negative health consequences, addiction concern, negative sensory experience, positive “smoking” experience, negative social experience)

Subscales: .77 - .94

Test Dimensionality (EFA & CFA); Convergence and divergence validity

Concurrent validity (associated with e-cigarette use susceptibility, current e-cigarette use, e-cigarette use dependence)

Pokhrel et al., 2018

Revised youth EC outcome expectancies (short)

Outcome expectancies

2 (positive outcome expectancies, negative outcome expectancies)

.87 - .92

Test Dimensionality (EFA & CFA)

Concurrent validity (associated with e-cigarette use susceptibility, current e-cigarette use, e-cigarette use dependence); construct validity (associated with e-cigarette outcome expectancy factors)

Morean et al., 2018

Self-report habit index (SRHI)

Habitual e-cigarette use

1 (habitual use)

0.91

Test Dimensionality (CFA & EFA); Measurement invariance;

Concurrent validity (associated with increased frequency of e-cigarette use)

Carey et al., 2018

Susceptibility to four product classes (EC, cigars, hookah and cigarettes)

Susceptibility to four product classes (e-cigarette, cigarettes, hookah and cigars)

4 (e-cigarettes, cigarettes, hookah, cigars)

0.74

Test Dimensionality (CFA)

Concurrent validity (associated with e-cigarette ever use)

Browne & Todd, 2018

Fagerström Test for Nicotine Dependence applied to Vaping

(FTND-V)–

E-cigarette dependence and consumption

.54

Test Dimensionality (CFA)

Hershberger et al., 2017

Comparing E-Cigarette and Cigarettes Questionnaire(CEAC)

Comparative beliefs of e-cig use and cigarette

smoking

3 (general benefits, general effects; health benefits

0.94

Subscales: .80-.88

Test Dimensionality (EFA & CFA); Measurement invariance

Waters et al., 2017

Perceived harms and social norms in the use of

electronic cigarettes

Perceived harms and social norms

1 (perceived harms), 1 (social norm)

.87 (perceived harms) .93 (social norm)

Test Dimensionality (EFA)

Concurrent validity (associated with e-cigarette use)

Persoskie et al., 2017

Direct and indirect measures of perceived harm of e-cigarettes and smokeless tobacco

Perceived harm of e-cigarettes

2 (direct and indirect)

Concurrent validity (associated with e-cigarette use)

Copeland et al., 2017

Risk and Benefits of E-cigarettes (RABE)

Perceived risk and benefits of e-cigarette use (RABE)

2 (perceived risks, perceived benefits)

Subscales: .89 - .92

Test Dimensionality (CFA);

Test differences by e-cigarette use status);

Correlation with Cigarette Dependent Index

Morean & L’Insalata, 2017

Short Form Vaping Consequences Questionnaire (S-VCQ)

Sensory vaping

expectancies

4 (Negative consequences, Positive reinforcement, Negative reinforcement, Appetite/weight control)

Subscale: .85 - .94

Test Dimensionality (CFA); Measurement invariance;

Correlation with smoking expectancy scale

Concurrent validity (associated with frequency of e-cigarette use)

Penzes et al., 2016

Motivations of intention to try EC

Motivations of intention to try e-cigarette

6 (health benefits/smoking cessation; curiosity/taste variety; perceived social norms; convenience; chemical hazards; danger of dependence)

subscale: .68-.90

Test Dimensionality (EFA)

Concurrent validity (subscales (curiosity/taste) associated with e-cigarette experimentation)

Hershberger et al., 2016

Nicotine and Other Substance Interaction Expectancy Questionnaire E-cig Revised version (NOSIE-ER)

Expectancies of combined e-cig and alcohol use

2 (alcohol use leads to e-cigarette use,

e-cigarette use leads to alcohol use)

.84 -.88 Subscale: .85-.94

Test Dimensionality (EFA & CFA)

Concurrent validity (associated with alcohol use disorder)

Hendricks et al., 2015

E-cigarette-specific Brief Smoking Consequences Questionnaire-Adult (BSCQ-A)

Hospitalized smokers' expectancies for electronic cigarettes

10 (negative affect reduction, stimulation/state enhancement, health risks, taste/sensorimotor manipulation, social facilitation, weight control, craving/addiction, negative physical feelings, boredom reduction, negative social impression)

.67 - .88

Test Dimensionality (CFA);

Correlations with tobacco use, and e-cigarette exposure and use

Concurrent validity (associated with intention to use e-cigarettes)

Chaffee et al., 2015

Conditional Risk Assessment of Adolescents’ Electronic Cigarette Perceptions

Perceived risk and benefits for use of e-cigarette

2 (perceived risks and benefits)

Test differences by e-cigarette use status; Correlation with Cigarette Dependent Index

Concurrent validity (associated with e-cigarette use)

Abbreviations: CFA: confirmatory factor analysis; EFA: exploratory factor analysis; PFA: Principal factor analysis

No competing interests reported.

Download PDF

Editorial decision: Major revision
24 Dec, 2021
Reviewers agreed at journal
30 Nov, 2021
Reviews received at journal
27 Oct, 2021
Reviewers agreed at journal
11 Oct, 2021
Reviewers invited by journal
13 May, 2021
Editor assigned by journal
13 May, 2021
Editor invited by journal
12 May, 2021
Submission checks completed at journal
12 May, 2021
First submitted to journal
05 May, 2021

You are reading this latest preprint version

Instruments to Measure E-Cigarette Related Constructs: A Systematic Review

Status:

Version 1

Abstract

Figures

Background

Methods

Results

Discussion

Conclusions

Abbreviations

Declarations

References

Tables

Additional Declarations

Status:

Version 1