The Multiscent-20: A Digital Odour Identification Test Developed with Item Response Theory

doi:10.21203/rs.3.rs-3136718/v1

Download PDF

Article

The Multiscent-20: A Digital Odour Identification Test Developed with Item Response Theory

https://doi.org/10.21203/rs.3.rs-3136718/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

Although validated and reliable psychophysical tests of olfactory function are available, an easy-to-use and feasible test has yet to be developed. This study aimed to design a digital odour identification test, evaluate its validity, establish a normative curve, and explore the impact of demographic factors.

The odour identification test was presented with the Multiscent-20, a hand-held, tablet-like digital scent device that features an integrated odour digital delivery system. The identification performance on the 20 odours was assessed using item response theory (IRT). The normative curve was established by administering the test to a large sample of participants (n = 1,299).

The mean identification score was 17.5 (SD = 2.1). The two-parameter logistic IRT model provided the best fit, revealing variation in item discrimination and difficulty parameters. Educational attainment influenced performance, with primary education associated with lower scores. Sex was marginally associated with performance, suggesting that men scored slightly lower than women.

This study provides initial evidence supporting the validity of use of the Multiscent-20 as a digital odour identification test. The test's automation and portability enable the standardized delivery of olfactory stimuli and efficient automatic recording and scoring of responses.

Biological sciences/Psychology

Health sciences/Biomarkers

Health sciences/Diseases/Neurological disorders

Health sciences/Diseases/Respiratory tract diseases

Health sciences/Anatomy/Nervous system/Sensory systems/Olfactory system

Physical sciences/Engineering/Biomedical engineering

Biological sciences/Neuroscience/Olfactory system

Biological sciences/Neuroscience

Biological sciences/Neuroscience/Sensory processing

Olfactory function assessment is vital for evaluating human health, providing information about an individual's neurological and cognitive status. At present, there are reliable and validated psychophysical tests for olfactory function assessment in clinical practice, including the University of Pennsylvania Smell Identification Test (UPSIT)¹, which employs microencapsulation of odorous liquids, and the Sniffin' Sticks test^2-4, which uses felt-tip pens infused with odorized fluids. While these tests are elegant and widely used, they have some limitations, such as the time required for testing, repeated use, cost, and the requirement of having a trained examiner conduct the tests. As a result, there is an increasing need for automated testing procedures capable of delivering stimuli, recording responses, and scoring assessments in a cost-effective manner. To overcome these challenges, we developed the Multiscent-20 (Noar, São Paulo, Brazil), a portable tablet computer with an integrated odour dispensing system; this device provides an efficient, mobile, and user-friendly method for olfactory function assessment.

A recent proof-of-concept study demonstrated the feasibility of using the Multiscent-20 to present an olfactory function assessment⁵. We enlisted 180 participants and compared performance on the Multiscent-20 with that on the 40-item UPSIT. The findings indicated that participants scored higher on the Multiscent-20 and completed this test in a shorter duration and that there was a strong correlation between performance on the two tests. The Multiscent-20 showed high test-retest reliability and was regarded as easy to use. Additionally, due to the COVID-19 pandemic, recent years have seen a surge in interest in olfactory function evaluation and research, underscoring the need for a universally accepted, feasible assessment tool that is validated across diverse populations globally^6,7. The creation of such a tool would substantially expedite cross-cultural research and ensure consistent evaluation criteria among diverse populations. To address this demand, we endeavoured to develop a new self-administered digital olfactory function test based on the classic 4-alternative forced-choice (AFC) paradigm to provide a universally applicable olfactory function assessment.

In this study, we describe the development, validation, and establishment of the normative curve of the Multiscent-20 olfactory identification test in its preliminary version. The selection of odours for this digital assessment was predicated upon the assumption that they embodied universally recognized and familiar odours across all five continents, thereby ensuring the potential to apply this assessment in a variety of cultures⁶.

Evaluating the reliability and identification performance on individual items in any psychophysical test is crucial to ensure the test's overall validity. Item response theory (IRT) is a framework for examining these elements, modelling the relationship between an individual's latent trait (olfactory ability, in this instance) and their responses to individual test items⁸. IRT enables assessment of the interplay between item parameters, such as difficulty and discrimination, in conjunction with the latent ability of tested individuals, thereby enhancing the precision of measurement and facilitating adjustments and adaptations for diverse test versions.

The objectives of this study are as follows:

(a) To develop an odour identification test utilizing the Multiscent-20 that can effectively and adequately measure individual olfactory identification ability.

(b) To assess performance on the 20 items of the Multiscent-20 through IRT analysis, ensuring accuracy, and examining item parameters such as discrimination and difficulty. Additionally, this assessment will gauge the quality of information generated by the test.

2.1 Participants

Table 1 displays the demographic and clinical data of a sample of 1,852 individuals. The majority of the participants reported that they did not have a cold or flu at the time of testing (85%), difficulty smelling things (90%), or major memory issues (98%). Similarly, most participants reported that they were literate (99%) and had never had a head injury (98%). Among those who underwent COVID-19 testing, 31% tested positive, and 19% of those who tested positive reported a loss of smell. The majority of participants had completed secondary education (69%), followed by those who had completed tertiary education (23%), and those who had completed only primary education (8%). Most participants were male (55%). The mean age of the sample was 37 years, with the male participants exhibiting a slightly higher mean age than the female participants (38.4 vs. 34.8 years, respectively).

Table 1

Demographic and clinical data of the study participants.
Characteristic	n	%
Sex (n = 1,832)
Male	1,005	54.8%
Female	827	45.2%
Age group
18–30 years	551	30.1%
30–40 years	438	23.9%
40–50 years	664	36.2%
50–60 years	157	8.6%
> 60 years	22	1.2%
Educational attainment (n = 1,852)
Primary education	144	7.8%
Secondary education (high school)	1,280	69.1%
Tertiary education (university)	428	23.1%
Cold or flu-like symptoms (n = 1,710)
No	1447	84.6%
Yes	243	14.2%
No answer/Not sure	20	1.2%
Literate
Yes	1,690	98.8%
No	16	1.0%
No answer/Not sure	4	0.2%
Difficulty smelling things
No	1,543	90.2%
Yes	135	7.9%
No answer/Not sure	32	1.9%
Severe memory problems
No	1,665	97.4%
Yes	34	2.0%
No answer/Not sure	11	0.6%
Head injury with loss of consciousness
No	1,674	97.9%
Yes	29	1.7%
No answer/Not sure	7	0.4%
COVID-19 test result
Negative	1,161	67.9%
Positive	531	31.1%
No answer/Not sure	18	1.0%
Loss of sense of smell with COVID-19
Yes	327	61.6%
No	202	38.0%
No answer/Not sure	2	0.4%
Regained sense of smell lost due to COVID-19
Yes	266	15.5%

2.2 Dimensionality assessment

The unidimensionality of test results was evaluated through confirmatory factor analysis (CFA). The factor model fit statistics were robust. The chi-square test for the factor model yielded a significant result (χ²(170) = 245.189, p < .001), indicating a good fit. The model also demonstrated strong performance in various other fit indices, including the comparative fit index (CFI = 0.983), Tucker‒Lewis index (TLI = 0.981), and Bentler-Bonett normed fit index (NFI = 0.947). Moreover, the model exhibited a low root mean square error of approximation (RMSEA) value of 0.015, with a 90% confidence interval ranging from 0.011 to 0.020. Additionally, the standardized root mean square residual (SRMSR) was well within acceptable limits (0.067). In summary, the CFA provided strong support for unidimensionality of test results.

2.3 Item-level IRT analysis

2.3.1 Selection of the most suitable IRT model

Three IRT models were compared to determine the best-fitting model for the data: the one-parameter logistic (1PL) model, the two-parameter logistic (2PL) model, and the three-parameter logistic (3PL) model. The goodness-of-fit statistics for each model included the M2, RMSEA, SRMR, TLI, and CFI values.

The 1PL model showed a relatively poor fit, with an M2 of 1046.848 (df = 189, p < 0.001), RMSEA of 0.0495, SRMSR of 0.0914, TLI of 0.8458, and CFI of 0.8466. In contrast, the 2PL model demonstrated a considerably better fit, with an M2 of 316.1901 (df = 170, p < 0.001), RMSEA of 0.0216, SRMSR of 0.0395, TLI of 0.9708, and CFI of 0.9739. The 3PL model showed a similarly good fit, with an M2 of 258.4386 (df = 150, p < 0.001), RMSEA of 0.0198, SRMSR of 0.0409, TLI of 0.9754, and CFI of 0.9806.

A comparison of the three models using the Akaike information criterion (AIC), Schwarz's Bayesian information criterion (SABIC), Hannan-Quinn information criterion (HQ), and Bayesian information criterion (BIC) revealed that the 2PL model had the lowest values for all criteria, indicating the best fit to the data. Furthermore, an analysis of variance (ANOVA) comparing the three models showed a significant improvement in fit for the 2PL model over the 1PL model (χ² = 327.282, df = 19, p < 0.001) but no significant difference in fit between the 2PL and 3PL models (χ² = 19.075, df = 20, p = 0.517).

Based on these results, the 2PL model was chosen as the most suitable model for the data. This model provides a more flexible approach than the 1PL model by allowing items to have different discrimination parameters, which can better capture the underlying relationships between the items and the latent trait. The lack of significant improvement in fit from the 2PL model to the 3PL model suggests that the added complexity of the 3PL model, which incorporates a guessing parameter, was unnecessary for adequate representation of the data.

2.3.2 Item and person fit

The item fit parameters for the 2PL IRT model were assessed using S-χ², RMSEA, and p values. The S-χ² statistic evaluates the discrepancy between observed and expected item responses, with higher values indicating a worse fit. RMSEA provides an estimate of the item fit error, with values closer to 0 indicating a better fit. The p value assesses the statistical significance of the item misfit.

In the present analysis, most items exhibited good fit, with nonsignificant p values (> 0.05) and low RMSEA values. However, a few items showed potential misfits: Item 3 (Coconut) with an S-χ² of 26.757, df = 12, RMSEA = 0.026, and p value = 0.008; Item 6 (Tire) with an S-χ² of 23.440, df = 12, RMSEA = 0.023, and p value = 0.024; and Item 14 (Strawberry) with an S-χ² of 23.926, df = 11, RMSEA = 0.025, and p value = 0.013.

The person-fit statistics for the 2PL IRT model indicated that the majority of individual response patterns were consistent with the model expectations, with 99.946% and 98.920% fitting based on infit and outfit statistics, respectively.

2.3.3 Discrimination and difficulty parameters

The discrimination (a) and difficulty (b) parameters of the 2PL IRT model for the 20 items provided meaningful insights into the characteristics of the test items. The discrimination parameters ranged from 0.38 to 2.43, indicating that items vary in their ability to differentiate between individuals with differing levels of the underlying latent trait. For instance, the item 'Vanilla' (a = 2.43) showed higher discrimination, meaning that it better distinguished between individuals along the ability continuum. In contrast, the item 'Rose' (a = 0.38) had lower discrimination and was less effective in this regard (Supplementary Table S2).

Figure 1 presents the item characteristic curves (ICCs) for the Multiscent-20 items, illustrating the relationship between individual ability and item difficulty on the same dimension. Figure 2 shows the item information curves (IICs), visually depicting the contribution of individual items to the overall precision of the scale.

The difficulty parameters ranged from − 3.22 to -0.60, providing information about the relative difficulty of each item. A more negative value of difficulty corresponds to an easier item, while a less negative value of difficulty denotes a more difficult item. For example, the item 'Coffee' (b = -3.22) is relatively easier to identify, whereas the item 'Rose' (b = -0.60) is more challenging for test-takers.

2.4 Test information curves

The test information curve (TIC; Fig. 3, left panel) illustrates the diagnostic test's precision across the spectrum of the latent trait, reflecting its capacity to differentiate between individuals possessing varying levels of the trait. The scale characteristic curve (SCC; Fig. 3, right panel) also serves as an invaluable resource for evaluating diagnostic tests, encompassing facets such as test reliability, precision, measurement gap identification, and test calibration and equating. The SCC revealed that individuals with a θ value of approximately 0, corresponding to average latent ability, obtained an identification score of approximately 17–18 points. In contrast, those with a θ value of -4 to -2, indicating very low levels of odour identification ability, attained identification scores between 4 and 12.5.

2.5 Influence of demographic factors

The one-way ANOVA revealed a statistically significant difference in the number of correct answers across the three educational attainment categories (F(2, 1,829) = 13.55, p < 0.001). The Tukey honestly significant difference (HSD) test showed lower performance in the primary education group than in the secondary and tertiary education groups (primary vs. secondary education: mean difference = 1.05, 95% CI: [0.53, 1.57], p < 0.001; primary vs. tertiary education: mean difference = 1.26, 95% CI: [0.69, 1.83], p < 0.001). However, no statistically significant difference was found between the secondary education (high school) and tertiary education (college) groups (mean difference = 0.21, 95% CI: [-0.10, 0.52], p = 0.257).

A multiple linear regression analysis was conducted to examine the relationship between the Multiscent-20 identification score (the dependent variable) and sex, age group, and educational attainment (independent variables). The results indicated that the educational attainment was significantly associated with the Multiscent-20 identification score (secondary education: p = 6.94e-06, tertiary education: p = 6.26e-07). Specifically, participants with secondary and tertiary education had higher scores than those with primary education.

However, no significant relationship was found between age group and Multiscent-20 identification score (30–40 years: p = 0.97, 40–50 years: p = 0.89, 50–60 years: p = 0.17, > 60 years: p = 0.50). Sex had a marginally significant association with the Multiscent-20 identification score (p = 0.096), suggesting that men had slightly lower scores than women. The multiple R² value of 0.018 indicated that the model explained only a small proportion of the variation in the Multiscent-20 identification score.

2.6 Descriptive data of the preliminary “normative” sample

A total of 533 individuals were excluded from the analysis of the preliminary normative curve. Exclusions were made according to the following criteria: recent infection (n = 240), illiteracy (n = 16), severe memory impairment (n = 34), prior head trauma (n = 107), nasal surgery (n = 101), persistent post-COVID-19 hyposmia (n = 4), self-reported hyposmia (n = 135), diagnosed neurological disease (n = 13), or scores of 23 or less on the Mini-Mental State Examination (MMSE) (n = 18).

The sample of 1,299 individuals had a mean Multiscent-20 identification score of 17.5, with a standard deviation of 2.1, as seen in Fig. 4. The median score was 18, and the trimmed mean was 17.8, indicating that the central tendency of the scores was relatively high. The scores ranged from a minimum of 2 to a maximum of 20. The skewness value of -2.3 suggests a negative (left) skew, indicating that the majority of the participants scored higher on the Multiscent-20 identification scale, with a smaller number of individuals scoring lower. The kurtosis value of 7.3 implies a leptokurtic distribution, meaning that the scores were more concentrated around the mean than in a normal distribution.

Table 2 presents descriptive data for the Multiscent-20, broken down by age group. The table provides information on the number of participants (N), mean scores, standard deviation (SD), minimum (Min) and maximum (Max) scores and percentiles (P1, P2, P5, P10, P25, P50, P75, P90, and P95) for each subgroup. The results revealed striking similarities in the mean scores and score distributions across age groups and sexes. While participants > 60 years old had a smaller sample size, their performance was generally consistent with that of other age groups, indicating that the ability to identify odours on the Multiscent-20 remained relatively stable across various demographic groups.

Table 2

Multiscent-20 identification scores of distinct age groups in the preliminary normative sample.
Age group (years)	N	Mean	SD	Min	Max	P1	P2	P5	P10	P25	P50	P75	P90	P95
18–30	393	17.6	2.1	2	20	9.7	12	14	15	17	18	19	19	20
30–40	322	17.5	2.1	5	20	10.2	11.4	14	15	17	18	19	20	20
40–50	459	17.5	2.1	7	20	10	12	14	15	17	18	19	20	20
50–60	107	17.3	2.2	9	20	10.1	12	13	15	16	18	19	20	20
> 60	18	17.2	2.1	13	20	13.2	13.3	13.9	14.7	15.3	18	18.8	20	20
Legend: N - number of participants; Mean - average score; SD - standard deviation of scores; Min - minimum score; Max - maximum score; P1–1st percentile score; P2–2nd percentile score; P5–5th percentile score; P10–10th percentile score; P25–25th percentile score; P50–50th percentile score (median); P75–75th percentile score; P90–90th percentile score; P95–95th percentile score.

In the present study, we developed a novel digital odour identification test presented on a dedicated tablet that releases odours and evaluated its psychometric properties. This new test is automated and portable, enabling standardized olfactory stimuli delivery, swift recording, and scoring of responses in a cost-effective manner. The validity of the test was evaluated with the IRT framework. The findings revealed that the 2PL model yielded the most appropriate fit to the data, indicating variations in item discrimination and difficulty. Furthermore, the adequate fit (based on infit and outfit statistics) showed that the majority of participant responses were in line with the model expectations.

The TIC and SCC provide insights into the performance of diagnostic tests in general and may also be used for odour identification tests such as in the present study. The TIC provides a visual representation of the test's capability to differentiate between individuals with different ability levels. For the Multiscent-20, the TIC revealed that the test has a high level of precision across lower levels of the latent trait, suggesting that it can effectively differentiate between individuals with hyposmia and anosmia. The SCC showed that individuals with an average odour identification ability obtained a score of 17–18 points, indicating that they correctly identified most of the scents presented. In contrast, individuals with the lowest levels of olfactory ability (θ values lower than − 2) obtained lower scores, ranging between 4 and 12.5 points.

The present findings also highlighted variations across demographic groups, consistent with the complexity of olfactory function testing. The data showed that while higher education levels were correlated with better performance on the Multiscent-20, sex was only marginally – but significantly – associated with olfactory ability. This is in line with previous research finding sex differences in olfaction with relatively small effect sizes⁹. In addition, previous research has shown that in odour identification, verbal ability plays a major role¹⁰.

The percentile-based interpretation of the normative values was guided by the contribution of the 'Sniffin' Sticks' test and the UPSIT, prompting us to consider a score below the 10th percentile as indicative of hyposmia^1,4. A similar standardized classification system for an objective assessment of participant performance based on percentiles is included in the guidelines of the American Academy of Clinical Neuropsychology consensus conference statement on labelling performance in tests with non-normal distributions¹¹. In this classification, scores above the 24th percentile (≥ 17 points) are classified “Within Normal Expectations Score", "Within Normal Limits Score", or “normosmia”, while scores ranging from the 10th to the 24th percentile (15–16 points) are labelled as "Low Average Score” but still normosmia. Scores falling between the 2nd and 8th percentiles are designated as "Below Average Score" or “hyposmia”, while scores below the 2nd percentile are classified as "Exceptionally Low Score" or "anosmia". Following these guidelines provided a consistent and objective assessment of subject performance in our study.

The olfactory function test presented with this digital device can enable large-scale administration without the need for examiners, as scoring is automatic, and the test is self-administered. This feature facilitates epidemiological screening in populations^12–14. Moreover, the equipment enables the collection of multidimensional data, including discrimination, threshold, familiarity, pleasantness, and preference for different odours. The collection and analysis of multidimensional data provide a more comprehensive and detailed understanding of olfactory responses and individual or collective patterns of odour perception¹⁵. Furthermore, the device facilitates the utilization of artificial intelligence, as analysis of the generated big data can provide novel insights into the biology of olfactory perception and the cognitive processes involved in odour identification¹⁶. The application of IRT may also be of interest in enabling computerized adaptive tests.

Moreover, incorporating "universal odours" in the Multiscent-20 and the initial exploration of cross-cultural research underlines the tool's potential to transcend cultural boundaries^17,18. Subsequent studies may refine the test by addressing any discrepancies in item calibration and investigating other potential factors that could account for variation in test performance.

This study also has some limitations. It is crucial to acknowledge that the multiple linear regression model explained only a small proportion of variance in Multiscent-20 identification scores. This emphasizes the necessity of future research to identify additional factors that may contribute to variation in olfactory ability. Moreover, it underscores the importance of refining the items of the Multiscent-20 to better represent the latent trait of olfaction. Furthermore, the test-retest reliability coefficient (0.82) obtained in the proof-of-concept study with 40 odours⁵ needs to be confirmed in this version of the test.

In conclusion, the findings of the present study support the validity of the Multiscent-20 as an odour identification test. The 2PL IRT model provides a robust framework for understanding the performance on test items. The test represents an important step forwards in olfactory function testing, with potential benefits for both clinical practice and research.

4.1 Study design and ethical approval

The study was approved by the Ethics Committee of the Faculty of Medicine of the University of Brasilia (Approval Number: 43188021-9-1001-5558). All methods were performed in accordance with the guidelines and regulations of the National Council of Research Ethics and the Declaration of Helsinki, and informed consent was obtained from all participants. Participant data are protected by national data protection legislation. The study had a cross-sectional observational design and utilized nonprobabilistic convenience sampling by quotas. In the following sections, we provide a detailed description of the experimental methodology and preliminary results.

4.2 Participant enrolment and inclusion criteria

To be eligible for participation, individuals had to be at least 18 years of age and native speakers of Portuguese. The study participants consisted of healthcare professionals from the university hospital, medical students, administrative staff, and employees from a glass factory. We excluded participants who provided careless or potentially biased responses to the olfactory function test, such as selecting answers at random or choosing the same option for all responses. We purposely did not exclude individuals who had a recent cold or flu or who reported olfactory loss because a sample with sufficient variability in test performance is essential to evaluate the test’s ability to measure the loss of the sense of smell.

However, for the normative curve of the test, other inclusion and exclusion criteria were established to ensure the inclusion of normosmic individuals. Participants had to be 18 or older, native speakers of Portuguese, and free of any known impairment of smell or taste. Participants who had been sick (with a cold or flu) within 15 days prior to the exam were excluded, as were those who reported neurological or psychiatric diseases and those with a history of head trauma. Participants who demonstrated negligent performance in the exam were also excluded. Additionally, those aged above 60 years and scoring below 24 points on the MMSE were excluded¹⁹.

4.3 Description of device features

The Multiscent-20 is a dedicated tablet featuring a 7-inch touchscreen specifically designed for use in the cosmetics industry to present perfume fragrances to customers (Fig. 5)⁵.

The Multiscent-20 is a portable tablet computer with an integrated hardware system consisting of a central processing unit, touchscreen, Wi-Fi antenna, USB dock, power supply, and rechargeable battery. Additionally, it contains an odour system composed of 20 microcartridges, an air filter, a system that generates a dry air stream, and an odour-dispensing opening. The device is capable of presenting 20 different odours from individual odour capsules, which store the olfactory stimuli by incorporating the odours into an oil-resistant polymer. The capsules are loaded through an insertion port on the back of the device.

A software-controlled processing unit governs the flow of dry air, generating a constant air stream that passes through the capsule, releasing individual odours through a small opening at the upper front of the device. In this study, each scent was presented for 5 seconds, with an interval of at least 6 seconds between odour presentations. Each capsule contained 35 µL of oil-based odour solution, allowing the device to maintain consistent odour intensity and identifiability for up to 100 activations, as per the manufacturer's instructions. The Givaudan Corporation prepared the odours used in the capsules.

A digital application (software) was developed to present odours and record olfactory function test results. The device delivers odours through a dry air system, leaving no residue in the environment or on users. Users can access the test using an application on the device (screen version) or mobile phone. The results are accessible through the integrated software, allowing for storage and analysis of the data.

4.4 Selection of the “universal” odours

For this study, 20 odours were selected based on the criteria used in the International Odour Identification Test for Children (Universal Sniff Test)⁶. These criteria encompassed factors such as global prevalence, stimulus-semantic specificity, exclusion of semantically generic odours, olfactory performance, familiarity among the general population, and commercial availability across five continents. In addition to the selected fragrances, universal semantic distractors were included (Supplementary Table S1). The selected odours were Smoked, Lavender, Coconut, Mint, Vanilla, Tire, Rose, Coffee, Grass, Bubble-gum, Menthol, Grape, Clove, Strawberry, Banana, Orange, Cinnamon, Garlic, Pizza, and Onion.

4.5 Experimental setup

The Multiscent-20 odour identification test commences with the participant reviewing the step-by-step instructions displayed on the initial screen. These instructions are presented below:

1. This assessment consists of a test containing 20 distinct odours.

2. Enter your identification information and press the "next" button.

3. The next screen will show a "Try it" button, the phrase "This smell resembles," and with four alternative answers. Please reading all the choices before pressing the "Try it" button.

4. Upon pressing the "Try it" button, a small opening at the top front of the device will release the odour for 5 seconds. You can press the "Try it" button up to three times. Please maintain the device approximately 10 cm from your nostrils after pressing the button. After perceiving the odour, please select one of the alternatives, touch the corresponding letter, and press the "next odour" button to proceed.

5. The number of correct responses will be displayed upon completion of the test.

This assessment was a 4-AFC test, meaning that participants were asked to choose one of the four alternatives to advance to the next odour. The software was specifically developed to present questions and manage responses on the device. After the test, the results are displayed, and the device automatically synchronizes with the database stored in the cloud.

4.6 Data analysis (descriptive statistics and dimensionality assessment)

For the descriptive analysis, categorical variables are presented as the frequency (n) and percentage (%). Continuous variables are either described as the mean and standard deviation (SD) or median and interquartile range (IQR), depending on the normality of the data, as determined with the Shapiro‒Wilk test. Data analysis was mainly performed in R software (v4.3.0, R Foundation for Statistical Computing, Vienna, Austria). All statistical tests were two-sided, with a significance threshold set at 0.05.

To assess the dimensionality of the dataset, a CFA was performed using JASP²⁰. Model fit indices, such as the SRMSR, TLI, and RMSEA, were examined to evaluate the model fit. This procedure aimed to evaluate the suitability of the dataset for further analysis using an IRT model.

4.7 Item response theory analysis

To conduct the IRT analysis, the 'mirt' package²¹ in R was utilized. The following steps were followed for the analysis:

(a) Data preparation: Data were organized into a matrix format, with rows representing individual participants, and columns representing dichotomous responses (correct/incorrect) to each of the 20 odour items.

(b) Model estimation: Using the 'mirt' function, the parameters of the 1PL, 2PL, and 3PL models were estimated. The 1PL model assumed a constant discrimination parameter across items, while the 2PL model allowed for variation in discrimination parameters. The 3PL model additionally included a guessing parameter, accounting for the possibility that participants may have guessed the correct answer by chance.

(c) Model comparison: To determine the best-fitting model for the present dataset, goodness-of-fit statistics were compared, such as the AIC and BIC. Lower values of these statistics indicated a better-fitting model.

(d) Item analysis: For the best-fitting model, the ICCs were examined. These provide information about the relationship between participant latent trait levels and their probability of correctly identifying each odour. Item information functions (IIFs) were examined to understand the precision of each item in measuring the latent trait.

4.8 Influence of demographic factors (sex, age, and educational attainment)

To investigate the relationship between participant educational attainment and the number of correct answers on the administered test, a one-way ANOVA was conducted. Participants were divided into three categories based on their educational attainment: primary (the fundamental level in Brazil), secondary (high school), and tertiary (college) education. The dependent variable was the number of correct answers in the odour identification test. Subsequently, Tukey's HSD test was employed to identify pairwise differences between the three educational attainment groups, provided that the ANOVA yielded significant results.

A multiple linear regression model was employed to investigate the relationship between the Multiscent-20 identification scores (dependent variable) and sex, age group, and educational attainment (independent variables). The dataset was analysed using R software, with the “lm” function utilized to create the regression model.

Data availability statement

The anonymized data that support the findings of this study are openly available in Zenodo public repository, at https://doi.org/10.5281/zenodo.8079860 reference number 8079860.

Acknowledgements

The authors gratefully acknowledge the study participants, as well as Fernando Araujo Rodrigues Oliveira, Ana Paula Matos Rodrigues, and Rosane Antunes Carreiro Chaves at the Clinical Research Center of the University Hospital of Brasília. We also thank Peter Gottschalk, Hugo Zambotti, Claudia Ciszevsk, Daiana Saraiva at the Noar Company. Finally, we thank Nature Research Editing Service for the English language editing and manuscript formatting.

Author contributions

M.N., G.S.M.F., C.G. performed the experiments. P.R.P.B. and L.G.A.V. analyzed data. M.N and P.R.P.B wrote and revised the manuscript. G.S.M.F., L.G.A.V., M.A.F., W.T.A.L., C.G., T.H. designed the study, made revisions, and edited the manuscript.

Additional information

Funding information Noar Company, Grant/Award Number: 43188021.9.1001.5558.

Competing interests

M.N. participated as a medical consultant and received speaker fees from Noar Company. C.G. is CEO shareholder of the Noar Company, the manufacturer of the Noar MultiScent device described in this article. The remaining authors have no competing interests to disclose.

Doty, R. L., Shaman, P. & Dann, M. Development of the university of pennsylvania smell identification test: a standardized microencapsulated test of olfactory function. Physiol. Behav. 32, 489–502 (1984).
Kobal, G. et al. "Sniffin' sticks": screening of olfactory performance. Rhinology 34, 222–226 (1996).
Hummel, T., Sekinger, B., Wolf, S. R., Pauli, E. & Kobal, G. ‘Sniffin’ sticks': olfactory performance assessed by the combined testing of odour identification, odor discrimination and olfactory threshold. Chem. Senses 22, 39–52 (1997).
Hummel, T., Kobal, G., Gudziol, H. & Mackay-Sim, A. Normative data for the “Sniffin’ Sticks” including tests of odor identification, odor discrimination, and olfactory thresholds: an upgrade based on a group of more than 3,000 subjects. Eur. Arch. Oto-Rhino-Laryngol. 264, 237–243 (2007).
Nakanishi, M. et al. The digital scent device as a new concept for olfactory assessment. Int. Forum Allergy Rhinol. 12, 1263–1272 (2022).
Schriever, V. A. et al. Development of an international odor identification test for children: the universal sniff test. J. Pediatr. 198, 265–272.e3 (2018).
Renaud, M. et al. Clinical outcomes for patients with anosmia 1 year after COVID-19 diagnosis. JAMA Netw. Open 4, e2115352 (2021).
Cai, L., Choi, K., Hansen, M. & Harrell, L. Item response theory. Annu. Rev. Stat. Appl. 3, 297–321 (2016).
Sorokowski, P. et al. Sex differences in human olfaction: a meta-analysis. Front. Psychol. 10, 242 (2019).
Lillqvist, M., Claeson, A.-S., Zakrzewska, M. & Andersson, L. Comparable responses to a wide range of olfactory stimulation in women and men. Sci. Rep. 13, 9059 (2023).
Guilmette, T. J. et al. American Academy of Clinical Neuropsychology consensus conference statement on uniform labeling of performance test scores. Clin. Neuropsychol. 34, 437–453 (2020).
Wysocki, C. J. & Gilbert, A. N. National geographic smell survey: effects of age are heterogenous. Ann. N. Y. Acad. Sci. 561, 12–28 (1989).
Oleszkiewicz, A. et al. Global study of variability in olfactory sensitivity. Behav. Neurosci. 134, 394–406 (2020).
Arshamian, A. et al. The perception of odor pleasantness is shared across cultures. Curr. Biol. 32, 2061–2066.e3 (2022).
Hummel, T. et al. Position paper on olfactory dysfunction. Rhinol. J. 54, 1–30 (2017).
Hedner, M., Larsson, M., Arnold, N., Zucco, G. M. & Hummel, T. Cognitive factors in odor detection, odor discrimination, and odor identification tasks. J. Clin. Exp. Neuropsychol. 32, 1062–1067 (2010).
Majid, A. Human olfaction at the intersection of language, culture, and biology. Trends Cogn. Sci. 25, 111–123 (2021).
Wysocki, C. J., Pierce, J. D. & Gilbert, A. N. Geographic, cross-cultural, and individual variation in human olfaction in Smell and taste in health and disease (eds. Getchell, T. V., Bartoshuk, L. M., Doty, R. L., & Snow, J. B.) 287–314 (Raven Pressp, 1991).
Folstein, M. F., Folstein, S. E. & McHugh, P. R. "Mini-mental state". A practical method for grading the cognitive state of patients for the clinician. J. Psychiatr. Res. 12, 189–198 (1975).
Love, J. et al. JASP: Graphical Statistical Software for Common Statistical Designs. J Stat Softw 88, 1–17 (2019).
Chalmers, R. P. Mirt: a multidimensional item response theory package for the R environment. J. Stat. Softw. 48, 1–29 (2012).

Competing interest reported. M.N. participated as a medical consultant and received speaker fees from Noar Company. C.G. is CEO shareholder of the Noar Company, the manufacturer of the Noar MultiScent device described in this article. The remaining authors have no competing interests to disclose.

Supplementarymaterials.docx

Download PDF

Editorial decision: Revision requested
25 Jan, 2024
Reviews received at journal
23 Jan, 2024
Reviewers agreed at journal
16 Jan, 2024
Reviews received at journal
15 Jan, 2024
Reviewers agreed at journal
30 Dec, 2023
Reviews received at journal
25 Oct, 2023
Reviewers agreed at journal
12 Oct, 2023
Reviewers invited by journal
10 Oct, 2023
Editor assigned by journal
21 Sep, 2023
Editor invited by journal
24 Jul, 2023
Submission checks completed at journal
24 Jul, 2023
First submitted to journal
03 Jul, 2023

You are reading this latest preprint version

The Multiscent-20: A Digital Odour Identification Test Developed with Item Response Theory

Status:

Version 1

Abstract

Figures

1 Introduction

2 Results

2.1 Participants

2.2 Dimensionality assessment

2.3 Item-level IRT analysis

2.3.1 Selection of the most suitable IRT model

2.3.2 Item and person fit

2.3.3 Discrimination and difficulty parameters

2.4 Test information curves

2.5 Influence of demographic factors

2.6 Descriptive data of the preliminary “normative” sample

3 Discussion

4 Methods

4.1 Study design and ethical approval

4.2 Participant enrolment and inclusion criteria

4.3 Description of device features

4.4 Selection of the “universal” odours

4.5 Experimental setup

4.6 Data analysis (descriptive statistics and dimensionality assessment)

4.7 Item response theory analysis

4.8 Influence of demographic factors (sex, age, and educational attainment)

Declarations

References

Additional Declarations

Supplementary Files

Status:

Version 1