A perspective of randomness in a clinical test of olfactory performance

doi:10.21203/rs.3.rs-2717827/v1

Background

Random walks describe stochastic processes that result from a sequence of indeterminate changes in a random variable that are not correlated with past changes. This report describes a random walk component of a clinical sensory test of olfactory performance. The formal description of the stochastic process during the clinical test allows precise cutoffs to be set for the diagnosis of olfactory loss.

Methods

While odor discrimination (D) and odor identification (I) are tested in the Sniffin`Sticks olfactory test battery with four- and three-alternative forced-choice designs, respectively, the odor threshold (T) test embeds a three-alternative forced-choice paradigm in a seven-alternative staircase paradigm. This was investigated in computer simulation scenarios and then formally described.

Results

Odor threshold assessment test consists of two sequential parts, the first of which determines the starting point for the next. Both parts can be described as a biased random walk with highly unequal probabilities of moving toward higher (11%) or lower scores (89%). The odor concentration from which the first test part begins, and the length of the random walk in the subsequent second part were critical factors in the probability of accidentally achieving high test scores.

Conclusions

By changing the odor concentration from where the starting point determination for the second test part begins, the current cut-off of a composite test score at T + D + I < 16 indicating anosmia was raised from representing the 87th quantile of random test scores to representing the 97th quantile. The results likely apply to other sensory tests using the staircase paradigm that can also be described as random walks.

Health sciences/Signs and symptoms

Health sciences/Medical research/Biomarkers/Diagnostic markers

Clinical testing of olfactory function is most commonly performed in ENT or neurology departments where olfactory dysfunction is the symptom leading to consultation or an early sign of disease ¹. SARS-CoV2 infections have recently increased interest in olfactory testing ^2,3. Common olfactory tests consist of a single test or battery that typically assesses combinations of three different components of the sense of smell, namely (i) the perception of odors at low concentrations (odor threshold), (ii) the nonverbal discrimination of different odors (odor discrimination), and (iii) the ability to name or associate an odor (odor identification). One of the established olfactory test batteries that assesses all of these components is the Sniffin` Sticks test ^4,5. It is clinically well established and has an extensive record of use in the published biomedical literature. A search of the PubMed database at https://pubmed.ncbi.nlm.nih.gov/ on March 16, 2023 for "(Sniffin`Sticks) OR (Sniff and Sticks) NOT(review[PT])" returned 1,023 results. From the first mention in 1996 in ⁵, the use of Sniffin` Sticks in publications per year has steadily increased to a recent maximum of 153 publications in 2022.

Like other olfactory tests, the Sniffin` Sticks test assesses olfactory function by presenting odors in a forced choice paradigm. The subject must either correctly name an odor known to the examiner from a set of alternatives, or discriminate an odor from also presented blanks. In either case, the test score is determined as the sum of the correct responses. The test result is then used to establish three olfactory diagnoses, i.e., lack of olfactory sensitivity, termed “anosmia”, reduced function, termed “hyposmia”, and normal function, termed “normosmia”. The boundaries defining these diagnoses were found by analyzing the distributions of sum scores in large cohorts of increasing size from n = 1,036 ⁶, with updates derived from testing n = 3,282 ⁷ to n = 9,139 subjects ⁸. The theoretical limit of anosmia is expected to be a defined confidence limit of results when olfactory test responses are only guesses. This is straightforward for the four-alternative and three-alternative forced-choice tasks, where the probability of guessing the correct result is ¼ and 1/3, respectively, and the confidence limits are easily calculated from common probability equations. However, determining the theoretical limit for the odor threshold is not as straightforward because the three-alternative forced-choice task is embedded in a complex staircase design that defies the application of standard probability equations. Because of this difficulty, previous publications have not reported an odor threshold that marks the 90% change in a guessed score; the threshold is reported only from observed scores in individuals diagnosed as anosmic by other means ⁶.

To overcome the empirical component of the diagnostic limits for anosmia, the present analyses approach the problem from the perspective of test results generated by a purely random process. A computational approach has been adopted that combines the analysis of the results of software-coded tests with the subsequent development of a probabilistic description of the underlying stochastic process as a one-dimensional random walk.

Description of the clinical olfactory test algorithm

The Sniffin` Sticks test uses felt-tip pens containing odors dissolved in odorless propylene glycol or just the solvent. The three subtests, designed to measure the three components of olfactory performance (T, D, I), each use 16 individual tests in different algorithms. While odor discrimination (D) and odor identification (I) are tested with four- and three-alternative forced-choice designs respectively, the odor threshold test (T) embeds a three-alternative forced-choice paradigm in a seven-alternative staircase paradigm. Possible ranges of subtest scores are [1, 16], [0,16] and [0,16] for odor threshold, discrimination and identification, respectively. The main outcome of the Sniffin` Sticks test is the TDI sum score. Pathological olfactory function is indicated by a TDI ≤ 30.5, with the empirical cut-off between hyposmia and anosmia at TDI = 16 ⁸. The latter is due to the finding in the first multicenter evaluation ⁶ that patients diagnosed with anosmia never exceeded a certain TDI value, specifically TDI > 16.

The more detailed test algorithm is as follows. Odor identification and odor discrimination are tested using a multiple forced-choice design, i.e., the tested individual has to choose from a set of options. For odor discrimination, three odors are presented, two of which are identical and one of which, the target, is different from the others. The task is to select the odor that smells different (three-alternative forced-choice design). For odor Identification, a single odor is presented along with a list of four odor names from which one must be selected (four-alternative forced choice design). In both tests, the sum of correct answers out of a total of d = 16 trails is the test result.

The odor threshold test is more complex. It is a three-alternative forced-choice design embedded in a staircase paradigm. An odor is presented at 16 concentrations, from a dilution of 4% and further diluted in a geometric series at a ratio of 1:2. For each dilution, a three-alternative forced-choice design is used in which the diluted odor (target) is presented together with two blanks at 3-s intervals. Which one is the target is randomly chosen by the investigator during the test. The test starts with the lowest concentration (T(0) = 16). After an incorrect response, the concentration is increased by 2 threshold levels (T(1) = 14) until the odor is correctly identified twice in a row at the same concentration. This is considered the starting point (T_start = T(n)) of the test and is recorded as the first turning point (staircase paradigm for staring point, SPSP). The odor concentration is then reduced in steps of one threshold score value, i.e., to T(n) + 1, until the odor is no longer detected, i.e., until it is not correctly identified twice in a row. This is the 2nd turning point from which the odor concentration increases again. In this way, two correct identifications in a row or one incorrect identification trigger a reversal of the staircase to the next higher or lower concentration step, respectively. If the test reaches the limit of T = 1, the lowest available odor dilution, the failure to correctly detect the odor is noted as a turning point and the next odor concentration presented is again T = 1. Successfully detecting the odor at T = 1 twice in a row triggers another increase in odor dilutions to T = 2, and so on. Similarly, if the test reaches the limit of T = 16, the highest available odor dilution, success in correctly detecting the odor twice in a row is noted as a turning point and the next odor concentration presented is again T = 16. Failure to detect the odor twice in a row at T = 16 results in T = 15 being presented next, and so on. The test ends after seven turning points. In the standard version of the Sniffin` Sticks, the odor threshold is the average of the last four of seven staircase reversals or turning points (Staircase Paradigm for Threshold, SPT).

Software coding of the clinical olfactory test algorithm

To study the behavior of the olfactory tests, in particular the consequences of successive random decisions in the odor threshold test that underlie the up and down movements to higher or lower odorant dilution steps during the staircase part of this test, a computer simulation was chosen. Coding was done by one author (JL) in the Python language ⁹ using Python version 3.8.15, available free of charge at https://www.python.org (accessed 19 January 2023). The main packages used for the simulations were the numerical Python package "numpy" (https://numpy.org ¹⁰), "pandas" (https://pandas.pydata.org ^11,12). To obtain as unbiased results as possible, the subsequent theoretical description of the olfactory test was independently coded by another author (AU) and in the matrix laboratory language using MATLAB (version 9.13.0.2049777 (R2022b) for Windows™, MathWorks, Natick, MS, USA).

Programming the odor discrimination and odor Identification tests was straightforward. The code for the odor discrimination test is shown as an example in Textbox 1; the odor identification test was programmed similarly, except that there are four alternatives to choose from instead of three. Programming the odor threshold assessment was more complex. Therefore, two different versions of Python code were written, based solely on the Sniffin` Stick manual with occasional consultation with the clinical author (TH) for test details. Python variant 1 mimicked the up and down behavior of the staircase test, while Python variant 2 considered the test as a flat sequence of random choices. Only after the empirical analyses had been completed and the theoretical description of the test had been developed based on the results of the simulations, were the results of the independent programming in Python and MATLAB (Textbox 3) compared.

Textbox 1: Python code for the simulation of random results in the odor discrimination test (JL). The odor identification test was implemented in a similar way, except for a four-alternative choice paradigm (line 5: “np.zeros((16, 4)”, line 7: l = (1, 0, 0, 0), line 12: randint(4)).

# %% Odor discrimination
Discrresult = []
for res in range(nPermut):
correct = np.zeros((16, 3))
for i in range(16):
l = [1, 0, 0]
random.Random(res + i).shuffle(l)
correct[i, :] = l
sumcorrect = 0
for i in range(16):
sumcorrect += correct[i, np.random.RandomState(res + i).randint(3)]
Discrresult.append(sumcorrect)

Textbox 2: Python code with two different programming implementations (StaircaseVariant == 1 or 2, starting at lines 5 and 79, respectively) to simulate random results in odor threshold testing (JL).

Startfrom = 16
StartSteps = 2
Alternate1516 = False
StaircaseVariant = 1
# %% Odor threshold staircase
# Guessing function
def ct1t2():
t1 = [1, 0, 0]
random.Random(res).shuffle(t1)
t1t2 = t1[np.random.randint(3)]
if t1t2 > 0:
t2 = [1, 0, 0]
random.Random(res).shuffle(t2)
t1t2 = t1t2 + t2[np.random.randint(3)]
return(t1t2)
Thresresult = []
dfThrTurningpointAll = pd.DataFrame(np.zeros((nPermut, 500)))
if StaircaseVariant == 1:
# Odor threshold staircase 1
def findStart(t1, StartSteps):
if StartSteps == -99: # -99 means use given value from Startfrom
Start = t1
else:
ThresStart = []
Start = t1
for i in range(math.ceil(t1 / StartSteps)):
ThresStart.append(ct1t2())
if max(ThresStart) > 1:
Start = Start - StartSteps * ThresStart.index(2)
else:
Start = 1
Start = max(1, Start)
return(Start)
for res in range(nPermut):
if Startfrom == 16 and Alternate1516 == True:
if list(np.random.RandomState(res).choice([0, 1], size=1, p=[0.5, 1-0.5]))[0] == 0:
Startfrom -= 1
Start = findStart(Startfrom, StartSteps)
ThrTurningpoint = [Start]
Thr = Start
for i in range(8):
if Thr < 16:
Thr += 1
while ct1t2() == 2:
Thr += 1
Thr = Thr if Thr < 16 else 16
if Thr == 16:
ThrTurningpoint.append(Thr)
ThrTurningpoint.append(Thr)
if Thr > 1:
Thr -= 1
while ct1t2() < 2:
Thr -= 1
Thr = Thr if Thr > 1 else 1
if Thr == 1:
ThrTurningpoint.append(Thr)
ThrTurningpoint.append(Thr)
Thresresult.append(np.array(ThrTurningpoint[3:7]).mean())
for i, tur in enumerate(ThrTurningpoint):
dfThrTurningpointAll.iloc[res, i] = tur
elif StaircaseVariant == 2:
# Odor threshold staircase 2
for res in range(nPermut):
if Startfrom == 16 and Alternate1516 == True:
if list(np.random.RandomState(res).choice([0, 1], size=1, p=[0.5, 1-0.5]))[0] == 0:
Startfrom -= 1
ThresChangeAll = []
for i in range(400):
if ct1t2() < 2:
ThresChangeAll.append(-1)
else:
ThresChangeAll.append(1)
if StartSteps == -99:
ThresChange = ThresChangeAll
Start = Startfrom
else:
ThresChangeStart1 = ThresChangeAll[:math.ceil(
Startfrom / StartSteps)]
if max(ThresChangeStart1) > 0:
ThresChangeStart = ThresChangeStart1[:ThresChangeStart1.index(
1)+1]
ThresChange = ThresChangeAll[ThresChangeStart1.index(1)+1:]
Start = Startfrom - StartSteps * ThresChangeStart.index(1)
else:
Start = 1
ThresChange = ThresChangeAll[math.ceil(
Startfrom / StartSteps):]
Thr = [Start, Start + 1 if Start < 16 else Start]
for i in ThresChange:
ThrBefore = Thr[-1]
ThrNew = ThrBefore + i
if ThrNew < 1:
ThrNew = 1
if ThrNew > 16:
ThrNew = 16
Thr.append(ThrNew)
ThrDelta = Thr[1:]
ThrDelta.append(0)
Thx = []
for i in range(len(Thr)):
Thx.append(ThrDelta[i] - Thr[i])
TurningpointYes = [1]
for i in range(1, len(Thx)):
if Thx[i] + Thx[i-1] == 0:
TurningpointYes.append(1)
else:
TurningpointYes.append(0)
Thr1or16 = [i for i, val in enumerate(Thr) if val == 1]
tur16 = [i for i, val in enumerate(Thr) if val == 16]
Thr1or16.extend(tur16)
for i in Thr1or16:
TurningpointYes[i] = 1
ThrTurningpointIdx = [i for i, val in enumerate(
TurningpointYes) if val == 1]
ThrTurningpoint = list(np.array(Thr)[ThrTurningpointIdx])
Thresresult.append(np.array(ThrTurningpoint[3:7]).mean())
for i, tur in enumerate(ThrTurningpoint):
dfThrTurningpointAll.iloc[res, i] = tur
else:
print("Which variant of staircase implementation?")

To verify that the results of the coded olfactory test were consistent with real observations, a large data set from a previous publication ¹³ was used to check whether the simulated data matched the real olfactory data. The dataset contained the results of the Sniffin` Sticks test from 10,714 subjects who presented with the symptom "loss of smell" at the Clinic for Smell and Taste, Dept. of ENT, TU Dresden, Germany. To avoid circular reasoning in the diagnosis of anosmia, the odor threshold was excluded and anosmia was determined from the odor discrimination and identification scores based on the empirical limits given in a publication that published the 90% percentiles of the random results for both tests, along with the advice to use the sum of both limits as the criterion, i.e., D + I < 16 ¹⁴.

Investigation of random results of the clinical olfactory test algorithm

Simulations were run with 100,000 replicates unless stated otherwise. The distributions of the random results of the soft-coded olfactory test were analyzed using transformations along Tukey's ladder of powers ^15,16. In addition, the significance of specific test algorithm details to random results were analyzed, in particular the starting point and the choice of turning points to be averaged for the final threshold score. Statistical analyses of the results were done in the Python or in the R language ¹⁷. For the latter, the R software package ¹⁸, version 4.2.2 for Linux, which is available from the Comprehensive R Archive Network (CRAN) at https://CRAN.R-project.org/.

First, the olfactory test was mirrored by a software implementation that produced purely random results on a large scale (e.g., 100,000 simulations), on which the behavior of the test could be studied before the observations were translated into a theoretical probabilistic solution.

Analyses of results of the soft-coded olfactory test on random choices

Correct operation of the software coded olfactory test algorithm for odor threshold assessment

The two Python implementations of the software-encoded test algorithm for the determination of odor thresholds produced practically identical results, as indicated by a quasi-linear QQ plot along the line of identity (Fig. 1). The agreement was also found in the details of the starting points of the olfactory tests and the turning points reached during the run through the test algorithm (Fig. 2). The simulations also agreed with observations of odor thresholds in 4,510 subjects who were classified as having an olfactory diagnosis of anosmia based on their odor discrimination and identification scores of D + I < 16.

Distributions of soft-coded olfactory tests on random choices

Distributions of odor threshold, discrimination and identification were analyzed with 10⁵ simulations. The resulting Box-Cox transformation with values of λ ≈ 1 indicated no transformation for the simulated scores of odor discrimination and identification. In contrast, a Box-Cox λ = -1.14 for the transformation of the odor thresholds suggested a reciprocal transformation according to the closest points on Tukey's ladder of powers. Interestingly, when the reciprocal transformation of odor thresholds was performed, the distribution was clearly not unimodal (Fig. 3). A mode at 1/T = 1 was accompanied by a second mode at lower values of 1/T ≈ 1/5. The rejection of unimodality was supported by a highly significant dip test ¹⁹ (dip = 0.086275, p < 10^− 293) using the Python “diptest” package available at https://pypi.org/project/diptest/.

Olfactory test score limits for the diagnosis of anosmia based on random choices

Current recommendations for the Sniffin` Sticks test use the empirically established criterion of a TDI ≤ 16 as an olfactory diagnosis of anosmia, i.e., a non-functioning sense of smell. The percentiles obtained in 100,000 random results indicate that this cutoff corresponds to the 87th percentile of TDI scores obtained as the sum of randomly generated test results for odor threshold, odor discrimination, and odor identification. In the actual test algorithm, the 90th percentile of simulated TDI scores corresponds to TDI = 17.

Significance of specific details of the odor threshold test to random olfactory test results

Starting point of the odor threshold test

The starting point according to the actual test protocol is determined from T = 16 in steps of T = 2, i.e., 16, 14, ..., 2, 1. The final TDI score was correlated with the starting point of the odor threshold testing at a value of Spearman’s ρ = 1 (Fig. 4). The higher the starting point, the more the distribution of final TDI scores was flattened and shifted to the right. Fitting the relationship between the TDI values marking the 90th or 95th percentile and the starting points with linear splines revealed two or three segment associations, based on significant goodness of fit tests for 1, ..., 4 linear spline segments. A prominent break point in these curves, from which the dependence of the final TDI scores on the starting point of the threshold test becomes weaker, was observed approximately at a starting point of the odor threshold test of T = 8 (Fig. 4A).

Choice of the number of turning points to be averaged for the odor threshold score

In an experiment with 10 turning points, there are 2¹⁰-1 = 1023 possible combinations of turning points from which the final odor threshold value can be averaged. The 1024th permutation would be zero turning points. An overview of the quantiles of odor thresholds associated with the use of different turning points for averaging to the final score (Fig. 5) showed that later turning points produced lower random scores, whereas the use of early turning points carried the risk of a high random T score. The current procedure, according to the test instructions, to use the 4th − 7th turning point is in the middle of the extremes and would be slightly improved if only the last two turning points were used for averaging instead of the last four. Details of the relationship between the choice of turning point and the thresholds when the test was answered randomly showed that the higher the position of the turning points used, either defined as the first, last or average position of the turning points, the lower the 90th and 95th quantiles of the randomly generated thresholds were (Fig. 6). The position range of thresholds used, or their total number had less influence.

Translation of the observed test behavior into a theoretical probabilistic solution

In the theoretical description of the olfactory test, the straight-forward tests of odor discrimination and odor identification were described using standard probabilistic equations, while the focus was on describing the odor threshold test. This required capturing the two parts of the test, determining the starting point for the subsequent part of the test, and then obtaining the subsequent turning points. The result should be olfactory scores consistent with the simulations above, including the importance of the starting point for the second part of the test procedure, the choice of turning points for averaging to the threshold score, and the explanation of the bimodal distribution of TDI scores generated when the olfactory test was performed with random choices.

Description of the odor identification and discrimination tests as probability function of the forced choice test design

Here, a standard equation applies. The probabilities of the random results, when the tests of odor discrimination or of identification are completely guessed, are given as a binomial probability distribution with \(P=\frac{n!}{k!*\left(n-k\right)!}*{p}^{k}*{\left(1-p\right)}^{n – k}\) where n is the number of trials of n = 16 odors/pairs, k is the number of correct test responses, and p is the probability of a correctly guessed response, with \(p = 0.\stackrel{-}{3}\) in the odor discrimination test and p = 0.25 in the odor identification test, according to the three- or four-alternative forced-choice designs of the discrimination and identification tests, respectively.

Description of the odor threshold test as a random walk

The starting point of the odor threshold test, according to the current test protocol, is determined from T = 16 in steps of T = 2, i.e., 16, 14, ..., 2, 1. The much less common alternative is a starting point at T = 15 with subsequent steps of T = -2, which was not considered further due to its infrequent use. The odor concentration step correctly detected on the first two consecutive times is the starting point T_start of the test. If this is not the case after eight trials, the subject has reached a value of T_start = 1 on the staircase. In fact, guessing with this algorithm usually leads to T_start = 1, which was the intended behavior during test development. However, the empirical probability of a higher starting point decreases from 16 to 2 (Fig. 2). Starting points with T_start = 16 occur in (1/3)² = 11% of the randomly generated test results. The starting point is also taken as the first turning point of the odor threshold test, according to the test design. Once the starting point is determined, odor thresholds are assessed using a staircase paradigm with steps up and down to higher or lower odorant concentrations. For a purely random outcome of an odor threshold test (random guessing), both processes can be described as random walks as follows.

The determination of the turning points, including the starting point, when the responses to the odor threshold test tasks are only random choices, is given by a random walk with probability p for decreasing the concentration of the next test step and q = 1 - p as the probability for increasing the concentration of the next test. Assuming independence of successive olfactory trials, which is reasonable if the subject cannot smell at all, p can be calculated as the probability of randomly guessing 1 out of 3 (= 1/3) on two successive trials, p = (1/3)² = 11.11%. For this type of so-called biased random walk, theoretical results exist and can be used ^20–22.

Let t = 0,...,N (time) be the length of the Biased Random Walk (BRW) i.e., the number of sniff tests with a guessed result. The time t = 0 is the start time of the BRW, which starts at T_start. For the staircase paradigm to determine the starting points (SPSP), the step width is S_w = 2 and the initial starting point is T_start = 16 at t = 0. The evolution of mean expectation and the 99.7% range in this algorithm shown in Fig. 7. For the staircase paradigm at threshold T, T_start is the result of the start point finding (SPSP) and S_w = 1. The distribution of the threshold pdf(t) reached at t can generally be calculated as a Gaussian pdf(t) = N(M,V) = N(M(t),V(t)), with mean M(t) = 2_*p _*t- s_w*t + T_start and variance V(t) = p_*q_*t.

The integral over all starting point searches for 100,000 trials yields the distribution shown in Fig. 3B and D, respectively for the staircase paradigm for starting points in Fig. 8. With this distribution of T_start, random walk theory yields the distributions of the evolving mean threshold and the 97% limit and as an integral over all distributions, both shown in Fig. 7. This confirms the existence of a second mode (region of high density) at T ≈ 5, which was found empirically (Fig. 3D). The MATLAB implementation of the theoretical description of the odor threshold test is shown in (Textbox 3).

Textbox 3: MATLAB code for the theoretical description of the odor threshold test as a random walk (AU).

clear all; close all; ProgramName ='StaircaseParadigmStartingPoints.m'
Comment = ['Staircase Paradigm for Starting Points(SPSP)']
p =0.33^2;
q = 1-p;
StepWidth = 2;
Tstart = 16;
MaxT = 8 ;
t = [0:MaxT]';
downt = flipud(t)+1;
NrOfT = length(t);
M = Tstart+ 2*p*t- t*StepWidth;
ProbT = ((downt/exp(1)+t)*p^2);
ProbT(end) = p;
ProbT(1) = 1-sum(ProbT(2:end));
Variance = p*q*t;
S =sqrt(Variance);
MplusS = M + 3*S;
MminusS= M - 3*S;
MminusS= max(1,MminusS);

clear all; close all; ProgramName ='StaircaseParadigmThresold.m'
Comment = ['Staircase Paradigm for Thresolds(SPT)']
% Start probabilities from Staircase Paradigm for Starting Points(SPSP)
StartPoints = [ 1, 2, 4, 6, 8, 10, 12, 14, 16]';
StartPointProb = [0.40635,0.046761,0.054258,0.061754,0.069251,0.076747,0.084244,0.09174,0.11];
p =(1/3)^2;
q = 1-p;
L = 1 ;
R = 16 ;
T = [0:0.01:25]';
NrOfStarts = length(StartPoints);
AllProb = T*0;
for i=1:NrOfStarts
StartPkt = StartPoints(i);
StartProb = StartPointProb(i);
MeanN1 = p*T ;
VarianceN1 = T*p*q;
SdevN1 = sqrt(VarianceN1);
MeanDistN = 2*MeanN1-T;
MeanN1 = MeanN1 +StartPkt;
MeanDistN = MeanDistN +StartPkt;
MplusS = MeanDistN + 3*SdevN1;
MminusS = MeanDistN - 3*SdevN1;
M1ind = min(find( MeanDistN<=L));
Mleft = T(M1ind);
Sleft = SdevN1(M1ind)*i %
pdfL = normpdf(T,Mleft,Sleft);
pdfL(isnan(pdfL))=0;
WeightedPdfL = pdfL*StartPointProb(i);
AllProb = AllProb+WeightedPdfL;
end; % for i

Comparison of theoretical expectations and empirical observations using the odor threshold test

Finally, to verify whether the theoretical model correctly describes the olfactory threshold test, the previously independent analyses, i.e., the empirical analyses of the soft-coded odor threshold test and the coding of the theoretical description of the underlying stochastic process, were compared. This first confirmed that odor thresholds generated by a pure random process in the olfactory test are bimodally distributed (Fig. 8). The second mode appears approximately at T = 5, which corresponds to the empirical observation in the software-encoded odor threshold test (compare Fig. 3). Second, the starting points with a maximum frequency at T = 1, but also with a decreasing frequency from T = 16 to T = 2 followed exactly the expectation from the description of the starting point determination as a random walk (Fig. 8). Therefore, it can be assumed that the theoretical description of the odor test, particularly the odor threshold subtest, correctly captures the true test.

Proposed test modification to reduce the likelihood of high random scores

Simulations and their theoretical translation have pointed to a crucial part of the test where improvements can be made to make it less likely that an anosmic person will pass the test with the result of being able to smell, although this was generated only by a rare series of correct guesses. That is, the strongest association of accidentally high TDI values is with the initial starting points of the threshold tests. The design of starting with the highest dilution of T_start = 16 was chosen in the original test to avoid premature adaptation and habituation of the subject, which was feared when starting with the highest concentration. The ideal starting point from a random results point of view, however, would be T_start = 1. To retain the original physiologically based idea, T_start = 8 instead of T_start = 16 is proposed. No further changes in the test algorithm are proposed. Thus, the starting point finding is performed only in the direction to T = 1C. If T = 8 is recognized twice in a row, the subsequent staircase part of the test will start immediately. Corresponding simulations showed that with the proposed modification, n = 4,107 (4.1%) instances were above the limit of anosmia at TDI = 16 by accident. TDI = 16 now represents the 97th percentile of random results. Expressed in terms of test performance to detect anosmic subjects, the modification would increase the accuracy from 80.1–95%. The availability of a theoretical background and the ability to fully simulate the outcomes of the olfactory test when run with random choices only allows cut-off limits for anosmia to be calculated for each possible combination of the three olfactory subtests. The corresponding values for all combinations of the T, D and I subtests are shown in Fig. 9 for the modified test algorithm.

Random walks are a common phenomenon that occur in many areas, such as molecular motion, the growth of bacterial colonies, the movements of microorganisms ^23–26. One-dimensional random walks are also a standard approach to analyzing stock price movements ^27–29. An early description of random walks goes back more than 100 years ³⁰, and several modifications and specifications have been added, from which the current description of the olfactory test could be derived. The present clinical test for assessing odor thresholds can be described as one-dimensional biased random walks (BRW) with highly unbalanced probabilities for upward (11%) and downward (89%) movements. The walk is complicated by the nesting of two components, the first consisting of the determination of the starting point for the next walk, which consists of the determination of the subsequent turning points for threshold calculation. The first component is unidirectional, i.e., the movement can only go in the direction of lower values. The second part of the test is an up and down movement.

The formal solution and the empirical soft-coded experiments consistently pointed to the starting point as a critical determinant of the subsequent turning points that ultimately determine the outcome of the odor threshold test. This is consistent with the inclusion of the starting point in an early formalization of random walks ³⁰ as \({X}_{n}={x}_{0}+\sum _{j=1}^{n}{Z}_{j}\), where \({\left({X}_{n}\right)}_{n\in {\mathbb{N}}_{0}}\)describes the stochastic process leading to the actual position in the walk after j steps from the start point \({x}_{0}\)In the present formalization (see Results section), the main determinants of the final results in terms of threshold score obtained by random choices are (i) the starting point, T_start, (ii) the length of the walk, t, and the probabilities in the bias component.

Starting from a high position involves a non-negligible chance of staying in high positions, even if the ability that would drive an upward movement is lacking. According to the rules of the test, the probability of a downward movement is still quite high (11%). On the other hand, if the starting position is low, it is very difficult to reach a high position by chance. Thus, according to the actual test instructions, starting at T = 16 includes a non-negligible chance of remaining without olfactory function at higher odor dilutions in the subsequent test, which applies to all published and unpublished applications of the Sniffin` Sticks test to date. As a result, the actual boundary of anosmia at TDI < 16 marks the 87th percentile. This behavior can be corrected by starting at T = 8, which is a result of the present analyses and raises the cutoff of anosmia at T < 16 to the 97th percentile of randomly generated TDI scores. Finally, the starting point alone, without the subsequent biased random walk, proved to be an inadequate approach to preventing guessed high TDI scores. If only the first part of the random walk, used to determine the starting point, was repeated 10 times and the average of these starting points was used as the threshold, the 90th percentile of the TDI scores would increase to TDI > = 20.

The formal solution and the empirical soft-coded experiments also emphasized that the length of the random walk, denoted as time t in the formula given in the results section of this report, is an important factor in the actual position in the staircase, which over time has a greater chance of being among the lower scores due to the unbalanced probabilities. In the current experiments, it appears that the use of later turning points for averaging to the final odor threshold shifts the 90th percentile to lower values, i.e., produces the desired reduction in the probability of high test scores possible by mere guessing at the test. However, a clinical trial that is performed on a patient cannot be continued for an indefinite period of time. In fact, the focus of olfactory test development over the past two decades has been on reducing test burden rather than increasing specificity to detect true loss of olfactory function, triggering proposals of so-called “short” olfactory tests ^31–36. Therefore, shifting the relevant turning points was considered second only to shifting the starting point in the present proposal to reduce the likelihood of false rejections of the diagnosis of anosmia due to chance results. Nevertheless, the present experiments indicate that attempts to shorten the olfactory threshold test by using earlier turning points ³⁶ should be undertaken with great caution. Given the importance of short test times in clinical practice, the change in probabilities of the bias component of the random walk was not further analyzed. Extending the forced-choice design beyond the current 3-alternative variant would certainly increase testing time and could at best be a rescue measure if other means fail, which, as discussed above, was not the case.

Random walks or mentions of the staircase paradigm are quite common in the biomedical literature. A search of the PubMed database on March 17, 2023 for “(((("staircase paradigm" OR "staircase approach" OR "staircase method" OR "random walk")) AND ((human or patient or subject or volunteer))) NOT ((injury OR accident))) NOT (review[Publication Type])" returned 1,464 results. “Random walk" alone without "staircase" returned 1,241 hits. Extraction of the most frequent words from the titles of publications, using the R package "PubMedWordcloud" (https://cran.r-project.org/package=PubMedWordcloud ³⁷), yielded seven words that were mentioned more than 60 times (arbitrarily chosen cut-off), namely diffusion, cancer, networks, gene, disease, cell, and dynamics. Among the less self-explanatory words in the present random walk context, genes were involved in the context of describing gene interaction network with random walk based models ³⁸, diseases are addressed from several perspectives by random walk approaches, such as improved diagnostics ³⁹ or identification of gene/mRNA disease interaction networks ⁴⁰, drug-target interaction networks in a pharmacological analysis of side effect prediction ⁴¹, and dynamics is used in several meanings, such as temporal dynamics ⁴² or molecular dynamics ⁴³.

The odor threshold subtest of the Sniffin` Sticks clinical olfactory test battery is not an isolated design in which random walks have been adapted. Sensory testing with random walks is rather common without the processes being named. In a recent report on changes in point pressure sensitivity as an early sign of Parkinson's disease, the authors specifically described the test design as a "state-of-the-art forced-choice staircase threshold test paradigm" ⁴⁴. Similarly, the determination of pain thresholds to mechanical or electrical noxious stimuli in a human experimental study was performed using a forced staircase paradigm similar to the olfactory test analyzed here ⁴⁵. Another example is the determination of cuff pain tolerance using a staircase paradigm ⁴⁶. The use of staircase paradigms for sensory testing extends to visual or acoustic stimuli for which the detection threshold in chicken has been determined using a staircase paradigm ⁴⁷. These are all random walks, although this type of process is barely mentioned by name in the sensory research context. Interestingly, a search for "("staircase paradigm") AND ("random walk")" returned an empty hit list.

Regarding randomness in a clinical olfactory performance test, one of its three components, the olfactory threshold test, was found to be a combination of random walks of a special type. To prevent the test from being too easy to pass with a high score, two consecutive trials with a choice of one out of three must be achieved. Statistically, this results in asymmetric probabilities of (1/3)² = 11% to achieve a higher score and 89% to achieve the next lower score. For pure guessing, the odor threshold test results in two successive biased random walk trials. In the present analyses of this process, empirical and formal approaches were applied independently and partly successively, partly in parallel. The concordant results provided mutual support for their correct implementation. This led to the proposal of a modification of the olfactory threshold test, which consists in shifting the starting point to the middle range of scores in order to reduce the risk of erroneously rejecting the diagnosis of anosmia based on purely guessed high scores in the odor threshold test. The availability of a theoretical background of random results in all three subtests of the Sniffin` Sticks test now makes it possible to establish precise cut-off values for the diagnosis of anosmia, i.e., the absence of olfactory function, that capture the test results possible by random selection at defined confidence limits. In addition, recognizing the staircase paradigm used in a variety of sensory and similar tests as a random walk provides a basis for estimating possible unexpected consequences of test modifications.

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Availability of data and material

The main parts of the Python and MATLAB code written to perform the present experiments and formal implementations are available as a part of this report.

Competing interests

The authors have declared that no competing interests exist.

Funding

JL was supported by the Deutsche Forschungsgemeinschaft (DFG LO 612/16-1). TH was supported by the Volkswagen Stiftung (project PERCEPTRONICS).

Author contributions

JL – Conceptualization of the project, software-encoding of the olfactory test, Python code, experimentation, mutual validation of empirical and formal results, writing of the manuscript, and creation of the figures.

TH – Clinical advice and discussion of details of the clinical olfactory test algorithm, writing of the manuscript.

AU – Developing of the formal solution, MATLAB code, mutual validation of empirical and formal results, writing of the manuscript.

Marin, C. et al. Olfactory Dysfunction in Neurodegenerative Diseases. Curr Allergy Asthma Rep 18, 42, doi:10.1007/s11882-018-0796-4 (2018).
Haehner, A., Draf, J., Dräger, S., de With, K. & Hummel, T. Predictive Value of Sudden Olfactory Loss in the Diagnosis of COVID-19. ORL J Otorhinolaryngol Relat Spec 82, 175-180, doi:10.1159/000509143 (2020).
Parma, V. et al. More Than Smell-COVID-19 Is Associated With Severe Impairment of Smell, Taste, and Chemesthesis. Chem Senses 45, 609-622, doi:10.1093/chemse/bjaa041 (2020).
Hummel, T., Sekinger, B., Wolf, S. R., Pauli, E. & Kobal, G. 'Sniffin' sticks': olfactory performance assessed by the combined testing of odor identification, odor discrimination and olfactory threshold. Chemical senses 22, 39-52 (1997).
Kobal, G. et al. "Sniffin' Sticks": Screening of olfactory performance. Rhinology 34, 222-226 (1996).
Kobal, G. et al. Multi-center investigation of 1036 subjects using a standardized method for the assessment of olfactory function combining tests of odor identification, odor discrimination, and olfactory thresholds. Eur.Arch.Otorhinolaryngol. 257, 205-211 (2000).
Hummel, T., Kobal, G., Gudziol, H. & Mackay-Sim, A. Normative data for the "Sniffin' Sticks" including tests of odor identification, odor discrimination, and olfactory thresholds: an upgrade based on a group of more than 3,000 subjects. Eur Arch Otorhinolaryngol 264, 237-243 (2007).
Oleszkiewicz, A., Schriever, V. A., Croy, I., Hahner, A. & Hummel, T. Updated Sniffin' Sticks normative data based on an extended sample of 9139 subjects. Eur Arch Otorhinolaryngol 276, 719-728, doi:10.1007/s00405-018-5248-1 (2019).
Van Rossum, G. & Drake Jr, F. L. Python tutorial. Vol. 620 (Centrum voor Wiskunde en Informatica Amsterdam, 1995).
Harris, C. R. et al. Array programming with NumPy. Nature 585, 357-362, doi:10.1038/s41586-020-2649-2 (2020).
McKinney, W. in Proceedings of the 9th Python in Science Conference 56 - 61 (2010 ).
pandas-dev/pandas: Pandas (Zenodo, 2020).
Lötsch, J., Hummel, T. & Ultsch, A. A unifying data driven model of human olfactory pathology representing known etiologies of dysfunction. Chem Senses (2016).
Kobal, G. et al. Multicenter investigation of 1,036 subjects using a standardized method for the assessment of olfactory function combining tests of odor identification, odor discrimination, and olfactory thresholds. Eur. Arch. Otorhinolaryngol. 257, 205-211 (2000).
Tukey, J. W. Exploratory Data Analysis. (Addison-Wesley, 1977).
Box, G. E. & Cox, D. R. An analysis of transformations. Journal of the Royal Statistical Society. Series B (Methodological), 211-252 (1964).
Ihaka, R. & Gentleman, R. R: A Language for Data Analysis and Graphics. Journal of Computational and Graphical Statistics 5, 299-314, doi:10.1080/10618600.1996.10474713 (1996).
R Development Core Team. R: A Language and Environment for Statistical Computing. (2008).
Hartigan, J. A. & Hartigan, P. M. The Dip Test of Unimodality. The Annals of Statistics 13, 70-84, doi:10.1214/aos/1176346577 (1985).
Erdös, P. & Révész, P. in Mathematical Statistics and Probability Theory: Volume B Statistical Inference and Methods Proceedings of the 6th Pannonian Symposium on Mathematical Statistics, Bad Tatzmannsdorf, Austria, September 14–20, 1986. 59-65 (Springer).
Hughes, B. D. Random Walks and Random Environments: Random walks. (Clarendon Press, 1995).
Papoulis, A. Probability, Random Variables, and Stochastic Processes. (McGraw-Hill, 1991).
Blumen, A., Klafter, J. & Zumofen, G. Models for Reaction Dynamics in Glasses.
Rucci, M. & Victor, J. D. The unsteady eye: an information-processing stage, not a bug. Trends Neurosci 38, 195-206, doi:10.1016/j.tins.2015.01.005 (2015).
Codling, E. A., Plank, M. J. & Benhamou, S. Random walk models in biology. J R Soc Interface 5, 813-834, doi:10.1098/rsif.2008.0014 (2008).
Kareiva, P. M. & Shigesada, N. Analyzing insect movement as a correlated random walk. Oecologia 56, 234-238, doi:10.1007/bf00379695 (1983).
Fama, E. F. Random Walks in Stock Market Prices. Financial Analysts Journal 21, 55-59, doi:10.2469/faj.v21.n5.55 (1965).
Cootner, P. H. The Random Character of Stock Market Prices. (M.I.T. Press, 1964).
Kendall, M. G. & Hill, A. B. The Analysis of Economic Time-Series-Part I: Prices. Journal of the Royal Statistical Society. Series A (General) 116, 11-34, doi:10.2307/2980947 (1953).
Pearson, K. The Problem of the Random Walk. Nature 72, 294-294, doi:10.1038/072294b0 (1905).
Toledano, A. et al. Development of a short olfactory test based on the Connecticut Test (CCCRC). Rhinology 47, 465-469, doi:10.4193/Rhin08.133 (2009).
Hummel, T., Pfetzing, U. & Lötsch, J. A short olfactory test based on the identification of three odors. J Neurol 257, 1316-1321, doi:10.1007/s00415-010-5516-5 (2010).
Jackman, A. H. & Doty, R. L. Utility of a three-item smell identification test in detecting olfactory dysfunction. Laryngoscope 115, 2209-2212 (2005).
Mueller, C. & Renner, B. A new procedure for the short screening of olfactory function using five items from the "Sniffin' Sticks" identification test kit. Am J Rhinol 20, 113-116 (2006).
Lötsch, J., Ultsch, A. & Hummel, T. How Many and Which Odor Identification Items Are Needed to Establish Normal Olfactory Function? Chem Senses, doi:10.1093/chemse/bjw006 (2016).
Croy, I. et al. Comparison between odor thresholds for phenyl ethyl alcohol and butanol. Chem Senses 34, 523-527, doi:bjp029 [pii] 10.1093/chemse/bjp029 [doi] (2009).
PubMedWordcloud: 'Pubmed' Word Clouds_. R package version 0.3.6, <https://CRAN.R-project.org/package=PubMedWordcloud> (2019).
Shen, F. et al. Prediction of Genetic Factors of Hyperthyroidism Based on Gene Interaction Network. Front Cell Dev Biol 9, 700355, doi:10.3389/fcell.2021.700355 (2021).
Arrais, J. P. & Oliveira, J. L. RecRWR: a recursive random walk method for improved identification of diseases. Biomed Res Int 2015, 747156, doi:10.1155/2015/747156 (2015).
Valentini, G., Paccanaro, A., Caniza, H., Romero, A. E. & Re, M. An extensive analysis of disease-gene associations using network integration and fast kernel-based gene prioritization methods. Artif Intell Med 61, 63-78, doi:10.1016/j.artmed.2014.03.003 (2014).
Hu, B., Wang, H. & Yu, Z. Drug Side-Effect Prediction Via Random Walk on the Signed Heterogeneous Drug Network. Molecules 24, doi:10.3390/molecules24203668 (2019).
Suki, B. & Frey, U. Temporal dynamics of recurrent airway symptoms and cellular random walk. J Appl Physiol (1985) 95, 2122-2127, doi:10.1152/japplphysiol.00636.2003 (2003).
Feng, Y., Zhang, L., Zhao, C., Li, N. & Chang, B. Investigation of the underlying mechanism of Zhibai Dihuang Pill in treating osteoporosis by using network pharmacology and molecular dynamics simulation. Endocr Metab Immune Disord Drug Targets, doi:10.2174/1871530323666221214144142 (2022).
Doty, R. L. et al. Point pressure sensitivity in early stage Parkinson's disease. Physiol Behav 138, 21-27, doi:10.1016/j.physbeh.2014.09.015 (2015).
Lötsch, J. & Angst, M. S. The mu-opioid agonist remifentanil attenuates hyperalgesia evoked by blunt and punctuated stimuli with different potency: a pharmacological evaluation of the freeze lesion in humans. Pain 102, 151-161 (2003).
Lindskou, T. A., Christensen, S. W. & Graven-Nielsen, T. Cuff Algometry for Estimation of Hyperalgesia and Pain Summation. Pain Med 18, 468-476, doi:10.1093/pm/pnw168 (2017).
Verhaal, J. & Luksch, H. Multimodal integration in the chicken. J Exp Biol 219, 90-95, doi:10.1242/jeb.129387 (2016).
Waskom, M. L. seaborn: statistical data visualization. Journal of Open Source Software 6, 3021 (2021).
Wickham, H. ggplot2: Elegant Graphics for Data Analysis. (Springer-Verlag New York, 2009).
ggforce: Accelerating 'ggplot2' (2020).

(Not answered)

A perspective of randomness in a clinical test of olfactory performance

Status:

Version 1

Abstract

Background

Methods

Results

Conclusions

Figures

Introduction

Methods

Description of the clinical olfactory test algorithm

Software coding of the clinical olfactory test algorithm

Investigation of random results of the clinical olfactory test algorithm

Results

Analyses of results of the soft-coded olfactory test on random choices

Correct operation of the software coded olfactory test algorithm for odor threshold assessment

Distributions of soft-coded olfactory tests on random choices

Olfactory test score limits for the diagnosis of anosmia based on random choices

Significance of specific details of the odor threshold test to random olfactory test results

Translation of the observed test behavior into a theoretical probabilistic solution

Description of the odor threshold test as a random walk

Comparison of theoretical expectations and empirical observations using the odor threshold test

Proposed test modification to reduce the likelihood of high random scores

Discussion

Conclusions

Declarations

Ethics approval and consent to participate

Consent for publication

Availability of data and material

Competing interests

Funding

Author contributions

References

Additional Declarations

Status:

Version 1