Improved tactile speech robustness to background noise with a dual-path recurrent neural network noise-reduction strategy

doi:10.21203/rs.3.rs-3395475/v1

Download PDF

Article

Improved tactile speech robustness to background noise with a dual-path recurrent neural network noise-reduction strategy

https://doi.org/10.21203/rs.3.rs-3395475/v1

This work is licensed under a CC BY 4.0 License

Journal Publication

published 28 Mar, 2024

Read the published version in Scientific Reports →

You are reading this latest preprint version

Many hearing-impaired people struggle to understand speech in background noise, making noise robustness critical for hearing-assistive devices. Recently developed haptic hearing aids, which convert audio to vibration, can improve speech-in-noise performance for cochlear implant (CI) users and assist those unable to access hearing-assistive devices. Haptic hearing aids are typically worn on the body rather than the head, allowing additional space for batteries and microprocessors. This means they can deploy more sophisticated noise-reduction techniques. In the current study, we assessed whether a real-time-feasible noise-reduction strategy, using a dual-path recurrent neural network (DPRNN), improves the noise robustness of haptic hearing aids. Audio was converted to vibration on the wrist using a previously developed vocoder method, either with or without noise reduction. In 16 participants, tactile-only sentence identification was measured for speech in quiet and in multi-talker background noise. The DPRNN improved sentence identification in noise by 8.2% on average and did not affect performance in quiet. This suggests that advanced techniques like the DPRNN could substantially improve outcomes with haptic hearing aids. Low cost, non-invasive haptic devices could soon be an important supplement to hearing-assistive devices such as CIs or offer an alternative for people who are unable to access CI technology.

Health sciences/Medical research/Translational research

Biological sciences/Neuroscience/Auditory system

Biological sciences/Neuroscience/Sensorimotor processing

Biological sciences/Neuroscience/Sensory processing

Hearing-assistive devices often fail to effectively extract speech from background noise¹. This means that users typically struggle to follow conversations in challenging listening environments, such as classrooms, cafes, and busy offices. Cochlear implant (CI) users tend to have particular difficulty in these scenarios². Several recent studies have shown that supplementing the electrical CI signal by providing speech information through haptic stimulation (“electro-haptic stimulation”³) can improve CI users’ speech-in-noise performance^3–9. There is also evidence that haptic hearing aids can be effective as standalone sensory substitution devices for the many millions of people around the world who could benefit from CI technology but are unable to access it due to medical, financial and other constraints^5,10–13. For both electro-haptic stimulation and sensory substitution to be clinically effective, audio-to-haptic conversion strategies that are robust to background noise must be developed.

Some previous studies have avoided the issue of noise robustness by converting the clean speech signal to haptic stimulation, rather than the speech-in-noise signal that would be received by the microphone of a haptic hearing aid^6–8. While these studies can provide a proof-of-concept, they leave unaddressed a major obstacle in translating laboratory benefits to the real world. Other studies have converted the speech-in-noise signal to haptic stimulation and have shown considerable benefits to speech recognition^3,4,9. The method used to reduce the impact of noise in these studies was envelope expansion, which enhances high intensity parts of the haptic signal. While envelope expansion is effective when speech is considerably more intense than noise^3,4,9, it breaks down at more challenging signal-to-noise ratios (SNRs)^3,4. If haptic signal extraction strategies that are robust in these more challenging listening scenarios can be developed, this would substantially widen the group of hearing-impaired people that could benefit from haptic hearing aids.

One advantage of haptic hearing aids compared to other hearing-assistive devices, such as CIs or acoustic hearing aids, is that they are worn on the body (e.g., the wrist) rather than the head. This means that they have significantly more form-factor flexibility and design space available. This allows the use of larger batteries and more powerful microprocessors that can deploy sophisticated noise-reduction techniques¹⁴. The current study explored whether an advanced real-time-feasible noise-reduction strategy, using a recurrent neural network, can effectively extract the haptic speech signal from background noise at challenging SNRs to improve tactile speech perception.

Noise-reduction or “speech enhancement” techniques, aim to enhance the intelligibility of speech in background noise. Traditional approaches using statistical models have failed to improve the intelligibility of speech in noisy environments commonly encountered in the real world, which contain competing speech and other fluctuating sounds¹⁵. Recently, data-driven machine-learning methods have revolutionised noise reduction, particularly for extracting speech in these more challenging scenarios. Such methods, which are typically based on deep learning (including variants of artificial neural networks), can substantially improve speech intelligibility for normal-hearing¹⁶, hearing-impaired¹⁷ and CI^18,19 listeners. One of the most effective deep-learning noise-reduction methods developed in this rapidly advancing field is the dual-path recurrent neural network (DPRNN)²⁰. The DPRNN operates on the long-term speech signal while retaining high temporal and spectral resolution. Critically, it also has excellent generalisation to unseen talkers and background noises.

In the current study, we assessed whether a real-time-feasible DPRNN can improve tactile-only sentence identification in a realistic background noise. The talkers and noise sample used during testing had not been used to train the DPRNN. For audio-to-tactile conversion, the audio was processed with a DPRNN before being converted to vibro-tactile stimulation on the wrist using a previously developed tactile vocoder approach^3,4,9,10. The tactile vocoder filters the audio into several frequency bands, and the amplitude envelope from each band is used to modulate the amplitude of one of several vibro-tactile tones. This approach has been shown to be effective for transferring phoneme information¹⁰ and improving speech-in-noise performance for CI users^3,4,9.

For testing speech-in-noise performance with and without the DPRNN, the SNR was set to 2.5 dB. Note, however, that this SNR is equivalent to around 0 dB SNR in many previous studies, because calculation was made after removing silences in the speech (see “Methods”). This represents the more challenging SNRs that hearing-impaired listeners encounter in the real world^21,22 and is a level at which most CI users have no appreciable speech intelligibility^3,9. The noise was a real-world recording from a party, which is representative of environments that hearing-impaired listeners typically find most challenging, with competing speech and other fluctuating and transient sounds. In addition to testing sentence identification in noise with and without the DPRNN, the effect of the DPRNN on sentence identification without background noise was tested to ensure that it did not impair performance in quiet listening conditions. The DPRNN was expected to have no effect in quiet but to produce a clear benefit in noise.

To complement the behavioural performance measures, the DPRNN performance was also assessed objectively. This was done by assessing the degree of correlation between the amplitude envelopes for the clean speech without the DPRNN and the envelopes with the DPRNN, either in quiet or in noise. This was intended to offer a quantification of the algorithm performance required to produce any behavioural change observed in the study. This could act as a reference point for future development of noise-reduction techniques for improving tactile speech-in-noise performance.

Behavioural performance measures

Figure 1 shows the percentage of sentences correctly identified in each condition for the 16 participants who took part in the study. The primary analysis was a repeated-measures analysis of variance (RM-ANOVA), with factors talker gender (male or female), noise (on or off), and noise reduction (on or off). A significant overall effect of noise (F(1, 15) = 77.0, p < .001) and of noise reduction (F(1, 15) = 7.1, p = .018) was found, as well as a significant interaction between noise and noise reduction (F(1, 15) = 15.8, p = .001). No main effect of talker gender (F(1, 15) = 1.5, p = .238) and no interaction between talker gender and any other factor was found (noise: F(1, 15) = 1.1, p = .311; noise reduction: F(1, 15) = 0.8, p = .780; noise and noise reduction: F(1, 15) = 0.1, p = .711).

Four planned t-tests were then run, with multiple comparisons correction applied (see “Methods”). The first established that the noise significantly reduced sentence recognition without noise reduction (t(15) = 7.4, p < .001). Performance reduced by 15.6% (ranging from 25.8 to 0%; standard deviation (SD): 8.5%), from 66.2% in quiet (ranging from 53.3 to 79.2%; SD: 7.9%) to 50.6% in noise (ranging from 45 to 57.5%; SD: 4.2%). Noise reduction improved performance in noise by 8.2% on average (ranging from − 10 to 25%; SD: 7.7%; t(15) = 4.3, p = .003), with performance increasing to 58.8% (ranging from 47.5 to 79.2%; SD: 7.7%). Performance in noise with noise reduction was worse than performance in quiet without noise reduction by 7.5% (ranging from − 3.3 to 16.7%; SD: 5.2%; t(15) = 5.7, p = < .001). In quiet, no significant change in performance with noise reduction was found (mean reduction in performance of 1.9%; t(15) = 1.3, p = .668).

No difference in the benefit of noise reduction was found across the male and female talker (t(15) = 0.4, p > 1). Noise reduction improved performance in noise by 7.5% for the male talker (ranging from − 18.3 to 21.7%; SD: 10.8%) and by 8.9% for the female talker (ranging from − 6.7 to 33.3%; SD: 9.9%). There was also no difference found in the effect of noise reduction in quiet between the talkers (t(15) = 0.1, p > 1), with average performance with noise reduction reducing by 1.8% for the male talker (ranging from a 16.7% reduction to a 10.0% benefit; SD: 8.4%) and by 2.1% for the female talker (ranging from a 15.0% reduction in performance to a 5.0% benefit; SD: 6.3%).

Finally, exploratory post-hoc correlation analyses were run (see “Methods”). These assessed the relationship between either participant age or vibro-tactile detection thresholds and either sentence identification scores in quiet without noise reduction or the improvement in sentence identification scores in noise with noise reduction. No evidence of a correlation between vibro-tactile detection thresholds and sentence identification was found (in quiet: r = 0.13, p = .632; benefit of noise reduction: r = 0.10, p = .713). There was also no clear evidence of a correlation between age and sentence identification in quiet (r = 0.37, p = .161). There was a trend for older participants to get more benefit of noise reduction, but this correlation between age and benefit did not reach significance (r = 0.47, p = .067).

Objective performance measures

Figure 2 shows the results of an objective assessment of the DPRNN performance. The correlation was measured between the clean tactile vocoder amplitude envelopes (without the DPRNN) and the envelopes in background noise, either with or without the DPRNN. Without the DPRNN, there was a mean r of 0.62 (ranging from 0.13 to 0.93; SD: 0.23) for band 1 (the lowest frequency band), 0.77 (from 0.26 to 0.93; SD: 0.13) for band 2, 0.74 (from 0.40 to 0.90; SD: 0.11) for band 3, 0.71 (from 0.05 to 0.94; SD: 0.18) for band 4, 0.76 (from 0.39 to 0.96; SD: 0.14) for band 5, 0.80 (from 0.40 to 0.96; SD: 0.13) for band 6, 0.76 (from 0.16 to 0.95; SD: 0.16) for band 7, and 0.60 (from − 0.03 to 0.98; SD: 0.24) for band 8. When averaged across all bands for each sentence, the mean r was 0.72 (from 0.49 to 0.88; SD: 0.08).

With the DPRNN, there was a mean r of 0.89 (ranging from 0.33 to 0.97; SD: 0.08) for band 1, 0.94 (from 0.82 to 0.98; SD: 0.03) for band 2, 0.95 (from 0.81 to 0.98; SD: 0.03) for band 3, 0.95 (from 0.71 to 1.00; SD: 0.05) for band 4, 0.96 (from 0.81 to 1.00; SD: 0.04) for band 5, 0.96 (from 0.77 to 1.00; SD: 0.04) for band 6, 0.94 (from 0.71 to 0.99; SD: 0.048) for band 7, and 0.77 (from 0.20 to 0.99; SD: 0.18) for band 8. When averaged across all bands, the mean r was 0.92 (from 0.82 to 0.97; SD: 0.04). This was an improvement of 0.20 (from − 0.04 to 0.46; SD: 0.09) compared to the tactile speech in noise without the DPRNN. The correlation for the DPRNN in quiet was greater than 0.999 (SD: <0.001) in all bands.

DPRNN noise reduction substantially improved sentence identification in background noise and had no effect on performance in quiet. The improvement was consistent across the male and female talkers tested, whose acoustic properties differed considerably (see “Methods”). The pattern of results observed with the DPRNN would therefore be expected to generalise to a wider range of talkers. The SNR and acoustic characteristics of the noise used in testing are representative of scenarios in which hearing-impaired people commonly struggle in their everyday life. They also represent a situation in which previously developed real-time noise-reduction methods for audio-to-tactile conversion are ineffective^3,4,9. The improved noise-robustness with the real-time-feasible DPRNN could dramatically increase the utility of haptic hearing aids, both when used as stand-alone sensory substitution devices and when used for sensory augmentation with CIs.

For speech in noise, an objective assessment of the DPRNN performance showed it improved the overall correlation with clean speech by an r of 0.2 (increasing from 0.72 without the DPRNN to 0.92 with the DPRNN). As this change was associated with a notable improvement in behavioural performance, it may act as a useful reference point for the design of future noise reduction algorithms for tactile speech. The objective assessment showed no appreciable degradation of the tactile speech in quiet signal by the DPRNN. This agrees with the finding of no decline in performance in quiet with the DPRNN in the behavioural assessment.

While behavioural performance in background noise was substantially improved by the DPRNN, it was unable to recover performance in noise fully so that it matched performance in quiet. There are several possible reasons for this. Figure 3 shows the amplitude envelopes from the tactile vocoder used in the current experiment for one of the sentences tested. Envelopes are shown with and without background noise and both with and without noise reduction applied. By comparing the clean speech without noise reduction to the speech in noise with noise reduction, it can be observed that, while strong noise rejection was achieved, some noise artefacts remained (for example, in channel 8 before the start of the sentence). These artefacts may have acted as distractors that increased temporal uncertainty^23–25 and disrupted estimation of key speech landmarks, such as sentence, syllable, and phoneme start and end points. Another subtler difference between the clean envelopes and the noise-degraded envelopes after noise-reduction processing is the temporal smoothing of some envelopes. An example can be seen in channel 2 for the words “…in a…”. This smoothing might reduce the salience of speech segments, particularly for speech tokens in which rapid envelope changes signal phoneme identity (plosive consonants) or aid speech segmentation.

Noise-reduction strategies balance removal of background noise and distortion of the speech signal²⁶. For practical applications, speech distortions can be mitigated by limiting the aggressiveness of the noise-reduction processing, for example by mixing the DPRNN output signal with the original speech-in-noise signal. The optimal balance will usually depend on multiple factors, such as the acoustic challenge the user typically faces (e.g., the SNR and the character of the background noise) and user characteristics (e.g., the amount of speech distortion that can be tolerated). Data on user characteristics is currently lacking for haptic hearing aids, and further work is required to establish the optimal balance of noise rejection and speech distortion.

It is possible that some of the factors that impaired performance in noise would have had reduced impact or been fully overcome if participants had received training with tactile speech. Previous studies both with tactile sensory substitution for speech in quiet¹¹ and sensory augmentation for speech in noise^3,4 have shown significant improvements in performance after training. This improved ability to extract speech information from tactile stimulation might improve robustness to noise artefacts. This suggestion is supported by studies in hearing, which have shown that training can reduce the impact of masking sounds on performance^27,28. Future work should establish the importance of training when assessing noise-reduction strategies for tactile speech, such as the DPRNN, to test whether listeners can acclimatise to the artefacts imposed by the noise-reduction processing.

The DPRNN might be further optimized for tactile speech to improve noise robustness and reduce computational complexity. One way to further optimise the DPRNN might be to place it later in the signal-processing chain. In the current study, the DPRNN was used as front-end noise reduction, being applied to the audio before the tactile vocoder. Future studies could explore whether, for example, the DPRNN is more effective and efficient if processing only the audio from the eight frequency bands in the tactile vocoder, which are designed to focus on frequencies most important to speech perception. Alternatively, the DPRNN could be trained specifically to extract the clean tactile speech envelopes from noise, rather than the clean audio. This would allow the speech synthesis (decoder) stage of the DPRNN algorithm to be removed and therefore for the computational complexity and processing latency to be reduced. Furthermore, the much smaller spectral resolution of the tactile signals (eight channels in the current study) compared to the spectral resolution of acoustic audio may allow a substantial reduction in the complexity of the DPRNN algorithm without compromising performance. Finally, the power efficiency of the DPRNN might be improved by selectively activating it only when background noise is detected. There are several existing environmental classification algorithms that could be used to achieve this (e.g., ²²).

Another approach to improving the effectiveness of the DPRNN might be to provide it with multiple microphone signals, rather than single-channel audio. In previous work, the audio received at microphones behind each ear was converted to haptic stimulation on each wrist so that spatial hearing cues could be exploited through haptics. This has been shown to improve sound localisation^29–32 and speech recognition for spatially-separated speech and noise in CI users⁹. The improvements in speech-in-noise performance were likely achieved, at least in part, through access to better speech information from the ear with the best SNR. In future iterations of the DPRNN for haptic hearing, this “better-ear-listening”, or more advanced multi-microphone noise-reduction methods (e.g., ^33,34) could be exploited.

There are important limitations to the current study that should be noted. Firstly, a new tactile sentence identification task was used, which used a forced-choice method to avoid floor effects and circumvented the need for prolonged tactile speech training. Unlike some previous methods for assessing tactile speech performance, which focus on phonemes¹⁰ or words¹¹, this method was intended to allow the use of a range of key speech processing factors. These include segmentation and recognition of phonemes and words in running speech and integration of sensory and higher-level (e.g., top-down) processing. Improved ability to discern sentences in background noise is expected to strongly predict whether haptic stimulation can work effectively, either when delivered alone or when supplementing a degraded acoustic signal (e.g., through a CI) or lip reading. However, further work is required to conclusively establish this.

Another limitation of the current study was that the participant group did not match the target user group. One difference was that the participants had no known hearing impairment. Previous studies have found no difference in tactile speech performance in quiet^10,12,35 or in background noise^3,4,9 between those with and without hearing loss. However, there is evidence that tactile sensitivity might be higher in those with congenital deafness³⁶. This might mean that the current results underestimate performance for some hearing-impaired groups.

The participant group also did not match the age range of the target user group. Previous studies have found no relationship between age and tactile speech performance^3,4,9,10. In the current study, there was also no evidence of a correlation between age (which spanned 19 years) and tactile speech performance in quiet. There was weak evidence that older users might perform better when extracting speech from background noise, but this result would need to be replicated in a larger study to become compelling. While previous work has found no worsening with age of either temporal gap detection³⁷ or intensity discrimination^31,38 for vibro-tactile tones, detection thresholds are known to worsen with age³⁹. This may mean that adjusting the dynamic range of tactile stimulation to take account of differences in sensitivity will be important for maximizing the effectiveness of haptic hearing aids.

The current study has shown that a real-time-feasible DPRNN noise-reduction method can substantially improve tactile sentence identification in noise, while not impairing performance in quiet. Advanced noise-reduction methods, such as the DPRNN, are more viable for haptic hearing aids because of the greater design space available compared than other hearing-assistive devices, such as hearing aids and CIs. The DPRNN could dramatically increase the effectiveness of haptic hearing aids both when used for sensory augmentation alongside devices such as CIs and when used to aid the many millions of people around the world who are unable to access CI technology.

Participants

Table 1

Participant characteristics. Each participant’s vibro-tactile discrimination thresholds (measured during screening), wrist temperature during testing, wrist dimensions, dominant hand, age, and sex are shown.
ID	31.5 Hz thresh. (m/s^− 2)	125 Hz thresh. (m/s^− 2)	Wrist temp. (°C)	Wrist height/ width (mm)	Wrist circum. (mm)	Dom. hand (L/R)	Age (yrs.)	Sex (M/F)
1	0.02	0.08	30.3	39/58	166	R	36	M
2	0.02	0.09	28.8	36/48	150	R	22	F
3	0.04	0.08	31.6	36/48	149	L	18	F
4	0.05	0.07	31.6	41/52	160	R	29	M
5	0.06	0.41	32.2	43/55	172	R	37	M
6	0.03	0.03	28.4	32/49	149	R	21	F
7	0.05	0.03	25.6	44/59	171	R	19	M
8	0.05	0.10	31.8	41/50	160	R	23	M
9	0.05	0.06	28.4	43/51	168	R	18	F
10	0.08	0.15	30.1	41/48	148	L	20	F
11	0.08	0.20	28.6	40/49	154	R	20	F
12	0.02	0.03	31	35/51	148	R	28	F
13	0.01	0.05	31	41/51	166	R	26	F
14	0.09	0.29	27.9	39/51	162	R	20	M
15	0.05	0.05	31.2	38/55	153	R	18	F
16	0.02	0.06	34	33/46	138	R	21	F
Mean	0.05	0.11	30.2	39/51	157	-	24	-

Table 1 shows the characteristics of the 16 adults who completed the experiment. There were 6 males and 10 females, with an average age of 24 years (ranging from 18 to 37 years). Participants all had normal touch perception, which was assessed through a screening questionnaire and by measuring vibro-tactile detection thresholds at the fingertip (see “Procedure”). All participants had British English as their first language and reported no hearing problems. Participants were paid an inconvenience allowance of £20 for taking part.

Stimuli

The tactile stimulus in the experiment phase (after screening), was generated using the EHS Research Group Sentence Corpus. This contained 83 sentences, each spoken by both a British English male and British English female talker. The sentences were taken from readings (connected discourse) of a public engagement article written in a semi-conservational style⁴⁰. They contained a range of natural variations in prosodic pattern, speaking rate, pitch, and phoneme pronunciation.

The long-term spectrum for each talker is shown in Fig. 4. The male talker had an average fundamental frequency of 147.9 Hz (ranging from 80.5 to 220.7 Hz; SD: 19.0 Hz) and the female talker had an average fundamental frequency of 205.5 Hz (ranging from 108.2 to 285.7 Hz; SD: 31.4 Hz). The fundamental frequency (estimated using a Normalized Correlation Function) and the harmonic ratio were determined using the MATLAB audioFeatureExtractor object (MATLAB R2022b). A 300-ms Hamming window was used, with a 30-ms overlap length. Samples were included in the analysis if their harmonic ratio was greater than 0.8.

The EHS Research Group Sentence Corpus was recorded in an anechoic chamber at the Institute of Sound and Vibration Research (UK). The audio was recorded using a Rode M5 microphone and RME Fireface UC soundcard (with a 96 kHz sample rate and a bit depth of 24 bits). The microphone was 0.2 m from the talker’s mouth.

Table 2

The subset of sentence pairs used in the experiment. For each pair, the table shows how much the syllable count differs between the sentences, how many syllables there are, and the sentence text. The spoken text is the same for the male and female talker.
Set	N Syl.	Sentence 1 text	Sentence 2 text
Match	3	However	Did it work?
Match	4	Blocked by your head	For example
Match	5	Over and over	And the one they chose
Match	5	For many people	Interestingly
Match	6	It tells you where things are	But to reach your left ear
Match	8	In India, for example	When you block it by closing a door
Match	8	If you're in a noisy hall	This can cause a lot of problems
Match	9	We were very pleased with what we found	They could help people across the world
Match	10	They keep on getting better and better	Together with a team of researchers
Match	13	We calculated how well they located the sound	And the music blaring from the speaker to your left
1 diff	2/3	So far	Who is shy
1 diff	2/3	Quieter	We hoped that
1 diff	4/5	To get a job	Through another sense
1 diff	4/5	Ears, nose, and mouth	To make this happen
1 diff	6/7	And who likes to show off	Between the correct speaker
1 diff	6/7	Touch and temperature	When using only their ears
1 diff	11/12	You listen all the more closely for footsteps	This difficulty isn't a temporary one
1 diff	11/12	And is always hungrily searching for more	Perhaps we can send the missing information
1 diff	14/15	They might be especially useful in poorer countries	They tended to be much closer to the correct location
1 diff	14/15	Which are a type of surgically fitted hearing aid	Less than a third of children with hearing problems go to school
2 diff	7/9	Our volunteers performed best	If you don't know where it's coming from
2 diff	7/9	Could overcome these problems	You can see this from the big blue bar
2 diff	7/9	As they go about their day	As illustrated in figure one
2 diff	7/9	That we took advantage of	The wristbands we are developing
2 diff	9/11	We measured this distance in degrees	Like the voice of the person in front of you
2 diff	9/11	That people can wear outside the lab	This is one of the main ways your brain works out
2 diff	10/12	Or expensive medical equipment	Adults with hearing problems in poorer countries
2 diff	10/12	Have been invented to solve this problem	And so are often forced to live in poverty
2 diff	11/13	But because a sense isn't working properly	If you're trying to follow a conversation
2 diff	11/13	Now we're looking to create a device	Were converted into vibration on the right wrist

A subset of 60 sentences from the EHS Research Group Sentence Corpus were used in the experiment (see Table 2). The cross-section of sentences contained a variety of prosodic patterns with different pitch contours, total lengths, and phoneme inventories. Each of the 60 sentences were spoken by both the male and female talker so that there were 120 speech samples in total. The sentences were grouped in pairs, with 10 of the pairs having the same number of syllables, 10 pairs differing by 1 syllable, and 10 pairs differing by 2 syllables. This was done to ensure trials spanned a wide range of difficulties.

For the conditions with background noise, a non-stationary multi-talker noise recorded by the National Acoustic Laboratories (NAL)⁴¹ was used. The noise sample was recorded at a party and has a long-term spectrum that matches the international long-term average speech spectrum⁴². This noise was selected to reproduce real-world challenges that haptic hearing aid users would face. For the speech-in-noise conditions, the speech and noise signals were mixed with a 2.5 dB SNR. Importantly, the RMS level of the speech was calculated with silences removed (though note that silences were not removed from the stimulus for presentation). Silent sections were identified by extracting the speech amplitude envelope using a Hilbert transform and a zero-phase 6th -order Butterworth low-pass filter, with a cut-off frequency of 23 Hz. Sections of the speech where the amplitude envelope dropped below 10% of the maximum were removed for the RMS level calculation. This meant that SNR setting in the current study was lower than for comparable studies where the silences were not removed for the SNR calculation. For the EHS Research Group Sentence Corpus, the SNR was 2.6 dB lower on average if silences are removed than if they were not (ranging across sentences from 0.9 to 4.7 dB lower; SD: 0.7 dB). For the more naturalistic speech material used in the current study – which contains a variety of prosodic characteristics, with differing syllabic stress patterns, speaking rate and overall modulation characteristics – silence-stripping was felt to be important to achieving a stable SNR across sentences.

On each trial, the masker duration was set to have a randomly selected delay of between 500 and 1,500 ms before and after the longest speech stimulus of the sentence pair (see Table 2). This was done to exclude the possibility that participants could learn when the sentence onset or offset would be and use total duration cues without detecting the speech signal. If the speech in the trial was the shorter sentence of the pair, it was located within the masker at a random point within the time window for the longer sentence. The speech and noise samples were ramped on and off with a 50-ms raised-cosine ramp. A new randomly selected section of the noise sample was used in each trial.

For conversion from audio to haptic stimulation, the audio was first downsampled to 16 kHz. This matches the typical sample rate available in compact wearable audio devices. For the conditions in quiet and in noise with noise reduction applied, the audio was first processed with the DPRNN algorithm shown in Fig. 5. This consisted of an end-to-end time-domain Audio Separation Network (TasNet) with three stages. In the first stage, a learned encoder block transformed the time-domain audio frames into a 2D feature space (similar to a time-frequency representation). The next stage consisted of a masking network that processed consecutive chunks within the latent feature space to estimate a noise-reduction mask. This mask was then applied to the noisy input speech representation to remove the background noise and to produce the enhanced speech signal. The final stage consisted of a learned decoder block, which transformed the enhanced speech signal back into a time-domain audio output signal.

The DPRNN algorithm used 1D-convolutional layers with 128 filters for the encoder and decoder blocks with a kernel size of 16 (1 ms) and a stride of 4 (0.25 ms) samples²⁰. The chunks processed by the masking network consisted of 20 frames, which limited the use of future information to 5.75 ms to allow real-time processing. The masking network consisted of 6 blocks of the DPRNN method, each with bottleneck and hidden dimensions of 256 units. Channel-wise normalisation was employed to ensure causal processing.

The DPRNN algorithm was implemented using the publicly available Asteroid PyTorch toolkit⁴³. It was trained on a large dataset of speech-in-noise stimuli comprising of speech utterances from the LibriSpeech corpus⁴⁴ mixed and noise samples from the WHAM! stimulus set⁴⁵ at various SNRs (total of 360 hours). The SNRs were sampled from a uniform distribution between − 6 to 10 dB, with 5% of the samples retained to ensure that performance for clean speech had not degraded. These speech and noise stimuli differed from those used during behavioural and objective assessment of the performance of the DPRNN. The training made use of the Adam optimizer⁴⁶ and the scale-invariant signal-to-distortion ratio⁴⁷ as loss functions for a total of 200 epochs. The training of the DPRNN algorithm was performed on three NVIDIA A100 40GB Tensor Core graphics processing units.

Following downsampling and, for some conditions, the application of noise reduction, the signal was converted to tactile stimulation using a tactile vocoder (following the method used previously by Fletcher, et al. ¹⁰). The audio was first passed through a 512th -order FIR filter bank with eight frequency bands, which were equally spaced on the auditory equivalent rectangular bandwidth scale⁴⁸ between 50 and 7,000 Hz. Next, the amplitude envelope was extracted for each frequency band using a Hilbert transform and a zero-phase 6th order Butterworth low-pass filter, with a cut-off frequency of 23 Hz (targeting the envelope modulation frequencies most important for speech recognition^49,50). These amplitude envelopes were then used to modulate the amplitudes of eight fixed-phase vibro-tactile tonal carriers.

The eight tactile tones had frequencies of 94.5, 116.5, 141.5, 170, 202.5, 239, 280.5 and 327.5 Hz. The frequencies remained within the frequency range that can be reproduced by the latest compact haptic actuators and were spaced so as to be discriminable based on frequency discrimination thresholds from the dorsal forearm⁵¹. A different gain was applied to each tone to make them equally exciting across frequency, based on tactile detection thresholds^10,52. The gains were 13.8, 12.1, 9.9, 6.4, 1.6, 0, 1.7, and 4 dB, respectively. Tactile stimuli were scaled to have an equal RMS amplitude, with a nominal level of 141.5 dB ref 10^− 6 m/s² (1.2 G). This intensity can be produced by a range of compact, low-powered haptic actuators. Pink noise was presented through headphones throughout speech identification testing at a level of 60 dBA to ensure any auditory cues were masked. During familiarisation, there was no masking noise, and the speech audio was played through the headphones at 65 dBA.

Apparatus

Participants sat in a vibration isolated, temperature-controlled room (mean temperature: 23°C; SD: 0.45°C). The room temperature and participant’s skin temperature were measured using a Digitron 2022T type K thermocouple thermometer, which was calibrated following ISO 80601-2-56:2017⁵³ using the method described in Fletcher, et al. ¹⁰. In screening, vibrotactile detection thresholds were measured with a HVLab Vibro-tactile Perception Meter⁵⁴. The circular probe was 6 mm in diameter and contacted the skin through a rigid surround with a 10 mm diameter. The probe gave a constant upward force of 1N. The downward force applied by the participant was measured using a force sensor built into the surround. This sensor was calibrated using Adam Equipment OIML calibration weights and the force applied was displayed to the participant. The Vibro-tactile Perception Meter output was calibrated using its built-in accelerometers (Quartz Shear ICP, model number: 353B43) and a Brüel & Kjær (B&K) Type 4294 calibration exciter. The system conformed to ISO-13091-1:2001⁵⁵ and the stimuli had a total harmonic distortion of less than 0.1%.

For the sentence identification task, the EHS Research Group haptic stimulation rig was used (see Fletcher, et al. ¹⁰). This consisted of a Ling Dynamic Systems V101 shaker suspended from an aluminium strut frame by an adjustable elastic cradle. The shaker had a downward facing circular probe with a 10-mm diameter, which contacted the participant’s dorsal wrist. A foam block with a thickness of 95 mm was placed below the shaker probe for participants to rest their palmar forearm on. The probe applied a downward force of 1N, which was calibrated using a B&K UA-0247 spring balance. The shaker was driven using a MOTU UltralLite-mk5 sound card, RME QuadMic II preamplifier, and HVLab Tactile Vibrometer power amplifier. The vibration output was measured using a B&K 4533-B-001 accelerometer and calibrated using a B&K type 4294 calibration exciter. All stimuli had a total harmonic distortion of less than 0.1%.

Masking noise in the experiment phase and speech audio in the familiarisation phase were played to the participant through Sennheiser HDA 300 headphones, driven by the MOTU UltralLite-mk5 sound card. A B&K G4 sound level meter with a B&K 4157 occluded ear coupler (Royston, Hertfordshire, UK) was used to calibrate the audio. Sound level meter calibration checks were carried out using a B&K Type 4231 sound calibrator.

Procedure

For each participant, the experiment was completed in a single session lasting approximately two hours. Participants first gave informed consent to take part in the study. They then completed a screening questionnaire, which ensured that they (1) had not had any injury or surgery on their hands or arms, (2) did not suffer from conditions that might affect their sense of touch, and (3) had not been exposed to intense or prolonged hand or arm vibration over the previous 24 hours. Their self-reported hearing health was also recorded.

Following this, the participant’s skin temperature on the index fingertip of the dominant hand was measured. When the participant’s skin temperature was between 27 and 35°C, their vibrotactile detection thresholds were measured at the index fingertip following BS ISO 13091-1:2001⁵⁵. During these measurements, participants applied a downward force of 2N, which was monitored by the participant and experimenter using the HVLab Vibro-tactile Perception Meter display. Participants were required to have touch perception thresholds in the normal range (< 0.4 m/s^-2 RMS at 31.5 Hz and < 0.7 m/s^-2 RMS at 125 Hz), conforming to BS ISO 13091‑2:2021⁵⁶. The fingertip was used for screening as normative data was not available for the wrist. If all screening stages were passed, the participant’s wrist dimensions were measured at the position they would usually wear a wristwatch, and they proceeded to the experiment phase.

In the experiment phase, participants sat in front of the EHS Research Group haptic stimulation rig¹⁰. They placed their arm on the foam surface with the shaker probe contacting the centre of the dorsal wrist, at the position where they would normally wear a wristwatch. During the experimental phase, participants completed a two-alternative focused-choice sentence identification task. This was done using a custom-built MATLAB (2022b) app with three buttons, one that allowed the user to play the stimulus for the current trial, and two that displayed the sentence text alternatives for the current trial. The app also displayed the instruction: “Select the text that matches the sentence played through vibration”. When the play button was pressed, both the buttons displaying the sentence text for the alternative response options turned green for the duration of the stimulus. The sentence text buttons were only selectable after the stimulus had been played at least once.

Before testing began, participants completed familiarisation to ensure they understood the task. In this stage, English male and female speech was used that was different from that used in testing. The male talker was from the ARU speech corpus (ID: 02)⁵⁷ and the female was from the University of Salford speech corpus⁵⁸. They each spoke four different sentences from the Harvard sentences set⁵⁹. Sentences were only played without background noise and the participant was permitted to feel the sentence through tactile stimulation or to hear the sentence through the headphones (without tactile signal processing). There was no limit placed on the number of times these sentences could be repeated and the participant was encouraged to ask the experimenter questions if they were unsure of the task. Once the participants selected one of the sentences, all buttons became inactive for 0.5 seconds and green text reading “Correct” was displayed if the response was correct, or red text reading “Incorrect” was displayed if the response was incorrect. Once the experimenter confirmed that the participant understood the task, they continued to the testing stage.

During testing, the participant did the same task as in familiarisation except that they could not opt to hear the audio stimulus, and the headphones played masking noise. Participants were also limited to a maximum of four repeats of the tactile stimulus per trial, after which the play button became inactive, and they were forced to select one of the two text alternatives. All sentences were tested for all four experimental conditions once (in quiet and in background noise, both with and without the DPRNN). This meant that there was a total of 480 trials for each participant in the testing stage. The trial order was randomised for each participant, with a rule that the same sentence could not appear in consecutive trials. None of the speech or background noise material used for testing was used during the training of the DPRNN.

The experimental protocol was approved by the University of Southampton Faculty of Engineering and Physical Sciences Ethics Committee (ERGO ID: 68477). All research was performed in accordance with the relevant guidelines and regulations.

Statistics

Behavioural performance measures

The percentage of correctly identified sentences was calculated for each condition for the male and female talkers. Primary analysis consisted of a three-way RM-ANOVA (see “Results”) and six two-tailed t-tests, with a Bonferroni-Holm multiple comparisons correction⁶⁰ applied. The t-tests compared: (1) identification in quiet with and without noise reduction; (2) identification in noise with and without noise reduction; (3) identification in quiet without noise reduction and identification in noise with noise reduction; (4) identification in quiet and in noise, both without noise reduction; (5) the difference in identification in quiet with and without noise reduction for the male and female talkers; and (6) the difference in identification in noise with and without noise reduction for the male and female talkers. Data were determined to be normally distributed based on visual inspection as well as Kolmogorov-Smirnov and Shapiro-Wilk tests.

Following the primary analysis, exploratory secondary analysis was performed. These analyses were not corrected for multiple comparisons, as no effects were anticipated, following results from previous studies (e.g., ¹⁰).

Objective performance measures

The effectiveness of DPRNN noise reduction was also assessed objectively for each audio frequency band in the tactile vocoder using a pairwise Pearson correlation coefficient. The amplitude envelopes for the temporally-aligned clean speech signal were compared to either (1) speech in quiet with noise reduction, (2) speech in noise with noise reduction, or (3) speech in noise without noise reduction. All audio samples used in the experiment were assessed.

Data Availability

The datasets generated during and analysed during the current study are available from the corresponding author on reasonable request.

Acknowledgements

Thank you to Ezzy Akis for assisting participant recruitment, and to Alex, Josh, and Helen Fletcher for assisting in the writing of the text.

Salary support for author M.D.F. was provided by the University of Southampton Auditory Implant Service (UK) and the UK Engineering and Physical Sciences Research Council (EP/W032422/1). Salary support for author S.W.P. was provided by the UK Engineering and Physical Sciences Research Council (EP/T517859/1) and the University of Southampton Auditory Implant Service. Author T.G. was supported by a Career Development Award from the UK Medical Research Council (MR/T03095X/1).

The DPRNN used in the experiment was produced using the Aristotle University of Thessaloniki High Performance Computing Infrastructure and the computational infrastructure of the MRC-CBU, University of Cambridge.

Author Contributions

M.D.F., S.W.P., and T.G. designed the experiment; M.D.F., I.T. and T.G. implemented the experiment; and S.W.P. collected the data. M.D.F. performed the data analysis and drafted the manuscript text. All authors reviewed and edited the manuscript.

Competing Interests

The authors declare no competing interests.

Lakshmi, M. S. K., Rout, A. & O'Donoghue, C. R. A systematic review and meta-analysis of digital noise reduction hearing aids in adults. Disabil Rehabil Assist Technol 16, 120-129, doi:10.1080/17483107.2019.1642394 (2021).
Carlyon, R. P. & Goehring, T. Cochlear implant research and development in the twenty-first century: A critical update. J Assoc Res Otolaryngol 22, 481-508, doi:10.1007/s10162-021-00811-5 (2021).
Fletcher, M. D., Hadeedi, A., Goehring, T. & Mills, S. R. Electro-haptic enhancement of speech-in-noise performance in cochlear implant users. Sci Rep 9, 11428, doi:10.1038/s41598-019-47718-z (2019).
Fletcher, M. D., Mills, S. R. & Goehring, T. Vibro-tactile enhancement of speech intelligibility in multi-talker noise for simulated cochlear implant listening. Trends Hear 22, 1-11, doi:10.1177/2331216518797838 (2018).
Fletcher, M. D. & Verschuur, C. A. Electro-haptic stimulation: A new approach for improving cochlear-implant listening. Front. Neurosci. 15, 581414, doi:10.3389/fnins.2021.581414 (2021).
Ciesla, K. et al. Immediate improvement of speech-in-noise perception through multisensory stimulation via an auditory to tactile sensory substitution. Restor Neurol Neurosci 37, 155-166, doi:10.3233/RNN-190898 (2019).
Ciesla, K. et al. Effects of training and using an audio-tactile sensory substitution device on speech-in-noise understanding. Sci Rep 12, 3206, doi:10.1038/s41598-022-06855-8 (2022).
Huang, J., Sheffield, B., Lin, P. & Zeng, F. G. Electro-tactile stimulation enhances cochlear implant speech recognition in noise. Sci Rep 7, 2196, doi:10.1038/s41598-017-02429-1 (2017).
Fletcher, M. D., Song, H. & Perry, S. W. Electro-haptic stimulation enhances speech recognition in spatially separated noise for cochlear implant users. Sci Rep 10, 12723, doi:10.1038/s41598-020-69697-2 (2020).
Fletcher, M. D., Verschuur, C. A. & Perry, S. W. Improving speech perception for hearing-impaired listeners using audio-to-tactile sensory substitution with multiple frequency channels. Sci Rep 13, 13336, doi:10.1038/s41598-023-40509-7 (2023).
Brooks, P. L., Frost, B. J., Mason, J. L. & Chung, K. Acquisition of a 250-word vocabulary through a tactile vocoder. J Acoust Soc Am 77, 1576-1579, doi:10.1121/1.392000 (1985).
Weisenberger, J. M. & Percy, M. E. The transmission of phoneme-level information by multichannel tactile speech perception aids. Ear Hear 16, 392-406, doi:10.1097/00003446-199508000-00006 (1995).
Perrotta, M. V., Asgeirsdottir, T. & Eagleman, D. M. Deciphering sounds through patterns of vibration on the skin. Neuroscience 458, 77-86, doi:10.1016/j.neuroscience.2021.01.008 (2021).
Fletcher, M. D. Using haptic stimulation to enhance auditory perception in hearing-impaired listeners. Expert Rev Med Devices 18, 63-74, doi:10.1080/17434440.2021.1863782 (2020).
Loizou, P. C. Speech Enhancement: Theory and Practice. 1 edn, (CRC Press, 2007).
Healy, E. W., Taherian, H., Johnson, E. M. & Wang, D. A causal and talker-independent speaker separation/dereverberation deep learning algorithm: Cost associated with conversion to real-time capable operation. J Acoust Soc Am 150, 3976, doi:10.1121/10.0007134 (2021).
Healy, E. W., Johnson, E. M., Pandey, A. & Wang, D. L. Progress made in the efficacy and viability of deep-learning-based noise reduction. J Acoust Soc Am 153, 2751-2768, doi:10.1121/10.0019341 (2023).
Goehring, T. et al. Speech enhancement based on neural networks improves speech intelligibility in noise for cochlear implant users. Hear Res 344, 183-194, doi:10.1016/j.heares.2016.11.012 (2017).
Goehring, T., Keshavarzi, M., Carlyon, R. P. & Moore, B. C. J. Using recurrent neural networks to improve the perception of speech in non-stationary noise by people with cochlear implants. J Acoust Soc Am 146, 705-718, doi:10.1121/1.5119226 (2019).
Luo, Y., Ghen, Z. & Yoshioka, T. Dual-Path Rnn: Efficient Long Sequence Modeling for Time-Domain Single-Channel Speech Separation. Int Conf Acoust Spee, 46-50, doi:10.1109/icassp40776.2020.9054266 (2020).
Wu, Y. H. et al. Characteristics of real-world signal to noise ratios and speech listening situations of older adults with mild to moderate hearing loss. Ear Hear 39, 293-304, doi:10.1097/AUD.0000000000000486 (2018).
Busch, T., Vanpoucke, F. & van Wieringen, A. Auditory environment across the life span of cochlear implant users: Insights from data logging. J Speech Lang Hear Res 60, 1362-1377, doi:10.1044/2016_JSLHR-H-16-0162 (2017).
Egan, J. P., Greenberg, G. Z. & Schulman, A. I. Interval of the time uncertainty in auditory detection. J. Acoust. Soc. Am. 33, 771–778, doi:10.1121/1.1908795 (1961).
Chang, P. & Viemeister, N. F. Temporal windows for signals presented at uncertain times. J. Acoust. Soc. Am. 90, 2248 (1991).
Dai, H. & Wright, B. A. Detecting signals of unexpected or uncertain durations. J Acoust Soc Am 98, 798-806, doi:10.1121/1.413572 (1995).
Kressner, A. A., May, T. & Dau, T. Effect of noise reduction gain errors on simulated cochlear implant speech intelligibility. Trends Hear 23, 2331216519825930, doi:10.1177/2331216519825930 (2019).
Neff, D. L. & Dethlefs, T. M. Individual differences in simultaneous masking with random-frequency, multicomponent maskers. J Acoust Soc Am 98, 125-134, doi:10.1121/1.413748 (1995).
Neff, D. L. & Callaghan, B. P. Effective properties of multicomponent simultaneous maskers under conditions of uncertainty. J Acoust Soc Am 83, 1833-1838, doi:10.1121/1.396518 (1988).
Fletcher, M. D., Cunningham, R. O. & Mills, S. R. Electro-haptic enhancement of spatial hearing in cochlear implant users. Sci Rep 10, 1621, doi:10.1038/s41598-020-58503-8 (2020).
Fletcher, M. D. & Zgheib, J. Haptic sound-localisation for use in cochlear implant and hearing-aid users. Sci Rep 10, 14171, doi:10.1038/s41598-020-70379-2 (2020).
Fletcher, M. D., Zgheib, J. & Perry, S. W. Sensitivity to haptic sound-localisation cues. Sci Rep 11, 312, doi:10.1038/s41598-020-79150-z (2021).
Fletcher, M. D., Zgheib, J. & Perry, S. W. Sensitivity to haptic sound-localization cues at different body locations. Sensors (Basel) 21, 3770, doi:10.3390/s21113770 (2021).
Gaultier, C. & Goehring, T. in INTERSPEECH 2023. 3497-3501.
Luo, Y., Han, C., Mesgarani, N., Ceolini, E. & Liu, S. C. Fasnet: Low-latency adaptive beamforming for multi-microphone audio processing. 2019 Ieee Automatic Speech Recognition and Understanding Workshop (Asru 2019), 260-267, doi:10.1109/asru46091.2019.9003849 (2019).
Weisenberger, J. M. & Kozma-Spytek, L. Evaluating tactile aids for speech perception and production by hearing-impaired adults and children. Am J Otol 12 Suppl, 188-200 (1991).
Levanen, S. & Hamdorf, D. Feeling vibrations: Enhanced tactile sensitivity in congenitally deaf humans. Neurosci Lett 301, 75-77, doi:10.1016/s0304-3940(01)01597-x (2001).
Van Doren, C. L., Gescheider, G. A. & Verrillo, R. T. Vibrotactile temporal gap detection as a function of age. J Acoust Soc Am 87, 2201-2206, doi:10.1121/1.399187 (1990).
Gescheider, G. A., Edwards, R. R., Lackner, E. A., Bolanowski, S. J. & Verrillo, R. T. The effects of aging on information-processing channels in the sense of touch: III. Differential sensitivity to changes in stimulus intensity. Somatosens Mot Res 13, 73-80, doi:10.3109/08990229609028914 (1996).
Verrillo, R. T. Age related changes in the sensitivity to vibration. J Gerontol 35, 185-193, doi:10.1093/geronj/35.2.185 (1980).
Fletcher, M. D. Listen with your wrists. Front Young Minds 9, 1-7, doi:10.3389/frym.2021.678405 (2021).
Keidser, G., Ching, T., Dillon, H., Agung, K., Brew, C., Brewer, S., Fisher, M., Foster, L., Grant, F., Storey, F. The National Acoustic Laboratories (NAL) CDs of speech and noise for hearing aid evaluation: Normative data and potential applications. Australian & New Zealand J Audiology 1, 16-35 (2002).
Byrne, D. et al. An international comparison of long-term average speech spectra. J Acoust Soc Am 96, 2108-2120, doi:Doi 10.1121/1.410152 (1994).
Pariente, M. et al. Asteroid: The PyTorch-based audio source separation toolkit for researchers. Interspeech 2020, 2637-2641, doi:10.21437/Interspeech.2020-1673 (2020).
Panayotov, V., Chen, G., Povey, D. & Khudanpur, S. in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 5206-5210 (IEEE).
Wichern, G. et al. in INTERSPEECH 2019. 1368-1372.
Kingma, D. P. & Ba, J. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, doi:10.48550/arXiv.1412.6980 (2014).
Le Roux, J., Wisdom, S., Erdogan, H. & Hershey, J. R. Sdr - Half-baked or well done? 2019 Ieee International Conference on Acoustics, Speech and Signal Processing (Icassp), 626-630 (2019).
Glasberg, B. R. & Moore, B. C. Derivation of auditory filter shapes from notched-noise data. Hear Res 47, 103-138, doi:10.1016/0378-5955(90)90170-t (1990).
Drullman, R., Festen, J. M. & Plomp, R. Effect of temporal envelope smearing on speech reception. J Acoust Soc Am 95, 1053-1064, doi:10.1121/1.408467 (1994).
Ding, N. et al. Temporal modulations in speech and music. Neurosci Biobehav Rev 81, 181-187, doi:10.1016/j.neubiorev.2017.02.011 (2017).
Mahns, D. A., Perkins, N. M., Sahai, V., Robinson, L. & Rowe, M. J. Vibrotactile frequency discrimination in human hairy skin. J Neurophysiol 95, 1442-1450, doi:10.1152/jn.00483.2005 (2006).
Rothenberg, M., Verrillo, R. T., Zahorian, S. A., Brachman, M. L. & Bolanowski, S. J., Jr. Vibrotactile frequency for encoding a speech parameter. J Acoust Soc Am 62, 1003-1012, doi:10.1121/1.381610 (1977).
ISO-80601-2-56:2017. in Medical electrical equipment — Part 2-56: Particular requirements for basic safety and essential performance of clinical thermometers for body temperature measurement (International Organization for Standardization, Geneva, Switzerland, 2017).
Whitehouse, D. J. & Griffin, M. J. A comparison of vibrotactile thresholds obtained using different diagnostic equipment: the effect of contact conditions. Int Arch Occup Environ Health 75, 85-89, doi:10.1007/s004200100281 (2002).
ISO-13091-1:2001. in Mechanical vibration–Vibrotactile perception thresholds for the assessment of nerve dysfunction–Part 1: Methods of measurement at the fingertips (International Organization for Standardization, Geneva, Switzerland, 2001).
ISO-13091-2:2021. in Mechanical vibration. Vibrotactile perception thresholds for the assessment of nerve dysfunction - Analysis and interpretation of measurements at the fingertips (International Organization for Standardization, Geneva, Switzerland, 2021).
Hopkins, C., Graetzer, S. & Seiffert, G. ARU adult British English speaker corpus of IEEE sentences (ARU speech corpus) version 1.0 [data collection]. Acoustics Research Unit, School of Architecture, University of Liverpool, United Kingdom, doi:10.17638/datacat.liverpool.ac.uk/681 (2019).
Demonte, P. in HARVARD speech corpus - audio recording (University of Salford, UK, 2019).
IEEE. in IEEE Recommended Practice for Speech Quality Measurements 1-24 (IEEE, New York, USA, 1969).
Holm, S. A Simple Sequentially Rejective Multiple Test Procedure. Scand J Stat 6, 65-70 (1979).

No competing interests reported.

Download PDF

Journal Publication

published 28 Mar, 2024

Read the published version in Scientific Reports →

Editorial decision: Revision requested
01 Nov, 2023
Reviews received at journal
18 Oct, 2023
Reviewers agreed at journal
16 Oct, 2023
Reviews received at journal
12 Oct, 2023
Reviewers agreed at journal
07 Oct, 2023
Reviewers agreed at journal
04 Oct, 2023
Reviewers agreed at journal
04 Oct, 2023
Reviewers invited by journal
04 Oct, 2023
Editor assigned by journal
04 Oct, 2023
Editor invited by journal
30 Sep, 2023
Submission checks completed at journal
30 Sep, 2023
First submitted to journal
28 Sep, 2023

You are reading this latest preprint version

Improved tactile speech robustness to background noise with a dual-path recurrent neural network noise-reduction strategy

Status:

Journal Publication

Version 1

Abstract

Figures

Introduction

Results

Discussion

Methods

Behavioural performance measures

Objective performance measures

Declarations

Data Availability

Acknowledgements

Author Contributions

Competing Interests

References

Additional Declarations

Status:

Journal Publication

Version 1