Musical note recognition in presence of harmonics without the fundamental

doi:10.21203/rs.3.rs-3934726/v1

Musical signals are complex periodic waveform characterized by the sum of different frequencies. The lower frequency is called fundamental while the other frequencies are called harmonics, and their frequency is multiple of the fundamental. The pitch of a sound is correlated with the value of the fundamental even though in many situations it can be impossible to perceive it. In these cases, it is possible to identify the pitch thanks to the perception of harmonics. The aim of this study was to evaluate the identification of the notes on the basis of the presence of consecutive harmonics only and to determine the importance of their distance from the fundamental.

The study was carried out on 30 normally hearing musicians without absolute pitch. The acoustic signal was characterized by the association of four consecutive and two consecutive harmonics of central notes of the piano keyboard.

The rate of correct identification ranged between and 8% and 100% with a better identification when stimuli were characterized by more harmonics and lower frequency.

The results confirm that it is possible to identify a note on the basis of the presence of harmonics near to fundamental especially if the fundamental frequency is under 2000 Hz.

Physical sciences/Energy science and technology

Physical sciences/Engineering

Physical sciences/Mathematics and computing

Physical sciences/Physics

harmonics

fundamental note

pitch

musical note identification

Pitch identification of an acoustic signal is allowed by the precise capacity of the organ of Corti to identify all the frequency composing the sound [1, 2]. This is possible thanks to the resonance characteristics of the basilar membrane; at this level each frequency places it in oscillation at different points with a distribution that goes from the apex to the base respectively for low and high frequencies [3]. Furthermore, the oscillation mechanism in relation to the frequency is made even more precise by the contractile activity of the external hair cells [1, 4].

On the basis of the basilar membrane vibration, it is easy to explain the pitch identification of a pure (sinusoidal) tone, characterized by the presence of a single frequency.

Musical note and voice are characterized by a more complex acoustic signal defined as complex periodic waveform (CPW) [5]. In CPW signal the waveform is characterized by the presence of multiple frequencies that sound together. The lowest frequency is named fundamental and represents, in the case of chordophones instruments and of the vocal folds, the vibration of the entire chord. The other frequencies, that are higher, are named harmonics and present a frequency, expressed in Hz, that are multiples of the fundamental [6.7]. In this case the basilar membrane vibrates in different points at the same time and each point is related with one of the tones that compose the CPW. In this way the cochlea perfectly reconstructs the musical or vocal acoustic signal [8, 9].

In the CPW the pitch is defined as the value, expressed in Hz, of the fundamental frequency. In the current convention, the reference note is the fifth A (defined A4) of the piano keyboard, which must have as fundamental frequency 440 Hz. On the basis of this value and of the mathematical relationship existing between the notes, based on the creation of 12 intervals doubling the frequency (octave) according with the equal temperament, it is possible to determine the fundamental frequency of each note. In the 88 keys of the piano keyboard the fundamental frequency of the note ranges from 27,5 and 4186 Hz [1, 10].

In presence of background noise, in which the acoustic pressure is concentrated on low frequencies [1, 5], or in listening filtered acoustic signals such as MP3 recordings or radio or telephone listening [10], it can be difficult to perceive the fundamental frequency of the notes, since it that can be masked or filtered.

In these cases, however, it is possible to identify the tonality of the notes thanks to the perception of the harmonics which, being higher in frequency, are less easily masked. Since the harmonics are in mathematical relationship with the fundamental, being theirs multiple, it is possible to reconstruct the fundamental frequency, even if not perceived, on the basis of the relationship existing between the perception of consecutive harmonics; for example, in the 500–600 Hz frequency couple the fundamental can only be 100 Hz [10].

The aim of this study is to evaluate, in a sample composed by musicians without absolute ear [11], the pitch identification of acoustic signals characterized by the presence of few (four and two) consecutive harmonics without fundamental. Moreover, we aimed to determine the importance of the distance from the fundamental of the harmonics presented in the pitch determination.

This study has been performed in accordance with the ethics standards laid down in the 1964 Declaration of Helsinki and informed written consent was obtained from all subjects.

The study was performed on a group of 60 subjects, 26 (43%) male and 34 (57%) female, aged between 21 and 75 years (mean of 37.2 years). Subjects were recruited with unprofessional ability to play an instrument but lacking perfect pitch.

Among the inclusion criteria there were also a negative history of ear diseases and an audiometric threshold equal to or less than 25 dB at frequencies between 125 and 8000 Hz.

The sound stimuli were presented by a generator of pure tones that gives the possibility to produce a complex waveform made up by two of more pure tones with the frequencies chosen by the Authors. The generator is based around a SGTL5000 Stereo Codec with Headphone Amplifier (https://www.nxp.com/docs/en/data-sheet/SGTL5000.pdf) that is mounted on an audio adaptor (https://www.pjrc.com/store/teensy3_audio.html) and controlled over a Freescale based (https://www.nxp.com/docs/en/data-sheet/K20P64M72SF1.pdf) Teensy 3.2 board (https://www.pjrc.com/store/teensy32.html). The SGTL5000 in our configuration is powered by 3.3V and is driving the analog output to the headphones in a 24bit data regime with a 44.1 kHz sampling. In this configuration using 32 Ohm headphones it could achieve a Signal to Noise Ratio (SNR) of 100 dB with a Total harmonic distortion plus noise (THD + N) of -88 dB and a frequency response of ± 0.11 dB. We have implemented a GUI (Globally Unique Identifier) with a full control over the generated frequencies and their mixing in the output waveform.

All frequencies produced by the generator had the same acoustic pressure and started simultaneously. The acoustic signal was sent to the subjects examined through Audiotechnica headphones that use wide-range bandpass filters at an intensity of 65 dB SPL.

The test was divided in two sessions.

In the first session the harmonics 1-2-3-4 of the notes C3 and G3 were presented to all participants, so they were invited to identify the note listened on an electronic keyboard without temporal limit (Table 1). Then, to the same participants were presented the harmonics 2-3-4-5 of the notes E3 and A3 and they were invited to identify the note following the same protocol (Table 2). In the second session two consecutive harmonics, 2–3, 3–4, 4–5, 5–6, 6–7, 7–8 and 8–9 of the notes A-4, D4, E4, F4, A4, E-4, D-4 (Table 3) were presented to all participants and they were asked to identify the note with the same protocol of the first session.

Table 1

Notes utilized for the test, frequency of their fundamentals according to the actual musical codification and frequency of the harmonics 1-2-3-4 present in the acoustic signal of the test.
Note	Fundamental (Hz)	Harmonics 1-2-3-4 (Hz)
C3	130,81	261,62 392,43 523,24 654,05
G3	196	392 588 784 980

Table 2

Notes utilized for the test, frequency of their fundamental according to the actual musical codification and frequency of the harmonics 2-3-4-5 present in the acoustic signal.
Note	Fundamental (Hz)	Harmonics 2-3-4-5 (Hz)
E3	164,81	494,43 659,24 824,05 988,86
A3	220	660 880 1100 1320

Table 3

Notes utilized for the test, frequency of their fundamental according to the actual musical codification, progressive number of the harmonic association presented and frequency of the two harmonics presented.
Note	Fundamental (Hz)	Presented harmonics	Frequencies of the presented harmonics (Hz)
A-4	466,16	2 3	1398,49 1864,64
D4	297,18	3 4	1485,90 1783, 08
E4	329,63	5 6	2307,41 2637,04
F4	349,23	4 5	2095,38 2444,61
A4	440	6 7	3520 3960
E-4	311,13	7 8	2800,17 311,30
D-4	277,18	8 9	2771,62 3048,80

The study was approved by the ethics committee of the University of Turin and to each participant the aim of the study was clearly explained in order to obtain the informed consent to be subjected to the test.

In Table 4 and 5 the rate of correct identification of the notes in relation to the group of harmonics presented is reported. The rate of correct identification of the note in absence of the fundamental is between 88% and 100% listening harmonics 1–4 and between 82% and 96% listening harmonics 2–5. The correct identification is higher in presence of harmonics nearer to fundamental and of nots with a higher fundamental frequency. Table 6 shows the rate of correct identification of the note in relation to the harmonics presented, respectively 2–3, 3–4, 4–5, 5–6, 6–7 and 7–8. The correct identification ranges between 6–76%. Even in this case the higher rate of correct identification has been obtained in listening the couple of harmonics nearer to the fundamental and with lower frequencies. This pattern is reported also in Fig. 1 where the rate of correct identification is related to the frequency of the lowest couple of harmonics presented to participants. The slope of the line clearly shows that the identification of the note is easier if the frequency of the first harmonic presented has a frequency less than 1500–2000 Hz, than the success rate of identification falls to lower values.

Table 4

Rate of correct identification of the notes C3 and G3 on the basis of the presentation of harmonics 1-2-3-4.
Note	Harmonics 1–4 (Hz)	Rate of correct identification of the notes Harmonics 1–4 (%)
C3	261,62 392,43 523,24 654,05	100
G3	392 588 784 980	94

Table 5

Rate of correct identification of the notes E3 and A3 on the basis of the presentation of harmonics 2-3-4-5.
Note	Harmonics 2–5 (Hz)	Rate of correct identification of the notes Harmonics 2–5 (%)
E3	494,43 659,24 824,05 988,86	94
A3	660 880 1100 1320	82

Table 6

Rate of correct identification of the notes in the different couple of harmonics presented to the participants.
Note	Harmonics	Frequency of the two presented harmonics (Hz)	Rate of correct identification of the Notes (%)
A-4	2–3	1398,49 1864,64	76
D4	3–4	1485,90 1783, 08	71
F4	4–5	2307,41 2637,04	35
E4	5–6	2095,38 2444,61	12
A4	6–7	3520 3960	35
E-4	7–8	2800,17 311,30	24
C-4	8–9	2771,62 3048,80	6

In the design of the study, we began to make listening the signals consisting of harmonics to musicians with perfect pitch in order to ask them to identify the presented note. In these subjects, unlike what happens in people without absolute pitch, the sensation generated by the stimulus sent was that of the simultaneous presence of single notes having the same fundamental frequency of the harmonics presented. Those who have perfect pitch, therefore, have the same perceptive behavior as a person without this characteristic but to whom the different frequencies sent are not sent simultaneously [12]. Since the aim of the study was to verify the possibility of note identification on the basis of the harmonics without fundamental, subjects with perfect pitch were excluded from the study.

We also tested some subjects lacking the ability to play an instrument but in many of these subjects the identification of the note on the keyboard created a lot of tension and anxiety for which the note was identified with considerable difficulty and the outcome could not be considered as reliable. Therefore, even these subjects were excluded by the study group.

Hence, we decided to test people who were able to play on a keyboard but did not have absolute pitch. In order to avoid interference with cochlear function we included in the study group only subjects free from ear pathologies and with normal audiometric threshold.

The results obtained from the study confirm that it is possible to identify a note on the basis of the presence of four harmonics [11]. The correct note identification was higher if the harmonics presented were nearer to the fundamental (1–4 versus 2–5). If the signal is composed by two harmonics only the identification is still possible, but the rate of correct identification drops to lower values (6–76%) and even in this case it is higher if tones presented are closer to the fundamental.

The identification of the note without the perception of the fundamental is possible thanks to the fact that two or more frequencies correlated with each other as consecutive multiples of a lower frequency allow the reconstruction of the missing part of the signal and confers the perceived tonal characteristic [11]. This modality of perception requires the simultaneous presentation of the single frequency components otherwise these are identified as signals not correlated with each other and heard each one as single associated note [13–15].

The critical point in note recognition listening only two harmonics seems to be placed between 1500 and 2000 Hz. In order to explain this pattern, it is necessary to remember that the fundamental identification on the basis of at least two harmonics can take place only if both harmonics can be heard. If the two harmonics are too close each other they are analyzed in points very near point along the basilar membrane. If the distance I lesser than 1 mm, the phenomenon of masking occurs, and one of the two tones is not perceived [16, 17].

The arrangement of the points of maximum cochlear oscillation does not follow a linear relationship with the frequency but is based on a logarithmic ratio in base 2, for which the distance between two points of maximum oscillation is constant, 4 mm, in relation to the doubling of the frequency [13]. It follows that the distance between the points of maximum oscillation of the basilar membrane induced by two tones characterized by a constant difference in frequency, as occur in our experiment, the distance between the point of maximum oscillation of the basilar membrane is lesser in the high frequency region; therefore in the high frequency range it is easier that one of the two tones could be masked by the other and hence it is more difficult to identify the fundamental.

In normal musical listening it is always possible to identify the fundamental even if not perceived while in our study this event does not always occur. The low performance reported in our experiment can be explained by the adoption of synthetic tones in which all the harmonics have the same intensity, situation far from normality. Moreover, in real music listening for a normally hearing subject, it is never possible that only two or four harmonics are perceived. Since the masking effect is mainly due to the background noise which is characterized by a higher acoustic pressure up to about 500 Hz [16] all the harmonics above 500 Hz, even for the notes whose fundamental is below this frequency (C5, i.e. the 52th note of the piano keyboard) can be heard, and in presence of more harmonics we have demonstrated a higher rate of correct identifications.

FUNDING

The authors received no financial support for the research, authorship, and/or

publication of this article.

FINANCIAL INTERESTS

The authors declare they have no financial interests.

CONFLICT OF INTEREST

The authors have no potential conflict of interest or financial disclosures pertaining to this article.

ACKNOWLEDGEMENTS

None

DATA AVAILABILITY STATEMENT

All data pertaining to this systematic review are available from the corresponding author upon reasonable request.

COMPLIANCE WITH ETHICAL STANDARDS

The authors deny potential conflict of interest. The review did not involve animals.

Informed consent was collected from all participants of the study.

AUTHOR CONTRIBUTIONS

Design of the work: R. A., S. L., A. Al.; writing: R. A., A., U.; data collection: A. Ap., L. M., A. Am., M. A.; final approval: A. A.

Benson DJ (2007) Music. A mathematical offering. Cambridge University Press.
Rugerro MA, Rich NC, Recio A, Narayan SS, Robles L (1997) Basilar membrane response to tones at the base of the chinchilla cochlea. JASA 101:2151–2163.
Chan WX, Lee SH, Kim N, Shin CS, Yoon YJ (2017) Mechanical model of an arched basilar membrane in the gerbil cochlea. Hear Res. 345:1–9.
Jabeen T, Holt JC, Becker JR, Nam JH (2020) Interactions between passive and active vibrations in the organ of Corti. Biophys J. 119(2):314–325.
Duifhuis H, Willems LF (1982) Measurement of pitch in speech. An implementation of Goldstein theory of pitch perception. JASA 71:1568–1580
Bennet WR, Morrison ACH (2008) The science of musical sound. Springer Cham.
Suzuki H (1986) Vibration and sound radiation of a piano sound-board. JASA 80:1573–1582.
Dai H (2000) On the relative influence of individual harmonics on pitch judgment. JASA 107:953–959.
Moore BCJ, Glasberg BR, Peters RW (1985) Relative dominance of individual partials in determining the pitch of complex tones. JASA 77:1853–1860.
Giordano N (1996) Sound production by a vibrating piano soundboard: experiment. JASA 103:2128–2133.
Kim J (2013) Analysis of factors affecting output levels and frequencies of MP3 players. Korean J Audiol 17:59–64.
Deutsch D, Henthorn T, Dolson (2004) Absolute pitch. Music perception 21:339–356.
Pierce JR (1983) The Science of Musical Sound. Scientific American Books New York.
Scheffers MTM (1983) Simulation of auditory analysis of pitch. An elaboration on the DWS pitch meter. JASA 74:1716–1725.
Preisler A (1993) The influence of spectral composition of complex tones and of musical experience on the precepibility of virtual pitch. Perception and Psychophysics 54:589–603.
Houtsama AJM, Smyrzynski J (1990) Pitch identification and discrimination for complex tones with many harmonics. JASA 87:304–310.
Greenwood DD (1961) Auditory masking and the critical band. JASA 33:484–502.

No competing interests reported.

Musical note recognition in presence of harmonics without the fundamental

Status:

Version 1

Abstract

Figures

Introduction

Materials and Methods

Results

Discussion

Declarations

References

Additional Declarations

Status:

Version 1