An English pronunciation and intonation evaluation method based on the DTW algorithm

doi:10.21203/rs.3.rs-2466432/v1

Download PDF

Research Article

An English pronunciation and intonation evaluation method based on the DTW algorithm

https://doi.org/10.21203/rs.3.rs-2466432/v1

This work is licensed under a CC BY 4.0 License

Journal Publication

published 27 Mar, 2023

Read the published version in Soft Computing →

You are reading this latest preprint version

In order to solve the problems of single feature dimension and low accuracy in traditional oral English evaluation, this study has used the support vector regression algorithm to achieve the effective integration of different dimension evaluation features. The study comprehensively evaluate the overall reading and pronunciation quality of students. This paper designs an evaluation model of spoken English pronunciation quality based on a dynamic time warping algorithm and introduces the evaluation algorithms of pronunciation standard, fluency, and intonation, respectively. A support vector regression algorithm is used to realize the effective fusion of different dimension evaluation features. In addition, the correlation between machine rating and manual rating was used as an evaluation index to evaluate students' oral pronunciation quality from different dimensions. The model shows superior performance in the pronunciation of both monosyllabic and polysyllabic words, which is beneficial to improving students' oral English learning effect.

Oral English

DTW

SVR

Evaluation model

English considerably facilitates the exchange of people from all over the world because it is a common language spoken by everyone. People are paying more attention to oral English tests and students' oral English abilities as China's inclusion into the global community progresses ^[1–2]. However, oral English has always been the weakest aspect of traditional English instruction because there is not enough time for students to practise speaking, and teachers cannot provide specific advice based on how different students pronounce things, which contributes to the low level of oral English proficiency among Chinese students ^[3]. Although most of them can get good results in the written examination which mainly focuses on English vocabulary, grammar and writing, few students can skilfully use English for practical and effective oral communication.

As the need arises, the Computer Assisted Language Learning system (CALL) is developed. In oral training, it gives students more time to learn and enough resources, which can help or take the role of teachers in directing students to perform more focused pronunciation exercises. In order to successfully increase students' oral learning efficiency and oral level, this paper identifies students' pronunciation problems, offers useful diagnosis feedback information, and assesses students' overall pronunciation level. By using a scoring algorithm and high-performance speech recognition technology, the test's accuracy can be increased, resulting in more objective, reliable, and effective scoring that assesses the students' overall pronunciation level in the form of a grade or score ^[4–5].

As a typical application of CALL system, pronunciation quality assessment has been widely concerned. The assessment of pronunciation quality can be based on Automatic Speech Recognition (ASR) technology to construct confidence evaluation features or based on acoustic phonetics knowledge ^[6]. But it's not about the quality of pronunciation that learners use, instead, the multi-dimensional assessment features at segmental and suprasegmental levels are integrated to simulate human experts' assessment of learners' overall pronunciation level. Among them, Goodness of Pronunciation (GOP) algorithm and its improved algorithm are widely used as evaluation features in different scenarios ^[7]. Cucchiarini et al. ^[8] used Hidden Markov Model (HMM) based speech recognition device to extract a series of temporal indicators, such as speaking speed, pronunciation speed, average length of speech flow, ratio of pronunciation time, number of pauses and total duration of pause. However, due to the limited number of non-native language training data specifically used for fluency assessment, the subspace Gaussian mixture model with compact parameter sharing mechanism is very useful. Zang ^[9] designed an adaptive evaluation model for pronunciation quality of English reading, which uses support vector back to realize feature fusion, and approaches the nonlinear relationship between scoring features and manual scoring, which can ensure the accuracy and reliability of pronunciation quality evaluation. Suzuki et al. ^[10] used word length ratio and logarithmic energy as rhythm features, the pitch frequency, logarithmic energy and their first derivative are used as intonation features. By performing forced alignment, the learners' pronunciation and the teacher's standard pronunciation are divided into word segments. On this basis, the distance between the two is calculated by Dynamic Time Warping (DTW) algorithm, and the final prosodic score is obtained by weighted summation.

While previous studies have focused on a single dimension such as pronunciation standard, fluency, or prosody, the main contribution of this paper is to evaluate students' English pronunciation quality in multiple dimensions. Therefore, in order to evaluate the learners' overall pronunciation quality more comprehensively, this paper designs an assessment model of oral English pronunciation quality and evaluates the quality of students' reading pronunciation. Considering that these assessment features can only represent the students' pronunciation quality in one aspect, this paper introduces the SVR algorithm to achieve the effective integration of different features.

The pronunciation quality evaluation is mainly to carry on the multi-dimensional comprehensive score to the student's English pronunciation of reading. This work offers a training tool for English pronunciation of unique terminology created in the MATLAB environment, with a default concentration on special language from signal processing and electronics. Accent, intonation, and voicing are the three phonetic factors that the tool can measure. Different computational techniques and algorithms are used for this, such as fundamental filtering, short-time energy, average magnitude difference function, and harmonic-to-noise ratio. In terms of phonetic parameters, spoken words are contrasted with the pronunciation of the reference text ^[11]. Speech recognition technology is used to create a portable tool for improving English pronunciation. This device evaluates the pronunciation correctness by computing the log-posterior probability of the segmented phonemes and corrects intonation by displaying the pitch contour. Not only are the phoneme substitution errors verified, but also the deletion and insertion errors. Because this technique uses both the English and Korean phoneme sets, it is also possible to determine the Korean phoneme substitution errors. This mobile device has a graphic LCD and is powered by an ARM7 CPU ^[12]. The study offers an evaluation model for spoken English pronunciation geared toward undergraduates and its associated system, with four evaluation criteria: accuracy, speed, rhythm, and intonation. This article demonstrates the value of each evaluation index, how it is computerized, and how much weight it has in the overall evaluation model. The evaluation index and model method used in this research are acceptable and trustworthy, as demonstrated by experiments ^[13]. In this study, a model for speech recognition is created using the Mel Frequency Cepstrum Coefficient (MFCC) feature and Hidden Markov Model (HMM) method. On the basis of its evaluation of English sentence pronunciation with a focus on rhythm and intonation, it then provides users with comments and recommendations regarding pronunciation issues. The accuracy and dependability of the model have been confirmed through experiments. It has been applied to a method for evaluating pronunciation that can help users ^[14]. The curve of the speech intensity, which reflects the change of intonation, is the primary indicator of how well English phrases are spoken in terms of both extent and intonation. As a result, this curve might be a character in the assessment of speech quality. In this paper, a brand-new pitch contour algorithm based on standards is proposed. Given that the duration and intensity of each individual's pronunciation vary, this algorithm creates a DTW (Dynamic Time Warping) model by aligning the evaluated sentences' duration and intensity with the standard sentences. It then outputs the difference between the standard sentences and the evaluated sentences as fractions. We used a test set of recordings of 5 sentences read by 10 speakers to assess how well our strategy worked. An experiment revealed that the outcome of this strategy is quite objective ^[15].

The processing flow is as follows:

The students' English reading pronunciation is preprocessed to extract the acoustic features;
Then the Viterbi alignment is performed with the given reading text. The Viterbi algorithm is to find the best of all the observed sequences
Support vector regression algorithm (SVR) is used to realize achieve the effective integration of different features, so as to give a score that can comprehensively reflect the overall quality of students' reading.

2.1 Overall design

The model consists of four parts: speech signal preprocessing module, acoustic feature extraction module, Viterbi search decoding module and pronunciation quality evaluation module. The overall structure of the model is shown in Fig. 1.

The speech signal preprocessing module firstly adjusts and modifies the input learners' English reading pronunciation. After a series of processing, such as pre-weighting, framing and windowing, the influence of various interfering factors on the reading pronunciation quality is eliminated; then, the acoustic feature extraction module extracts the most accurate feature parameters which can describe the acoustic characteristics of the speech signal from the preprocessed pronunciation. The most commonly used feature parameter is Mel frequency cepstrum coefficient (MFCC); Then Viterbi search decoding module is constrained and guided by the information provided by the knowledge base composed of acoustic model, language model and pronunciation dictionary. The results of phoneme segment recognition are segmented by forcibly aligning the learners' pronunciation with the given reading text. On this basis, various evaluation features required for the evaluation of pronunciation quality are calculated and sent to the module of pronunciation error detection and pronunciation quality evaluation respectively, so as to evaluate the quality of English reading; finally, the results of pronunciation quality evaluation are output and fed back to learners.

2.2 Audio feature extraction

Many researchers refer to the human auditory system and put forward the Mel frequency closely related to sound, which is nonlinear corresponding to the common sound frequency (Hz). MFCC feature extraction method uses this relation to calculate the cepstrum coefficients in Mayer frequency domain. MFCC feature extraction process includes pre-weighting, framing, windowing, Fourier transform, Meyer filter bank, discrete cosine transform, dynamic feature, and so on ^[16]. Among them, fast Fourier transform and filter bank are the two most important processes, which mainly play the role of feature dimension reduction. Figure 2 shows the process of MFCC extraction.

Some audio in the audio data set is not clear due to recording or saving problems. It is necessary to manually remove the unqualified audio one by one to avoid adverse effects on the detection results. The format of audio data set is unified as WAV file, the sampling rate is set to 50 kHz, and the channel is dual channel. To make the ensuing audio feature extraction procedure easier, the most crucial preprocessing step is to draw attention to the features of the audio signal and eliminate any unnecessary information. In contrast to the conventional acoustic model, the laborious forced alignment step can be dropped during the preprocessing phase.

After pre-emphasis, the voice signal needs to be divided into frames. In this paper, the frame length is 25ms and the frame shift is 10ms. In fact, the sound signal is segmented by a fixed length window. There is a 15ms overlap between two adjacent audio frames, which is the dynamic process of analog sound. There are still dramatic changes between frames, therefore, the signal is divided into two parts.

2.3 Evaluation index design

2.3.1 Standard

For the task of speech quality assessment, this paper mainly uses two algorithms, logarithmic posteriori probability and GOP, to achieve higher consistency with manual scoring ^[17].

For phoneme \({\text{q}}_{\text{i}}\), assuming that its duration is \({\text{d}}_{\text{i}}\), the logarithmic posterior probability score calculated is \(\text{L}\text{P}\text{P}\left({\text{q}}_{\text{i}}\right)\), then the logarithmic posterior probability score of the whole sentence is the mean logarithmic posterior probability score of all phonemes in the sentence:

\(\text{L}\text{P}\text{P}=\frac{1}{\text{N}}\sum _{\text{i}=1}^{\text{N}} \frac{\text{L}\text{P}\text{P}\left({\text{q}}_{\text{i}}\right)}{{\text{d}}_{\text{i}}}\)

(1)

Where, N is the total number of phonemes in the sentence.

Similarly, the GOP score of the phoneme \({\text{q}}_{\text{i}}\) is normalized by phoneme duration, and then the mean value is calculated to obtain the GOP score of the whole sentence:

\(\text{G}\text{O}\text{P}=\frac{1}{\text{N}}\sum _{\text{i}=1}^{\text{N}} \frac{\text{G}\text{O}\text{P}\left({\text{q}}_{\text{i}}\right)}{{\text{d}}_{\text{i}}}\)

(2)

2.3.2 Fluency

In this paper, we use cucchiarini et al's assessment method of fluency of pronunciation ^[8]. Specifically, the speech recognition framework is used to extract various evaluation features that can effectively represent the fluency of pronunciation, and the support vector regression scoring model is trained by manual scoring, so that these evaluation features are fused and mapped into the final score of fluency similar to manual scoring.

Chinese students are not familiar with the accurate pronunciation of some words or are nervous in the actual English reading process. Some students are likely to lengthen the pronunciation of some phonemes when reading these words. The length of the pronunciation of these words is not appropriate, and the whole reading sound is not fluent. Take the word "particular" as an example. When students read the word, due to nervousness or hesitation, the pronunciation duration of the first two sounds "P" and "ər" may be lengthened, resulting in a large deviation between the actual pronunciation segment length of the whole word and the standard pronunciation segment length. When the number of such words is large, the fluency of the whole speech will be reduced.

In view of the actual pronunciation situation, this paper designs a word duration ratio evaluation feature to measure the fluency of students. The formula is defined as follows:

\(\text{ WDR}\text{=}\frac{1}{\text{n}}\sum _{\text{i}=1}^{\text{n}} \text{l}\text{n}\frac{\text{d}\text{u}{\text{r}}_{\text{i}}}{\text{D}\text{u}{\text{r}}_{\text{i}}}\)

(3)

Where, \(\text{d}\text{u}{\text{r}}_{\text{i}}\) refers to the physical length of Chinese students' pronunciation section, \(\text{D}\text{u}{\text{r}}_{\text{i}}\) refers to the length of section corresponding to word i.

In order to calculate the standard pronunciation segment length of words, we took the following steps:

(1) We first used speech recognizer to calculate the segment length of each phoneme in ARCTIC Corpus developed by CMU,

(2) Then classified the statistical results by phoneme, and finally averaged the segment length of each phoneme and took it as the standard pronunciation segment length of each phoneme.

(3) After obtaining the statistics of these phoneme segment lengths, the standard segment lengths of all phonemes in each word are summed, take it as an estimate of the standard pronunciation segment length of the word.

(4) In order to facilitate the calculation and obtain a more statistically significant distribution, we take the natural logarithm of the ratio of the actual length of a word to the length of a standard pronunciation segment.

2.4 Algorithm design

2.4.1 DTW-based intonation assessment algorithm

In the assessment of intonation, this paper adopts the method of feature comparison, that is, to measure the quality of learners' pronunciation and Intonation by comparing the differences between learners' pronunciation and the corresponding reference standard pronunciation. The specific assessment process is as follows: firstly, pitch feature representing intonation characteristics is extracted from learners' speech and reference standard speech respectively. At the same time, the acoustic characteristics of MFCC are extracted; Then, the learners are divided into two parts: first, the students are aligned with the standard phonetic information, and then the learners are segmented from the standard phonetic information; Afterwards, using the obtained phoneme boundary information, the two speech segments are aligned once on the phoneme segments with similar contents, and then DTW is used to aligns the pitch feature sequences of the two in each phoneme segment, and calculates the similarity between them; Finally, using the trained score mapping model, the similarity between the two is mapped to the final intonation score. The block diagram of tone assessment is shown in Fig. 3.

DTW algorithm is based on the idea of dynamic programming and combines time warping with distance calculation, which can effectively solve the matching problem between time series of different lengths ^[18].

Assuming that the Pitch sequence extracted by learners is \(\text{T}=\left\{{\text{t}}_{1},{\text{t}}_{2},\dots ,{\text{t}}_{\text{N}}\right\}\), the corresponding reference standard speech Pitch sequence is \(\text{R}=\left\{{\text{r}}_{1},{\text{r}}_{2},\dots ,{\text{r}}_{\text{M}}\right\}\), N and M are the number of speech frames of two sequences respectively, and the value in the sequence is the Pitch value extracted from each frame.

DTW algorithm is to find a regular path \(\text{W}=\left\{\left({\text{t}}_{\text{i}},{\text{r}}_{\text{i}}\right),\text{i}=\text{1,2},\dots ,\text{K}\right\}\), which minimizes the distance between two sequences along this path. Where, K is the path length, \({\text{w}}_{\text{i}}=\left({\text{t}}_{\text{i}},{\text{r}}_{\text{i}}\right),\text{i}=\text{1,2},\dots ,\text{K}\) means that the i-th point \({\text{t}}_{\text{i}}\) in sequence T matches the i-th point \({\text{r}}_{\text{i}}\) in sequence R. Therefore, DTW algorithm is to find a time warping function \(\text{w}=\left\{{\text{w}}_{1},{\text{w}}_{2},\dots ,{\text{w}}_{\text{K}}\right\}\), where the learner's speech Pitch sequence T is mapped nonlinearly to the reference standard speech Pitch sequence R, and the cumulative distortion between the two sequences is minimized.

Considering the timing characteristics of speech signal, the time warping function W needs to meet certain constraints, typical of which are monotonicity, continuity and starting point and ending point constraints. In addition, in order to reduce the computational complexity of DTW algorithm and improve the system accuracy, there are usually some restrictions on the region where the regularization function is located, so there will be different constraint paths. The DTW algorithm starts from the beginning of the sequence and recursively calculates the cumulative distance until the end of the matching sequence. In intonation assessment, the DTW distance between two pitch sequences is calculated to measure the intonation similarity between learners' speech and reference standard speech. After the DTW distance calculation is completed, it is mapped to the intonation score similar to the manual score by Formula (4).

\(\text{ score }\text{=}\frac{100}{1+\text{a}(\text{d}\text{i}\text{s}\text{t}{)}^{\text{b}}}\)

(4)

Where dist refers to the DTW distance between the two Pitch sequences, a and b are the parameters to be trained, and score is the final intonation score, whose value falls within the range of 0 to 100.

2.4.2 Oral scoring algorithm based on SVR

Support vector regression (SVR) algorithm is a machine learning algorithm based on structural risk minimization criterion. It makes full use of the advantages of machine learning and can learn complex data patterns with only limited training samples ^[19].

Given the training data set \(\left\{\left({\text{x}}_{1},{\text{y}}_{1}\right),\left({\text{x}}_{2},{\text{y}}_{2}\right),\cdots ,\left({\text{x}}_{\text{m}},{\text{y}}_{\text{m}}\right)\right\}\), where \({\text{x}}_{\text{i}}\in {\text{R}}^{\text{n}}\) represents the n-dimensional feature vector extracted from the i-th paragraph spoken speech, \({\text{y}}_{\text{i}}\in \text{R}\) is the artificial score corresponding to \({\text{x}}_{\text{i}}\), m is the total number of samples in the training dataset. The goal is to find a regression function \(\text{y}=\text{f}\left(\text{x}\right)\) as flat as possible to approximate the relationship between the two by training all sample pairs \(\left({\text{x}}_{\text{i}},{\text{y}}_{\text{i}}\right)\) in the data set and minimize the prediction error.

For nonlinear data \(\text{x}\), it is difficult to be linearly separable in the original space. SVR algorithm uses a nonlinear function \({\Phi }\left(\text{x}\right)\) to map x to a high-dimensional feature space for processing. The regression function is defined as:

\(\text{f}\left(\text{x}\right)=⟨\text{w},{\Phi }\left(\text{x}\right)⟩+\text{b}\)

(5)

Among them, \(\text{w}\) is the weight vector, \(\text{b}\) is the offset term, \(⟨\ast ,\ast ⟩\) is an inner product operation.

The process of pronunciation quality assessment based on SVR algorithm is as follows:

Based on the Chinese students' English reading pronunciation data collected in the experiment, the pronunciation standard, fluency and prosodic features are extracted, and the feature scores are calculated;

The cubic polynomial function \({\text{a}}_{1}{\text{x}}_{\text{i}}^{3}+{\text{a}}_{2}{\text{x}}_{\text{i}}^{2}+{\text{a}}_{3}{\text{x}}_{\text{i}}+{\text{a}}_{4}\) is used to normalize each characteristic score \({\text{x}}_{\text{i}}\), so that it is consistent with the interval of manual score;

The SVR training sample set is constructed with multi-dimensional evaluation feature score as input and manual score as output;

The parameters of SVR scoring model are trained;

The trained SVR scoring model is used to fuse their features, so as to achieve an effective evaluation of students' overall reading pronunciation quality.

3.1 Experimental parameters

The pre-emphasis coefficient is 0.97, and the Hamming window with a length of 25ms is used to smooth the speech frame signal. The duration of each frame is 25ms, and the overlap between adjacent voice frames is 15ms. The acoustic characteristic parameters of speech signal adopt 39 dimensional MFCC features (12 cepstrum parameters, 1 logarithmic energy, and their first-order and second-order difference coefficients).

The acoustic model uses a context independent monophonic phoneme model. The HMM model corresponding to each phoneme consists of three emission states from left to right. The probability distribution of MFCC acoustic eigenvectors associated with HMM States is simulated by GMM with eight Gaussian components. The language model uses the ternary language model, and the pronunciation dictionary uses a dictionary of CMU. Based on the sphinx4 speech recognition engine of CMU University, this paper develops an English reading pronunciation quality evaluation model suitable for Chinese students.

3.2 Experimental data

Two subsets, BWC and NCC, are selected from L2-arctic corpus. A total of 2261 speech data are used as the intonation evaluation database. The speech data format is converted into the 16 kHz sampling rate and 16bit mono channel wav format required by sphinx4 speech recognition engine. At the same time, an English teacher is invited to give a full score of 100 for the intonation of these speech data. When assessing intonation, the corresponding American pronunciation in CMU ARCTIC corpus is taken as the reference standard pronunciation.

Every five sentences constitute a reading passage with a length of about 50 words. Students from a university are invited to read these essays according to the normal speech speed. At the same time, the pronunciation is as clear as possible. Finally, a total of 450 reading voice data are recorded and saved in 16KHz sampling rate and 16bit mono channel wav format. English teachers from foreign language colleges are invited to give a full score of 100 on the overall pronunciation quality of these phonetic data.

4.1 Model validation

The experimental results of pronunciation error detection are shown in Fig. 4. Among them, log posteriori probability refers to the method to calculate the log posteriori probability score of phonemes by using the standard American tone acoustic model, and "GOP" refers to using the standard American tone acoustic model, The GOP score of phonemes is calculated and compared with the unified threshold of phonemes. The "phoneme independent threshold" refers to the use of adjusted and optimized acoustic model to calculate the GOP score of phonemes and set a separate decision threshold for each type of phoneme.

The performance of GOP algorithm is better than that of log posteriori algorithm on the whole. When precision takes values of 0.25, 0.3 and 0.35 in the middle of performance curve, the recall value of GOP algorithm is higher than that of log posterior probability algorithm, which indicates that the performance improvement effect of GOP algorithm is relatively stable. When the Precision value ranges from 0.3 to 0.4, the Recall value is greatly improved. In addition, the point in the performance curve where the precision value is equal to the recall value can also be used to measure the performance of different methods and the corresponding points of the three performance curves are 0.3, 0.316 and 0.357, respectively.

4.2 analysis of pronunciation quality

In the assessment of pronunciation fluency, we compared the features extracted above, including five temporal indicators, the length of phoneme segment and the word segment length ratio feature designed for Chinese students' actual pronunciation. The performance of different features in fluency assessment is shown in Fig. 5.

The results show that the first five temporal indicators ( Speech speed, Articulation speed, Articulation time ratio, Average pause length, Phoneme segment length) have better performance in assessing the fluency of pronunciation, among which speaking speed has the best effect, and the correlation is 0.431, which indicates that the students with higher pronunciation level speak faster and read more smoothly, which also conforms to the subjective feeling of teachers' score, so this feature can obtain a higher correlation with manual score. Average pause length had the lowest correlation at 0.352, this may be because compared to students whose pronunciation is more smoothly, the pronunciation is not smooth when the students are reading will be more pause times indeed, but basically is a short pause, the relatively short duration. The correlation between the new feature and the artificial score is 0.419, which measures the fluency of students' pronunciation by calculating the ratio between the actual length of each word and the length of standard pronunciation segment, and then extends to the sentence level. Finally, when all the features are combined, the evaluation performance can be greatly improved, and the correlation degree reaches 0.551.

Eight monosyllabic words and eight polysyllabic words are selected to analyze the data of manual scoring and system scoring. The results are shown in Fig. 6.

It can be seen from the figure that the man-machine rating based on DWT algorithm has excellent performance in the pronunciation of monosyllabic and polysyllabic words. The similarity between the system score and the English teacher's score is not much different, which basically conforms to people's subjective feeling. Generally speaking, after the corresponding test, the system is basically qualified, and can meet the requirements of English word phonetic comparison in middle school, which is helpful for students to learn English pronunciation better after class, and compared with the traditional listening to English pronunciation without giving real-time feedback, it is better for students' oral learning effect.

This paper designs an evaluation model of spoken English pronunciation quality and introduces the evaluation algorithms of pronunciation standard, fluency, and intonation. The SVR algorithm is used to effectively fuse evaluation features of various dimensions. In addition, the correlation between the machine score and the manual score is used as the evaluation index to evaluate the students' oral pronunciation quality from different dimensions. The experimental results show that the performance of these one-dimensional assessment features is relatively limited, and can only represent the students' pronunciation quality in a certain way. However, after the multi-dimensional evaluation features are fused by the SVR algorithm, the overall performance of the pronunciation quality assessment can be significantly improved. In addition, DWT-based human-machine scoring in monosyllabic and polysyllabic word pronunciation shows superior performance, which is conducive to improving students' oral English learning effect. After the corresponding test, the system is basically qualified and can meet the requirements of the comparison of middle school English words and sounds, which is conducive to students' learning the pronunciation of English words after class.

Funding: This work received no funding.

Conflict of Interest: The authors declare that there are no conflicts of interest.

Ethical approval : All applicable international, national, and/or institutional guidelines for the care and use of animals were followed.

Ethical approval: All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Ethical approval: This article does not contain any studies with human participants performed by any of the authors.

Informed consent: Informed consent was obtained from all individual participants included in the study.

Author Contributions: Jing Duan was responsible for the conception of research ideas and the writing of the first draft. Zhu He was responsible for the project administration.

Data Availability: The dataset can be accessed upon request.

Jia Fei. Problems and Countermeasures of oral English Teaching in Higher Vocational Colleges, Popular literature, 2022 (05): 146-148(in Chinese)
Zhai Linlin. On the practice of college oral English Teaching under the background of artificial intelligence, English Square, 2021 (31): 125-127(in Chinese)
Chen Lin. Practical research on improving the oral English expression ability of secondary vocational school students, Modern vocational education, 2022 (08): 145-147(in Chinese)
Golonka E M, Bowles A R, Frank V M, et al. Technologies for foreign language learning: a review of technology types and their effectiveness. Computer assisted language learning, 2014, 27(1): 70-105.
Van Doremalen J, Boves L, Colpaert J, et al. Evaluating automatic speech recognition-based language learning systems: a case study, Computer Assisted Language Learning, 2016, 29(4): 833-851.
Zhang Feng, Chen Yining, Luo Quanfeng. Application of speech recognition in English pronunciation error correction, Computer applications and software, 2011,28 (02): 40-43(in Chinese)
Cheng Deqiang, Xu Chao, Li Yan, you Dalei, Chen Gang. MPEG video quality evaluation method based on time domain weighted processing, Progress in laser and optoelectronics, 2018,55 (08): 268-273(in Chinese)
Cucchiarini C, Strik H, Boves L. Quantitative assessment of second language learners’ fluency by means of automatic speech recognition technology, The Journal of the Acoustical Society of America, 2000, 107(2): 989-999.
[1] ZANG Ruiting. Study on the design of adaptive evaluation model of English Pronunciation Quality, Automation technology and application, 2021,40(10):117-120.
Suzuki M, Konnot, ITO A, et al. Automatic evaluation system of English prosody based on word importance factor, Journal of Systemics, Cybernetics and Informatics, 2008, 6(4): 83-90.
Malucha J. Software Tool for Pronunciation Training of Specific English Terminology. In2022 New Trends in Signal Processing (NTSP) 2022 Oct 12 (pp. 1-5). IEEE.
You K, Kim H, Chang H, Lee J, Sung W. A handheld english pronunciation evaluation device. In2005 Digest of Technical Papers. International Conference on Consumer Electronics, 2005. ICCE. 2005 Jan 8 (pp. 267-268). IEEE.
Li XG, Chen JH, Chen Z, Chen Y. An approach to evaluation index and model of undergraduates' spoken English pronunciation. In2016 12th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD) 2016 Aug 13 (pp. 2189-2193). IEEE.
Li X, Chen J, Yao M, Shen D, Lin F. English sentence pronunciation evaluation using rhythm and intonation. InThe 2014 2nd International Conference on Systems and Informatics (ICSAI 2014) 2014 Nov 15 (pp. 366-371). IEEE.
Min-feng Y, Xin-guang L, Jiang-hao L, Si-zhe Y. A new approach to pronunciation assessment of English sentences. In2011 6th IEEE Joint International Information Technology and Artificial Intelligence Conference 2011 Aug 20 (Vol. 1, pp. 177-180). IEEE.
Tan Zhenyu. End to end English pronunciation error detection based on multimodality, Donghua University, 2021(in Chinese)
Li Y, Huang G. An English Pronunciation Quality Evaluation Model Based on Multi-dimensional Features, Journal of Physics: Conference Series. IOP Publishing, 2022, 2224(1): 012061
Wang Guolin. DTW based speech evaluation system and its application in English Teaching of middle school students, Central China Normal University, 2017(in Chinese)
Zhu Hongtao. A study on the quality of English pronunciation evaluation, Guilin University of Electronic Science and technology, 2020 (in Chinese)

Download PDF

Journal Publication

published 27 Mar, 2023

Read the published version in Soft Computing →

Editorial decision: Major Revision
13 Feb, 2023
Reviewers agreed at journal
29 Jan, 2023
Reviewers invited by journal
16 Jan, 2023
Editor assigned by journal
16 Jan, 2023
First submitted to journal
10 Jan, 2023

You are reading this latest preprint version

An English pronunciation and intonation evaluation method based on the DTW algorithm

Status:

Journal Publication

Version 1

Abstract

Figures

1 Introduction

2 Evaluation Model Of Oral English Pronunciation Quality

2.1 Overall design

2.2 Audio feature extraction

2.3 Evaluation index design

2.3.1 Standard

2.3.2 Fluency

2.4 Algorithm design

2.4.1 DTW-based intonation assessment algorithm

2.4.2 Oral scoring algorithm based on SVR

3 Experiment

3.1 Experimental parameters

3.2 Experimental data

4 Results And Discussion

4.1 Model validation

4.2 analysis of pronunciation quality

5 Conclusion

Declarations

References

Status:

Journal Publication

Version 1