A Quantitative Analysis of Vocabulary Taught in Japanese EFL Textbooks

doi:10.21203/rs.3.rs-688772/v1

Download PDF

Original article

A Quantitative Analysis of Vocabulary Taught in Japanese EFL Textbooks

https://doi.org/10.21203/rs.3.rs-688772/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

This research evaluates vocabulary taught in government-approved Japanese EFL textbooks intended for senior high school students in two different ways: by calculating the proportion of high-frequency words in textbooks and by calculating how many types of core high-frequency words are taught in textbooks. To this end, vocabulary words in textbooks are compared to those in the New General Service List (NGSL), which consists of the top 2,801 high-frequency words in general English. Results show that textbooks are largely made up of words in the NGSL at a statistically significant level with larger than 92% lexical coverage; however, textbooks do not sufficiently cover words in the NGSL at a statistically significant level with up to 38%. Overall, research findings indicate that vocabulary words textbooks provide may be what learners frequently come across in the real world, but those words cannot suffice in order for learners to read English texts intended for native speakers. Therefore, language teachers need to give learners additional input of core words through homework or classroom activities to widen their vocabulary size.

Special Education

Japanese EFL textbooks

lexical coverage

New General Service List

textbook corpus

In reaction to the increasingly globalized society, the revision of the Couse of Study Guidelines this time has a strong awareness of the necessity of improving learners’ English skills regardless of their occupations as English will be supposed to be used in many more situations than ever (the Ministry of Education, Culture, Sports, Science and Technology [MEXT], 2018; MEXT, 2019). To cultivate students’ English skills, one area where researchers can contribute lies in the development of textbooks that have the potential to teach English effectively because for those who study English as a foreign language (EFL), textbooks used in the classroom serve as a main source of English input (Bouhlal, Horst, & Martini, 2018); this is also the case in Japan (Matsuda, 2002). In other words, the improvement of textbooks potentially has a direct contribution to that of learners’ English skills.

Vocabulary, whose essentialness in language learning has been long acknowledged by many researchers (Alqahtani, 2015; Schmitt, Schmitt, & Clapham et al., 2001; Zimmerman, 1997), was investigated in this study. According to Nation (2006, 2013), a reasonable learning target is neither mastering all of the English words nor trying to master as much vocabulary as do native speakers of English. Rather, non-native speakers should aim to achieve vocabulary thresholds where comprehension occurs for English texts in which they are interested. If one wants to read an authentic English text such as novels or newspapers, they need to get 95% coverage (Laufer, 1989; Liu & Nation, 1985) yet ideally 98% coverage of the text (Laufer & Ravenhorst-Kalovsk, 2010); these coverage levels are provided with 4,000–5,000 and 8,000 of the most frequent word families respectively.

For developers of textbooks, this information on coverage levels would be of much use. Putting this clearly, by counting how many types of high-frequency words are treated in their textbooks, they can ascertain how close their textbooks can potentially make learners’ vocabulary size to those required vocabulary sizes. Even if it is found that their textbooks do not cover high-frequency words so much, they can make the best use of that information for the development of textbooks in the future.

Furthermore, textbook developers can prevent themselves from developing textbooks having a large number of words that are not worth learning. If the lexical coverage of a textbook calculated with core high-frequency words is far behind the vocabulary thresholds mentioned above, the textbook might have an inclination to teach vocabulary that learners will be less like to meet in the real world and would therefore need some modifications. In sum, examining vocabulary in a textbook by calculating the proportion of high-frequency words and by counting the number of types of core high-frequency words can possibly offer designers of textbooks some insight into what vocabulary to teach in their textbooks. This research, therefore, evaluated vocabulary occurring in textbooks in these two different ways.

Research into Vocabulary Taught in Japanese EFL Textbooks

Vocabulary taught in Japanese EFL textbooks is now becoming one of the research areas where researchers show an interest. In this section, we look at how vocabulary in teaching materials has been evaluated.

Chujo (2004) measured vocabulary levels of not only the combinations of junior and senior high school EFL textbooks, but also college qualification tests, textbooks and articles used at colleges, and English proficiency tests by using the British National Corpus (BNC). She defined words occurring 100 times or more as high-frequency words and extracted those that met this criterion from the BNC for the creation of a benchmark word list. High-frequency words extracted from the BNC were then stratified into frequency bands, each of which consists of 100 lemmas, for a total of 140 bands. Then, it was investigated how many of the word bands readers needed to be familiar with for reading each of the materials. Results showed varying vocabulary demands of the materials, with a low of the top 2,800 lemmas to a high of the top 15,000 lemmas. For the combinations of junior and senior high school textbooks, it was found that the 3,200 most frequent lemmas were required.

Not being at the center of his research, Underwood (2010) also measured vocabulary levels of Japanese EFL textbooks. He compared textbooks approved by MEXT (hereafter, MEXT-approved textbooks) with the National Center Exams, which are the most common university admission tests in Japan, in terms of readability, vocabulary levels, reading skills which were supposed to be required to solve comprehension questions, and types of comprehension questions. In his study, vocabulary levels were measured by referring to the General Service List (GSL; West, 1953) and the Academic Word List (AWL; Coxhead, 2000); it was observed that textbooks were largely made up of high-frequency words. Because he reported not vocabulary levels of textbooks but the average frequency of occurrence of the GSL and the AWL words per chapter of textbooks, I calculated the lexical coverage of textbooks by the word lists based on the reported average scores and found almost 90% coverage was provided with the first and second 1,000 most frequent words in the GSL. He concluded this pervasiveness of high-frequency words in textbooks might allow students to get regular exposure to them.

Wongsarnpigoon (2018) measured vocabulary levels of MEXT-approved junior high school textbooks as well as those of senior high school entrance examinations by using Heatley et al.’s (2002) software, Range. For the study, he created two different corpora: a corpus of MEXT-approved junior high school textbooks and a corpus of senior high school entrance examinations. He found that 95% coverage of textbooks was provided with the top 2,000 high-frequency words in the BNC. He thus considered textbooks to be appropriate materials for learning high-frequency words. Additionally, a comparison between the two corpora identified that the vocabulary demands of entrance examinations were less than those of textbooks, leading him to consider entrance examinations to be appropriate learning targets.

By measuring vocabulary levels, these previous studies offered a clear picture of the density of high-frequency words in textbooks (and other materials) as well as their vocabulary loads. Pedagogically speaking, these previous studies are noteworthy. As suggested by Underwood (2010) and Wongsarnpigoon (2018), teachers in Japan can use textbooks analyzed in these studies in order to impart high-frequency words to students. Following them, this study also measured vocabulary levels of textbooks.

In the first place, textbooks are not reading materials but teaching materials, so the primary focus of research into vocabulary in textbooks should be on the investigation of how many types of high-frequency words students can potentially learn through textbooks, rather than that of how many vocabulary words in textbooks are high-frequency. Surprisingly, this area of research has not as yet directly investigated. This would be an area to explore since without that research, textbook developers could not notice the potential deficiency of supply of vocabulary required for reading an authentic text in their textbooks and would unintentionally leave it as it is. The following two research questions were addressed:

How much vocabulary in MEXT-approved textbooks is what learners are more likely to meet in the real world?
If Japanese students are able to master all of the vocabulary words taught in MEXT-approved EFL textbooks, is that enough for them to be able to read authentic texts?

Vocabulary Size Needed to Read English Texts

In regards to the second research question, it would be necessary to make it clear how much vocabulary is required for reading an authentic text.

At the end of the 20^th century, it was suggested that 95% of running words in a text should be known for reading a text (Laufer, 1989; Liu & Nation, 1985) and guessing from context (Liu & Nation, 1985). Subsequently, Hu and Nation (2000) administered reading comprehension tests to 66 learners and found that 98%-99% of running words in a text should be known for reading a text for pleasure. At the same time, they observed comprehension took place for a few learners with 90% and 95% of lexical coverage. 90% coverage would be provided by the second 2,000-3,000 most frequent word families along with proper nouns, marginal words (e.g., “er”), and transparent compounds (e.g., “forever” and “lifespan”) (Nation, 2013). According to Laufer and Ravenhorst-Kalovsk (2010), 95% and 98% of lexical coverage require readers to know 4,000-5,000 and 8,000 of the most frequent word families respectively. Although 98% coverage is an optimal threshold for learners (Laufer & Ravenhorst-Kalovsk, 2010), it seems that 90 coverage would be a minimum threshold where comprehension occurs. This research thus adopted 90% coverage as the first target in vocabulary learning and addressed the second research question by exploring whether or not textbooks can suffice in order for learners to achieve this threshold.

Textbooks Under Scrutiny

This study examined textbooks intended for senior high school students. According to the current Couse of Study Guidelines (as of 2021), senior high school English classes are roughly divided into two types of course according to their course aims (MEXT, 2009): Komyunikeisyon Eigo (directly translated as English communication) and Eigo Hyogen (directly translated as English expression). The former aims to develop students’ comprehensive English skills; the latter aims to develop their productive skills (i.e., writing and speaking). Textbooks used in the former course are composed of reading passages exclusively, and new words show up frequently. Most textbooks used in the latter course are, on the other hand, largely composed of decontextualized sentences accompanied by their Japanese translations or explanations of grammar rules written in Japanese. That is, it can be said that textbooks used in the former course serve as a main source of vocabulary input rather than those in the latter course. To ensure a representative sample of Japanese learners’ vocabulary input, textbooks used in the former course were solely examined.

Komyunikeisyon Eigo course is further divided into three subgroups according to their course levels, namely, Komyunikeisyon Eigo 1, 2, and 3. Generally, they are intended for 1st-, 2nd-, and 3rd-year senior high school students respectively. By referring to the website of Tokyo Metropolitan Board Education (2020), the top three best-selling MEXT-approved textbooks in Tokyo in 2021 were chosen from each of the three levels, for a total of nine textbooks as shown in Table 1.

Table 1

Textbooks Analyzed in This Study

Grade	Publisher	Textbook
SH1	Sanseido	VISTA English Communication Ⅰ New Edition (Kaneko et al., 2018a)
	Tokyosyoseki	All Aboard! English Communication Ⅰ (Kiyota et al., 2017)
	Suken	Revised COMET English Communication Ⅰ (Ikeno et al., 2017)
SH2	Sanseido	VISTA English Communication Ⅱ New Edition (Kaneko et al., 2018b)
	Keirinkan	Revised ELEMENT English Communication Ⅱ (Ushiro et al., 2018)
	Suken	Revised COMET English Communication Ⅱ (Ikeno et al., 2018)
SH3	Keirinkan	LANDMARK Fit English Communication Ⅲ (Tanabe et al., 2018)
	Keirinkan	Revised ELEMENT English Communication Ⅲ (Ushiro et al., 2019)
	Tokyosyoseki	All Aboard! English Communication Ⅲ (Kiyota et al., 2019)

Word List

Thanks to researchers’ sincere efforts, there are currently numerous freely available word lists; for example, the GSL (West, 1953), the University Word List (Xue & Nation, 1984), the AWL (Coxhead, 2000), and more. As a benchmark word list, this research adopted one type of word list representing general English, namely, the New General Service List ver. 1.01. (NGSL; Browne, Culligan, & Phillips, 2013a), which is sourced from more than 273 million words within the Cambridge English Corpus. The NGSL is composed of the 2,801 most frequent words in general English plus 52 supplemental words such as days of the week, months of the year, and numbers. A remarkable superiority of the NGSL over others lies in that it can cover a larger number of words in a text with a smaller number of words. For example, the NGSL can cover around 92% of words of general texts with the 2,801 words whereas the GSL, which is one of the most well-known word lists, covers only 84% of words of general texts with approximately 3,600 words.

Components of the NGSL

English words are classified and counted in several ways such as types, tokens, word families, and lemmas. Browne, Culligan, and Phillips (2013a) differently counted words than existing word lists for the creation of the NGSL. Specifically, they used a modified lexeme approach, which corresponds to the counting unit of flemmas (McLean, 2017). Their modified lexemes and flemmas are distinguished from lemma counting in that those include the inflected forms of a word in different parts of speech in addition to those in the same part of speech.

Another point to mention is that the NGSL deleted “proper nouns, abbreviations, slang and other noise.” (Browne, 2013, p. 15) Text data was thus so arranged that this study could fairly compare the textbooks with the NGSL, making data analysis time-consuming as closely explained in the next section.

Procedure

To make the textbooks analyzable, I firstly scanned them with OCR software; then, they were converted into .text files. Text data were gathered solely from reading passages in the textbooks. Because OCR software cannot provide 100% text compilation accuracy, I went through every page and manually modified errors.

There are now several corpus-analysis tools that are capable of calculating the lexical coverage of a text by the NGSL; for example, AntWordProfiler 1.5.1 (Anthony, 2021), the Online Graded Text Editor (OGTE; Waring & Browne, n.d.), VocabProfiler (Cobb, 2021). Unfortunately, these tools cannot compute the coverage level of the NGSL words by the textbooks, in part because those are not meant to analyze word lists themselves. Besides, there are cases where they incorrectly identify intended meaning of homographs (e.g., “found” used as the past tense form of “find” or the meaning of “establish”) or components of contracted forms (e.g., “I’d” used as “I would” or “I had”), indicating the necessity of analyzing those words while considering the surrounding context.

For these reasons, this study adopted a unique approach by using Cobb’s text analysis tool, Text Lex Compare ver. 4.2. (2021) to compare the textbooks with the NGSL as reasonably as possible. This tool compares two different texts and informs us of how many tokens, word types, word families, and word phrases are used in and unique to each text. By replacing words in the textbooks with their headwords based on the modified lexeme approach, the number of word types in a text counted by that tool becomes equal to that of flemmas in the text, making it possible to reasonably compare vocabulary words between the NGSL and the textbooks.

As a preliminary preparation for analysis, every word in the textbooks was replaced with its headword based on the modified lexeme approach. In so doing, the replacement function of Microsoft Word 2016 was used (see Figure 1). This was also used to separate semantically transparent contracted forms (e.g., “I’m” into “I am”) as those words are composed of two different flemmas. Semantically opaque ones (e.g., “I’d” used as “I would” or “I had”) and homographs were replaced with their headwords while considering the surrounding context.

Text data after these treatments were loaded into a WordList module which is part of Wordsmith 7 (Scott, 2016). Word lists produced by this module were used to visually ascertain whether or not every word occurring in the textbooks was successfully replaced with its headword.

When calculating the lexical coverage of the textbooks, the 52 supplemental words were added to the 2,801 words in order to avoid the possibility that their frequent recurrences in the textbook underestimate its lexical coverage. On the other hand, they were not added when counting the number of the NGSL words covered by the textbooks. One of the justifications for this was that even if this research argues the deficiency of high-frequency words in the textbooks based on evidence that they do not cover all of numerals or months of the year, no one would be convinced because presenting all those words in a single textbook would not be possible as well as of much help to students. Figure 2 shows how lexical coverage was computed by using Cobb’s Text Lex Compare.

According to Figure 2, 61 tokens were unique to the second text (i.e., the textbook), meaning that 1403 tokens (calculated by 1464 - 61) were covered by the first text (i.e., the NGSL). The lexical coverage of the textbook was then calculated by 1403 ÷ 1464 95.8%.

Lexical Coverage of the Textbooks by the NGSL

To address the first research question, we will look at the lexical coverage of the textbooks by the NGSL. Let us bear in mind that if the textbooks had been developed with an intention to teach high-frequency words, most words appearing in the textbooks should belong to the NGSL. Table 2 reports the number of tokens and that of word types used in and unique to each of the textbooks; the lexical coverage of the textbooks; the proportion of tokens to those unique to the textbooks as well as that of word types to those unique to the textbooks.

Table 2

Results of Lexical Coverage Analysis

Textbook	Number of tokens/(tokens unique to the book)	Proportion	Number of word types/(types unique to the book)	Proportion	Coverage
VISTA Ⅰ	1633(118)	7%	512(74)	14%	92.8%
All Aboard! Ⅰ	1464(61)	4%	451(41)	9%	95.8%
COMET Ⅰ	1257(50)	4%	384(30)	8%	96%
VISTA Ⅱ	1755(74)	4%	486(53)	11%	95.8%
ELEMENT Ⅱ	6149(395)	6%	1153(232)	20%	93.6%
COMET Ⅱ	2475(144)	6%	644(84)	13%	94.2%
LANDMARK Fit Ⅲ	4923(335)	7%	1064(192)	18%	93.2%
ELEMENT Ⅲ	7457(487)	7%	1388(300)	22%	93.5%
All Aboard! Ⅲ	3164(217)	7%	786(125)	16%	93.1%

Seemingly, all the textbooks are largely composed of the NGSL words with larger than 92% coverage. Looking at the columns showing the proportion of word types appearing in the textbooks to those unique to them, a notable finding can be that the textbooks provided words outside the NGSL in varying proportions ranging from 8% (COMET Ⅰ) to 22% (ELEMENT Ⅲ); however, most of those off-list words seem to have rarely recurred in the textbooks as can be inferred from the small differences in frequency of occurrence between tokens unique to the textbooks and word types unique to them. If word types unique to the textbooks had recurred many times, lexical coverage would have been lower than what was shown in the table. To see whether or not the differences in frequency of occurrence between words in the NGSL and those outside the NGSL were statistically significant, a Chi-square goodness of fit test was performed on the data by using Mizumoto’s (2015) web-based application, langtest.jp.

Table 3

Statistical Test Results (NGSL Words vs. off-list words)

Textbook	χ²	df	Cohen’s W
VISTA Ⅰ	1195.1*	1	0.855
All Aboard! Ⅰ	1290.8*	1	0.917
COMET Ⅰ	1065*	1	0.92
VISTA Ⅱ	1471.5*	1	0.916
ELEMENT Ⅱ	1932.5*	1	0.884
COMET Ⅱ	4670.5*	1	0.872
LANDMARK Fit Ⅲ	3674.2*	1	0.864
ELEMENT Ⅲ	5636.2*	1	0.869
All Aboard! Ⅲ	2355.5*	1	0.863
Note. df = degree of freedom. Cohen’s W refers to effect sizes ranging from 0–1; .10. means a small effect size; .30. means a medium effect size; .50. means a large effect size (Cohen, 1992).
* p < .05.

Statistical significance was indicated for all the differences with a large effect size, meaning that at least the textbooks under investigation were largely made up of the NGSL words at a statistically significant level.

Number of the NGSL Words Covered by the Textbooks

Next, to explore the question of whether or not vocabulary words taught in the textbooks can suffice in order for learners to read English texts intended for native speakers, we look at how many of the NGSL words were covered by each of the textbooks (see Figure 3).

The textbooks covered the NGSL words with considerably different proportions ranging from 12% (COMET Ⅰ) to 37% (BIG DIPPER Ⅲ). These proportions might be low enough for us to insist on the deficiency of the NGSL words in the textbooks. To corroborate the results, a Chi-square goodness of fit test was performed.

Table 4

Statistical Test Results (NGSL Words in the Textbooks vs. NGSL Words Outside the Textbooks)

Textbook	χ²	df	Cohen’s W
VISTA Ⅰ	1370*	1	0.693
All Aboard! Ⅰ	1448.7*	1	0.713
COMET Ⅰ	1612.7*	1	0.752
VISTA Ⅱ	1383.9*	1	0.696
ELEMENT Ⅱ	358.26*	1	0.354
COMET Ⅱ	1052.7*	1	0.607
Textbook	χ²	df	Cohen’s W
LANDMARK Fit Ⅲ	431.08*	1	0.389
ELEMENT Ⅲ	160.65*	1	0.237
All Aboard! Ⅲ	821.58*	1	0.537

Note. df = degree of freedom. Cohen’s W refers to effect sizes ranging from 0-1; .10. means a small effect size; .30. means a medium effect size; .50. means a large effect size (Cohen, 1992).

* p < .05.

This statistical test found statistical significance for all the differences with varying effect sizes from small to large. The results indicate that a large part of the NGSL is not covered by the textbooks. The combined results of the lexical coverage analysis of the textbooks and the coverage analysis of the NGSL indicate that although most words in the textbooks belonged to the NGSL, they were only part of the NGSL.

The results of the lexical coverage analysis showed that vocabulary words taught in the nine types of textbooks under investigation were largely made up of the NGSL words at a statistically significant level with larger than 92% lexical coverage. Pedagogically speaking, this is good news because vocabulary words that learners can come across in textbooks would be what they are more likely to meet in the real world. Putting that finding another way, those who have mastered core high-frequency words would be able to achieve 90% coverage of the textbooks, which is a minimum requirement for comprehension, and therefore can read them without a heavy reliance on a dictionary or teacher’s explanation. This is contrary to Browne’s (1998) findings. He analyzed the lexical coverage of MEXT-approved senior high school textbooks by the GSL (West, 1953) and found that they contained so many new and low-frequency words that even learners with the 2,000 most frequent words in the GSL could hardly read a text and guess word meaning from context. Although these contrasting findings may be because this research used a different word list than Browne’s study, it would not be a groundless argument that MEXT-approved senior high school textbooks have come to reflect high-frequency words in the real world over the past two decades.

By counting how many of the NGSL words were treated in the textbooks, it was found that all the textbooks did not sufficiently cover the NGSL words at a statistically significant level. At the same time, the textbooks were found to be insufficient in order for learners to achieve the minimum threshold (i.e., 90%) for reading an authentic text. Hence, they may have to bridge a gap by themselves between vocabulary size that they can potentially learn through textbooks and it required for reading an authentic English text. Language teachers would thus need to give learners additional input of basic words through homework or classroom activities. For example, they can ask their students to watch English movies (Nation, 2006) or to read English texts extensively. Through extensive reading, students can improve their reading comprehension skills (Mason & Krashen, 1997; Tanaka & Stapleton, 2007), their writing performance (Elley, 1991; Mason & Krashen, 1997), or their grammatical competence (Elley, 1991). In these ways, providing students with opportunities for self-learning can possibly lead them to become independent English learners, which is related to learners’ success in vocabulary learning (Gu & Johnson, 1996).

The insufficient supply of the NGSL words in the textbooks may be partly because MEXT-approved textbooks treat a wide variety of topics and are too short to cover all of the NGSL words. In general, they consist of several chapters accompanied by two or three sections; each chapter introduces one type of topic such as emperor penguins’ ecosystem, an environmental problem, a famous Japanese tennis player, and so on. As the use of vocabulary words largely depends on language genres where they occur (Hsu, 2009; Nation, 2006), the greater the number of topics, the greater the number of topic-dependent words. Conversely, if textbooks treat a relatively narrow variety of topics, those words may rarely show up, meaning that it becomes possible to cover a greater number of high-frequency words.

For textbooks to cover core vocabulary words, another possible way is simply to increase the number of words treated in textbooks as the new Course of Study Guidelines declares. Prior to looking at this point more closely, please note that MEXT’s word counting way is almost identical to yet slightly different from that of the NGSL in that MEXT does not count comparatives and superlatives of irregular adjectives as members of their positive forms. Considering that in English there are not so many irregular adjectives, it would be fair to say that the numbers of words of a text counted by MEXT and the NGSL are comparable to each other. The current Couse of Study Guidelines declares that around 3,000 words need to be learned before learners graduate senior high school (MEXT, 2008, 2009). The new Course of Study Guidelines (MEXT, 2017a, b; MEXT 2018), on the other hand, intends to make students familiar with 4,000–5,000 words in total through 10 years of English learning beginning from age 8 or 9. This addition would increase the possibility that MEXT-approved textbooks treat more high-frequency words.

Here, it can be interesting to report results of a survey conducted by the Benesse Educational Research and Development Institute (2014), which is one of the largest companies involved in education in Japan. According to their survey results, junior and senior high school students spend the longest time during their preparation for a lesson looking up new words. Besides, the vast majority of English teachers in Japan felt that students had an aversion to memorizing vocabulary words (Benesse Educational Research and Development Institute, 2015). Although it is a widely accepted idea that an increase in vocabulary size has a positive effect on one’s reading proficiency (Hsu, 2009; Hu & Nation, 2000; Laufer & Ravenhorst-Kalovski, 2010), the survey results suggest that increasing the number of different words in a textbook might just end up imposing a heavy burden on learners. In order to avoid the possibility of saturating students with vocabulary, it is possible that MEXT sets a vocabulary learning target of the 2,801 NGSL words before they graduate senior high school, rather than having them learn 4,000–5,000 words as the new Course of Study Guidelines (MEXT, 2017a, b, 2018) declares.

Although there is not so much research into the lexical coverage of English texts by the NGSL, the value of mastering the NGSL words can be seen in Browne, Miyata, and Kawauchi’s (2016) study. They identified the NGSL words plus 40 keywords provided approximately 98% coverage for a TV show, Friends; for Star Wars- Episode 1, approximately 94% of running words were covered by the NGSL words plus 36 keywords. In their study, keywords referred to words occurring frequently in the text yet outside the NGSL. One may question whether or not the NGSL can satisfy the vocabulary demands of other language genres, too. To answer this question, further research needs to be conducted for other language genres.

Consulting Webb and Rodgers’s (2009a, b) and Nation’s (2006) research may allow us to infer an answer to it. Webb and Rodgers (2009a, b) investigated the vocabulary demands of TV programs and movies; Nation (2006) investigated those of novels, newspapers, graded readers, children’s movies, and unscripted spoken English. These studies analyzed texts by using the same analysis software, Range (Heatley et al., 2002); lexical coverage was calculated by using word families; text data included proper nouns as well as marginal words. Results observed in those studies would be thus fairly comparable. Based on their research findings, the vocabulary demands are summarized as below.

Table 5

Summary of Nation’s (2006) and Webb and Rodgers’s (2009a, b) Findings

Research	Language genre	95% coverage (word family)
Webb and Rodgers (2009a)	Television programs	3,000
Webb and Rodgers (2009b)	Movies	3,000
Research	Language genre	95% coverage (word family)
Nation (2006)	Novels	4,000
	Newspapers	4,000
	Graded readers	2,000
	Children’s movies	4,000
	Spoken English	3,000

As can be seen from this table, several genres require readers to know more words than TV shows or movies. To understand a diverse range of English texts, students would also need to be familiar with words outside the NGSL, that is to say, mid- or low-frequency words. Acknowledging the necessity of learning words other than high-frequency words, this study suggests in the rest of this paper that students should learn the NGSL words first.

Learning English vocabulary is not merely an act of memorizing word meaning (Nagy, 1997). In this regard, Nation (2013) gave a detailed description of multidimensional aspects involved in one word. Briefly explained, three aspects are involved in vocabulary knowledge, namely, form, meaning, and use; each has three components. Form refers to knowledge related to (a) pronunciation, (b) orthography, and (c) morphology; meaning refers to knowledge regarding (a) the meaning of the word form and a word form which can express the meaning, (b) concepts the word has and referents of the concept, and (c) other words semantically associated with the word; use refers to knowledge related to (a) grammatical functions, (b) collocations, and (c) registers to use the word or the frequency of occurrence of the word. Teaching all of these aspects involved in one word at a time is not a reasonable approach (McCarten, 2007; Nation, 2013), and an oversupply of new words within a single textbook possibly prevents language teachers from imparting those aspects to learners as they may unintentionally spend a huge amount of time explaining word meaning. As Nagy (1997) argues, definition-based vocabulary learning should be avoided as much as possible because students may end up learning little about one word. What people involved in English education should do is thus giving students a proper amount of vocabulary.

“Low-frequency words tend to have fewer family members than high-frequency words” (Nation, 2006, p. 65), indicating that more time should be allocated to high-frequency words than low-frequency words so that learners can become well acquainted with at least core high-frequency words. Accordingly, MEXT should give multiple occurrences to high-frequency words, rather than increasing the number of words to teach. By so doing, students can know much about high-frequency words because “repeated meeting can have the effects of strengthening and enriching previous knowledge” (Nation, 2013, p. 127).

These arguments are not to say mid- or low-frequency words are not of high value for learners. Learning those words will be undoubtedly necessary because when they enter a university or college, they will need to be familiar with academic words or technical words in their majors. However, it would take many years for Japanese learners to master high-frequency words (Browne, 2021), asking them to learn mid- or low-frequency words including specialized vocabulary words is impractical. According to Nation (2013), those words can be effectively learned by paying special attention to them. Fortunately, we can now have access to numerous word lists focusing on particular contexts; for example, the AWL (Coxhead, 2000), the Business Word List (Browne & Culligan, 2016a), the TOEIC Word List (Browne & Culligan, 2016b), the Newspaper Word List (Chung, 2009), the Academic Spoken Word List (Dang, Coxhead, & Webb, 2017), and more. High-frequency words in general English are more dominant even in such specialized contexts than specialized words (Coxhead, 2000; Nation, 2013), again indicating the importance of mastering high-frequency words first as Nation (2013) suggests. Such high-frequency words as the NGSL words can potentially allow students to prepare for their English learning in the future.

Limitations

There are several limitations to note. It is general that senior high school students in Japan take English classes not for just one year but for three years. In other words, they learn vocabulary through three types of textbooks. Therefore, the evaluation of vocabulary should have been done not by textbook but by a set of three textbooks to represent learners’ vocabulary input at senior high school. Due to the availability of information on which set was used by students the most, this research decided to consult the marketing data of textbooks in Tokyo.

Second, the number of textbooks examined in this study was limited to 10 types of MEXT-approved senior high school textbooks. To generalize the results, further research should be conducted for textbooks other than those analyzed in this study. The research findings will be, then, more reliable.

This study has evaluated vocabulary in MEXT-approved senior high school textbooks from two different perspectives: the lexical coverage of the textbooks by the NGSL and the coverage of the NGSL by each of the textbooks. The overall research findings indicated that it was possible for students to learn high-frequency words through textbooks, but those words would not be sufficient enough for learners to read an authentic text. In reaction to these findings, it was suggested that textbooks should teach the NGSL words first in order to help students master at least core high-frequency words. Without these words, learners would not become able to read any kind of text because “any significant gap in knowledge of these ultra-high frequency words would make it virtually mathematically impossible to achieve 90% or higher coverage.” (Browne, 2021, p. 3).

Certainly, both mid- and low-frequency words would be indispensable for learners to understand a wide range of English texts; teaching them to learners who have not probably mastered core words yet would not be impractical. A necessary step toward letting students move on to those words is to explore how many words learners can reasonably acquire before they graduate senior high school. By so doing, it would be possible to offer designers of textbooks more valuable insights into an optimal amount of vocabulary in a textbook. Without that study, trying to teach more vocabulary words would increase the risk of inundating students with vocabulary. To avoid developing textbooks that have the risk of saturating learners with vocabulary, their vocabulary capacity would be a research area to explore as soon as possible.

AWL: Academic Word List; BNC: British National Corpus; EFL: English as a foreign language; GSL: General Service List: MEXT: Ministry of Education, Culture, Sports, Science and Technology; NGSL: New General Service List; OGTE: Online Graded Text Editor

Availability of Data and Materials

The dataset of word list analyzed during the current study are available from http://www.newgeneralservicelist.org, but textbook data analyzed during the current study are not publicly available due to copyright.

Competing Interests

The author declares that he has no competing interests.

Funding

This study did not receive any funding.

Authors’ Contribution

This study was conducted by a single author. The author designed the study, reviewed the literature, gathered and analyzed the data, and interpretated the analysis results. The author(s) read and approved the final manuscript.

Acknowledgments

Not applicable.

Alqahtani, M. (2015). The importance of vocabulary in language learning and how to be taught. International Journal of Teaching and Education, 3(3), 21-34. https://doi.org/10.20472/TE.2015.3.3.002
Anthony, L. (2021). AntWordProfiler (Version 1.5.1) [Computer software]. Waseda University. https://www.laurenceanthony.net/software Accessed 30 March 2021.
Benesse Educational Research and Development Institute. (2014). Tyukousei no eigokyouiku ni taisuru zittai tyousa 2014 [Teenagers English learning survey 2014]. https://berd.benesse.jp/global/research/detail1.php?id=4356 Accessed 7 February 2021.
Benesse Educational Research and Development Institute. (2015). Tyukou no eigogakusyu ni kansuru zittai tyousa 2015 [Teenagers English learning survey 2015]. https://berd.benesse.jp/global/research/detail1.php?id=4776 Accessed 7 February 2021.
Bouhlal, F., Horst, M., & Martini, J. (2018). Modality in ESL textbooks: Insights from a contrastive corpus-based analysis. Canadian Modern Language Review, 74(2), 227-252. http://doi:10.3138/cmlr.3075
Browne, C. (1998). Japanese high school textbooks: How readable are they? Temple University Japan Working Papers in Applied Linguistics, 12, 1-13.
Browne, C. (2013). The new general service list: Celebrating 60 years of vocabulary learning. The Language Teacher, 7(34), 13–16.
Browne, C. (2021). The NGSL project: Building wordlists and resources to help EFL learners (and teachers) to succeed. In E. Forsythe (Ed.), Teaching with Technology 2020 Selected papers from the JALTCALL2020 Conference (pp. 1-18). Tokyo: The JALT CALL SIG. https://doi.org/10.37546/JALTSIG.CALL2020.1
Browne, C., & Culligan, B. (2016a). The business service list. http://www.newgeneralservicelist.org. Accessed 10 February 2021.
Browne, C., & Culligan, B. (2016b). The TOEIC service list. http://www.newgeneralservicelist.org. Accessed 10 February 2021.
Browne, C., Culligan, B., & Phillips, J. (2013a). The new general service list. http://www.newgeneralservicelist.org. Accessed 12 February 2021.
Browne, C., Culligan, B., & Phillips, J. (2013b). The new academic word list. http://www.newgeneralservicelist.org. Accessed 10 February 2021.
Browne, C., Miyata, Y., & Kawauchi, Y. (2016, September 12-14). Does the NGSL provide good coverage for authentic listening materials such as TV shows and movies? A quick look at Star Wars and Friends [Paper presentation]. Vocab@Tokyo Conference, Tokyo, Japan.
Chung, M. (2009). The newspaper word list: A specialised vocabulary for reading newspapers. JALT Journal, 31(2), 159-182. https://doi.org/10.37546/JALTJJ31.2-2
Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155–159. https://doi.org/10.1037/0033-2909.112.1.155
Cobb, T. (2021). Text Lex Compare v.4.3 [Computer software]. https://www.lextutor.ca/cgi-bin/tl_compare/ Accessed 20 February 2021.
Cobb, T. (2021). Web VocabProfile [Computer software]. https://www.lextutor.ca/vp/comp/　 Accessed 15 June 2021.
Coxhead, A. (2000). A new academic word list. TESOL Quarterly, 34(2), 213-238. https://doi.org/10.2307/3587951
Chujo, K. (2004). Measuring vocabulary levels of English textbooks and tests using a BNC lemmatised high frequency word list. In J. Nakamura, N. Inoue, & T. Tabata (Eds.), English corpora under Japanese eyes (pp. 231-249). Amsterdam: Rodopi.
Dang, T. N. Y., Coxhead, A., & Webb, S. (2017). The academic spoken word list. Language Learning, 67(4), 959-997. https://doi.org/10.1111/lang.12253
Elley, W. B. (1991). Acquiring literacy in a second language: The effect of book-based programs. Language Learning, 41(3), 375-411. https://doi.org/10.1111/j.1467-1770.1991.tb00611.x
Gu, Y., & Johnson, R. K. (1996). Vocabulary learning strategies and language learning outcomes. Language Learning, 46, 643-679. https://doi.org/10.1111/j.1467-1770.1996.tb01355.x
Heatley, A., Nation, I. S. P., & Coxhead, A. (2002). Range [Computer software].
http://www.victoria.ac.nz/lals/staff/paul-nation/nation.aspx Accessed 20 February 2021.
Hsu, W. (2009). College English textbooks for general purposes: A corpus-based analysis of lexical coverage. Electronic Journal of Foreign Language Teaching, 6(1), 42-62.
Hu, M., & Nation, I. S. P. (2000). Unknown vocabulary and reading comprehension. Reading in a Foreign Language, 13(1), 403-430.
Hu, R. J. S. (2011). The relationship between demotivation and EFL learners’ English language proficiency. English Language Teaching, 4(4), 88-96. http://dx.doi.org/10.5539/elt.v4n4p88
Ikeno et al. (2017). Revised comet English communication Ⅰ. Tokyo: Suken.
Ikeno et al. (2018). Revised comet English communication Ⅱ. Tokyo: Suken.
Kaneko et al. (2018a). Vista English communication Ⅰ new edition. Tokyo: Sanseidou.
Kaneko et al. (2018b). Vista English communication Ⅱ new edition. Tokyo: Sanseidou.
Kiyota et al. (2017). All aboard! English communication Ⅰ. Tokyo: Tokyosyoseki.
Kiyota et al. (2019). All aboard! English communication Ⅲ. Tokyo: Tokyosyoseki.
Laufer, B. (1989). What percentage of text lexis is necessary for comprehension? In C. Lauren & M. Nordman (Eds.), Special language: From humans thinking to thinking machines (pp. 316–323). Bristol: Multilingual Matters.
Laufer, B., & Ravenhorst-Kalovski, G. C. (2010). Lexical threshold revisited: Lexical text coverage, learners’ vocabulary size and reading comprehension. Reading in a Foreign Language, 22(1), 15-30.
Liu, N., & Nation, I. S. P. (1985). Factors affecting guessing vocabulary in context. RELC Journal, 16(33), 33-42. https://doi.org/10.1177/003368828501600103　
Matsuda, A. (2002). Representation of users and uses of English in beginning Japanese EFL textbooks. JALT journal, 24(2), 182-200. https://doi.org/10.37546/JALTJJ24.2-5
Mason, B., & Krashen, S. (1997). Extensive reading in English as a foreign language. System, 25(1), 91-102. https://doi.org/10.1016/S0346-251X(96)00063-2
McCarten, J. (2007). Teaching vocabulary: Lessons from the corpus, lessons for the classroom. Cambridge: Cambridge University Press. http://dx.doi.org/10.14393/DL15-v8n1a2014-39　
McLean, S. (2017). Evidence for the adoption of flemma as an appropriate word counting unit. Applied Linguistics 2018, 39(6), 823-845. https://doi.org/10.1093/applin/amw050
Ministry of Education, Culture, Sports, Science and Technology. (2008). Chugakkou gakusyu shidou youryou kaisetsu gaikokugo hen [Expository comments on the course of study guideline for foreign languages in junior high schools]. Kyoto: Higashiyamashobo.
Ministry of Education, Culture, Sports, Science and Technology. (2009). Koutougakkou gakusyu shidou youryou gaikokugo hen eigo hen [Expository comments on the course of study guidelines for foreign languages in senior high schools]. Kyoto: Higashiyamashobo.
Ministry of Education, Culture, Sports, Science and Technology. (2017a). Shougakkou gakushu shidou youryou kaisetsu gaikokugo katsudou gaikokugo hen [Expository comments on the course of study guidelines for foreign languages in elementary schools]. Tokyo: Touyoukan.
Ministry of Education, Culture, Sports, Science and Technology. (2017b). Chugakkou gakusyu shidou youryou kaisetsu gaikokugo hen [Expository comments on the course of study guideline for foreign languages in junior high schools]. Kyoto: Higashiyamashobo.
Ministry of Education, Culture, Sports, Science and Technology. (2018). Koutougakkou gakusyu shidou youryou gaikokugo hen eigo hen [Expository comments on the course of study guidelines for foreign languages in senior high schools]. Kyoto: Higashiyamashobo.
Mizumoto, A. (2015). Langtest.jp (Version 1.0) [Computer software]. http://langtest.jp/ Accessed 20 February 2021.
Nagy, W. E. (1997). On the role of context in first- and second-language vocabulary learning. In N. Schmitt & M. McCarthy (Eds.), Vocabulary: Description, acquisition and pedagogy (pp. 64-83). Cambridge: Cambridge University Press.
Nation, I. S. P. (2001). Learning vocabulary in another language. Cambridge: Cambridge University Press.
Nation, I. S. P. (2006). How large a vocabulary is needed for reading and listening? Canadian Modern Language Review/ La Revue canadienne des langues vivantes, 63(1), 59-81. http://dx.doi.org/10.1353/cml.2006.0049
Nation, I. S. P. (2013). Learning vocabulary in another language (2nd ed.). Cambridge: Cambridge University Press.
Schmitt, N., Jiang, X., & Grabe, W. (2011). The percentage of words known in a text and reading comprehension. The Modern Language journal, 95(1), 26-43. https://doi.org/10.1111/j.1540-4781.2011.01146.x
Schmitt, N., Schmitt, D., & Clapham, C. (2001). Developing and exploring the behavior of two new versions of the vocabulary levels test. Language Testing, 18(1), 55-88. https://doi.org/10.1177/026553220101800103
Scott, M. (2016) WordSmith tools version 7 [Computer software]. Stroud: Lexical Analysis Software. https://www.lexically.net/wordsmith/ Accessed 10 March 2021.
Tanabe et al. (2018). Landmark fit English communication Ⅲ. Osaka: Keirinkan
Tanaka, H., & Stapleton, P. (2007). Increasing reading input in Japanese high school EFL classrooms: An empirical study exploring the efficacy of extensive reading. The reading matrix, 7(1), 115-131.
Tokyo Metropolitan Board Education (2020). Reiwa 3 nendo shiyou toritsu koutougakkou oyobi toritsu tyutougakkou (koukikatei) you kyoukasho kyoukabetsu saitaku kekka (kyoukasho betsu gakkousuu) [Textbooks adopted in senior high schools and upper secondary schools in Tokyo in 2021]. https://www.kyoiku.metro.tokyo.lg.jp/press/press_release/2020/release20200827_01.html Accessed 5 February 2021.
Underwood, P. (2010). A comparative analysis of MEXT English reading textbooks and Japan’s National Center Test, RELC Journal, 41(2), 165-182. https://doi.org/10.1177/0033688210373128
Ushiro et al. (2018). Revised element English communication Ⅱ. Osaka: Keirinkan.
Ushiro et al. (2019). Revised element English communication Ⅲ. Osaka: Keirinkan.
Waring, R. & Browne, C. (n.d.). Online graded text editor [Computer software]. https://www.er-central.com/ogte/ Accessed 20 February 2021.
West, M. (1953). A general service list of English words. London: Longman, Green & Co.
Webb, S., & Rodgers, M. P. H. (2009a). Vocabulary demands of television programs. Language Learning, 59(2), 335-366. https://doi.org/10.1111/j.1467-9922.2009.00509.x
Webb, S., & Rodgers, M. P. H. (2009b). The lexical coverage of movies. Applied Linguistics, 30(3), 407-427. https://doi.org/10.1093/applin/amp010
Wongsarnpigoon, I. (2018). Vocabulary in junior high school textbooks and exams. In P. Clements, A. Krause, & P. Bennett (Eds.), Language teaching in a global age: Shaping the classroom, shaping the world. Tokyo: JALT.
Xue, G., & Nation, I. S. P. (1984). A university word list. Language Learning and Communication, 3, 215–229.
Zimmerman, C. B. (1997). Historical trends in second language vocabulary learning. In J. Coady & H. Thomas (Eds.), Second language vocabulary acquisition: A rationale for pedagogy (pp. 5-19). Cambridge: Cambridge University Press.

Download PDF

Editorial decision: Major revision
06 Aug, 2021
Review #1 received at journal
26 Jul, 2021
Review #2 received at journal
26 Jul, 2021
Reviewer #2 agreed at journal
11 Jul, 2021
Reviewers invited by journal
11 Jul, 2021
Reviewer #1 agreed at journal
11 Jul, 2021
Editor assigned by journal
04 Jul, 2021
Submission checks completed at journal
04 Jul, 2021
Editor invited by journal
04 Jul, 2021
First submitted to journal
03 Jul, 2021

You are reading this latest preprint version

A Quantitative Analysis of Vocabulary Taught in Japanese EFL Textbooks

Status:

Version 1

Abstract

Figures

introduction

Literature Review

Methodology

Results

Lexical Coverage of the Textbooks by the NGSL

Discussion

Conclusion

Abbreviations

Declarations

References

Status:

Version 1