3.1 Participants
For the quantitative phase of the study, regarding the efficiency of computer software on vocabulary development and listening comprehension ability of L2 learners, 60 intermediate EFL learners (i.e., 24 male and 36 female) were conveniently chosen from a private English language institute situated in Amol city, Mazandaran province of Iran. Initially, the participants’ homogeneity was checked through their scores gained from the records of the institute where they were studying. Besides, all the participants were between the age range of 18 to 25 to see how young EFL learners would respond to technology enhanced instruction. It should also be noted that the participants were divided into one experimental group (n=30) and one control group (n=30) to compare their scores in case of the effectiveness of instruction through computer software on their vocabulary learning and listening comprehension ability.
For the qualitative phase of the study, 30 intermediate EFL learners from the same experimental group were kindly invited to join the study for semi-structured interviews. Hopefully, 24 of them (i.e., fourteen female & ten male ones) agreed to participate in the study entirely voluntarily, in order to explain and share their experiences of technology-enhanced language instruction and its effects on their vocabulary knowledge and listening comprehension with the researchers of the study. Their ages ranged from 19 to 24 (M= 22.31, SD= 1.42). All the attendees were reassured that the data obtained from them would be merely used for the sake of research and academic purposes, and their personal information would be kept entirely confidential.
3.2 Design
The present study is based on concurrent design (i.e., mixed-methods) whereby the quantitative data and qualitative data are collected and analyzed separately but at approximately the same time (Ary et al.’s classifications, 2019). Initially, the homogeneity of the learners was checked through the records of the institute where they were doing their General English Course. Then, the quantitative phase of the study commenced, and the vocabulary pre-test as well as the listening comprehension pre-test were given to the participants to check their initial ability in vocabulary knowledge and listening comprehension. Then, those participants assigned as the experimental group, were given vocabulary and listening instruction through computer software, trying to provide opportunities for the learners to be interactively involved in the listening instruction and simultaneously getting mastery over vocabularies emerged within the listening tasks. Finally, the experimental group and the control group (receiving no CALL instruction) took the vocabulary and listening post-test to investigate the effect of technology enhanced instruction through computer software on the so-called language sub-skill and skill. Later, the qualitative phase commenced and the EFL learners of the experimental group were invited to semi-structured interviews to explain and share their experiences of technology enhanced language learning with the researchers of the study.
3.3 Instruments
The following instruments were used for the quantitative phase of the study to conduct data collection procedure concerning the purpose of the study. For the qualitative phase, interview was run.
3.3.1 Listening Comprehension Pre-Test and Post-Test
In order to check the learners’ initial listening comprehension ability, they took the listening pre-test, which was adopted from Cambridge Key English Test (KET). It contained 25 listening comprehension items with various formats, including multiple choice, fill in the blanks, and matching questions, played for the testees twice. The participants were given 30 minutes to do the pre-test. Prior to the administration of the test, the teacher provided sufficient explanations regarding the test and how they were expected to answer to the items. Similar to the pre-test, another version of KET was used as the post-test to check the effect of treatment sessions of technology enhanced instruction through computer software on the learners’ listening comprehension ability. The number of items as well as the formats were the same as the pre-test.
3.3.2 Vocabulary Pre-Test and Post-Test
The researcher-made vocabulary pre-test was taken by the participants before the treatment. The pre-test was based on the course syllable content. It was in a form of 15 multiple choice questions to check their initial knowledge of the target vocabularies. Similar to the pre-test, the researcher-made vocabulary post-test was used to look into the effectiveness of the treatment sessions of CALL instruction. In fact, the post-test served as a measurement of the students’ progress after the treatment. It is notable that the post-test items were the same as the pre-test with some small changes in the order of the questions and replacing the responses. As to the reliability coefficient of the vocabulary test, a pilot study was conducted with the participation of 5 intermediate students (from a private institute with similar characteristics of the participants of the present study) to check test score consistency. Reliability coefficient was found to be 0.74 (using KR-21 formula), which appeared to be a reasonable value in terms of consistency of scores as highlighted in Farhady, Jafarpour, and Birjandi (1994). The reliability of the vocabulary test is shown in Table 1.
Table 1. Reliability of Vocabulary Test
N
|
Mean
|
SD
|
Variance
|
Reliability
|
5
|
7.2
|
2.86
|
8.2
|
0.74
|
3.3.3 Language-learning Software
Rosetta Stone is a language-learning software produced by Rosetta Stone, Ltd. The Rosetta Stone Software utilizes a combination of images, texts, and sounds in order to teach various vocabulary terms, listening skills and grammatical functions intuitively without drills or translation and the difficulty levels increase as the language learner progresses. Instruction takes the form of a unit of lessons consisting of ten group of four images each, with an associated word or sentence both written and spoken aloud by a native speaker of the language. Lesson topics range from grammatical concepts such as verb tense to specific topics. Within each lesson there are sets of exercises testing vocabulary and listening comprehension (for which the computer must have a speaker and a microphone). The user is offered either text, sound or image and later, video, to match the four possibilities. With a mark and chosen by the reader from the preferences menu, the program indicates whether the right or wrong choice has been selected.
3.3.4 Interview
In order to deeply look into the efficiency of computer software on vocabulary development and listening comprehension ability of L2 learners, semi-structured interviews were arranged and utilized. According to Klassen et al. (2012), open-ended questions and the responses received could produce qualitative data which are rich in detail and would provide in-depth data that goes beyond ordinary quantitative data. The interview questions of this study were prepared and arranged in a top-down manner beginning from general ideas and experiences, and then continuing with more specific and detailed notions (Herndon, 2001), regarding how participants felt, what they practiced, and what they experienced in technology-based classes. One obvious advantage of this method is that the researcher formulates questions prior to the interview meetings so that the conversation can be smooth and have direction to avoid loss of focus. To equally obtain answers from the participants, the researcher asked the same questions from all of the participants. As Lindlof (1995, p. 172) maintains, "by asking the same questions of all participants in the same order, the researcher minimizes interviewer effects and achieves greater efficiency of information gathering" (Gilbert & Stoneman, 2015). In addition, the content validity of the interview-questions was verified by three experts who are seasoned enough in conducting research studies in Teaching English as a Foreign Language (TEFL). As Klassen et al. (2012) declare, experts in the field can serve to establish content validity for questionnaires. Each question had an acceptable universal agreement of .81 or above (Polit & Beck, 2006). The interview sessions were run in a face-to-face manner at a language institute center located in Amol, Iran. The researchers met each of the individuals in a separate meeting in which general prompt questions were asked allowing participants verbalize their perceptions, attitudes, beliefs, emotions, and experiences. Besides, the interview sessions were held in the interviewees’ native language (i.e., Persian) in order to make verbalization more relaxed and more comfortable in a less stressed atmosphere.
3.4 Procedures
The present study benefited from mixed-methodology to look into the learners’ possible improvement in vocabulary learning and listening comprehension as a result of being exposed to CALL environment (i.e., The Rosetta Stone Software). In fact, the participants of the study, who were doing their General English Course, received vocabulary input and listening instruction through computer software to help them get mastery over these language skill and sub-skill. Before initiating the treatment sessions, participants took the vocabulary and listening comprehension pre-tests to check their initial ability in the knowledge of vocabulary and listening comprehension. It is noteworthy that the reliability of the vocabulary tests was checked by a pilot study with the participation of similar students. After the pre-tests, the participants received CALL instruction through computer software to experience a fairly various educational environment. Learners were provided with computer software slides full of vocabulary and listening tasks to raise their knowledge in an interactive way. Prior to being exposed to listening instruction, target vocabularies related to the listening tasks were also taught by the teacher-researcher to help the learners develop their understanding of target words. In fact, vocabulary instruction was done before the learners’ comprehension of listening tasks by asking various questions and making related examples. In this way, learners were exposed to conscious understanding of the vocabularies as they were asked to apply them in their own sentences. After exposure to 10 two-hour sessions of CALL instruction during 5 weeks, the participants took the required post-tests to examine the effectiveness of treatment sessions on their vocabulary learning and listening comprehension. It should be also mentioned that the participants in the control group underwent normal conventional classroom sessions in which the learners experienced no technology-enhanced learning environment and the same teacher offered the same treatment in a conventional manner.
Secondly, the qualitative phase commenced and was based on the interpretations of the data obtained through 24-fifteen-minute interviews with the EFL learners of the same experimental group, which took 32 days to fulfil. In this regard, all the participants were separately invited to attend an interview session, at their best convenience, in the same language institute in which they had underwent CALL instruction (located in Amol city in Mazandaran Province, Iran). And those invited participants who missed the meetings were offered a second chance to be interviewed online via Skype– an easy and user-friendly application specifically used for video calling. Each interview session, whether in person or virtual, lasted for nearly 15 minutes on average. Following greetings, each interview began with a short announcement of the objectives of the current study to assist interviewees discern what was going on and feel more comfortable (Isaee & Barjesteh, 2023). Then, general questions were asked which would naturally lead to more specific ones. All the interview sessions were entirely audio-recorded for further analysis. Great care was taken by the researchers to offer as much time needed to the respondents to freely express or reiterate their ideas, opinions, and feelings without unnecessary interruptions. In order to make verbalization more relaxed and in a less stressed atmosphere, the interview sessions were held in the interviewees’ native language (i.e., Persian). Later, the recorded interviews were transcribed and later, translated into English by one of the researchers (i.e., Associate Professor Hamed Barjesteh). As some errors might probably occur during the translation process, this phase was considered a challenging issue since the meanings of the original messages could be distorted; however, every line was double-checked to ensure that there were no inconsistences between the recorded message and the translated message. Then, the transcripts were carefully read several times by the researchers to aid them become sufficiently familiar with the content of the data. Finally, through principles of conformability, dependability, credibility, and transferability, essential precautions were taken to ensure trustworthiness of the data.
3.5 Data Analysis
In order to analyze the data, quantitative methodology as well as qualitative methodology were adopted. Unlike secondary data in which the data are collected from other sources (e.g., articles, publications, websites, books), the data collected for this study is known as primary data (Ary et al., 2019), which were gathered by the researchers themselves. In the current study, the first-hand primary data were obtained through pre-test, post-test, and interviews.
Initially, the data obtained through pre-test and post-test were subjected to statistical analysis (i.e., descriptive and inferential statistics) through SPSS version 22, in order to find the effectiveness of CALL instruction on the EFL learners’ vocabulary and listening comprehension improvement. Besides, a series of paired sample t-tests were employed to compare experimental group’s performance to the control group’s, on the pre-test and post-test.
Secondly, the qualitative data obtained from the interview sessions were thematically analyzed via Nvivo Application to make sure that all the important aspects of the data were captured. The creation of themes is a crucial process where important and recurring points are derived from the transcribed texts in relation to the research objectives (Terry et al., 2017). The interview transcripts of this study underwent five steps: 1) getting sufficiently familiar with the information, 2) open coding, axial coding, and selective coding, 3) identifying themes, 4) naming and defining themes, and 5) getting the reports on paper. It is worth mentioning that during open coding, meaning-carrying and significant parts of the transcripts were coded and when plausible, groups of codes were assigned as a category. During axial coding, meaningful connections between the categories were established. And at last, during selective coding, a unified category of the EFL learners’ perceptions of technology-based classes was achieved.
Each research question of the study has been investigated separately, and the results have been tabulated in 10 Tables. To facilitate grasp of the data, two figures are also depicted as follows.