As indicated above, 109 people filled in the questionnaire, out of which 60 were females and 49 were males. Most of the respondents were between 20 and 39 years old (81 people/74%), followed by 17 (16%) people at the age between 40 and 59 years and eight (7%) people at the age between 0 and 19 years. Only three (3%) people were older than 60 years. According to the responses, the majority of the respondents (82/75%) were classified as beginners (A2 level) in a foreign language. A slightly smaller number (24/22%) have basic proficiency (A1 level) and only three (3%) respondents identified themselves as intermediate (B1 level). If a respondent had a higher or advanced level of a foreign language, s/he was excluded from the survey as s/he did not belong to the target group of this paper.
The findings also show that the most popular chatbot is Duolingo because 100 (92%) of the respondents know and use it, followed by ChatGPT with 66 (61%) users. In addition, Memrise is popular among 42 (39%) users, HelloTalk among 41 (38%) users, and Replika among 21 (19%) users. Tandem, Babbel, Lingvist, and Busuu have relatively few users, with 18 (17%), 15 (14%), and 7 (6%) users respectively. The respondents were also asked about the features they appreciate most when using these chatbots. Their responses are listed in Table 1 below in order of their frequency.
Table 1
An overview of the beneficial features of a chatbot for learning a foreign language
Feature | Number of responses |
Simplicity | 29 (27%) |
Quick practice, short lessons | 25 (23%) |
Free of charge | 17 (16%) |
Gamification features | 11 (10%) |
A good learning tool | 11 (10%) |
Use anywhere and at anytime | 9 (8%) |
Appealing design | 8 (7%) |
Fun lessons | 8 (7%) |
Easy to use | 8 (7%) |
Saves time | 5 (5%) |
I don´t know | 4 (4%) |
Word translations | 1 (1%) |
Vocabulary learning | 1 (1%) |
Good feedback | 1 (1%) |
Thus, the findings indicate that the main reason for using chatbots appears to be their simplicity, quick practice, gamification features, accessibility, and the fact that their use is free of charge.
On the contrary, students complained about the advertisements (24/22% users) they had to face when using these chatbots, as well as that some of the chatbots were paid (15/14% users) and some contained complex lessons (12/11% users). Other, lesser criticisms included the incomprehensibility of some features, slowness, lessons on talking, and limitations of lives. However, a total of 33 (30%) users did not find any negative features. Overall, the results of the table suggest that users expect chatbots to be easy to use, and provide relevant and useful content without too many advertisements, with a minimum of restrictions.
Figure 1 below then shows the frequency of using chatbots for learning a foreign language. The findings indicate that most people (56/51%) use chatbots for learning a foreign language at least once a month, and 21 (19%) of them at least once a week to practice a foreign language.
The respondents were also asked about the period of using the chatbot. Most respondents (97/89%) actively use chatbots for no more than half an hour, which may indicate that chatbots are more suitable for short interactions. At the same time, however, there is no one using chatbots for more than an hour, which might indicate that users increasingly prefer human interaction for longer interactions. Some respondents (12/11%) use chatbots for 31–60 minutes. Overall, the results suggest that chatbots are useful for short-term interactions.
Furthermore, the results reveal that almost all respondents (107/98%) use a chatbot to develop their foreign language at home. 45 (41%) then also use it when traveling, 24 (22%) at work, and only 7 (6%) outdoors (Fig. 2). Thus, most people prefer to use chatbots at home, which might be connected with the convenience and ease of access to these chatbots. On the contrary, outdoor environments seem less suitable for using chatbots, which could be caused by less available internet connections or less suitable conditions for using these chatbots.
Figure 3 below then shows which learning method helps students learn a foreign language best. The findings indicate that most respondents (92%) learn a foreign language through spoken communication, followed by writing (70%) and reading (63%). On the other hand, listening (15%) is not very popular among respondents. This may be due to the fact that people often do not want to or cannot listen to the sound; it is also demanding on concentration. It is also possible that listening is sometimes only in the paid version and users do not want to pay for additional features. Moreover, learning a foreign language on the basis of grammar structures, vocabulary development, and translation of words seems to be also a marginal issue.
When developing a chatbot for learning a foreign language, interaction with a chatbot appears to be an important issue for the development of a personalized chatbot. The findings of this study revealed that the most appropriate way for respondents to interact with chatbots is a choice of options (60/55% of respondents), followed by writing messages (48/44% of respondents). The least popular is speaking into a microphone (1/1% respondent), which may be due to fear of misunderstanding or inability to use this feature. However, the results also demonstrate that the most effective way of interaction for students seems to be writing messages (58/53% of respondents), which is followed by a choice of options (51/47% of respondents). Therefore, when compared with the previous results, one can observe that some respondents find a certain way of interaction to be the most appropriate, but find another option to be the most effective.
Figure 4 below then illustrates the students´ goals for learning a foreign language by using a chatbot. The results reveal that the highest number of responses include improving vocabulary, indicating that most respondents (41/28%) wish to expand their vocabulary and learn more words and phrases in a foreign language. The second highest number of responses (39/36%) was recorded for improving the ability to understand a foreign language, indicating that many respondents wish to improve their ability to read, listen and understand a foreign language. The third, also a high number, was recorded for improving their grammar (27/25%). Conversely, only two people (2%) responded that their goal was to improve their conversational skills.
Moreover, the findings reveal that respondents prefer to receive feedback immediately after each answer (105/96% of respondents), suggesting that they want to know straight away whether they have made a mistake and whether there has been any misunderstanding. Only 4 (4%) respondents prefer to receive feedback after the conversation has been completed.
The respondents were also asked how they would like new words and phrases to be explained to them. Most respondents prefer verbal definitions to examples of use or pictures. Specifically, 106 (97%) respondents prefer verbal definitions, while only 2 (2%) respondents prefer examples of use and 1 (1%) respondent prefers pictures. In practice, one can interpret this as users wanting explanations of new words or phrases using definitions.
Thus, in summary, it can be concluded that the respondents were roughly equally divided by gender. The main group was made up of people aged between 20 and 39 years old and most of them were classified as beginners and advanced beginners of English. The chatbots that were familiar to the respondents included Duolingo and ChatGPT, while other chatbots were not as widespread. The conclusion from the survey showed that chatbot users valued free, easy-to-use, and quick-to-learn features the most, while complex features, the presence of advertisements, and paid versions were less popular. More than half of the respondents use chatbots regularly, usually at home for less than 30 minutes. Respondents prefer learning a foreign language through writing and reading while listening exercises and speaking are not as popular.
The method of interacting with the chatbot that has proven most effective is the choice of options, however, texting is also very popular. When it comes to effectiveness, texting was identified as the most effective method of interaction. Chatbot users aim to improve their vocabulary and understanding of a foreign language. It is important that the chatbot responds to replies in text form and explains new words and phrases using word definitions.
These survey results suggest that chatbots are a valuable resource for users learning a foreign language. Free, easy-to-use, simple, and fast learning are key factors that chatbot user value. Since most respondents use chatbots more than once a month, it seems that these applications are very effective for foreign language learning. The survey results also showed that typing messages is the most effective way to interact with a chatbot, but the choice of options is also very popular.
Table 2 below describes the appropriate persona of the foreign language user.
Table 2
The persona of a foreign language user
Category | Data |
Sex | Female |
Age | 22 years |
Language level | A1 |
The environment where a chatbot is used | At home |
Frequency of use | Several times a month |
Period of use | 0–30 minutes |
Favourite learning method | Speaking, writing, and reading |
Goal of use | Enhancement of vocabulary |
Other requirements | Free, easy to use, simple, fast |