Context of the study
This study was conducted on written feedback the thesis supervisors’ offer to their supervisees in Teaching English as a Foreign Language (TEFL) while conducting their M.A thesis. The study took place in four public universities in Ethiopia. Among the universities of Ethiopia, Arbaminch University, Wolayeta Sodo University, Hawassa University, and Dilla University were selected for this study due to their relative proximity to the researchers’ workplace, Arbaminch. The data were collected in 2020 between January and October.
Research design
The objective of this study was to examine EFL supervisors’ written feedback focus, and language functions at some selected public universities in Ethiopia. To address the objective of the study, a descriptive exploratory design with a qualitative approach was adopted. Since the experience of written feedback at the graduate level is less studied, a descriptive exploratory design was considered appropriate.
Participants of the study
Selected EFL thesis supervisors in the four universities and their supervisees were the participants of the study. From the total of 255 supervisees, 55 were selected through a simple random sampling technique for a questionnaire survey. However, four supervisors were also selected based on their willingness for in-depth interviews.
Data collection instruments
This study aimed to explore the feedback focus and language functions of supervisors’ written feedback on supervisees’ thesis at selected universities in Ethiopia. To address these objectives, a questionnaire, interviews with selected supervisors, and in-text feedback comments on thesis drafts were utilized. A detailed description of each tool is provided below.
Questionnaire for supervisees
The purpose of the supervisees’ questionnaire was to obtain quantitative data on their supervisors’ feedback focus. A close-ended questionnaire was prepared based on the literature on a five-point Likert scale that ranged from never (1), rarely (2), sometimes (3), often (4), and to always (5). The questionnaire constituted of 15 items in three sections. The first section covered (n=6) items on linguistic accuracy and appropriateness (form), and the second and the third sections formed content knowledge (n=5) items and genre knowledge (n=4) items, respectively. The reliabilities of the sections were also checked. The Cronbach alpha value of 0.91 was obtained for linguistic accuracy and appropriateness while Cronbach alpha values of 0.88 and 0.89 were gained for content knowledge and genre knowledge, respectively. These indicated that the dimensions for feedback focus are highly reliable for data collection.
Interview with supervisors
Semi-structured interview questions were utilized to intensively probe supervisors’ view of feedback language functions in their feedback practices. Thus, the purpose of the interview was to generate elaborated in-depth information on the issue mentioned. To this effect, face-to-face individual interviews were held.
In-text feedback comments on thesis drafts
To generate valuable data about supervisors’ feedback focus and language functions, the in-text feedback comments on supervisees’ theses drafts were used. To this effect, supervisees’ in-text feedback comments on five theses were selected randomly from each university assuming adequate data would be obtained from them.
The in-text written feedback comprised of all comments offered by the supervisor provided in the margin of the text most of the time. The in-text feedback comments were transcribed word for word and are available in the appendix section. Each comment, phrase, or word that communicated a single message to the supervisee was taken as one piece of communication.
Methods of data analysis
The objective of this study was to examine EFL supervisors’ written feedback focus, and language functions at some selected public universities in Ethiopia. The data from the written text was arranged and coded into categories. To this effect, the focus of written feedback comments were grouped as content knowledge (if comments are on the accuracy, completeness, and relevance of thoughts and clarity of ideas), linguistic accuracy, and appropriateness (if comments highlighting grammar, punctuation, tenses, surface structure errors, and word choices), and writing genre (if comments are on the functions of different parts of a thesis which includes literature synthesis, argument building, reflective writing, and referencing style, etc) according to Bitchener and Basturkmen (2010) categorization of feedback focus.
The categorization and organization of feedback language functions into the directive and expressive functions and the subcategories were adapted from earlier studies (Kumar & Stracke, 2007). As to Kumar and Stracke (2007), directive feedback is ordering the supervisees to do something and is sub-categorized into clarifications and instructions. On the other hand, the expressive function of feedback is conveying feelings and is comprised of praise, criticism, and opinion /suggestion. It is comprised of praise/approval, criticism/disapproval, and opinion/suggestion (Hyland & Hyland, 2001; Kumar & Stracke, 2007). Consequently, feedback comments which appreciate or credit student work was coded as praise, while comments that show of dissatisfaction or negative were coded as criticism, and comments that give ways to do the work were coded as a suggestion (Hyland & Hyland, 2001).
To develop an appropriate categorization and coding, the researchers tried the categorization numerous rounds individually and then in pairs. The comments were, therefore, double-coded to ensure the credibility of the coding. The organized data obtained through supervisees’ questionnaire and in-text written feedback comments on the thesis were quantified and organized into frequency counts and percentages.
The data obtained through interviews from the supervisors were also coded and organized into the directive and expressive functions of language and their sub-categories. The organized data were then analyzed qualitatively as per their themes to address the language functions of written feedback.
Data analysis and discussion
This section introduced data analysis and discussion on thesis supervisors’ feedback language functions and focus on their feedback comments to the supervisees
Supervisors’ feedback functions
This subsection examines thesis supervisors’ feedback language functions in their feedback comments to the supervisees. To this effect, on-script feedback comments, and interviews with the supervisors were employed. The data have been analyzed and presented below.
On-scripts feedback functions
Table 1: Feedback distribution based on speech act functions
Function
|
Number
|
Percent
|
Directive
|
397
|
74.6
|
Expressive
|
135
|
25.4
|
Total
|
532
|
100
|
As shown in table1, the written feedback students received on their theses are directive and expressive. Sums of 532 feedback comments were found in the written drafts of the students. Among which, 397 (74.6) written feedback is directive which urges the receiver for action while the expressive feedback category account 135 (25.4). Thus, it seems that the supervisors use the directive function most dominantly in their feedback provision.
Table 2: Supervisors’ feedback language functions on students’ theses
Function
|
Number
|
Percent
|
Directive clarification
|
221
|
41.5
|
Direct instruction
|
135
|
25.4
|
Expressive approval
|
3
|
0.6
|
Expressive disapproval
|
88
|
16.5
|
Expressive suggestion
|
85
|
16
|
Total
|
532
|
100
|
Table 2 shows the sub-categories of directive and expressive feedback language functions. Fix feedback sub-categories were obtained from feedback comments on theses of the students which included directive-instruction, directive-clarification, expressive-approval, expressive-disapproval, and expressive-suggestion.
As to feedback language functions sub-categories (table2), the results indicated that supervisors’ feedback on students’ theses was dominantly directive-clarification 221 (41.5 %) followed by directive-instruction 135 (25.4%). Directive-clarification feedback requests the students for the clear elucidation of ideas in the paper for direction and revision. Some of the directive-clarification common in the comments comprise: ‘how?’, ‘what do you mean?’, and ‘evidence?? How do you relate these (teachers’ traditional teaching and students’ reading) to VLS? ‘.
The second most commonly offered feedback was directive-instruction feedback. This feedback type gives direction to the students to make necessary amendments in the texts. Among the many feedback comments of directive-instruction provided in the theses, some of them which may exemplify the entire are: ‘please check spelling’, ‘follow SGS format for the cover page and title page’, ‘put this in its right place, not here’, and ‘consistently (in all) capitalize initial letters of content words in titles and sub-titles.’
Expressive-disapproval feedback was the third frequently provided feedback which accounts for 88 (16.5%) of the total comments. Some of the expressive-disapproval feedback comments the students received includes: ‘this can’t be a sound justification’, ‘faulty parallelism’, ‘this is inappropriate citation; no two names should be used like this’, ‘your background looks like literature review, not study background’, ‘it is also too long and lacks focus’, and ‘your questionnaire as an instrument lacks a description.’
The expressive-suggestion was the fourth that took 85 (16%) of the total comments. For instance, some of the expressive-suggestion comments provided in the theses of the students are: ‘you need to relate your research find with previous research findings. You should show the differences and the similarities between the finding of your study and the previous studies’, ‘try to shorten the background by focusing on only the pertinent issues which have a direct bearing on your title’, and ‘you need to specifically state what methods you employed to ensure the validity of each tool’s data; then state the same about reliability.’
On the contrary, expressive-approval feedback was insignificant 3 (0.6%) in the written feedback comments the students received. These include: ‘this is a good objective’; ‘you already started talking about similar local studies. Keep on doing this’, and ‘the background seems good.’ This implies that the provision of negative comments (criticisms) seems to be double and is overriding in the feedback comments as compared to opinions for improvement and acknowledgment of the strength of students’ attempts.
Supervisors’ interview data analysis on feedback functions
Teachers were interviewed on the focus and language functions of feedback comments. Concerning the focus of feedback comments, one of the interviewed supervisors underscored that he pays attention to content, genre, and linguistic accuracy and appropriateness though he gives more attention to content and noted the following:
I mainly focus on organization, the relevance of concepts included (content), and formatting issues. Sometimes, I also consider mechanics and diction as well. I often give due emphasis on content and organization as they are the most important pillars of the learners’ written research report.
Describing the focus of feedback comments he offers to his supervisees, the other supervisor noted:
I focus on all parts of the candidates' research work but I give special attention to the links among the research title, the stamen of the problem, the objectives, the methodology section, and the analysis section with the major findings. My reason for doing this is that focusing on these parts may give me the picture of the research; how things are woven together in a thread-the flow of the entire research work.
The third interviewed supervisor believes that he focuses only on essential aspects. The specific aspects he focuses on include adequacy and relevance of contents, word choices and organizational issues, writing styles, and conventions. This supervisor seems that he is eclectic in his focus on feedback comments.
The last interviewed supervisor emphasized that effective feedback should indicate where the problems lie and how these problems should be addressed. He further noted that supervisors’ written feedback comments are meant for improving supervisees’ work and hence they should use them as much as possible. Accordingly, he mentioned that he focuses on content, genre, and linguistic accuracy and appropriateness although he pays more attention to content in his feedback comments. The interviewee additionally noted the following:
I focus on all aspects of a thesis while giving feedback comments. Truly speaking most of the time I focus on content. But this does not mean I ignore other aspects. For instance, I suggest students adhere to the school of graduate studies (SGS) guidelines of my university. Besides, I give due attention to the language aspect as well though I give more attention to that of contents.
On language functions of feedback comments, the interviewed supervisors replied that they use directive language in their feedback comments. Specifically, they mentioned that they apply directive clarification most of the time. Besides, they claimed that they use directive instruction feedback comments sometimes as well. In the elaboration of these, one of the interviewees noted the reason behind his choice for directive clarification over directive instruction, and noted:
I use both [directive clarification and directive instruction] though I tend to use directive clarification in my written feedback most of the time. The reason for this is that this [directive clarification] kind of feedback comments is very much important to encourage students to search for knowledge or to create insight learning (which important to discover solutions to problems). Besides, it gives the candidates to see their works and to revisit them in light of the comments.
The other interviewed supervisor replied that his feedback language most of the time is that of directive clarification which requests the supervisees to elaborate and substantiate what they are writing. Also, he reported he uses expressive suggestions to give them room to revisiting their work as well rather than praise or criticisms. Moreover, he mentioned that he sometimes suggests the resources useful to address the comments given.
However, the interviewees mentioned that they do not often use expressive approval and suggestions though they abundantly apply criticisms in their feedback comments. As to his use of expressive suggestion feedback comments, one of the interviewees stated he uses suggestions rarely when he feels that the learner is somehow confused.
Supervisors’feedback focus
This subsection examines thesis supervisors’ feedback focus on supervisees’ theses. To this effect, on-script feedback comments, and interviews with the supervisors, and a questionnaire to the supervisees were employed. The data have been analyzed and presented below.
Supervisees’ views of their supervisors’ feedback focus
Table 3: Linguistic accuracy and appropriateness
|
|
Number (Percent)
|
|
Items
|
Always
|
Often
|
Sometimes
|
Rarely
|
Never
|
1
|
Appropriateness of word choice
|
18(32.7)
|
21(38.2)
|
11(20)
|
5(9.1)
|
--
|
2
|
Appropriateness of voice
|
16(29.1)
|
27(49.1)
|
7(12.7)
|
2(3.6)
|
3(5.5)
|
3
|
Grammatical accuracy
|
9(16.4)
|
19(34.5)
|
15(27.3)
|
9(16.4)
|
---
|
4
|
Coherence and cohesion
|
13(23.6)
|
23(41.8)
|
15(27.3)
|
4(7.3)
|
--
|
5
|
Development of ideas
|
17(30.9)
|
24(43.6)
|
10(18.2)
|
4(7.3)
|
--
|
6
|
Stance in the thesis work
|
17(30.9)
|
18(32.7)
|
10(18.2)
|
7(12.7)
|
3(5.5)
|
|
Total
|
90 (27.5)
|
132(40.4)
|
68 (20.8)
|
31(9.5)
|
6(1.8)
|
Table 3 depicts that six attributes were used to measure thesis supervisees’ view of their supervisors’ feedback focus on their theses. The majority of the supervisees (40.4% and 27.5%, on average) replied that their supervisors’ feedback focuses are on linguistic accuracy and appropriateness as frequently as ‘often’ and ‘always’, respectively. Specific to the attributes of linguistic accuracy and appropriateness, a significant number of the supervisees indicated that their supervisors were less concerned to focus on grammatical accuracy, and coherence, and cohesion as compared to the four attributes. Supervisors’ feedback focus on the contents of supervisees’ theses is presented below.
Table 4: Content knowledge
|
|
Number (Percent)
|
|
Items
|
Always
|
Often
|
Sometimes
|
Rarely
|
Never
|
7
|
Gaps in the content and coverage
|
10(18.2)
|
22(40)
|
14(25.5)
|
9(14.4)
|
---
|
8
|
irrelevance of issues in the research
|
6(10.9)
|
22(40.0)
|
15(27.3)
|
12(21.9)
|
-----
|
9
|
gaps in the justification of arguments
|
8(14.5)
|
19(34.5)
|
22(40.0)
|
6(10.9)
|
----
|
10
|
Theoretical frameworks of the thesis
|
12(21.8)
|
17(30.9)
|
14(25.5)
|
12(21.8)
|
----
|
11
|
Gaps in coverage of new literature
|
7(12.7)
|
20(36.4)
|
17(30.9)
|
11(20)
|
----
|
|
Total
|
43(15.6)
|
100(36.4)
|
82 (29.8)
|
50(18.2)
|
0
|
Table 4 shows five indicators that were used to measure supervisees’ view of their supervisors’ feedback focus on their thesis works. The majority of the supervisees (36.4% and 15.6%, on average) demonstrated that their supervisors’ feedback focuses are on the content of the thesis as recurrently as ‘often’ and ‘always’, respectively. However, the data indicated that still, large percentages of the respondents rated that their supervisors’ feedback focus on the content of their thesis works ranges between sometimes (29.8%) and rarely (18.2%). Concerning particular indicators of feedback focus on content, an undeniable percent of the respondents showed their supervisors rarely focus on relevance/irrelevance of issues, theoretical framework, and gaps in the coverage of the literature. Supervisors’ feedback focus on the genre of theses is presented below.
Table 5: Genre knowledge
|
|
Number (Percent)
|
|
Items
|
Always
|
Often
|
Sometimes
|
Rarely
|
Never
|
12
|
placement of units or topics of contents in the thesis
|
14(25.5)
|
20(36.4)
|
13(23.6)
|
8(14.5)
|
---
|
13
|
Rational on including particular content in a particular part-genre
|
6(10.9)
|
27(49.1)
|
16(29.1)
|
6(10.9)
|
---
|
14
|
Functions and contents of different parts of the thesis
|
10(18.2)
|
20(36.4)
|
19(34.5)
|
6(10.9)
|
----
|
15
|
the use of other samples for contents of different parts of the thesis
|
14(25.5)
|
21(38.2)
|
9(16.4)
|
7(12.7)
|
4(7.3)
|
|
Total
|
44 (20)
|
88 (40)
|
57 (26)
|
27(12.3)
|
4(1.8)
|
As shown in table5, four indicators were used to measure supervisees’ view of their supervisors’ feedback focus on their thesis works. The majority of the supervisees (40 % and 20%, on average) revealed that their supervisors’ feedback focuses are on the genre as regularly as ‘often’ and ‘always’, respectively. Nevertheless, there are still large percentages of supervisees who rated their supervisors’ feedback focus on genre knowledge as frequently as sometimes (26%) and rarely (12.3%). On indicators of feedback focus on genre, the supervisees have proven that their supervisors’ focus on the rationale to including particular content in a particular part-genre, and functions and contents of different parts of the thesis as often as sometimes and rarely.
In conclusion, the average percentage in the tables above showed that the supervisees perceived that these supervisors focused dominantly on linguistic accuracy and appropriateness, followed by genre, and content respectively as frequently as ‘ always’ and ‘often’. However, it seems essential to examine supervisors’ feedback focus on supervisees’ theses to make an accurate conclusion of the issue. Therefore, supervisors’ feedback focus on supervisees’ theses is presented in table 6 below.
On-scripts data analyses on feedback focus
Table 6: Supervisors’ feedback focus on students’ theses
Feedback type
|
Number
|
Percent
|
Content Knowledge
|
301
|
66.5
|
Genre Knowledge
|
93
|
20.5
|
Linguistic accuracy and appropriateness
|
59
|
13
|
Total
|
453
|
100
|
Supervisors’ written feedback focus on students’ theses was collected from in-text comments of the theses. The data were collected and organized into content knowledge (its accuracy, completeness, and relevance), genre knowledge (the functions of different parts of a thesis), and linguistic accuracy and appropriateness. Table 6 shows content knowledge (66.5%) was the most frequently observed written feedback which requires students’ conceptual understanding, accuracy, completeness, and relevance. A few of these comments, as illustration include: ‘indicate how your work is different from others?’, ‘not clear?’, and ‘can this be a reason for the sampling?’
The second most frequent feedback focus, though it is one-third of content knowledge, was genre knowledge (20.5%) which deals with referencing and citations, the functions of different parts of a thesis, and the relevance and appropriateness of the thesis for scientific research. Some of these include: ‘do you think this discussion is appropriate here?’, ‘bring the discussion here’, and ‘it must be the title of the journal which should be bold.’
The third frequent feedback focus was on linguistic accuracy and appropriateness (13%). A few exemplars of the written comments asked students to revise, edit, or use the correct and formal language which include: ‘this is not an appropriate term in research’, ‘check the completeness of your sentence’, ‘check language?’, and ‘not parallel???’ are some to mention.
In conclusion, it seems that the majority of the comments focus on content knowledge, followed by genre knowledge, while a limited percentage of feedback focuses on linguistic accuracy and appropriateness.