The study's results are now presented in relation to the research questions that guided the conduct of the systematic literature review.
RQ1. What are the most investigated aspects in the education domain with respect to sentiment analysis?
The opinions of students are a useful instrument for gaining important knowledge on different educational entities, such as lecturers, institutions, classes, etc. and teaching approaches involving these entities. It is crucial to recognize these aspects as they are expressed in students' textual remarks because it helps decision-makers take the necessary steps to address them specifically. In this context, we looked at and categorized the reviewed articles according to the issues that the authors wanted to look into and that were relevant to students. Specifically, we discovered three groups and associated teaching aspects that were the focus of these studies' research. The first group of research looked at how students responded to different qualities of their teachers, such as their knowledge, behavior, pedagogy, etc. The second group includes publications addressing other facets of the three distinct entities, such as courses, teachers, and institutions. Tuition costs, the campus, student life, and other characteristics connected to the institution entity were examples of course-related features. Course-related aspects comprised dimensions like course content, course structure, evaluation, etc. meanwhile the third group includes Papers examining the perspectives and attitudes of students toward institutional entities. From our findings as illustrated in Table 5, we found out that 76% of the papers that were reviewed were based on extracting students’ thoughts, opinion and attitudes toward teachers and 16% were based on extracting students’ opinion towards courses and institution, whereas the remaining 8% were based on extraction student opinion towards institution.
Table 5: Student Feedback Aspects Examined in the Reviewed Papers
Students’ opinion
|
Towards Teacher
|
Towards Institutions
|
Towards courses and institutions
|
Percentage
|
76%
|
8%
|
16%
|
RQ2. Which techniques and models are extensively researched for using sentiment analysis in the area of education?
A wide variety of techniques and models have been used to conduct sentiment analysis. These techniques are generally classified into three group which are supervised learning, unsupervised learning and lexicon-based techniques. While some researchers decide to use either supervised, unsupervised or the lexicon-based techniques, other researcher decide to use a hybrid of two of the three major techniques. Table 6 shows the learning techniques used for sentiment analysis in the area of education
Table 6: Learning techniques used for sentiment analysis in education domain
Learning Techniques
|
Papers
|
Supervised
|
[7], [8], [9], [10], [11], [12], [13], [2], [14], [15], [16], [17], [18], [19], [20], [21], [22], [23], [24], [25], [26], [27], [28], [29].
|
Unsupervised
|
[30], [31], [32], [33], [34].
|
Lexicon-based
|
[35], [36], [37], [38], [39], [40], [41], [42], [43], [44], [45], [46], [47].
|
Supervised and unsupervised
|
[48], [49], [50], [51].
|
Supervised and lexicon-based
|
[52], [53], [54], [55], [56], [57], [58], [59], [60], [61].
|
Unsupervised and lexicon based
|
[62], [63], [64].
|
Table 7 show emphasis on supervised learning models that are wildly studied for sentiment analysis in education domain. These models include the Decision Tree (DT), Support Vector Machine (SVM), K Nearest Neighbor (KNN), Naïve Bayes (NB) and Neural Network (NN).
Table 7 Supervised learning models that are wildly studied for sentiment analysis in education domain
Supervised learning models
|
Papers
|
DT
|
[7], [16], [18], [19], [24], [25], [41], [56], [59].
|
SVM
|
[2], [7], [8], [9], [12], [15], [16], [17], [19], [20], [21], [24], [25], [38], [41], [48], [51] [52], [56], [57], [59], [60], [64].
|
KNN
|
[9], [15], [18], [19], [23], [52], [64].
|
NB
|
[9], [12], [15], [16], [17], [20], [21], [22], [23], [24], [25], [26], [34], [38], [41], [51], [52], [56], [57], [58], [59], [64], [65].
|
NN
|
[9], [11], [13], [14], [17], [19], [23], [38], [41], [59], [61].
|
Additionally, as shown in Table 6, lexicon-based learning approaches, also referred to as rule-based sentiment analysis, were frequently used in a number of research and were frequently linked to either supervised learning techniques or unsupervised learning techniques. We observed that the Valence Aware Dictionary and Sentiment Reasoner (VADER) and Sentiwordnet were used far more frequently than TextBlob, MPQA, Sentistrength, and Semantria in Table 8's list of the most commonly used lexicons elaborated among the examined publications.
Table 8: Frequently used lexicons
Lexicon-Based
|
Papers
|
VADER
|
[36], [38], [42], [43], [48].
|
Sentiwordnet
|
[46], [56], [57], [65].
|
Semantria
|
[45], [58].
|
Sentistrength
|
[44].
|
TextBlob
|
[38], [49].
|
MPQA
|
[20].
|
RQ3. What are the most common metrics for measuring the effectiveness of sentiment analysis systems?
Systems designed for sentiment analysis were commonly evaluated using metrics based on information retrieval such as precision, F1-score, and recall. Additionally, other research used measures based on statistics to evaluate the precision of systems.
Comparing the number of articles that utilized a certain assessment measure to evaluate the performance of systems with the number of articles that either performed no evaluation or chose not to stress the employed metrics is highly intriguing. Table 9 shows the percentage of articles defined for each of the assessment metrics.
Table 9: Percentage of evaluation metrics applied in the reviewed papers
Evaluation metrics
|
Information retrieval-based metrics (accuracy, precision, f1-score, and recall)
|
Kappa
|
Pearson R-value
|
N/A
|
Papers (%)
|
67%
|
4%
|
3%
|
26%
|
Table 9 shows that 67% of the publications either simply featured the accuracy or other evaluation metrics such as the precision, recall, and F1-score. Kappa on the other hand was employed in just 4% of the research, while Pearson R-value in 3%, and no assessment metrics were specified in 26% of the research.
RQ4. What are the most popular methods for gathering student feedback?
During the process of reviewing the papers in this study, we found different data source and we divided them into three different categories based on their characteristics. These categories are:
- Questionnaires/Survey: This category of dataset was collected by providing questionnaires in order to gather feedbacks from students or conducting survey among teachers and students.
- Social media and blogs: This category of dataset comprises od data that are collected through social media platforms like Facebook, Twitter, and blogs
- Education/research platforms: In this dataset category, data are extracted through online educational and research platforms such as edX, Coursera, ResearchGate, Kaggle, LinkedIn, etc.
Based on the reviewed paper, just about a three-quarter of the of the papers disclosed the data source while about one-third of the papers did not disclosed information about the source of dataset collected. A tabular representation of these papers and dataset source is shown in Table 10.
Table 10: Dataset sources that have been used by the reviewed papers
S/N
|
Category of dataset
|
Papers
|
Description
|
1
|
Questionnaires/ Surveys
|
[7], [10], [19], [22], [23], [29], [30], [31], [35], [40], [42], [43], [51], [52], [57], [58], [59], [61].
|
This category of dataset was collected by providing questionnaires in order to gather feedbacks from students or conducting survey among teachers and students.
|
2
|
Social media and blogs
|
[12], [21], [32], [34], [39], [41], [46], [47], [48], [59], [60], [62], [64], [66].
|
This category of dataset comprises od data that are collected through social media platforms like Facebook, Twitter, and blogs
|
3
|
Research platforms/Education
|
[8], [9], [13], [24], [25], [26], [28], [50], [56],
|
In this dataset category, data are extracted through online educational and research platforms such as edX, Coursera, ResearchGate, Kaggle, LinkedIn, etc.
|