A Recommendation System Involving a Hybrid Approach of Student Review and Rating for an Educational Video

doi:10.21203/rs.3.rs-2375194/v1

Download PDF

Research Article

A Recommendation System Involving a Hybrid Approach of Student Review and Rating for an Educational Video

https://doi.org/10.21203/rs.3.rs-2375194/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

In face-to-face learning, lecturers directly manage their students and use all their emotional expressions to assess their understanding of courses. Contrastingly, in e-learning, online platforms suffer from the absence of emotional communication. Therefore, this paper addresses educational videos because they are the top closed type of delivered data that can express and clarify details about a subject's meaning. This paper's authors propose an educational video recommendation system based on the hybridization of student historical data. The system will recommend educational videos on learning platforms (the Coursera platform is a case study). We use latent Dirichlet allocation (LDA) for probabilistic topic models. LDA extracts topics from online video content. Simultaneously, we use sentiment analysis of learners' comments on educational videos to make new rating predictions and then hybrid the original user-rating matrix and predicted rating matrix to obtain a better recommendation.

Recommendation systems

sentiment analysis

text reviews

educational video

Online learning platforms are learning methods that use modern technologies like the Internet. This method of teaching in e-learning allows you to offer courses to students anytime from anywhere and interact with students efficiently and straightforwardly. To keep up with the rapid development of educational institutions worldwide, e-learning has joined the essential requirements of the educational process. This learning has also been important in the development of education [1]. Among the most often-used machine learning applications on modern websites and platforms is recommendation systems. Due to the rapid growth of e-commerce, personal recommendations to customers are necessary for an e-store to stand out. Recommendation systems have constituted the foundation of online services like Spotify, Netflix, YouTube, and Amazon.com [2]. The recommender system (RS) is a new big data technology. Recommendations in the form of user-friendly guidance help students succeed in an online learning environment. Many of the recommendation system's strategies use both collaborative and content-based filtering. Both of these methods may be used for online learning, and additional methods like hybrid, information-based, and so forth [3]. The findings of this study demonstrated that numerous research communities continue to be interested in studying RS. Nowadays, the e-learning recommendation system is frequently utilized to improve learning. With e-learning, we have the opportunity to learn nearly anywhere, at any time, via any device, such as a desktop computer, laptop, iPad, smartphone, and so forth (at home, colleges, schools, universities, while travelling, in the garden, etc.) [4].

The educational system is improving with the aid of the Internet and e-learning. The video material helps the user learn easily or to teach quickly. Therefore, a recommendation system with user reviews or ratings based on video material in the e-learning platform can accomplish this task effectively [5]. The rise in online video consumption has led to a new problem: information overload. This necessitated developing an automatic video recommendation system to provide learners with relevant videos that match their requirements [6]. As a result, many academic and commercial institutions have focused on developing video RSs. YouTube, Yahoo! Video, and Bing Videos are commercial sites that use deferent information formats to build their RSs, frequently relying on textual video data and registered user profiles. Generally, good video RSs should be built on analysing users' interests based on their usage history or profiles. [7]. Since 2020 (during the COVID-19 epidemic), more people have signed up for popular online learning platforms, including Coursera1, Khan Academy2, lecture video channels on YouTube3, and various university platforms like Stanford and MIT [8]. The number of online learner reviews on each academic site is enormous. These reviews describe how students feel about the teaching tools. Fffectively utilizing resources and user representation may make user-generated text reviews of learning resources worthy of addition to recommendation systems. Numerous studies demonstrate how the user reviews can improve rating prediction performance by minimizing sparsity difficulties since text reviews more clearly represent user interests and resource information. As was indicated, learners have been expressing their ideas about learning resources via free-text reviews on e-learning platforms. These student reviews are a useful source of data for an effective learning recommendation system since they provide more thorough, trustworthy, and nuanced information about student preferences. Most recommendation system approaches rely on an overall rating as the primary recommendation source [10]. In this research, we used information from user reviews and comments to address one of the problems RSs frequently experience. RSs use a single discrete scale to model customers' thoughts about an item, which is not entirely accurate for expressing the user's preferences.

This section will give a background on related educational video recommendation system work, particularly research focused on our research problem, which deals with recommendation systems based on student reviews and comments on educational videos.

1.1 Background

1.1.1 Video Recommendation Systems on an E-learning Platform

The user has recommended videos using a video recommendation engine that analyzes videos the user has seen or is currently watching. Besides saving customers from having to browse through many movies to find their favourites, this technique increases network traffic and user stickiness on video websites. The most crucial component of a video recommendation system is the video recommendation algorithm, which primarily comprises two filtering types: collaborative and content-based [9], [10]. In handling information overload, recommendation systems have proven successful. Rapid expansion of information on the Internet. Users were frequently presented with abundant products and e-learning resources. Therefore, personalization is a key tactic for enhancing the user experience. These systems have proven essential in different web domains, including e-learning websites [11], [12]. Aided by the Internet and e-learning, the educational system has improved. So the video material helps the user whether student to learn the subject easily or the teachers to teach the subject easily. Therefore, the recommendation system with user reviews or user ratings based on the video material on the e-learning platform can accomplish this task effectively [13], [14].

1.1.2 Recommendation System-based User Rating

El Mabrouk et al. [15] proposed a hybrid intelligent recommendation system for e-learning systems based on data mining. The recommendation system's objective is to direct users of e-learning platforms toward the most important content. Making material easier to obtain helps people develop their centre of interest. The first module is for implicit and explicit data collecting. The second module is used to process the data gathered and construct the learner profile, user categorization, and content classification. The fourth module creates log files and user history data for use in the forthcoming recommendation processes, while the third module generates recommendations based on the suggested model.

Geng [16] conducted a study of online education institutions that provided online teaching resources and offered free courses online. The online education sector is experiencing rapid expansion. Data about university resources for innovation and entrepreneurship education, as well as student information, were immediately obtained via a crawler tool. A total of 36,967 pieces of information on online learners were collected, including the learners' majors, educational history, locations, friend networks, learning styles, course names, and assessment topics. Following data screening, accounts that had been abandoned or with less than three monthly logins were eliminated. Finally, the useful data set produced had 2,400 students, and e-students were splited into two groups in an 8:2 ratio. The test data set included information on 480 students, whereas the training data set included information on the remaining 1,920. They showed that with more than 20 suggested students, the proposed technique surpasses the other three recommendation algorithms in accuracy and root mean square error. When there are more than 35 recommended students, the proposed technique and the knowledge-based recommendation algorithm have comparable recall rates, but the novel strategy outperforms the other two methods significantly.

1.1.3 Recommendation System–Based User Reviews

Islek and Oguducu [17] researched recommendation systems, which are regarded as essential elements of the e-commerce industry owing to their direct financial influence. Items considered relevant to the client discovered by utilizing their prior activities. These items were presented appealingly, thanks to recommendation algorithms. By considering consumers' most recent purchase history, Islek and Oguducu [17] described the architecture of the recommendation system. They also suggested a hierarchical item recommendation system for the e-commerce industry. Two layers comprise the suggested hierarchical structure. The recommendation level is the second level of the first level of the recommendation system, which has three phases. In this system, the input to the recommendation model was enhanced using cutting-edge embedded learning techniques. Creating a document for each item based on its text is the first level's first stage. Several text-preprocessing procedures were performed on these documents in this level's second phase. Each item document is transformed into a fixed length-embedding vector in the third stage, item embedding construction. In the second level of the suggested hierarchical system, our last component is a self-attentive sequential recommendation model. Instead of an item id, each item at this level of the self-attentive network was represented by a fixed-length embedding vector. We can use the textual data associated with each item to produce a document for each in the first phase of the hierarchical system's first level. Every e-commerce platform has an item detail page to provide comprehensive information. Typically, this page includes pictures of the item and some text, like the title, description, and reviews. A buyer views an item's detail page to decide whether to buy it.

Mohsin and Rashid [18] delivered suggestions to e-learners when they browse e-learning websites in the form of text, images, audio, or videos. They conduct a new idea of "Online based Multimedia Recommendation system for e-learning website" proposed in the field of web recommendations. Users of such websites require direction and suggestions to locate their preferred topics, comprehend them, and complete their coursework more effectively and efficiently without having to spend much time browsing the entire website or the entire web. To create suggestions, we've built a search strategy based on the topic's title or tags. Currently, this system only offers suggestions based on the information found on our e-learning website, but future work will be based on adopting the suggested method, which would gather recommendations from the entire web. The front view or index page of the "Web based Multimedia Recommendation System for e-Learning Website". The title is at the top of the page in header format, while the navigation and categories are on the left. When a user clicks on a category, related or requested content appears. The multimedia recommendations, which are this research's focus, are at the bottom of the page.

1.2 Related Work

Murad et al. [19] examined the RS papers produced by academics in the domains of computer science, information systems, and information technology. The data mining, sentiment analysis, learning analytics, and application RS on large data is the study that discovered problems in RS domain research. Herath and Jayaratne [20] suggested that we must motivate e-learners to actively engage in an e-learning environment to improve education. This system uses web mining techniques, particularly web usage mining, to identify the navigational preferences of e-learners and web content mining to find pertinent online materials. The subsequent parts of the web mining process include data cleaning, integration, transformation, and pattern recognition. The first step is data purification, a part of pre-processing. The server's access logs should be cleaned up in this phase to remove unnecessary and duplicated references. Data integration is the subsequent phase. This is because not all of the information from web mining can be found in the server's access logs. Different e-learners must be linked to their sessions or transactions, online content data, profiles, results data, assigned task data, and other relevant data stored in the database throughout this stage. Thakker et al. [21] presented in this paper the research movie recommender systems. Websites must provide individualized services to each user to increase customer happiness and the quality of the customer's time engagement when a user has a wide range of service options. Users are often known for being unable to choose; thus, when several options are available, they choose nothing or choose poorly. According to a poll, a typical Netflix or other movie-streaming customer loses interest after 60 to 90 seconds of perusing up to 20 films. If this occurs, it generates consumer unhappiness, ultimately causing the loss of the client. Therefore, it is crucial to create an effective recommender system that greatly aids users in making decisions. The evolution of recommendation systems has been seen. These systems have been created utilizing unique methods, and applying such approaches has generally helped e-portals grow their companies. Suggestions made by the site increase the likelihood that users will believe them. They showed the user interfaces of two widely used streaming services—Netflix and Amazon Prime Video—and how they present the outcomes of suggestions made specifically for the user.

Kim et al. [22] suggested a technique for producing a new suggestion candidate for users by using a learning model that anticipates ratings across several categories and total ratings from reviews. Through word embedding in the user review data, CNN-BiLSTM was used to predict the user's rating for each criterion, and a linear regression model was used to predict the user's total rating. We determined and applied the user and item priority of the criteria during this step. The weights taken from the linear regression model are used for the user's priority of criteria, while the average value is generated by averaging the ratings for each criterion based on the item used for the item's priority of criteria. The anticipated overall rating of the user's item was then synthesized to create a recommendation candidate. Using the Tripadvisor dataset, the suggested approach was applied to the user's hotel suggestions. The experiment verified the suggested model's excellent performance.

Shen et al. [23] briefly described the proposed recommendation algorithm. A new CNN was built as level two. It was necessary to first solve the CNN's input and output before training it. A language model was used for the input. We suggested the L1-norm-regularized latent component model for the output. Our recommendation algorithm's architecture. Our system for making recommendations has three layers. Level one describes how the CNN creates its input and output from historical data. Level two shows how the CNN's input and output are used to train its parameters. Levels one and two make up the training procedure. A new learning resource recommendation procedure is completed by level three. The past rating scores between the students and the learning resources serve as the training data for LFM. The rating scores may be either explicit—marked by the students—or implicit—inferred from their actions. Text information from available learning materials serves as the language model's input data.

Kandakatla and Bandi [24] developed a new RS based on the CBNR technique to accomplish learning personalization. Utilizing the suggested RS in an e-learning system completes the important stage. By ensuring the suggested items are in the present learning environment, the system's accuracy is boosted, and learner performance is enhanced. To compare the performance of the proposed CBNR approach to other comparable e-learning RSs, experiments are being done. Additionally, the data collected demonstrated that, compared to the methods currently in use, learning performance has improved in terms of accuracy and suggestion quality. Implementing negative learner ratings into the e-learning RS has a significant positive influence on the RS's accuracy and the learner performance. Future research will use cutting-edge technologies that primarily depend on RS hybridization with negative evaluations to optimize and improve suggestions' outcomes. The introduction of our e-learning RS in this part is divided into four steps. Getting the students' ratings from the database is the first stage in the procedure. In the pre-processing stage, the negative rating-based profiles removed from the collected data and stemmed. These two methods for getting the input data ready accelerate the subsequent processing stages. The material processed during the semantic indexing phase has been reduced in dimension by the major operations carried out. The system's learner profile was constructed in the next stage by modelling the learners' unfavourable ratings. The building process constructs the learner's profile by merging negative rating-based and semantic-based profiles. The suggestion aws computed during the prediction phase using these characteristics.

We proposed a recommender system based on the educational videos' content and learner reviews to satisfy what students actually need to learn. Our model builds on a combined content-based recommendation system and convolutional neural network (CNN) that runs on three levels, as illustrated in figure 1.

Level one in figure 1 on the left line is an input of a CNN training model. We used a language model of educational videos after proposing steps for source videos. Where the right line is an output of the CNN training model, we use the latent factor model (LFM) for learners' historical data. Learners' reviews of educational videos were used as training data for LFM. At this level, we made a preprocessing step to formulate the CNN model's input and output data. Level two is a training level at which all CNN parameters will be trained by the input and output data. Level 3 is the recommendation level. At this level, a new educational video will be recommended.

2.2 Educational Video Preprocessing

To build a language model (input of the CNN model), we must train it on textual data. Thus, we preprocessed source educational videos to convert them to text data. The first step was extracting audio from educational videos; second, we exploited Google's API to extract textual data from audio, keeping track of the original contents. This transformation done automatically by speech-to-text operation (see figure 2). API borrowed from Google had an audio length limitation, which was a challenge for our model, so we fixed it with our source code, breaking down long audio sequences into small junk. Then, we combined text from all junk belonging to the same source video. This process keeps the original content of the source video stable, as we proved in section 5.1.2.

2.3 CNN Model

The language model will provide the input training data for the CNN model. We used LDA to train a language model on textual data extracted from the source video and classify it according to topics based on video content (e.g., science and math) and subtopic (e.g., linear algebra, vectors and matrices, et cetera). LFM will be the output of CNN, trained from learners' histories represented by free text reviews on educational videos. The user can express their opinion and preference by writing data more than numerical data (rating data), which abbreviates what he thinks in one number. For instance, a user may rate a particular item highly but have a negative opinion on a small part of the video, like the teacher's teaching style. Those details would be invisible for numerical data but are very important, especially for the RS. The CNN model built on four layers: The first layer is a convolutional layer with multiple feature maps. The second layer is a pooling layer, and in this layer, we apply a max-pooling function to reduce diminution. The third layer is a convolutional layer, and the last layer is a fully connected layer. Figure 3 shows the CNN model.

2.3.1 Language Model

As we mentioned, the input of the CNN model is a language model. Here, we use topic modelling to justify the input of CNN. When recommending suitable online learning videos to students, each video has given content. LDA mostly used for topic modelling to extract the subject and semantic structure that the video covers. Because LDA enables us to acquire a topic from each educational video, we can use its output as a feature vector for educational videos to configure items according to a content-based recommendation system. After we convert the source video to its textual data, each video represented by a document. A document contains latent topics. Every single topic represented by a probabilistic distribution of words. LDA assists with extracting topics related to that document; it scans the document and discovers the statistics of the words, then represents those words as a probability of topics [25]. Figure 4 below shows the educational video recommendation model.

2.3.2 Latent factor model (LFM)

LFM use to justify the output of the CNN model. LFM extracts the educational videos' features and learner preference from historical data; these data represented in two matrices: U refers to user preference, and V refers to features of educational videos. With LFM, we can predict the user rating on any educational video based on some indirect factors and detect the learners' interests. Figure 5 shows LFM for educational videos and learners' reviews. Tradition LFM reduces solution space by L2-norm regularization. That cased in over-smoothing problem. Thus, in our propped model, we use LFM by L1-norm regularization to indicate educational resource features and learners’ preferences. Matrix factorization modified according to equation 1 below.

j(U,V) is the fidelity part, and the rest of equation 1 is the regularization part.

U: It is a matrix of relations between learners and latent factors.

V: It is a matrix of relations between educational videos and latent factors.

r_ij:It is a rating given by learner i on educational video j.

λ1, λ2: are the regularization parameters to balance the fidelity and regularization parts.

As we mentioned previously, we applied these computations on non-empty cells in the matrix

To minimize the non-convex function in the equation, cross-validation and a grid search were used to optimize parameters. A cross-validation procedure runs for the proposed recommendation algorithm, and then the LAD model captures the mathematical framework.

We run our proposed model on the configuration of Python 3.8.8 on the Anaconda environment and with a spider. A personal computer had the following description: HP Elite x2 1012 G1, Processor: Intel(R) Core(TM) m7-6Y75 CPU @ 1.20GHz 1.51 GHz, Installed RAM: 8.00 GB (7.88 GB usable). The system type is a 64-bit operating system with an x64-based processor. For further video processing and saving, we need to rent a server (GPU DEDICATED SERVER) from interserver.net[1]Company with the following description in figure 6 below:

[1] https://www.interserver.net/

We worked on the same dataset in the previous two versions of this article, [26], [27], which were from the Coursera platform; the data size was 1.45 million comments and reviews written by learners on educational videos^{^[2]}. The data are described in tables 1 and 2. Table 1 contains all data about 622 learning courses, while table 2 contains all data on 1.45 million learner scorings and comments.

Table 1: Dataset Course Disruption

Variable	Class	Description
Name	Character	Course Name
Institution	Character	Reviewer Name
course_url	Character	Course URL
course_id	Character	Course ID

Table 2: Dataset Student Review Description

variable	class	description
reviews	character	course review
reviewers	character	the name of reviewer who wrote the review
date_reviews	date	date the review was posted
rating	integer	the rating score given to the course by the reviewer
course_id	character	course ID

For testing purposes, we reduced the data size and selected three courses from the dataset. Those three courses contain 178 educational videos, 198856 learners' comments, and the same number of learner ratings from 1 to 5 stars on source videos. The three courses used can be accessed via the Coursera platform as follows:

machine-learning course from Stanford University URL: https://www.coursera.org/learn/machine-learning
python-data from University of Michigan URL: https://www.coursera.org/learn/python-data
e-learning from University of Illinois at Urbana-Champaign URL: https://www.coursera.org/learn/elearning

These courses have been chosen for many reasons, such as the courses' experts, authority and quality of spoken English. All data used to train and test our proposed model were saved in two CSV files. The first file contains learner reviews (198856 reviews) and their original ratings (from 1 to 5 stars) on the 178 educational videos in the three selected courses as shown in table 3. The second file contains details and information related to the educational course in table 4.

Table 3: CSV File of Learners' Reviews and Ratings

reviews	reviewers	date_reviews	rating	course_id
Dr. Chuck is the best mentor I've ever been taught by. This course helped me a lot and surely will help me afterwards also <3 thank you COURSERA	By Dhiraj J	28-May-20	5	python-data

Table 4

name	institution	course_url	course_id
Machine Learning	Stanford University	https://www.coursera.org/learn/machine-learning	machine-learning

^{^[1]} https://www.kaggle.com/imuhammad/course-reviews-on-coursera/metadata

5.1 Educational Video Prepressing

In this initial step, educational videos are converted to textual data, as mentioned in section 2.2, figure 2, by a sequence of converting operations: video to audio, then audio to text by speech-to-text Google API. To deal with API restrictions on the audio length, we divide long audio into small chunks based on the lengths of the teacher's silences in the video, creating split points when a speaking person stops for five seconds. After that, all split texts for the specific video collected into one document representing all textual data of that video without any information loss. That needs to be approved; hence, we compare the final collected text file resulting from this step with the transcript text file provided by the Coursera site on the web page below each video. As an example, figure 6 shows the converted text, and figure 7 shows the original text from the Coursera site for the educational video titled “Lists” from the “python-data” course accessed by the link “https://www.coursera.org/learn/python-data/lecture/ss3cl/8-1-lists” on 25 November 2022.

5.1.2 Originality Source Data Evaluation

In this experiment, we try to prove that our source input data preprocessing preserves information originality and accuracy. Therefore, we calculate the distance (similarity) between textual data after the conversion process and the original transcript text provided by Coursera below every educational video in the dataset. We apply two strategies of similarity. First, we use the Python and Cython-written natural language processing module SpaCy library. Researchers in [28] demonstrated that the SpaCy library is significantly faster than many other libraries. Then, we have included cosine similarity because it is a simple, reliable similarity metric used with the TF/IDF weighting scheme [29].

Since the videos in our dataset for the three courses differ in size, length, and type of material (science, math, etc.), we apply these measurements to different video samples from the source dataset. For more experiment interpretation, table 5 shows a small sample of videos. Since the data set is so large, only a very small sample can be displayed here for documentation purposes (see table 5), which is used for result display and discussion.

Table 5: Details about small sample of educational videos from dataset

Video ID	Title	Length (min.)	Size (MB)	Course ID	Video URL
Vd. 1	Manipulating Lists	9:13	12.6	Python-ata	https://www.coursera.org/learn/python-data/lecture/IUkAn/8-2-manipulating-lists
Vd. 2	Processing Files	11:42	40.0	Python-ata	https://www.coursera.org/learn/python-data/lecture/h0aMz/7-2-processing-files
Vd. 3	Counting with Dictionaries	11:52	14.7	Python-ata	https://www.coursera.org/learn/python-data/lecture/HLgKK/9-2-counting-with-dictionaries
Vd. 4	Tuples	17	20.0	Python-ata	https://www.coursera.org/learn/python-data/lecture/0ou0N/10-tuples
Vd. 5	Active Knowledge Making, Part 2C: Memory Work in Learning	5:13	8.67	e-learning	https://www.coursera.org/learn/elearning/lecture/HtmyN/active-knowledge-making-part-2c-memory-work-in-learning
Vd. 6	Recursive Feedback, Part 4B: Summative Assessment vs Formative Assessment	10:38	16.2	e-learning	https://www.coursera.org/learn/elearning/lecture/4wxub/recursive-feedback-part-4b-summative-assessment-vs-formative-assessment
Vd. 7	Metacognition, Part 6B: Metacognition in e-Learning Ecologies	11:40	17	e-learning	https://www.coursera.org/learn/elearning/lecture/OZevN/metacognition-part-6b-metacognition-in-e-learning-ecologies
Vd. 8	Applications of Machine Learning	4:28	9.51	Machine-learning	https://www.coursera.org/learn/machine-learning/lecture/IjrpM/applications-of-machine-learning
Vd. 9	Gradient Descent for Multiple Linear Regression	7:45	7.23	Machine-learning	https://www.coursera.org/learn/machine-learning/lecture/ltMMp/gradient-descent-for-multiple-linear-regression
Vd. 10	Logistic Regression	9:48	8.60	Machine-learning	https://www.coursera.org/learn/machine-learning/lecture/zNxaw/logistic-regression

In figure 8, we use the same small sample as table 5 to explain and present the results because the dataset is too large to exhibit here.

Figure 8 illustrates the great similarity between the two texts; some brief—especially non-mathematical videos—reach a similarity of 100%. Therefore, we can observe the following:

In all 10 shown educational videos, spacy is a built-in module in the version of Python that we used to create our model. There is no information loss throughout the computation process, and as a result, spacy provides higher similarity than cosine.
Some videos performed more similarly than others in both metrics, fully driven by the video's content. Where there is a lot of arithmetic in the videos, it cannot be convert to text 100% of the time since, for instance, symbols may be translated into words or formulas translated into sentences.

Since all evaluated videos had an index of similarity greater than 95%, the overall result appears satisfactory.

5. 2 Leaner historical data preparing

User historical ratings are a common basic input source for a recommendation system. These data are numerical, which seems restricted in expressing user opinions and interests. Therefore, free text written by the user will be good data to extract his/her preference to build an accurate RS. This is why we build our educational video recommendation system based on learners' reviews through sentiment analysis of their opinion to extract preferences from all information in written text comments on learning resources to match their learning needs. In this study, we develop our learner comment-based RS model for a variety of reasons [30] that are briefly discussed in the following items:

We can use learners' comments and reviews to deal with sparsity issues, where text comments offer helpful information in the absence of a rating that can be used to predict a rating.
Free review text is a good choice to fix cold-start problems by providing all necessary information about new items or new users.
Learners' reviews are the best choice for dense data problems, causing it to deliver more features to make an accurate recommendation system performance by guaranteeing high-quality ratings in case of conflict between ratings and comments of the same user on the same source; for example, user x rates item y 5 stars incorrectly, and the same user x on the same item y writes a very negative comment.
User reviews powered in fine-grained sentiment can help in extracting more features about a single item that will be very flexible to personalized use/item models.

We use sentiment analysis to classify user comments as positive, natural, and negative. For this purpose, we apply natural language processing tools. VADER (Valence Aware Dictionary for Sentiment Reasoning) is one of the most suitable sentiment analyzers [31]. VADER is a lexicon employed to make a sentiment analysis of textual data in different domains, such as social media, education, and news. It is a rule-based sentiment analysis tool.

5.2.1 Review Sentiment Analysis

We analyze a sample of 198,856 negative and positive learners’ reviews on 178 educational videos from three different courses. This sample is handpick from the bigger real-world dataset of 1.45 million reviews on 622 courses presented by the Coursera online platform. The NLP toolkit of sentiment analyzer VADER is use to classify and score the reviews based on English adjectives in learner comments. Additionally, VADER's lexicon allows qualitative analysis that can strongly detect all characteristics and proprieties of text review that influence how strongly the text's sentiment is perceive. Then we have five syntactical and grammatical heuristics that can modify and change the intensity of review sentiments as well, exemplified and highlighted in red in table 6. These heuristics take into account term relationships that are sensitive to word order. The five heuristics listed are as follows:

Exclamation point (!) punctuation, which keeps semantic orientation stable but increases the intensity score. For example, the learner comment “this course is good!!” will be a more intense comment than “this course is good”.
Full capitalization, or ALL CAPS, which also keeps semantic orientation stable and will highlight the sentiment of the relevant term. That increases the intensity score of the sentiment. For example, “this course is GOOD” will be more intense than the comment “this course is good”.
Adverb degree. Some words intensify the sentiment intensity or reduce intensity. For example, the comment "this course is extremely good" will be more intense than "this course is good."
"But" conjunction, which affects shifting sentiment scoring; the text that comes after "but" will be the main learner opinion. In this case, the comment contained a mixed sentiment, such as "lecture is great, but need more explanation," and the second part of the comment identified the sentiment polarity.
Most words can influence sentiment polarity, and flipped text scoring is a negation "not". For example, consider the comment "This course is not good."

Table 6: Syntactical and Grammatical Heuristics of Sentiment Analysis

Review	Reviewer Name	Date of Review	Rating	Course	URL of Review Access Webpage
Happy to learn and feeling motivated to explore more!!	By Charu A	11-Jul-2020	5	python-data	https://www.coursera.org/learn/python-data/reviews?star=5&page=93
GREAT COURSE, very detail and great discussion board	By Gary C	2-Oct-2018	5	python-data	https://www.coursera.org/learn/python-data/reviews?completers=true&sort=helpful&star=5&page=252
Though-provoking and extremely useful to anyone working in education!	By Athina	30-Dec-020	5	e-learning	https://www.coursera.org/learn/elearning/reviews?star=5&page=3
not useful	By Dr S	18-Apr-20	1	e-learning	https://www.coursera.org/learn/elearning/reviews?page=1&star=1
Lacks coding , but its very nice in terms of theory	By Francisco S	30-Sep-2022	3	machine-learning	https://www.coursera.org/learn/machine-learning/reviews?star=4&page=3

5.2.2 Rule-Based Sentiment Analysis

A rule-based sentiment analysis or sentiment lexicon consists of a set of rules called a lexicon which can classify words as negative, positive, and natural according to intensity [32]. To apply a rule-based sentiment analysis, we need to perform the following steps on learners' text reviews:

Data extraction.
Text tokenization, splitting text into separate words.
It removed stop words, such as "a", "an", "the", "they", etc., that were not used in the sentiment analysis.
Some of the punctuation not used in the sentiment analysis should be removed.
Text preprocessing based on sentiment lexicon to detect the polarity of emotion.

Manual Example:

For more explanation and clarification, let us take the learner's comment, "This course is good," to apply the previous rule-based sentiment analysis steps.

5.3 Rating Prediction

Sentiment polarity was employed to rate the learner's comment. The text review already had been classified by the VADER lexicon as positive, negative, or natural. To detect the polarity sentiment in an individual review, a compound value was used. We used (1) as a threshold of compound value to classify learners' comments where:

If the compound value > 0.001, then the sentiment was positive, and polarity = 1.
If the compound value > -0.001 and < 0.001, then the sentiment was natural, and polarity = 0.
If the compound value < -0.001, then the sentiment was negative, and polarity = -1.

Table 7 shows the CSV file's details resulting from text review classification and rating.

Table 7: Header information of CSV file from new rating prediction

Text review

Reviewer name

Review date

User rating

Course ID

Sentiment

type

Predicted rating

Learner text review

Name of learner who wrote the comment

Date of comment

Original rating

Title of the course

Positive, negative, or natural

New rating

VADER lexicon scores are between +4 and -4, where +4 is a very positive sentiment and -4 is a very negative sentiment. In our proposed framework, we put polarity between +1 and -1. For normalization and comparative purposes, we need to formulate scoring fraction values between +1 and -1 to integer numbers from 1 to 5, as shown in figure 10. This value will represent a new rating protected from text-review sentiment analysis.

Cold starts are among the most challenging aspects of any recommender system. Hence, in our proposed model, we try to solve it or at least reduce its effects on recommender system performance. We use a hybrid approach incorporating users' historical data: rating and text review data. From text review data, we predict a new matrix rating. As previously mentioned, we analyzed learners' reviews by sentiment lexicon to predict the user's new matrix rating. However, not all users will comment on educational videos, which means if a user rates an educational video, they are not necessarily giving feedback; they may have provided a blank or non-readable comment that yields nothing or a wrong rating prediction. Thus, we will have empty cells in the new predicted matrix. In the original matrix, we also have empty cells and maybe the wrong rating value. For this reason, we make a hybrid matrix factorization of the original and predicted ratings in case one of these reasons happen. In the new prediction rating, we will substitute it with the original rating in an equivalent cell in the hybrid matrix; otherwise, we will keep using the new rating.

In table 10, we present different samples of the source datasets pre-described in section 4. Samples were selected to clarify the deference and accuracy between the original and predicted ratings. We carefully chose rating pairs with a high deference rating in the resulting matrix. In examining table 8, we see four deferent cases of rating degradation between the user and new predicted ratings that are fixed in the new proposed model; all four cases will be discussed in section 6.1.

Table 8: Selective sample of rating prediction from users' reviews using sentiment analysis

Review-id	Text-reviews	Reviewer name	Review date	User rating	course_id	Predicted rating
1.	One of the best courses i studied in Coursera	By Baccouche M S	25-Feb-19	5	machine-learning	5
2.	good	By SRUTHI P N	4-Jun-20	1	machine-learning	4
3.	A bit boring!	By Abu M R	15-May-20	4	machine-learning	2
4.	great course	By onkar p	3-Apr-18	4	machine-learning	5
5.	Excellent...!	By RAHUL S	16-May-19	5	machine-learning	5
6.	Nice	By Rupak K p	10-Apr-20	1	python-data	4
7.	best course	By rajyagurusaisri k	3-May-20	2	python-data	5
8.	Good Lectures	By Himanshu V	12-Sep-20	3	python-data	4
9.	Great Course for starting!	By Mahmud M	23-Jul-18	4	python-data	5
10.	very good course actually!	By Dipu	26-Sep-16	4	python-data	5
11.	It's good learning course	By Yogesh u	2-Aug-20	4	python-data	4
12.	I really loved it.	By Szabolcs S	12-Mar-20	5	python-data	5
13.	Dr. Chuck is amazing. I'm too dumb to code.	By Puja G	1-May-20	2	python-data	4
14.	Interesting overview.	By Dorottya B	30-May-18	4	e-learning	4
15.	I'd love to learn such topics in future as well.	By Manisha o	29-Jul-20	5	e-learning	5
16.	Excellent start for passionate educators – those who want to make a difference in education systems.	By Nida A	1-Jun-20	4	e-learning	5

For more research reliability and transparency, figure 11 and figure 12 presented a real sample of learner reviews, and feedback in table 8 was captured directly from the Coursera site. In figure 11, we captured the user comment in row 14 of table 8, "Interesting overview." This was accessed through the link https://www.coursera.org/learn/elearning/reviews?page=1&star=4, which clearly shows that user ratings and our model's rating predictions match; both are equal to 4 stars.

In figure 12, we captured the user comment "great" in row 13 of table 8, which can be accessed through the link https://www.coursera.org/learn/python-data/reviews?completers=true&sort=helpful&star=2. This clearly shows the conflict between the user rating (2 stars) and our model's rating prediction (4 stars). The new predicted rating is more reasonable according to the sentiment analysis of the learner's comment.

6.1 Discussion

Table 8 displays a different sample of a new rating prediction made by a proposed model and shows the users' original ratings of the learning resource. Figures 11 and 12 are real-world samples from the Coursera site to support table 8. The results in table 8 and displayed user ratings and comments in figures 11 and 12 clearly explain that users' text reviews gave the recommender system greater efficiency and accuracy than rating. Additionally, we combined the users' ratings and reviews to get a filtered matrix factorization, directly influencng features extracting educational videos and learners' preferences, thus improving the recommender system. Table 8 contains four cases in which the proposed model presented a positive rating prediction over the original user rating:

Case 1: The user wrote a positive review but gave a low rating. A new predicted rating will have a high rate according to the learner's opinion in the text comment. For example, see rows 2, 6, 7 and 13 in table 8.
Case 2: The user wrote a negative review but gave a high rating. The new predicted rating will be low according to the learner's opinion in the text comment. For example, see row 3 in table 8.
Case 3: The user review and rating matched, regardless if positive or negative. A new predicted rating will be exactly equal to the original rating. For example, see rows 1, 5, 11, 12, 14 and 15 in table 8.
Case 4: The user wrote a positive or negative review, and the rating was almost equivalent to the given opinion. The new predicted rating will be slightly different from the original rating but more expressive about the user's opinion because the sentiment analysis of the review takes into account all the information in the text review. For example, see rows 4, 8, 9, 10 and 16 in table 8.

As we can notice in Cases 1 and 2, a big conflict exists between the original and predicted ratings possibly caused by different reasons like a wrong rating if the user clicked on the wrong number. On the other hand, it could happen if the user was in a hurry to click the rating stars.

From the presented results, we can show the degradation between the original users' ratings and the new predicted ratings, especially in cases 1, 2 and 4, so we measure the distance similarity between the user rating matrix and the predicted rating matrix. That explains why using user data reviews will help to build a good learning RS more than using numerical rating data. Cosine similarity and TF/DF are used for similarity calculations [29]. These calculations are made for all source datasets, which already contain different text reviews that vary in text length, size and type of content. In figure 13, we show a similar result for samples from each of the four cases.

In contrast, the similarity in the overall dataset was 98%, which means there is an incompatibility between learners' ratings and preferences reflected in the text reviews. This proves the difference can transform what is recommend or not recommend to the user. In other words, this changes the behavior of the recommender model.

We proposed a hybrid learning recommender system of rating and text-review data provided by learners in educational videos. The proposed content-based learning recommender suggests the most suitable learning videos to learners per their interests. Our model runs in two parallel lines (input and output data lines). Each line comprises multiple phases: The input-line phases are learning data preparation, textual data preprocessing, and language model building, while the output-line phases are learners' historical data preparation, text-review sentiment analysis and new rating prediction, rating hybridization, and LFM. The last phase of the two lines will be used by the CNN model for the recommendation process. Using text reviews increased the accuracy of the recommendation system because this type of data provides more details about user interests, which can be employed to devise a good recommender system that meets user preferences. We used a hybrid of both types of historical user data (rating and comment) in the presented model to reduce cold-start problem effects. We used a real-world dataset to train and test the proposed model (70% training and 30% testing). This data came from the Coursera platform. Results showed that using online user reviews is more accurate than using numerical user ratings, which yields a high-performing recommendation system. In the short term, we will use a transformer instead of a CNN. We tried to use a transformer but faced challenges due to its limitations with text sequencing, so we will look into how to deal with this.

Authors’ contributions

Mounir zrigui carried out the literature review. Manar joundy developed the mathematical models. Mohsen maraoui and Manar joundy jointly conducted the simulations and drafted the manuscript. Moonir zigui provided useful remarks and critically reviewed the manuscript. All authors read and approved the final manuscript.

Competing interests

The authors declare that they have no competing interests.

Availability of data and materials

All data used in this paper can be request from corresponding author

M. J. Hazar, Z. H. Toman, and S. H. Toman, "Automated Scoring for Essay Questions in E-learning," in Journal of Physics: Conference Series, 2019, vol. 1294, no. 4, doi: 10.1088/1742-6596/1294/4/042014.
F. Ricci, L. Rokach, and B. Shapira, "Recommender Systems: Techniques, Applications, and Challenges," Recomm. Syst. Handb., pp. 1–35, 2022, doi: 10.1007/978-1-0716-2197-4_1.
Y. Afoudi, M. Lazaar, and M. Al Achhab, "Hybrid recommendation system combined content-based filtering and collaborative prediction using artificial neural network," Simul. Model. Pract. Theory, vol. 113, pp. 102–375, Dec. 2021, doi: 10.1016/J.SIMPAT.2021.102375.
P. Vaibhav Kulkarni DrVishwanath Karad MITWPU Pune, S. Rai, P. V. Kulkarni, and D. Kale, "Recommender system in elearning: a survey," Springer, pp. 119–126, 2020, doi: 10.1007/978-981-15-0790-8_13.
A. A. M. A. Ali, M. Mabrouk, and M. Zrigui, "A Review: Blockchain Technology Applications in the Field of Higher Education," J. Hunan Univ. Nat. Sci., vol. 49, no. 10, pp. 88–99, Oct. 2022, doi: 10.55463/ISSN.1674-2974.49.10.10.
L. L. Dias, E. Barrére, and J. F. de Souza, "The impact of semantic annotation techniques on content-based video lecture recommendation," J. Inf. Sci., vol. 47, no. 6, pp. 740–752, Dec. 2021, doi: 10.1177/0165551520931732.
A. Yassine, L. Mohamed, M. A. A. S. M. P., and undefined 2021, "Intelligent recommender system based on unsupervised machine learning and demographic attributes," Elsevier, Accessed: Nov. 10, 2022. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S1569190X20301374.
L. Chen, L. Liu, P. T. C. in the Schools, and undefined 2022, "Using Multilayer Videos for Remote Learning: Videos of Session Guidance, Content Instruction, and Activity," Taylor Fr., Accessed: Nov. 10, 2022. [Online]. Available: https://www.tandfonline.com/doi/abs/10.1080/07380569.2021.1989220.
J. Xiao, M. Wang, B. Jiang, and J. Li, "A personalized recommendation system with combinational algorithm for online learning," J. Ambient Intell. Humaniz. Comput., vol. 9, no. 3, pp. 667–677, Jun. 2018, doi: 10.1007/S12652-017-0466-8.
S. S. Kundu, D. Sarkar, P. Jana, and D. K. Kole, "Personalization in Education Using Recommendation System: An Overview," Intell. Syst. Ref. Libr., vol. 197, pp. 85–111, 2021, doi: 10.1007/978-981-15-8744-3_5.
A. El Moustamid, E. En-Naimi, and J. El Bouhdidi, "Integration of data mining techniques in e-learning systems: Clustering profile of learners and recommender course system," ACM Int. Conf. Proceeding Ser., vol. Part F129474, Mar. 2017, doi: 10.1145/3090354.3090453.
V. Tolety, E.P.B. of E. E. and Informatics, and undefined 2022, "Hybrid content and collaborative filtering based recommendation system for e-learning Platforms," beei.org, Accessed: Nov. 11, 2022. [Online]. Available: https://www.beei.org/index.php/EEI/article/view/3861.
S. Phongsatha, A. C. A. eJournal of, and undefined 2017, "ELEARNING TEACHERS' AND STUDENTS' PERCEPTION TOWARD LIVE-VIDEO CONFERENCE IN AN ELEARNING ENVIRONMENT," assumptionjournal.au.edu, vol. 2, 2017, Accessed: Nov. 11, 2022. [Online]. Available: http://www.assumptionjournal.au.edu/index.php/eJIR/article/view/4237.
J. K. W. Wong, O. T. Oladinrin, C. M. F. Ho, E. Guilbert, and R. Kam, "Assessment of video-based e-learning in a construction measurement course," Int. J. Constr. Manag., vol. 22, no. 1, pp. 1–7, 2022, doi: 10.1080/15623599.2018.1435152.
M. El Mabrouk, S. Gaou, and M. K. Rtili, "Towards an intelligent hybrid recommendation system for E-learning platforms using data mining," Int. J. Emerg. Technol. Learn., vol. 12, no. 6, pp. 52–76, 2017, doi: 10.3991/ijet.v12i06.6610.
L. G.-C. I. and Neuroscience and undefined 2022, "The Recommendation System of Innovation and Entrepreneurship Education Resources in Universities Based on Improved Collaborative Filtering Model," hindawi.com, Accessed: Nov. 11, 2022. [Online]. Available: https://www.hindawi.com/journals/cin/2022/7228833/.
I. Islek, S. O. E. C. R. and Applications, and undefined 2022, "A hierarchical recommendation system for E-commerce using online user reviews," Elsevier, Accessed: Nov. 11, 2022. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S1567422322000151.
S. Mohsin, R. R. I. J. A. N. and, and undefined 2010, "Web-based multimedia recommendation system for e-learning website," academia.edu, 2009, Accessed: Nov. 11, 2022. [Online]. Available: https://www.academia.edu/download/87611986/4.2.pdf.
ieeexplore.ieee.org, Accessed: Nov. 11, 2022. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/8691088/.
D. Herath, L. J.-2017 N. I. Technology, and undefined 2017, "A personalized web content recommendation system for e-learners in e-learning environment," ieeexplore.ieee.org, Accessed: Nov. 11, 2022. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/8285650/.
U. Thakker, R. Patel, and M. Shah, "A comprehensive analysis on movie recommendation system employing collaborative filtering," Multimed. Tools Appl., vol. 80, no. 19, pp. 28647–28672, Aug. 2021, doi: 10.1007/S11042-021-10965-2.
koreascience.or.kr, Accessed: Nov. 11, 2022. [Online]. Available: https://www.koreascience.or.kr/article/JAKO202126651526257.page.
ieeexplore.ieee.org, Accessed: Nov. 11, 2022. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/7685589/.
D. Kandakatla, K. B.-P. of the International, and undefined 2018, "A content-based filtering and negative rating recommender system for e-learning management system," researchgate.net, 2018, Accessed: Nov. 11, 2022. [Online]. Available: https://www.researchgate.net/profile/Mahesh-Kandakatla/publication/331829560_A_Content_Based_Filtering_and_Negative_Rating_Recommender_System_for_E-learning_Management_System/links/5c8f2f8192851c1df948174f/A-Content-Based-Filtering-and-Negative-Rating-Recommender-System-for-E-learning-Management-System.pdf.
R. Ayadi, M. Maraoui, and M. Zrigui, "Latent Topic Model for Indexing Arabic Documents," https://services.igi-global.com/resolvedoi/resolve.aspx?doi=10.4018/ijirr.2014040104, vol. 4, no. 2, pp. 57–72, Jan. 1AD, doi: 10.4018/IJIRR.2014040104.
M. Hazar, M. Maraoui, M. Z.-J. of H. U. Natural, and undefined 2022, "Recommendation System Based on Video Processing in an E-Learning Platform," jonuns.com, Accessed: Nov. 10, 2022. [Online]. Available: http://jonuns.com/index.php/journal/article/view/1087.
M. J. Hazar, M. Zrigui, and M. Maraoui, "Learner comments-based Recommendation system," Procedia Comput. Sci., vol. 207, pp. 2000–2012, 2022, doi: 10.1016/J. PROCS.2022.09.259.
C. Treude, M. Sicard, M. Klocke, and M. Robillard, "TaskNav: Task-Based Navigation of Software Documentation," Proc. - Int. Conf. Softw. Eng., vol. 2, pp. 649–652, Aug. 2015, doi: 10.1109/ICSE.2015.214.
A. Mahmoud and M. Zrigui, "Semantic Similarity Analysis for Corpus Development and Paraphrase Detection in Arabic," Int. Arab J. Inf. Technol., vol. 18, no. 1, 2021, doi: 10.34028/iajit/18/1/1.
Y. Zhang, R. Liu, A. L.-2015 4th N. C. on, and undefined 2015, "A novel approach to recommender system based on aspect-level sentiment analysis," atlantis-press.com, 2016, Accessed: Jun. 21, 2022. [Online]. Available: https://www.atlantis-press.com/proceedings/nceece-15/25847127.
C. J. Hutto and E. Gilbert, "VADER: A Parsimonious Rule-based Model for Sentiment Analysis of Social Media Text," 2014, Accessed: Jun. 21, 2022. [Online]. Available: http://sentic.net/.
P. Pandey, "Simplifying sentiment analysis using VADER in Python (on social media text)," Retrieved from Anal. Vidhya website https//medium. com/analytics-vidhya/simplifying-socialmedia-sentiment-analysis-using-vader-in-python-f9e6ec6fc52f, 2018, [Online]. Available: https://medium.com/analytics-vidhya/simplifying-social-media-sentiment-analysis-using-vader-in-python-f9e6ec6fc52f.

No competing interests reported.

Download PDF

Version 1

posted

You are reading this latest preprint version

A Recommendation System Involving a Hybrid Approach of Student Review and Rating for an Educational Video

Status:

Version 1

Abstract

Figures

Introduction

1. Literature Review

1.1.2 Recommendation System-based User Rating

1.1.3 Recommendation System–Based User Reviews

2. Proposed Approach

3. Simulation Setting

4. Dataset

5. Experiments And Results

6. Rating Hybridization

6.1 Discussion

Conclusion

Declarations

References

Additional Declarations

Status:

Version 1