Enhancing MOOCs Personalized Recommendation with Graph Neural Networks and Attention Mechanisms

doi:10.21203/rs.3.rs-3780549/v1

Download PDF

Research Article

Enhancing MOOCs Personalized Recommendation with Graph Neural Networks and Attention Mechanisms

https://doi.org/10.21203/rs.3.rs-3780549/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

Massive open online courses (MOOCs) have revolutionized education, providing unprecedented access to knowledge and skills to learners worldwide. While traditional methods have achieved comparable performance in personalized recommendations, they suffer from two key limitations. Firstly, they fail to capture the rich relationships between courses and users embedded within the MOOC graph structure. Secondly, they disregard the sequential nature of user learning, neglecting the evolving preferences and interests over time. These methods often overlook the recency of items, potentially neglecting relevant and trending courses. This paper presents a personalized recommendation approach for MOOCs that combines the effectiveness of an Attention mechanism with the capabilities of a Graph Neural Network, namely AGNN, to tackle this problem. This novel recommendation system in MOOCs leverages GNNs for rich learner-course relationships and LSTM for dynamic user preferences, culminating in personalized recommendations through MF-BPR learning. Real-world course data experiments demonstrate AGNN’s ability to significantly improve recommendation performance. An in-depth ablation study further underscores the critical influence of attention mechanisms, highlighting the model’s ability to dynamically adapt to evolving user preferences and prioritize recent, relevant items, ultimately leading to more personalized and effective recommendations.

graph neural network

attention mechanism

MOOCS recommendation system

Massive open online courses (MOOCs) have revolutionized education by democratizing access to knowledge and skills for learners worldwide, fostering an era of unprecedented global learning opportunities (Soraya et al., 2019). This unprecedented surge in online learning opportunities has presented a new challenge: helping learners navigate the vast landscape of courses and identify those that align with their unique interests and learning goals (Suresh & Srinivasan, 2020). Recommendation systems have emerged as a powerful solution to this challenge, employing sophisticated algorithms to offer students appropriate courses based on their past performance and preferences.

Traditional recommendation systems often rely on collaborative filtering (Li et al., 2020) or content-based filtering techniques (Choi et al., 2010). Collaborative filtering algorithms identify similar learners and recommend courses that those similar learners have enjoyed (Zheng et al., 2018). Content-based filtering algorithms recommend courses that align with a learner’s interests and past enrollment history, identifying courses similar to those they have previously engaged with (Ghauth & Abdullah, 2009). While these techniques have proven effective in various recommendation scenarios, they have limitations in capturing the complex relationships between learners and courses in the context of MOOCs.

MOOCs provide a rich tapestry of graph-structure data that can be leveraged to develop more effective recommendation systems (Jiang et al., 2023). In addition to explicit learner preferences and course descriptions, MOOCs generate a wealth of implicit data (Tian & Liu, 2021), such as course enrollments, video interactions, and forum discussions. This implicit data encodes valuable information about learners’ interests, learning styles, and engagement patterns. Unfortunately, this hidden information is not well used to improve recommendations.

Secondly, traditional recommendation systems for MOOCs often fail to capture the sequential nature of learners’ course enrollments, leading to suboptimal recommendations (Khalid et al., 2021). This is because these systems rely on content-based or collaborative filtering algorithms, which do not specifically take learners’ sequence of course enrollment into account (Wang et al., 2017). As a result, these systems may recommend courses that are not aligned with learners’ current interests or learning goals. For example, a learner who has enrolled in a series of introductory programming courses is more likely to be interested in intermediate programming courses than a learner who has enrolled in a random assortment of courses from different disciplines. The LSTM attention mechanism is able to capture these sequential patterns by learning long-range dependencies in learners’ course enrollment history (Hu & Rangwala, 2019).

Graph Neural Networks (GNNs) have become a robust tool for modeling intricate relationships within graph-structured data, providing an effective solution to challenges associated with leveraging the information embedded in such data structures. (Wang et al., 2019). GNNs are able to effectively capture the interactions between learners and courses, taking into account the rich network of relationships that exists among them (Chang et al., 2022). However, GNNs alone may not be sufficient to capture the sequential patterns of learners’ course enrollments. The long-short-term memory (LSTM) attention mechanism emerges as a key player in resolving the issue (Van Houdt et al., 2020). LSTMs, a variant of recurrent neural networks (RNNs), are particularly adept at capturing patterns and dependencies in sequential data (Moghar & Hamiche, 2020). They are able to learn long-range dependencies in time series data, making them ideal for capturing the temporal dynamics of learners’ course enrollments. The LSTM attention mechanism (Shewalkar et al., 2019) addresses this problem by learning long-range dependencies in learners’ course enrollment history, enabling the system to identify patterns in learners’ course preferences and provide more accurate and personalized recommendations.

In this article, the researchers propose an Attention Graph Neural Network (AGNN) with a long-short-term memory (LSTM) for recommendation system in MOOCs. The AGNN captures the interactions between learners and courses, while the LSTM attention mechanism focuses on the sequential patterns of learners’ course enrollments. Specifically, (1) to discover the complex connections found in MOOC graph-structured data between students and courses, we represent learners and courses as nodes, and learner learning behaviors and course associations are represented as edges so that we can use GNN to iteratively aggregate information from neighbor nodes to learn node representations. (2) to address both the evolving user preferences and the recency of items, first, we create a course sequence graph that includes all learners. We then learn course representations using a modified graph attention network with LSTM attention mechanism. Subsequently, we input these representations into a sequence encoder that is based on a GRU to deduce their short-term patterns, and we extract the final hidden state as the sequence-level learner embedding that has been learnt. Finally, we use BPR learning to recommend courses based on the representations of users and courses generated by the attention mechanism GNN, as well as the short-term course sequence representations. The proposed model outperforms traditional recommendation methods, demonstrating the effectiveness of incorporating graph-based and sequential information for course recommendation in MOOCs. Our contributions are as follows:

We propose a novel AGNN with an LSTM attention mechanism for a recommendation system in MOOCs.

We employed real-world MOOC datasets to benchmark the proposed model against traditional recommendation methods, showcasing its superior performance.

We conducted an in-depth analysis of the role of attention mechanisms in model recommendation.

The subsequent sections are organized as follows: Section 2 delves into relevant research on graph neural networks and long-short-term memory (LSTM) attention mechanisms for recommendation systems. Section 3 constitutes the core of our research methodology, meticulously detailing the proposed AGNN framework. Section 4 presents a comprehensive performance evaluation of AGNN, encompassing the experimental setup, data description, model comparisons, and insightful discussions. Section 5 concludes the study by summarizing the experimental findings and offering valuable suggestions for future research endeavors.

2.1 Attention mechanisms

Attention mechanisms in the context of machine learning and deep learning are computational mechanisms (Shi et al., 2023) that allow models to accomplish a task while selectively focusing on particular portions of the input data (Chaudhari et al., 2021). Inspired by human attention, these mechanisms enable the model to dynamically weigh and prioritize information by giving distinct items in the input sequence varied degrees of relevance (De Santana Correia & Colombini, 2022). Attention mechanisms are particularly useful in tasks like image recognition and natural language processing because they improve a model’s capacity to represent intricate linkages and dependencies in data. They play a crucial role in tasks like sequence-to-sequence processing, where the model needs to selectively attend to different parts of the input sequence when generating an output sequence (Hafiz et al., 2021). Overall, attention mechanisms contribute to the interpretability and performance of machine learning models by allowing them to focus on relevant information within the input data (Gao & Ruan, 2021).

The recurrent neural network (RNN) architecture known as long-short-term Memory (LSTM) is a popular attention mechanism that was created to solve the vanishing gradient problem and identify long-term dependencies in sequential data. (Wang et al., 2020). LSTM was first proposed by Hochreiter & Schmidhuber (1997). Introduced as an improvement over traditional RNNs, LSTMs have gated cells that enable them to selectively learn, forget, and update information over extended sequences (Sherstinsky, 2020). The key components of an LSTM include a memory cell and three gates: the input gate, forget gate, and output gate (Staudemeyer & Morris, 2019). These gates control the flow of information by regulating what information should be stored, discarded, and outputted at each time step. The architecture’s ability to maintain and update long-term memory makes LSTMs well-suited for tasks involving sequential data, such as natural language processing and time-series prediction, where capturing context and dependencies over extended sequences is crucial (Sherstinsky, 2020).

LSTM networks have significantly advanced the field of sequential modeling, particularly in the context of recommendation systems. Originally proposed by Hochreiter & Schmidhuber (1997), LSTMs address the challenges of learning long-term dependencies in sequential data, making them well-suited for tasks that involve modeling temporal dynamics and evolving user behaviors. Researchers, such as (Lee et al., 2017), have successfully applied LSTMs to session-based recommendation scenarios, where capturing the temporal context is essential for accurate predictions. The unique architecture of LSTMs (Van Houdt et al., 2020), featuring memory cells and gating mechanisms, enables the network to selectively remember and forget information, making them effective in capturing nuanced patterns in user-item interactions over time. Recent advancements, such as the work by (Wang et al., 2020), have explored attention-augmented LSTMs, emphasizing the importance of focusing on relevant information within sequential data for improved recommendation accuracy. Furthermore, the integration of LSTMs into hybrid architectures, as proposed by (Chen et al., 2022) the hybrid convolutional neural network with an LSTM attention mechanism for recommendation systems in e-learning. This hybrid approach combines the strengths of LSTMs in modeling sequential dependencies with the capabilities of CNN and attention mechanisms, showcasing the potential for enhancing the accuracy of recommendation in dynamic learning environments like e-learning. But this method cannot use the graph-structure datasets to improve the recommendation.

2.2 GNN-based Recommendation Systems

Graph neural networks (GNNs) have emerged as a powerful tool for recommendation systems, offering a promising approach for capturing complex relationships between graph structure data (Gao et al., 2023). Unlike traditional recommendation methods, which often rely on collaborative filtering or content-based filtering techniques, GNNs can effectively model the interactions between users and items, taking into account the rich network of relationships that exist among them (Wu et al., 2022). This ability to capture complex relationships has led to significant improvements in recommendation performance in various domains. Foundational works, such as Graph Convolutional Networks (GCNs), laid the groundwork by introducing the idea of aggregating information from neighboring nodes for improved node representations (Gong et al., 2020). Researchers have increasingly recognized the potential of GNNs in recommendation scenarios, where user-item interactions form a graph-like structure. Pioneering studies, including those by (Liang et al., 2023), demonstrated the efficacy of GNNs in collaborative filtering tasks, sparking a wave of exploration into their application in diverse recommendation domains.

Recent research has further explored the potential of GNNs in recommendation systems. Researchers have proposed novel GNN architectures that incorporate attention mechanisms, temporal dynamics, and knowledge graphs to enhance recommendation accuracy and personalization. For instance, a study by Romero (2018) introduced an attention-based GNN that focuses on the most relevant neighbors for each user, thereby enhancing recommendation performance under cold-start conditions. Furthermore, Luo et al. (2023) proposed a GNN-based model that incorporates temporal dynamics to capture user preferences over time. Attention mechanisms, as demonstrated by studies like (Knyazev et al., 2019), empower GNNs to prioritize relevant nodes during information propagation, leading to improved recommendation performance. Moreover, a hybrid GNNs with LSTM networks, as explored by researchers such as Li and Xu (2020), represents a cutting-edge direction. This hybrid approach combines the strengths of GNNs in capturing graph dependencies with the sequential modeling capabilities of LSTMs, presenting a comprehensive solution for recommendation systems in MOOCs. These advancements demonstrate the increasing sophistication of GNN-based recommendation systems and their potential to revolutionize the way recommendations are generated and delivered.

In this section, we introduce our proposed approach AGNN to recommendation in the MOOCs dataset. We utilize the graph neural networks framework based on the LSTM to generate the user and item’s embedding. The final prediction is ${\widehat{y}}_{u,i}$, which denotes the predicted top-N of item i which the user u may like. As shown in Fig. 1, our proposed framework contains three major components and we will discuss each component in the following paragraphs.

3.1 User Embedding

User embeddings are vector representations of users that capture their preferences and interests. These embeddings are used by the recommendation system to predict which courses c a user u is likely to be interested in. User embeddings are generated using the following process. Initially, the graph convolutional network (GCN) is utilized to extract pertinent information from the user-item interaction graph. The graph is composed of nodes representing users and items, linked by edges that signify interactions between them. The GCN aggregates information from a user’s neighbors to update its embedding. Secondly, The LSTM attention mechanism (Li & Xu, 2020) is used to capture the sequential patterns of a user’s course enrollments. Each context vector represents the importance of a particular course enrollment in the user’s overall enrollment history.

$${e}_{u} = {\sum }_{t}^{{a}_{t}}{f}_{LSTM}\left({x}_{t }{,ℎ}_{t-1}{,c}_{t-1}\right)$$

where ${a}_{t}$ is the attention weight at time t, ${x}_{t }$is the input at time t, ${ℎ}_{t-1}$is the hidden state of the LSTM cell at time t-1, ${c}_{t-1}$is the cell state of the LSTM cell at time t-1, ${f}_{LSTM}$is the long-short-term memory function.

3.2 Item Embedding

In the context of recommendation systems for MOOCs, the item embeddings are used to represent the characteristics of each course, allowing the model to determine which courses are similar to each other. This information is then used to make recommendations to users based on their past course enrollments and preferences.

Generating item embeddings involves extracting relevant information from item attributes and utilizing it to create vector representations of items. These item embeddings capture the characteristics of each course and enable the model to determine similarities between courses, which is crucial for making effective recommendations.

To extract meaningful features from sequences while mitigating the impact of noise, we choose n filters${F}^{k}\in {\mathbb{ℝ}}^{ℎ\ast d}$, $k\in (1,\cdot \cdot \cdot n),$ h represents the filter’s height, L represents the number of layers. The convolution layer’s ultimate output:

${c}^{k}$ = [${c}_{1}^{k}{c}_{2}^{k}\bullet \bullet \bullet {c}_{L-ℎ+1}^{k}$] (2)

To extract relevant features from the sequence while mitigating the influence of noise, each filter is followed by a max pooling operation. This process identifies and preserves the most salient features, enhancing the vector representation of the sequence.

${e}_{i}$ =${w}_{i}$[i ; max(${c}^{k}$)] (3)

This formula indicates that the item embedding ${e}_{i}$ is computed by applying a linear transformation (represented by the trainable weight matrix ${w}_{i}$) to the item’s attributes i, followed by applying the activation function ${c}^{k}$. The activation function introduces non-linearity into the model, allowing it to capture more complex relationships between the item’s attributes.

3.3 Model Prediction

Finally, the model adopts a short-term sequential attention mechanism fusion before BPR ranking to generate user and item embeddings. ${w}_{u,i}$represents the attention weight of user u on historical interaction i. The user’s potential interest in the target item is assessed as follows:

$${\widehat{r}}_{u,i} = {e}_{u}^{T}{{w}_{u,i}e}_{i}$$

Lastly, we employed the pairwise BPR (Bayesian Personalized Ranking) (Rendle et al., 2009)loss function, a widely used approach in nowadays. BPR operates under the assumption that observed interactions in the user-item interaction graph mirror user preferences. Specifically, it pairs each observed item with an unobserved item and aims to optimize the ranking of these pairs (J. Wang et al., 2021). The objective function for this optimization is as follows:

L = $\sum _{(u,i,j)\in o}-ln\sigma ({\widehat{r}}_{u,i}-{\widehat{r}}_{u,j})+\lambda {‖\varTheta ‖}^{2}$ (5)

The objective function, denoted as o, employs a pairwise learning approach during training. The positive items are represented by ${c}^{+}$ and opposite is represented by ${c}^{-}$. The σ is denoted as an activation function and $\varTheta$ is a set of parameters, the L2 regularization strength λ is employed to alleviate overfitting. In order to optimize the process, the Adam optimizer is used to train the model.

4.1 MOOCCube Datasets

MOOCCube dataset (Yu et al., 2020) is chosen by many researchers (Piao, 2021) for course recommendation algorithm research due to its advantages of large scale, rich data, high data quality and flexible data structure. It outshines existing databases for similar educational resources in terms of size, richness, and diversity. This comprehensive bedataset encapsulates meticulously detailed information about student behavior, including the amount of time spent learning, the quantity of learning attempts, and the duration between watching videos. The dataset encompasses around 200,000 students who actively used the site out of approximately 5 million students who recorded their video viewing for educational purposes. This rich data collection encompasses 2,005 users, 21,037 concepts, 600 courses, 22,403 videos, 137 schools, 138 teachers, and their intricate relationships. The dataset includes 930,553 total interactions between users and ideas, of which 858,072 are part of the training set and 72,481 are part of the test set. Table 1 provides an overview of the dataset’s statistics.

Table 1

Details of the MOOCCube dataset for our research.
Entities	Statistics	Entities	Statistics
users	2,005	videos	22,403
concepts	21,037	schools	137
courses	600	teachers	138
Training set	858,072	Testing set	72,481

4.2 Experimental Setting

The entirety of our experiments were conducted utilizing a single NVIDIA RTX2050 GPU, with Pyorch as the implementation framework. Network optimization was achieved through the employment of mini-batch Adam (Kingma & Ba, 2015) along with momentum. Adopting a similar data splitting approach to (Wang et al., 2021), we divided the dataset for training and testing. The learning rate is fixed at 0.0001, and the hidden size is set at 100.

4.3 Baseline

To demonstrate the effectiveness of AGNN, we compare it with several state-of-the-art methods, including:

POP (He et al., 2017): The POP method recommends the most popular items to all users, regardless of their online behavior, and serves as a simple yet effective baseline in recommender systems.
BPR: The algorithm, developed by (Rendle et al., 2009), is a widely used pairwise ranking approach for recommender systems. While it typically excels at recommending non-sequential items, its effectiveness can be limited when applied to sequential data like user interaction histories.
PinSage (Berg et al., 2017): This model effectively captures user and item characteristics and relationships to generate meaningful feature representations, leading to enhanced recommendation accuracy.
LightGCN (He et al., 2020): The LightGCN algorithm employs a simplified graph convolution network architecture that effectively captures user preferences and item similarities, resulting in enhanced recommendation performance.
TP-GNN(J. Wang et al., 2021): This is a Top-N personalized recommendation model by using GNN, addressing issues in existing methods by explicitly learning the structural relations of items and incorporating attention mechanisms for improved representation of user preferences.

4.4 Evaluation Metrics

The performance of top-k concept predictions for users is evaluated using the following widely used metrics: Recall (Rec@k) (Powers, 2020)and Normalized Discounted Cumulative Gain (NDCG@k). The value of k is set {5, 10, 15, 20} to assess performance at varying recommendation lengths.

4.5 Results

To assess the overall performance of AGNN, we compare it against several state-of-the-art baseline methods on the real-world MOOCCube dataset. Table 2 presents the top-k recommendation results, while Fig. 4 illustrates the complete performance curves of AGNN and the baselines. Based on these results, we can observe that:

While traditional algorithms showcase the superior performance of Bayesian Personalized Ranking (BPR), highlighting the significance of contextual information about items in sequences, simpler models such as POP, relying on item frequencies for recommendations, tend to display comparatively lower efficacy, particularly in session-based recommendation scenarios.

Outperforming PinSage, the LightGCN and TP-GNN baseline methods demonstrate the efficacy of GCNs in extracting implicit relationship information from user-item interactions. While PinSage is also a GCN-based method, its performance is hindered by the anonymity in sessions, which limits its ability to model users’ implicit relationship features.

AGNN, our proposed model, consistently demonstrates superior performance compared to all the baseline methods. For instance, it is noteworthy that AGNN outperforms all baseline models by a substantial margin of 1.73% (37.68→37.04), 18.06% (47.40→40.15), 15.04% (52.94→46.02) and 14.01% (57.23→50.20) of Recall metric for Top-k recommendation. As for the NDCG metric, AGNN achieves the obviously highest score when k = 5, 10, 15, 20 with the improvements of 6.04%, 5.04%, 7.86%, and 4.79%, respectively. These demonstrate the efficacy of our GNN-based approach and the advantages of the models that included the LSTM attention mechanism.

Table 2: The recommendation accuracies of method measured by Recall and NDCG. Bold denotes indicate the best model performance, and underline indicates the second-best model performance. Red font the empirical improvements against second-best (underline) models.

Methods	Rec@5	Rec@10	Rec@15	Rec@20	NDCG@5	NDCG @10	NDCG @15	NDCG @20
POP	10.21%	16.99%	22.45%	28.62%	6.77%	8.98%	10.42%	11.87%
BPR	31.34%	40.15%	46.02%	50.20%	25.16%	27.99%	29.56%	30.55%
PinSage	10.58%	19.66%	22.69%	28.31%	7.13%	9.99%	10.78%	12.13%
LightGCN	24.20%	35.69%	42.98%	47.69%	15.28%	18.97%	20.92%	22.03%
TP-GNN	37.04%	41.12%	45.50%	50.55%	28.50%	31.76%	32.30%	34.22%
AGNN (ours)	37.68%	47.40%	52.94%	57.23%	30.22%	33.36%	34.84%	35.86%
Improve	1.73%	18.06%	15.04%	14.01%	6.04%	5.04%	7.86%	4.79%

As the value of K increases, AGNN consistently maintains its superior performance across the benchmarks. The results presented in Fig. 4 demonstrate the effectiveness of AGNN in enhancing the performance of the ranking recommendation task, specifically Top-K recommendation. These findings indicate that GNN effectively captures the underlying information embedded within graphs, while LSTM effectively captures the sequential nature and short-term preferences of users towards items, both of which contribute to improved ranking recommendation performance.

In subsequent studies, we delve deeper into the impact of the hyperparameter sequence length L and perform ablation experiments to demonstrate the significance of LSTM within the model.

4.6 Experimental analysis

To further understand the effectiveness of multi-layer graph neural networks in learning hidden information, we changed the depth of the layers in {1,2,3}. AGNN1 represents the model with a single embedding layer, and other layers use similar symbols. Table 3 shows the performance affected by the layers. The specific findings are as follows:

Different propagation layers capture different levels of information. AGNN1 only considers direct connections between items, while AGNN2 and AGNN3 incorporate higher-order relationships, leading to improved performance.

Increasing the depth of propagation layers from two to three negatively impacts AGNN’s performance. This is attributed to the introduction of higher-order information carrying noise, masking reliable laten information.

Two layers of propagation provide the optimal balance for our dataset. This balance allows for capturing high-order latent information without over-smoothing the item representations.

Table 3

Effect of graph convolution layers.
Methods	Rec@5	Rec@10	Rec@15	Rec@20	NDCG@5	NDCG @10	NDCG @15	NDCG @20
AGNN-L1	36.14%	46.39%	52.15%	56.32%	29.87%	32.18%	33.49%	35.12%
AGNN-L2	37.68%	47.40%	52.94%	57.23%	30.22%	33.36%	34.84%	35.86%
AGNN-L3	37.58%	46.54%	52.39%	57.11%	30.33%	31.59%	33.48%	34.73%

To demonstrate the importance of the attention mechanism LSTM in the model, we also conducted ablation experiments. Table 4 shows the results of the ablation experiments.

The results show that LightGCN is a graph convolution network model that does not incorporate the attention mechanism, which results in poor performance in both REC@K and NDCG@K. TP-GNN adds the attention mechanism before user embedding, which results in better performance than LightGCN. Our model also incorporates the attention mechanism into user embedding, and at the same time, a short-term sequential attention mechanism is used before BPR ranking. The best results show that the method of combining attention mechanisms can effectively improve the recommendation performance.

Table 4

The results of the ablation study.
Methods	Rec@5	Rec@10	Rec@15	Rec@20	NDCG@5	NDCG @10	NDCG @15	NDCG @20
LightGCN	24.20%	35.69%	42.98%	47.69%	15.28%	18.97%	20.92%	22.03%
TP-GNN	37.04%	41.12%	45.50%	50.55%	28.50%	31.76%	32.30%	34.22%
AGNN	37.68%	47.40%	52.94%	57.23%	30.22%	33.36%	34.84%	35.86%

In conclusion, this study proposes an attention graph neural network (AGNN) framework with an LSTM attention mechanism for recommendation systems in MOOCs. The AGNN effectively captured user preferences and course similarities by incorporating both user-item interactions and course content information. The LSTM attention mechanism further enhanced the model’s ability to focus on relevant aspects of user interaction history. Experimental results on the MOOCCube dataset demonstrated that the AGNN outperformed baseline methods in terms of recommendation accuracy. This research contributes to the development of effective recommendation systems for MOOCs by leveraging the power of graph neural networks and attention mechanisms. Future work could explore different graph representations or incorporate additional context information to further improve recommendation performance.

Author Contribution

ZUO. Y and LUO.H. wrote the main manuscript text and XU.L. searched the relevant references. All authors reviewed the manuscript."

This work was supported by Yunnan Province Local Universities Joint Special Youth Project (202101BA070001 270) and Teaching Quality and Teaching Reform Project of Baoshan University in 2022-2023(ZHP202344)

Berg, R. van den, Kipf, T. N., & Welling, M. (2017). Graph Convolutional Matrix Completion. http://arxiv.org/abs/1706.02263
Chang, Y., Chen, C., Hu, W., Zheng, Z., Zhou, X., & Chen, S. (2022). MEGNN: Meta-path extracted graph neural network for heterogeneous graph representation learning. Knowledge-Based Systems, 235. https://doi.org/10.1016/j.knosys.2021.107611
Chaudhari, S., Mithal, V., Polatkan, G., & Ramanath, R. (2021). An Attentive Survey of Attention Models. ACM Transactions on Intelligent Systems and Technology, 12(5), 1–33. https://doi.org/10.1145/3465055
Chen, Y. H., Huang, N. F., Tzeng, J. W., Lee, C. A., Huang, Y. X., & Huang, H. H. (2022). A Personalized Learning Path Recommender System with LINE Bot in MOOCs Based on LSTM. 2022 11th International Conference on Educational and Information Technology, ICEIT 2022, 40–45. https://doi.org/10.1109/ICEIT54416.2022.9690754
Choi, S. H., Jeong, Y. S., & Jeong, M. K. (2010). A hybrid recommendation method with reduced data for large-scale application. IEEE Transactions on Systems, Man and Cybernetics Part C: Applications and Reviews, 40(5), 557–566. https://doi.org/10.1109/TSMCC.2010.2046036
de Santana Correia, A., & Colombini, E. L. (2022). Attention, please! A survey of neural attention models in deep learning. Artificial Intelligence Review, 55(8), 6037–6124. https://doi.org/10.1007/s10462-022-10148-x
Gao, C., Zheng, Y., Li, N., Li, Y., Qin, Y., Piao, J., Quan, Y., Chang, J., Jin, D., He, X., & Li, Y. (2023). A Survey of Graph Neural Networks for Recommender Systems: Challenges, Methods, and Directions. ACM Transactions on Recommender Systems, 1(1), 1–51. https://doi.org/10.1145/3568022
Gao, Y., & Ruan, Y. (2021). Interpretable deep learning model for building energy consumption prediction based on attention mechanism. Energy and Buildings, 252, 111379. https://doi.org/10.1016/j.enbuild.2021.111379
Ghauth, K. I. Bin, & Abdullah, N. A. (2009). Building an e-learning recommender system using Vector Space Model and good learners average rating. Proceedings - 2009 9th IEEE International Conference on Advanced Learning Technologies, ICALT 2009, 194–196. https://doi.org/10.1109/ICALT.2009.161
Gong, J., Wang, S., Wang, J., Feng, W., Peng, H., Tang, J., & Yu, P. S. (2020). Attentional Graph Convolutional Networks for Knowledge Concept Recommendation in MOOCs in a Heterogeneous View. SIGIR 2020 - Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, 79–88. https://doi.org/10.1145/3397271.3401057
Hafiz, A. M., Parah, S. A., & Bhat, R. U. A. (2021). Attention mechanisms and deep learning for machine vision: A survey of the state of the art. ArXiv Preprint ArXiv:2106.07550. http://arxiv.org/abs/2106.07550
He, X., Deng, K., Wang, X., Li, Y., Zhang, Y. D., & Wang, M. (2020). LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation. SIGIR 2020 - Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, 639–648. https://doi.org/10.1145/3397271.3401063
He, X., Liao, L., Zhang, H., Nie, L., Hu, X., & Chua, T. S. (2017). Neural collaborative filtering. 26th International World Wide Web Conference, WWW 2017, 173–182. https://doi.org/10.1145/3038912.3052569
Hochreiter, S., & Schmidhuber, J. (1997). long-short-term Memory. Neural Computation, 9(8), 1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735
Hu, Q., & Rangwala, H. (2019). Academic performance estimation with attention-based graph convolutional networks. EDM 2019 - Proceedings of the 12th International Conference on Educational Data Mining, 69–78.
Jiang, L., Liu, K., Wang, Y., Wang, D., Wang, P., Fu, Y., & Yin, M. (2023). Reinforced Explainable Knowledge Concept Recommendation in MOOCs. ACM Transactions on Intelligent Systems and Technology, 14(3). https://doi.org/10.1145/3579991
Khalid, A., Lundqvist, K., Yates, A., & Ghzanfar, M. A. (2021). Novel online Recommendation algorithm for Massive Open Online Courses (NoR-MOOCs). Plos One, 16(1), e0245485.
Kingma, D. P., & Ba, J. L. (2015). Adam: A method for stochastic optimization. 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings, 1–13.
Knyazev, B., Taylor, G. W., & Amer, M. R. (2019). Understanding attention and generalization in graph neural networks. Advances in Neural Information Processing Systems, 32(NeurIPS).
Lee, D., Lim, M., Park, H., Kang, Y., Park, J. S., Jang, G. J., & Kim, J. H. (2017). long-short-term memory recurrent neural network-based acoustic model using connectionist temporal classification on a large-scale training corpus. China Communications, 14(9), 23–31. https://doi.org/10.1109/CC.2017.8068761
Li, W., & Xu, B. (2020). Aspect-Based Fashion Recommendation with Attention Mechanism. IEEE Access, 8, 141814–141823. https://doi.org/10.1109/ACCESS.2020.3013639
Li, X., Li, X., Tang, J., Wang, T., Zhang, Y., & Chen, H. (2020). Improving Deep Item-Based Collaborative Filtering with Bayesian Personalized Ranking for MOOC Course Recommendation. International Conference on Knowledge Science, Engineering and Management, 247–258.
Liang, F., Zhao, H., Wang, Z., Fang, W., & Shi, C. (2023). Retrieving GNN Architecture for Collaborative Filtering. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management (CIKM ’23), October 21â•ﬁ25, 2023, Birmingham, United Kingdom (Vol. 1, Issue 1). Association for Computing Machinery. https://doi.org/10.1145/3583780.3615035
Luo, H., Husin, N. A., & Aris, T. N. M. (2023). ROME: A Graph Contrastive Multi-View Framework From Hyperbolic Angular Space for MOOCs Recommendation. IEEE Access, 11(November 2022), 9691–9700. https://doi.org/10.1109/ACCESS.2022.3232552
Moghar, A., & Hamiche, M. (2020). Stock Market Prediction Using LSTM Recurrent Neural Network. Procedia Computer Science, 170, 1168–1173. https://doi.org/10.1016/j.procs.2020.03.049
Piao, G. (2021). Recommending Knowledge Concepts on MOOC Platforms with Meta-path-based Representation Learning. The 14th Educational Data Mining.
Powers, D. M. W. (2020). Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation. 37–63. http://arxiv.org/abs/2010.16061
Rendle, S., Freudenthaler, C., Gantner, Z., & Schmidt-Thieme, L. (2009). BPR: Bayesian personalized ranking from implicit feedback. Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence, UAI 2009, 452–461.
Romero, A. (2018). GRAPH ATTENTION NETWORKS. ICLR 2018, 1–12. https://dblp.org/rec/journals/corr/abs-1710-10903%0Apapers3://publication/uuid/BDBBF9BC-355C-4AC9-95AA-0D04929D5290
Sherstinsky, A. (2020). Fundamentals of Recurrent Neural Network (RNN) and long-short-term Memory (LSTM) network. Physica D: Nonlinear Phenomena, 404(March), 1–43. https://doi.org/10.1016/j.physd.2019.132306
Shewalkar, A., nyavanandi, D., & Ludwig, S. A. (2019). Performance Evaluation of Deep neural networks Applied to Speech Recognition: Rnn, LSTM and GRU. Journal of Artificial Intelligence and Soft Computing Research, 9(4), 235–245. https://doi.org/10.2478/jaiscr-2019-0006
Shi, Y., Lin, T., Chen, B. et al. (2023). Structured deep learning based object-specific distance estimation from a monocular image. International journal of machine learning and cybernetics 14, 4151-4161. https://doi.org/10.1007/s13042-023-01887-6
Soraya, K., Purnawarman, P., & Suherdi, D. (2019). Revisiting Massive Open Online Courses Concept in the 21st Century Era. Proceedings - 2019 2nd International Conference of Computer and Informatics Engineering: Artificial Intelligence Roles in Industrial Revolution 4.0, IC2IE 2019, 79–82. https://doi.org/10.1109/IC2IE47452.2019.8940849
Staudemeyer, R. C., & Morris, E. R. (2019). Understanding LSTM -- a tutorial into long-short-term Memory Recurrent Neural Networks. 1–42. http://arxiv.org/abs/1909.09586
Suresh, K., & Srinivasan, P. (2020). Massive Open Online Courses – Anyone Can Access Anywhere at Anytime. Shanlax International Journal of Education, 8(3), 96–101. https://doi.org/10.34293/education.v8i3.2458
Tian, X., & Liu, F. (2021). Capacity Tracing-Enhanced Course Recommendation in MOOCs. IEEE Transactions on Learning Technologies, 14(3), 313–321. https://doi.org/10.1109/TLT.2021.3083180
Van Houdt, G., Mosquera, C., & Nápoles, G. (2020). A review on the long-short-term memory model. Artificial Intelligence Review, 53(8), 5929–5955. https://doi.org/10.1007/s10462-020-09838-1
Wang, H., Leskovec, J., Zhang, F., Zhao, M., Li, W., Zhang, M., & Wang, Z. (2019). Knowledge-aware Graph Neural Networks with Label Smoothness Regularization for Recommender Systems. Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 968–977. https://doi.org/10.1145/3292500.3330836
Wang, J., Xie, H., Wang, F. L., Lee, L. K., & Au, O. T. S. (2021). Top-N personalized recommendation with graph neural networks in MOOCs. Computers and Education: Artificial Intelligence, 2(September 2020), 100010. https://doi.org/10.1016/j.caeai.2021.100010
Wang, Q., Peng, R. Q., Wang, J. Q., Li, Z., & Qu, H. B. (2020). NEWLSTM: An Optimized long-short-term Memory Language Model for Sequence Prediction. IEEE Access, 8, 65395–65401. https://doi.org/10.1109/ACCESS.2020.2985418
Wang, W., Ye, C., Yang, P., & Miao, Z. (2020). Research on Movie Recommendation Model Based on LSTM and CNN. Proceedings - 2020 5th International Conference on Computational Intelligence and Applications, ICCIA 2020, 28–32. https://doi.org/10.1109/ICCIA49625.2020.00013
Wang, X., Zhang, Y., Yu, S., Liu, X., Yuan, Y., & Wang, F. Y. (2017). E-learning recommendation framework based on deep learning. 2017 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2017, 2017-Janua, 455–460. https://doi.org/10.1109/SMC.2017.8122647
Wu, S., Sun, F., Zhang, W., Xie, X., & Cui, B. (2022). Graph Neural Networks in Recommender Systems: A Survey. ACM Computing Surveys, 55(5). https://doi.org/10.1145/3535101
Yu, J., Luo, G., Xiao, T., Zhong, Q., Wang, Y., Feng, W., Luo, J., Wang, C., Hou, L., Li, J., Liu, Z., & Tang, J. (2020). MOOCCube: A large-scale data repository for NLP applications in MOOCs. Proceedings of the Annual Meeting of the Association for Computational Linguistics, 3135–3142. https://doi.org/10.18653/v1/2020.acl-main.285
Zheng, L., Lu, C. T., Jiang, F., Zhang, J., & Yu, P. S. (2018). Spectral collaborative filtering. RecSys 2018 - 12th ACM Conference on Recommender Systems, 311–319. https://doi.org/10.1145/3240323.3240343

No competing interests reported.

Download PDF

Version 1

posted

You are reading this latest preprint version

Enhancing MOOCs Personalized Recommendation with Graph Neural Networks and Attention Mechanisms

Status:

Version 1

Abstract

Figures

1. Introduction

2. Related Work

2.1 Attention mechanisms

2.2 GNN-based Recommendation Systems

3. Method

3.1 User Embedding

3.2 Item Embedding

3.3 Model Prediction

4. Experimental

4.1 MOOCCube Datasets

4.2 Experimental Setting

4.3 Baseline

4.4 Evaluation Metrics

4.5 Results

4.6 Experimental analysis

5. Conclusion

Declarations

Author Contribution

References

Additional Declarations

Status:

Version 1