Robust Enhanced Collaborative Filtering Without Explicit Noise Filtering

.


Introduction
Learning vector representations of users and items is crucial in modern recommender systems [1,2] that enable mitigating the issue of information overload.Collaborative filtering (CF) is a traditional recommendation model that relies on user feedback; however, its co-occurrence matrix often suffers from sparsity due to limited user behavior data, resulting in subpar recommendations.Matrix factorization (MF) [3] addresses this problem by decomposing the co-occurrence matrix into low-rank matrices.While deep learning has revolutionized computer vision, its application to CF in the recommendation field has been limited.Neural collaborative filtering (NCF) [4] introduces neural networks to traditional CF, utilizing a multilayer perceptron to capture user-item interactions with promising results.However, existing approaches are still deficient in learning high-quality user representations, as they only map a single ID or attribute of the user or item.Graph structures naturally capture useritem interaction behavior, and graph neural networks [5,6] (GNNs) excel at learning representations from such structures.Consequently, GNNs have gained traction in recommendation systems.NGCF [7] proposed a CF framework based on neural graphs, leveraging a bipartite graph structure to enhance higher-order connectivity in the user-item graph and improve recommendation performance.HMLET [8] combined linear and nonlinear CF using graph convolutional networks (GCNs) to address training difficulties and excessive smoothing.ULtraGCN [9] simplified the GCN structure and introduced two auxiliary loss functions, achieving significant results.Although GCN have been successful in recommender systems, they also have noticeable limitations.Existing recommender systems typically employ implicit feedback data for training, which introduces false interactions [10][11][12] and a gap between implicit feedback and real user satisfaction.The limitations can be categorized into three main aspects: (1) vulnerability to adversarial noise attacks in implicit feedback data, such as simulated interactions that do not reflect genuine user preferences, (2) a significant portion of interactions driven by user mistakes and curiosity, and (3) unobserved interactions, which may not indicate user disinterest but rather lack of exposure to items.Current approaches that fit implicit feedback to recommender systems without considering inherent noise, particularly in the context of GCN, fail to effectively filter noise and inadvertently amplify its impact, leading to decreased recommendation performance and misinterpretation of user preferences [13].To address the issue of implicit feedback noise, recent studies have explored denoising methods, including resampling [14][15][16][17][18] and reweighting [19,20] While resampling methods focus on learning user preferences by designing a more efficient sampler to sample clean samples, they are heavily dependent on the sample distribution.Reweighting methods define clean data and noisy interaction data mainly based on the loss values during training, attributing lower weights to noisy data with high losses.For example, ADT [21] proposes an adaptive denoising training strategy that adaptively prunes the interactions with large losses.Furthermore, SGDL [22] based studies have proposed a novel denoising strategy, selfguided denoising learning, which uses two phases (memory phase and self-guided phase) to learn clean and rich information.However, the existing denoising approaches often rely on auxiliary information [23][24][25] or additional loss functions that are sensitive to hyperparameters and may result in the exclusion of difficult samples, thereby reducing model performance.
In this study, we present a research proposal for a robustness-enhanced CF GNN model to address the problem of adversarial interaction noise in implicit feedback data.Rather than explicitly filtering the noise, our approach involves randomly selecting nodes and replacing them with virtual users or items to counteract the noise and enhance the overall view.To achieve this, we simulate virtual users or items using random noise and leverage the principle of mutual information maximization.We utilize the InfoNce [26] loss optimization function to maximize the consistency between different views of the same node, thereby reducing the impact of adversarial interaction noise in hidden feedback.The key contributions of our research are as follows: • We propose a method to simulate user-item interactions that do not occur in practice by introducing random noise and replacing selected nodes with simulated user-item interactions.
• By maximizing the consistency between augmented graphs based on mutual information maximization, we mitigate the effects of adversarial noise in hidden feedback.
• Experimental studies are conducted in three publicly available datasets and the experimental results are analyzed to demonstrate the superiority of our model.

PRELIMINARIES
This section provides an introduction to traditional CF methods based on matrix decomposition in recommender systems, as well as the application of GCNs in CF.Additionally, we present a model of lightweight graph convolutional neural network (LightGCN).

Traditional CF Algorithm
CF is a widely-used technique in modern recommender systems.It operates on the assumption that there are similar users or items and leverages these similarities to make recommendations.MF is a traditional CF algorithm that addresses the challenges posed by sparse matrices and high dimensionality of similarity matrices.The MF algorithm decomposes the user-item interaction scoring matrix into two matrices: a matrix of user hidden vectors and a matrix of item hidden vectors.This decomposition maps the original matrix into a lower-dimensional hidden factor space of dimension f.The interaction between the user and item hidden vectors is modeled as an inner product of the respective vectors, as shown in the following equation: where p u is a vector representation of each user, and q i is the vector representation of each item, the user and item.Each dimension of the vector represents the strength of a specific hidden factor.For example, if the first dimension represents the action movie, then the value of the first dimension of the vector represents the user's preference for the action movie.The first dimension of the vector, item represents the number of action components.The better the fitting of vector rui for a user-item pair, the higher the rating.In other words, the better the vectors of the user and item match, the higher the predicted rating for that item by the user.
Common matrix decomposition methods, such as singular value decomposition (SVD), have been widely used.However, applying SVD to user-item matrices requires dense matrices, and since these matrices are typically sparse, early systems used estimation techniques to fill in missing values.However, these estimations are often not accurate.As a result, most recommender systems only consider observed interaction data for modeling and incorporate regularization techniques to prevent overfitting.The optimization problem for CF can be expressed as follows: To solve the optimization problem, stochastic gradient descent algorithms are commonly employed.In addition to traditional CF algorithms, there have been efforts to enhance their performance by incorporating additional information.One such example is SVD++ [27], which combines the strengths of MF with additional factors to improve recommendation accuracy.However, traditional CF algorithms still suffer from limitations due to their reliance on limited information.As a result, these algorithms may not effectively capture the unique characteristics of individual users.

GCN
With the advent of deep learning, there has been a surge in the application of deep learning-based methods in recommender systems.GCNs have gained significant attention due to their ability to extract meaningful representations from graph structures.Given that user-item interactions can be represented as graph structures, GCNs have been employed in CF to capture higher-order information.
In the graph convolution-based CF model, user-item interactions are represented by a bipartite graph.As shown in Figure 1,Each user and item is considered as a node in the bipartite graph, while the interactions between users and items are represented as edges connecting the corresponding nodes.This bipartite graph provides a comprehensive representation of the user-item relationships.Given a bipartite graph of user-item interactions, let the set of users be U and the set of items be I;the interaction between users and items is denoted as the set O + ,and O + = {yui | uϵU, iϵI}, where yui represents the user u, and project i is generated.Furthermore, where V = U ∪I represents the nodes of all users and items, and ε = O + represents the generated interaction.The basic idea of GCN is to aggregate the features of the neighboring nodes on the graph and represent them as new nodes, whose first-order aggregation operation is denoted as following: where H represents the neighbor aggregation function,Z 0 represents the initial embedding representation of users and items, and Z 1 is the representation of the nodes in layer 1 after aggregation; similarly, after several iterations of convolution, where l represents the node representation of convolutional layer l.

LightGCN
LightGCN is based on the basic concept of GCN and simplifies two common operations in graph convolution to capture the higher-order information of the graph and accelerate the training by aggregating higher-order neighbor information to update the representation of self-nodes and form the final embedding.In this study, we adopt the basic architecture of LightGCN.The two unnecessary redundant GCN operations of feature matrix transformation and nonlinear activation function are analyzed, proved, and further simplified into the final iterative formulation as follows: where the item embedding at layer k+1 is represented as the sum of the neighboring user embeddings at layer k.To prevent the increasing size of the embedding a normalization operation is performed on the nodes.After stacking the multilayer graph convolutional neural network, the aggregation is performed using a weighted summation based on the following weighting formula: Ultimately, the final codes of the user and item representations are multiplied to obtain the score as follows: ŷu,i = e T u e i (10)

METHODOLOGY
In this section, we introduce our proposed GNN model for robustness-enhanced CF without explicit noise filtering.Our aim is to mitigate the impact of adversarial interaction noise present in implicit feedback data.To achieve this, we adopt a strategy where we randomly select certain nodes and introduce virtual users or items with random noise to simulate interactions that do not exist in real situations.These simulated interactions are considered as adversarial interaction noise in the implicit feedback.We then replace the selected nodes with these simulated user or item nodes.To optimize the model, we utilize the principle of mutual information maximization.Specifically, we employ InfoNce as the loss optimization function for the joint training of our model.The overall framework of our proposed model is depicted in Figure 2, providing a visual representation of the model architecture.

Contrastive Interaction Noise
In current recommendation systems, implicit feedback is widely utilized for learning recommendation models.The abundance of implicit feedback helps overcome the data sparsity issue in recommendation systems.However, a common challenge arises in the form of adversarial interaction noise.This noise refers to the inclusion of illegitimate samples that are mistakenly treated as legitimate by the system.Examples of adversarial interaction noise include users' accidental clicks, simulated interactions performed by merchants, and interactions that users have not been exposed to.In practical scenarios, this noise is often considered as negative examples, but it is typically treated as positive examples in recommendation models.To address this issue, we propose a strategy to mitigate the impact of interaction noise in implicit feedback.
Initially, we simulate users or items by introducing random noise.This approach helps reduce the distortion of the original graph information.Rather than simply removing or adding interactions to simulate adversarial noise, which may compromise the modeling of users' true preferences, we simulate user or item nodes.Accordingly, we generate interactions that do not exist in real-world situations, serving as adversarial interaction noise.To implement this strategy, we randomly select a subset of nodes, as expressed by the following equation: where G randint is a function for random sampling of all nodes, and V represents all user and project nodes; furthermore, for the perturbation of the original graph minimization, we set a hyperparameter p that represents the proportion of random sampling, and the nodes extracted are denoted by V R .After obtaining the randomly extracted nodes, we proceed to simulate user-item interactions that do not exist in real-world scenarios using random noise.These simulated entities are subsequently replaced with the randomly extracted nodes, serving as adversarial interaction noise.Mathematically, this process can be expressed as follows: where △ ′ represents the random noise.Furthermore, we set a hyperparameter p noise for control in order to maintain a consistent order of magnitude between the extracted nodes and the interactions simulated by random noise.This hyperparameter allows us to control the scaling of the extracted nodes.G represents the substitution function.

Mutual Information Maximization
Next, we leverage the principle of mutual information maximization to mitigate the impact of adversarial interaction noise.This is achieved by maximizing the consistency between two augmented graphs.Mutual information quantifies the degree of dependence between two variables, specifically measuring how much information one variable provides about the other variable.Mathematically, mutual information can be expressed using the following formula: where p(x, y) denotes the variables x and y joint distribution, the p(x), and p(y) denote the respective marginal distributions, respectively.Unlike general similarity measures, mutual information can capture nonlinear statistical correlations between variables, making it a suitable measure for reflecting true dependence.However, directly optimizing mutual information is challenging; therefore, it is often approximated using lower bounds.Accordingly, we use the general optimization function InfoNce that is formulated as follows: where Z ′ i and Z ′′ i denote different augmented representations of the same node that we treat as positive samples.While the consistency of Z ′ i and Z ′′ i should be maximized, the consistency of Z ′ i and Z ′ j , which are negative samples, should be minimized.

Joint Training
In order to improve the training efficiency, we used a multi-task learning mechanism for the BPR loss task and the InfoNce loss task.The overall loss function is calculated as follows:

EXPERIMENT
This section presents the extensive experiments conducted to verify and analyze the performance of our model in order to establish the validity of our model.

Dataset
We perform experimental evaluation on three publicly available benchmark datasets (MovieLens-1M, Yelp, and Ta-Feng), and the number of interactions and sparsity of each dataset are shown in the following Table 1.

Hyper-parameter Settings
All the models in our study were developed using the Recbole library [28].We initialized the model parameters using the Xavier [29] method.The ADAM optimizer [30] was employed for model optimization.The embedding size for users and items was set uniformly to 64, and the batch size was set to 4096.Regarding the dataset scale, we followed the common practice of splitting the interactions into training, validation, and test sets in an 8:1:1 ratio.For evaluating the performance of our models, we used two widely-used metrics: Recall and normalized discounted cumulative gain (NDCG).The top-k recommendations were considered, with k set to both 10 and 20.

Contrast Model
We have compared the following models: BPR [31]:A matrix decomposition based BPR loss function is used.NGCF: It is a GNN-based collaborative recommendation framework that aims to capture higher-order connectivity in user?item graphs.It explicitly incorporates collaborative signals from user-item interactions into the embedding process, enabling more effective modeling of user preferences and item characteristics.
LightGCN [32]: It is a graph convolution framework that simplifies the design of GCNs by eliminating the need for a feature transformation matrix and nonlinear activation functions.This simplification reduces training complexity and facilitates efficient representation learning in CF tasks.
SGL [33]: A self-supervised auxiliary task is added to enhance the representation of learning of nodes by self-identification.
HMLET: This is a hybrid model of linear and nonlinear CF based on GCN, which analyzes the linear and nonlinear determinants of embedding propagation.
RGCF [34]: Based on CF of GNNs, the two main technical modules of graph denoising and diversity preservation are designed to achieve noise reduction for unreliable interactions.

Contrast Analysis
Table 2 shows the overall performance of all the compared models on three different datasets.Bold represents the best performance.Through the comparative analysis of the effects of different models, we are able to draw the following conclusions: • Compared with the performance of BPR, the performance of the recommendation system is substantially improved after the introduction of GNN, indicating that more higher-order information can be mined through graphs.
• Compared with NGCF, LightGCN improves the model significantly by removing two redundant operations?the nonlinear activation function and the feature transformation matrix.
• SGL exhibits superior performance, illustrating the effectiveness of introducing self-supervised auxiliary tasks and contrast learning, where contrast loss leads to improved uniformity of embedding distribution, resulting in improved performance.
• GCN-based HMLET combines linear and nonlinear activation for CF and exhibits better results than LightGCN, illustrating that using nonlinear activation in embedding propagation can lead to enhanced results.
• RGCF further improves the robustness and effectiveness of the model by reducing the effects of noise interactions using the graph denoising module and enriching the denoised graphs using the diversity maintenance module.
• Our models exhibit better results than all models, demonstrating that simulating users or items with noise as adversarial interaction noise in line with the principle of maximizing mutual information can effectively mitigate the effects of adversarial noise in implicit feedback and consequently improve the performance of the models.

Analysis of Key Parameters
In this section, we investigate the effects of three important hyperparameters in the model.Here we use the experimental setup used in Section 4.1.2.

Effect of the Replacement Ratio p
We use the replacement ratio control parameter p to control the number of replaced nodes.We set p as [0.00, 0.35].The impact on the model performance is shown in Figure 3, where it is observed that the performance improves as p increases to 0.02 in MovieLens-1M, 0.04 in Ta-Feng, and 0.04 in Yelp, and subsequently, the performance degrades.Therefore, it can be concluded that an excessively large p will increase the corruption of the original image information, thereby reducing the model effect, whereas an exceedingly small p will be ineffective for training.The temperature parameters play an important role in the differentiation of difficult and easy samples.Figure 4 shows the different τ parameters on the model performance curves; we set τ to [0.05, 0.40] and observe the change trend.It can be seen that the performance of the model deteriorates as τ increases, which is insufficient in distinguishing difficult samples, and similarly too small τ also makes the model's performance worse.

Robustness to Noise Interactions
In this section, the focus is on evaluating the robustness of the model to noisy interactions.To achieve this, a certain percentage (15%) of adversarial examples, which represent negative user-item interactions, are introduced into the training and validation sets of the three datasets.This contamination helps simulate the presence of noise in real-world scenarios.At the same time, the test set remains unaffected to ensure the reliability and accuracy of the evaluation results.The specific approach involves randomly discarding 15% of the existing observed user-item interaction records from the training and validation sets.Additionally, 15% of false interactions are artificially generated and injected as noise into the dataset.

Contrast Model
We compared the models listed in Section 4.1.3.Table 3 shows the performance of all the compared models on each of the three injected-noise datasets.From the analysis, we can confirm the following: • In the experiments where 15% noise was injected into the training data, our proposed models demonstrated a remarkable improvement of approximately 10% in performance compared to the best baseline model.This result indicates that our models are robust and effective in handling noisy interactions and can still provide high-quality recommendations even in the presence of such noise.
• However, in the models based on the GNN approach, the performance of all models experienced a significant degradation when evaluated on the contaminated dataset.This outcome can be attributed to the repeated execution of graph convolution operations in GNN, which amplifies the effect of the injected noise.Consequently, the models based on GNN were more susceptible to the adverse impact of noise compared with other models.
• When comparing our proposed model with existing approaches such as Light-GCN, SGL, and RGCF, the decline rate in performance for our model was significantly lower than that of LightGCN and SGL.This implies that our model exhibits higher resilience to noise and maintains more stable performance in the presence of noisy interactions.In comparison with RGCF, although our model achieved slightly better overall performance, its decline rate in the contaminated dataset was higher than that of RGCF.

RELATED WORK
In this section, we introduce two main tasks related to this study: graph CF and denoising recommendations.

Graph CF
A graph-based CF method is employed in this study that specifically focuses on modeling user-item interactions as a bipartite graph and utilizes the graph structure for recommendation purposes.GNNs have demonstrated exceptional capability in extracting information from graph-structured data, making them widely adopted in successful recommendation systems.In particular, the LightGCN model enhances the GCN architecture by removing two redundant operations.This simplification not only streamlines the model but also leads to notable performance improvements.Furthermore, self-supervised learning techniques, as seen in SGL, have been applied to graph CF to enhance model generalization.SGL achieves better recommendation performance by introducing auxiliary tasks that improve the learning of node representations.However, despite these advancements, the iterative nature of graph convolutional operations amplifies the impact of adversarial noise, limiting further performance improvement.To address this issue, our proposed model introduces virtual interactions and employs mutual information maximization to mitigate the adverse effects of noise arising from graph convolution operations.This approach allows for additional performance gains in the CF task.

Denoising Recommendations
Existing recommendation methods commonly utilize implicit feedback data for training, which inevitably contains noise.In recent research, efforts have been made to address the issue of noise in implicit feedback.Two strategies have emerged: resampling and reweighting.These methods, such as ADT dynamic loss and large loss interaction, often require auxiliary information or the introduction of additional functions.In this paper, we propose a denoising scheme that does not rely on explicit filtering techniques.Our approach effectively mitigates the impact of noise without the need for auxiliary information or additional functions.

CONCLUSION AND FUTUREWORK
In this study, we address the issue of adversarial interaction noise commonly found in implicit feedback.To tackle this problem, we propose a robustly enhanced CF GNN model that does not rely on explicit noise filtering techniques.Our approach involves simulating users or items that are not present in real situations by introducing random noise as adversarial interaction noise in the hidden feedback.Additionally, we replace these simulated entities with randomly selected nodes, allowing us to obtain different enhanced views.To ensure consistency between the original and enhanced views, we employ the principle of mutual information maximization and utilize InfoNCE as the optimized loss function.Furthermore, we enhance the model's effectiveness by incorporating a multi-task learning strategy.
In future work, we plan to further explore the challenges associated with noise in implicit feedback and investigate various approaches to improve the model's resilience to such disturbances.

Declarations
Ethical Approval This article does not contain any studies with human participants or animals performed by any of the authors.All texts, pictures and tables in the article belong to the original author and follow ethical guidelines, no academic misconduct.

Fig. 2
Fig. 2 Overall framework of the model

Fig. 3
Fig. 3 Substitution of scaling control parameters p effect on model performance

Fig. 4
Fig. 4 Temperature parameters τ effect on model performance

Table 1
Basic information of the data set

Table 2
Comparison of effects with different models

Table 3
Comparison of the effects of different models on the dataset injected with noise