A hybrid approach to three-way conversational recommendation

Conversational recommendation is ubiquitous in e-commerce, while three-way recommendation provides friendly choices for service providers and users. However, their combination has not been studied yet. In this paper, we introduce the three-way conversational recommendation problem and design the hybrid conversational recommendation (HTCR) algorithm to address it. First, a new recommendation problem is defined by considering the man–machine interaction as well as the misclassification and promotion costs. The optimization objective of the problem is to minimize the total cost. Second, a popularity-based technique is designed for user cold-start recommendation, where the user maturity is responsible for deciding when HTCR turns to the second technique. Third, an incremental matrix factorization technique is designed for regular recommendation. It is efficient since only a few rounds of training are needed for newly acquired user feedback. Experiments were carried out on four well-known datasets, including Jester, MovieLens 100K, MovieLens 1M and Yelp. Results demonstrated that our algorithm outperformed state-of-the-art ones in terms of average cost.


Introduction
Conversational recommendation Smyth et al. 2004) refers to the scenario that items are recommended to a user based on the feedback. Compared to traditional rating prediction tasks (Sun and Zhang 2018), this scenario focuses on continuous negotiation between the recommender system (RS) and the user. Hence, it is a better model for real-world e-commerce as well as other online RSs. There are some research trends on this topic. To improve the usability, compound critiques Zhang and Pu 2006) help users to understand the suggestions from the RS. By incorporating richer information, sentiment analysisbased dialog (Li et al. 2018) helps in decreasing root mean squared error (RMSE), and chatbot and natural language processing-based approaches (Iovine et al. 2020) provide users with diverse product recommendations. By making use of human-machine interaction, interactive collaborative filtering (CF) with the probabilistic matrix factorization (Zhao et al. 2013) improves the prediction accuracy, and query-based recommendation (Sun and Zhang 2018) helps in decreasing the normalized discounted cumulative gain (NDCG) (Manning et al. 2008) and increasing success rate. However, these work only considered only two actions, i.e., "recommend" or "not recommend" items to users.
Three-way recommendation (Zhang and Min 2016) introduces the third action "promote" to the system. This action indicates that RS can consult a user's purchase interest by coupons or discounts. In fact, this action is widely used by most sellers and service providers to increase profit or equivalently, to decrease cost. Naturally, three-way recommendation achieves lower total cost than two-way ones. This problem has also attracted some research interests. The regression-based three-way recommendation ) handles numeric rating data through determining two rating prediction thresholds. Matrix factorization-based approaches (Liu and Ye 2020) obtain better prediction as well as lower total cost. Granular computing-based approaches (Ye and Liu 2021) obtain average cost lower than traditional CF algorithms. Pair-wise learning-based approaches  improve the performance of RS in terms of weighted accuracy, weighted error and total cost ratio. User reputation-based approaches (Qian et al. 2019) decrease RMSE and mean absolute error (MAE) compared with other popular algorithms. Yet they did not consider the interaction between users and RSs.
In this paper, we introduce the three-way conversational recommendation problem. There are four aspects of the new problem. First, we adopt the new user assumption, so that the whole process of conversation can be tracked. Second, there is a 3 × 2 cost matrix indicating promotion and misclassification costs. Third, conversation is implemented as follows: During each round, the RS provides a list of recommended and promoted items to the user, and the user provides ratings to some items. This process repeats until the user is not interested in any item in the current list. Fourth, the evaluation measure is the total cost. Then we design the hybrid conversational recommendation (HTCR) algorithm to address the new problem. It consists of two techniques. The popularity-based technique is designed for user cold-start recommendation, where popularity refers to the times that each item has been rated (Yin et al. 2012;Xu et al. 2017). This is because in the early stage, the user's personal information is so little that any distancebased approach would fail. In order to obtain the lowest cost, we actively learn a pair of popularity thresholds π α , π β , where π α > π β . According to the three-way decision (Yao 2010;Li et al. 2016), the thresholds partition the remaining items into three groups: The recommendation candidate pool contains items with popularity not less than π α , the promotion candidate pool contains items with popularity within (π β , π α ), and the non-recommendation candidate pool contains all other items. We randomly select L items from first two pools to form the recommendation list, where L is a predefined constant such as 10 or 20 depending on the system settings . Meanwhile, the RS gradually learns more about the user preferences and calculates the user maturity. When the user maturity is no less than a predefined threshold, the regular recommendation technique is triggered.
The incremental matrix factorization (IMF) technique is designed for regular recommendation. At this stage, the RS recommends items to the user according to historical data. Conventional nonnegative matrix factorization algorithms have a high complexity and cannot be incrementally updated when more ratings come (Bucak and Gunsel 2009;. We adopt IMF (Luo et al. 2012; due to its low time complexity and adaptability to the surge of the amount of new ratings. Some modifications are made to meet our requirements. We also learn a pair of prediction rating thresholds R α , R β in consideration of cost, where R α > R β . Similar to the popularity-based technique, the candidate item pool are also partitioned into three groups: The recommendation candidate pool contains items with the predicted rating not less than R α , the promotion candidate pool contains items with the predicted rating within (π β , π α ), and the non-recommendation candidate pool contains all other items. We also randomly select L items from the first two groups to form the recommendation list.
Experiments were carried out on four datasets including Jester, MovieLens 100K, MovieLens 1M and Yelp. The source code and data are available at https://github.com/ FanSmale/TCR. Results demonstrated that the new algorithm was better than other rough set model-based algorithms and popular non-hybrid algorithms in terms of average cost.
The rest of this paper is organized as follows. Section 2 reviews some related work, including recommender systems and the three-way recommendation. Section 3 defines a three-way conversational recommendation problem. Section 4 describes our new algorithm with a running example. Section 5 shows the experimental results. Section 6 concludes and points out some further works.

Related work
This section reviews some basic knowledge about recommender systems and three-way recommendation.

Recommender system
RSs are often used to mine user preferences in e-commerce. There are two main techniques of RSs: content-based and CF. Content-based recommendation adopts the contents of items and establishes the profiles of user preferences for recommendations (Yao et al. 2014). Thus, it is inappropriate for cold-start. CF is a technique to recommend items to customers who share the same interests or experiences. It has different types, including memory-based, model-based and hybrid (Breese et al. 1998;Chen et al. 2014) ones. Memorybased algorithms predict based on the similarity between users or items. In contrast, model-based algorithms use the user database to estimate or learn a model for prediction (Breese et al. 1998). Hybrid algorithms combine at least two approaches to improve the performance (Chen et al. 2014;Paradarami et al. 2017).
Conversational recommendation refers to the scenario that there are multiple rounds of recommendations between users and RSs. Dialog-based approaches (Sun and Zhang 2018;Iovine et al. 2020) formalize it as a dialog scenario and mine users' preferences based on the natural language. Com-pound critique-based approaches Zhang and Pu 2006) help users to understand the recommendations from RSs. Visualization-based approaches Devendorf et al. 2012) provide users with friendly interfaces. Unfortunately, there is still a lack of formal definition of the scenario.
Nonnegative matrix factorization (NMF) (Lee and Seung 2001) is a popular approach to build CF recommenders. It is also employed in other fields such as face recognition (Meng and Torre 2013) and image processing (Yang et al. 2020). NMF uses the product of two low-rank matrices to approximate the original rating matrix. Some techniques are incorporated into NMF. Sparseness constraint helps in finding part-based representations and converging quickly to the optimal solutions in a few iterations (Hoyer 2004). Bayesian probabilistic models help in avoiding overfitting (Hernando et al. 2016).
Incremental matrix factorization (IMF) (Bucak and Gunsel 2009) is designed to deal with the high complexity of NMF and the incremental feature of online data. Suppose that the rating matrix has a size of m × n, the rank of factorization is k, and the number of iterations for convergence is s. In case of new data arrival, the time complexity of NMF is O(mnks) for matrix updating. In contrast, when the new data of only one user arrive, the time complexity of IMF is O(nks) for matrix updating. Hence, IMF is more appropriate for the conversational scenario of our work.

Three-way recommendation
Three-way decision (Yao 2010;Liu et al. 2011) induces positive, negative and boundary rules for the decisions of acceptance, rejection and abstaining. It has been introduced in typical classification fields. Sequential three-way decisions for face recognition seek a low decision cost rather than low misclassification error (Li et al. 2016;Zhang et al. 2021). Three-way Email spam filtering provides a more sensible feedback to users for precautionary handling their incoming emails (Zhou et al. 2010). Moreover, three-way recommendations provide a third action "promote" to decrease the average cost (Zhang and Min 2016;Zhang et al. 2017).
Existing three-way recommendation systems (Zhang and Min 2016;Liu and Ye 2020;Xu et al. 2017) include the following four parts: (1) define the user's two preferences: X and X means "like" and "dislike," respectively; (2) define a set of actions A = {a P , a B , a N } for items, where a P , a B and a N represent "recommend," "promote" and "not recommend," respectively; (3) predict the probability that a user likes an item; and (4) after comparing the probability with the threshold pair (α, β), the action for the item is decided. The optimization objective is to minimize the average cost.
In cost-sensitive learning, cost is a more general evaluation index than accuracy. Existing research proves that incorporat-ing misclassification cost can help in improving the accuracy of testing (Tapkan et al. 2016), and minimizing test cost is considered as the goal of attribute reduction (Min et al. 2014). Existing three-way recommendations (Zhang and Min 2016;Zhang et al. 2017) consider the cost of misclassification and promotion: The former is the penalty for misrecommendation, while the latter is the cost of coupon distribution and discount.

Problem statement
In this section, we present the data model, illustrate recommendation scenario and define the problem. Table 1 lists notations used throughout the paper.

Data model
We consider a recommender system with m users and n items. Let U = {u 1 , u 2 , . . . , u m } be the set of users and T = {t 1 , t 2 , . . . , t n } be the set of items. The rating function is given by where Ω is the rating scale. For convenience, we represent the rating system as an m × n matrix R = (r i j ) m×n , where .m] and j ∈ [1..n] is the rating of u i to t j . In most applications, R is a sparse matrix with a few valid ratings. Table 2 lists a rating system where Ω = {1, 2, 3, 4, 5}, and 0 means that the user has not browsed or bought the item.
We set the threshold θ to determine whether the user likes the item or not. With this threshold, the user preference matrix is given by Δ = (δ i j ) m×n , where where 1, 0 and −1 indicate like, dislike and unknown, respectively.
We also define a list called gray list, denoted by G = {g 1 , g 2 , . . . , g n } for the current user. It records whether an item is still available for recommendation or promotion. Here g j = 0, if t j has been recommended or promoted; 1, otherwise.
(3) The threshold to determine whether a user likes an item or not Data model The matrix of user preference Data model

A
The set of actions Data model The normalized popularity of item t j Algorithm variable The maturity of user u i Algorithm variable q The number of recommended items Algorithm variable Γ The recommendation list in each round Algorithm output L The length of the recommendation list Algorithm setting The recommendation matrix Algorithm output The total cost Algorithm optimization objective Table 2 Rating matrix R U \T t 1 t 2 t 3 t 4 t 5 t 6 t 7 t 8 t 9 t 10 t 11 t 12 t 13 t 14 t 15 In the three-way recommendation scenario, the recommendation matrix is given by D = (d i j ) m×n , and where a P , a N and a B indicate "recommend," "not recommend" and "promote." R α and R β are ratings thresholds mapped from the probability threshold α and β. p i j is the predicted rating for r i j . Table 3 lists the cost matrix as the foundation of three-way decisions.
With the recommendation matrix D and the like threshold θ , the set of user item pairs with known ratings is partitioned into six subsets: Consequently, the total cost is given by where | · | denotes the cardinality of a set. Figure 1 illustrates the conversational recommendation scenario. The left side of the dotted line is the behavior of a user, and the right side is the behavior of the RS. These behaviors are explained in detail as follows.

The conversational recommendation scenario
2 The RS predicts the user's ratings to items. 3 The RS recommend a list of items to the user. 4 The user evaluates the recommended items. 5 If no recommended item favors the user, she will quit the system. 6 If the user finds some favorite items in the list, she buys them. 7 The user provides the feedback by numeric rating. 8 The recommender records the choices of the user and updates gray list (mentioned in Section 3.1). 9 The RS enters the next round of recommendation.

The problem
We also consider the leave-user-out training-testing scenario. That is, during the period that a new user enters and exists the system, the information of other users remains unchanged. Naturally, to obtain good recommendation results for all users (i.e., min η), we only need to consider the optimization for each user. For user u i , we have Consequently, the cost of u i is given by Now we can define the new problem as follows.
Problem 1 Three-way conversational recommendation Input: A rating system R, a cost matrix C, the current user u i . Output: The recommendation lists Γ 1 , …, Γ k where k is the round of interactions. Optimization objective: min η i .
Here we have a few lists Γ 1 , …, Γ k for different rounds. Suppose that Algorithm A produces a better Γ 1 than Algorithm B, while Algorithm B produces a better Γ 2 than Algorithm A, which algorithm is better? The answer is uncertain. In other words, local optima does not assure global optima for the user.

The algorithm
In this section, we first describe the algorithm and two recommendation techniques, then present a running example.
According to Problem 1, the number of rounds s indicates the adherence of the user to the system. With a larger s, more items have the opportunity of being recommended; hence, |N P| is smaller and |P P| is bigger, which decrease the total cost. At the same time, |P N | is bigger, which increases the total cost. Hence, we need to adjust the recommendation list to achieve a tradeoff among different costs.
Recommender systems usually have two tasks: exploiting and exploring. Exploiting refers to finding items that favor the user, while exploring refers to finding items that potentially favor the user. To balance them, we divide each recommendation list into two parts: one for high-confidence items for the exploiting task and the other for medium-confidence items for the exploring task.

Algorithm description
Algorithm 1 presents the one-round recommendation under the conversational scenario. Specifically, the input ξ is the maturity threshold deciding the recommendation technique to employ, and the output Γ is the recommendation list.
Line 1 initializes both the recommendation (RC) and promotion (PC) candidate sets to be empty.
Line 2 decides which technique to use. Lines 3-9 handle cold-start recommendation with the popularity-based approach. Popularity refers to the times that an item has been rated. A more popular item often favors more people. In the previous work (Xu et al. 2017), we have found that recommending some items with lower popularity can decrease costs. So we actively learn a pair of popularity thresholds (π α , π β ), where π α > π β , to divide candidate items into three groups: items with popularity no less than π α form the recommendation candidate set; items with popularity within the interval (π β , π α ) form the promotion candidate set; and other items form the remaining set.
Line 10 updates the user's maturity χ(i). We will discuss this issue in more detail in Sect. 4.1.1.
Lines 12-21 handle regular recommendation with an IMFbased approach. We also learn a threshold pair R α , R β to partition candidate items into three groups, while the semantics of the thresholds is changed to accommodate rating prediction.
Finally, Line 23 randomly selects q and (L −q) items from the recommendation and promotion candidate sets, respectively. Figure 2 describes the recommendation process from another perspective. Figure 2a shows that the maturity is responsible for switching the recommendation techniques. if (g j = 1 and ψ( j) ≥ π α ) then 5: RC = RC ∪ t j ; 6: else if (g j = 1 and ψ( j) ∈ (π β , π α )) then 7: PC = PC ∪ t j ; 8: end if 9: end for 10: update χ(i); 11: else 12: for (t j ∈ T ) do 13: if (g j = 1) then 14: Predict P i -based IMF; //P i is the predicted ratings for u i in Algorithm 2; 15: if (  The two threshold pairs (π α , π β ) or (R α , R β ) are considered as the input of each algorithm. Figure 2b-c show that the candidate items are divided into three parts: recommendation candidates, promotion candidates and non-recommendation candidates. Finally, Fig. 2d shows that q and (L − q) items are selected randomly from RC and PC to form Γ .

The popularity-based recommendation technique
In the first stage, the popularity-based recommendation is employed to deal with the cold-start problem. We use user maturity to switch between the two techniques. Equation 9 indicates the maturity matrix Z = z 00 z 01 z 10 z 11 .
According to , both existing and nonexisting ratings are beneficial for RS to understand users. In case that a user has not rated an item with high popularity, it is likely that the user knows the item but is unwilling to rate it. So it can be inferred that she does not like it. Therefore, items without ratings also have implicit feedback information, so it is considered as follows: (1) for each recommendation to existing rating, the maturity increases by z 00 ; (2) for each recommendation to nonexisting rating, the maturity increases by z 01 ; (3) for each promotion to existing rating, the maturity increases by z 10 ; and (4) for each promotion to nonexisting rating, the maturity increases by z 11 . So the maturity of u i is given by If χ(i) > ξ, we choose the IMF-based technique to make recommendations. The steps of popularity-based recommendations are as follows. First, we define the popularity as the number of times each item has been rated. Then, we use the maximum popularity to normalize the popularity of each item. The function ψ( j) represents the normalized popularity of item t j . Third, (a) (b) (c) (d) Fig. 2 Three-way hybrid recommendation we obtain a pair of popularity threshold (π α , π β ) by active learning, where π β < π α . If ψ( j) ≥ π α , t j enters the set of recommendation candidates. If ψ( j) ∈ (π β , π α ), t j enters the set of promotion candidates. Otherwise, t j can be ignored.

The IMF-based recommendation technique
In the second stage, IMF is employed for regular recommendations. In this stage, the user is not new and there are more new rating coming into our datasets. IMF is more efficient than conventional NMF for handling incremental data. The time complexity of IMF is lower than conventional NMF for handling incremental data. NMF (Lee and Seung 2001) approximately factorizes R into two factor matrices V (V ∈ R m×k ) and H (H ∈ R k×n ) where k is the rank of factorization. V and H denote that users and items are mapped into a k-dimensional latent space. So, we call V as user subspace and H as item subspace. Generally, the NMF technique minimize loss function F to optimize the results of factorization arg min where · F means Frobenius norm. It has a high complexity of O(mnk) in each iteration. Meanwhile, it does not work for online datasets and incremental data. So we consider IMF and make some modifications to suit our needs. We review the optimal objective function of the regularized MF (Guan et al. 2011) arg min where means inner product and · means L 2 norm. W = (ω i j ) m×n records the existing entries of R. If r i j exists, then ω i j = 1, otherwise 0. ε 1 controls the strength of regularization, which are usually an L 2 norm to prevent overfitting. Algorithm 2 describes our IMF algorithm for u i . The input V and H are two subspaces obtained by NMF on the original dataset R. k is the rank of subspaces. e is the convergency threshold. ε 2 is the learning rate. The output V and H are the new subspaces updated by IMF.
Line 1 employs the input V and H as initial values in the first iteration.
Line 2 assigns the maximum iteration times as a constant N .
Line 3 computes the predictive value P i by V (s−1) i and H (s−1) . The subscript i indicates the ith row of the matrix.
Lines 4-7 exclude the nonexisting ratings. Line 8 computes the residual. Lines 9-12 update two subspaces. Line 10 updates V by the values of V and H from last iteration. Line 11 updates H by the value of V from Line 10 and the value of H from the last iteration.
Lines 13-15 judge whether the iteration converges. Line 13 computes 1-norm of the difference between R (s) i and P (s−1) i . Line 14 expresses that if the norm is below e, the iteration is terminated.
Lines 18-19 return the values of V and H after the end of the iteration.
According to the three-way decision, there are a pair of probability threshold (α, β) to decide d i j . However, we use popularity and prediction rating to decide d i j , respectively, in two stages, so we design a function Eqs. (14)-(16) to map probability to popularity and rating, (π * α , π * β ) and (R * α , R * β ) are the optimal solutions. Algorithm 2 Incremental matrix factorization (IMF) algorithm.

Time complexity analysis
The time complexity of our HTCR algorithm is the sum of the popularity-based and the IMF-based recommendation algorithms. We will elaborate it as follows.

Proposition 1 The time complexity of the popularity-based recommendation is O(n).
Proof Lines 3-11 in Algorithm 1 describe the popularitybased recommendation. The for loop of Lines 3-9 takes O(n) of time. Line 11 updates the maturity of u i by Eq. (10)  If we recommend M rounds, the time complexity is O(Mn). The upper bound of M is n/L . Due to the maturity threshold ξ , M is usually a small number in reality. It is hardly influenced by n. So we will not replace it with n/L in the following analysis.

Proposition 2 The time complexity of the IMF-based recommendation is O(N nk).
Proof Lines 12-22 in Algorithm 1 describe the IMF-based recommendation. Line 12 makes n loops with O(n) of time. Line 14 predicts ratings for u i by Algorithm 2. Table 4 describes the detailed complexity of Algorithm 2, and we will analyze it as follows.
In Algorithm 2, Line 3 obtains the product of V i and H . Since the scan is relevant to n and k, the time complexity is O(nk). Lines 5-7 judge whether the current rating exists; hence, the time complexity is O(1). Lines 9-12 update the i th row of V and the j th column of H , and the time complexity is O(k). Lines 13-15 scan each entry of R i in two iterations to compute 1-norm; hence, the time complexity is O(k). Suppose it takes N rounds to converge and the total time complexity of Algorithm 2 is

Overall time complexity of the HTCR algorithm
The performance of the HTCR algorithm should be evaluated across the whole dataset. Hence, we need the complete leaveuser-out training-testing process. That is, each time we select one user as the testing data and all the others as the training data. In this way, the recommendation process will repeat m times. We have the following corollary.

Corollary 1 In a complete leave-user-out training-testing process, the overall time complexity of the HTCR algorithm is O(mn(M + N k)).
Proof According to Proposition 1, the cold-start recommendation process takes O(nM) of time for each user. According to Proposition 2, the IMF recommendation process takes O(n N k) of time for each user. Hence, the time complexity for all users is O(m) * O(nM + n N k) = O(mn(M + N k)). Figure 3 shows the entire recommendation process of u 2 of Table 2, including three rounds of conversational recommendation. The main parameters are as follows. π α , π β , R α , R β , ξ and Z are 0.7, 0.3, 4, 2.5 and 5 2.5 4 2 , respectively.
Round III continues to use the IMF-based recommendation. In this round, RC = {14}, PC = {2} and Γ = (14, 2). Only t 2 is successfully promoted. At the end, we cannot find any more items for u 2 and the recommendation is terminated. Hence, the total cost for u 2 is 120.

Experiments
In this section, we present the experimental results to answer these following questions.
(3) Is the three-way decision model superior to other rough set models?

Datasets and experiment settings
2. The paper can also compare with one more dataset and provide the details. detail: the popularity threshold pair π α , π β , the rating threshold pair R α , R β the maturity threshold ξ . Table 5 shows basic information of three datasets Jester, MovieLens 100K, MovieLens 1M and Yelp. Only user IDs, item IDs and ratings were considered in our experiments. For example, in MovieLens 100K, there were 1682 movies in total, 563 were rated less than 10 times, 526 were rated 10-50 times, 259 were rated 50-100 times, 302 movies were rated 100-300 times, and only 32 movies were rated more than 300 times. Consequently, 80% of the films were rated less than 100 times. After normalized popularity, π α and π β we learned were 0.4 and 0.1, respectively.
We employed the following two cost matrices for experimentation:  Figure 4 shows the runtime change of our HTCR algorithm with different numbers of users or items on the MovieLens 1M dataset. The runtime increased linearly wrt the number of users or items. Figure 4b shows that when the number of users changes from 500 to 6040, the runtime increases from 1.5095 to 14.6970 sec. For example, when the number of users was 6040, the total runtime was 14.6970 sec. It took about 5.6723 sec to perform overhead operations such as data loading and space allocation. We also noticed that when the number of users increased by 500, the runtime increased by 1.5 sec on average. Figure 4(b) shows that when the number of items changes from 400 to 3952, the runtime increases from 1.5611 to 14.6940 sec. When the number of items increased by 400, the runtime increased by 1.2 sec on average. Table 6 shows the average costs of HTCR on three datasets are less than those of non-hybrid algorithms. "Pop" and "IMF" mean that we only employ the popularity-based or the IMFbased recommendation, respectively. Table 6a shows that when inputting C 1 on Jester, the average cost of HTCR was 11.5235% less than Pop and 4.9280% less than IMF. Table 6b shows that when inputting C 2 on Jester, the average cost of HTCR was 3.5380% less than Pop and 13.6245% less than IMF. So, the superiority of our HTCR algorithm is obvious. Tables 7 and 8 show the experimental results of Pawlak, VPRS and three-way models. Table 7 demonstrates that average costs of the three-way model on three datasets are less than those of Pawlak model. Table 7a, b shows that on Movie-Lens 1M, the average cost of three-way model is 23.7030% or 22.3197% less than that of Pawlak, respectively. So, the superiority of three-way model is still obvious.

Comparison with other rough set models
For simplicity, Table 8 shows only the average costs of MovieLens 1M under VPRS model. Both R α and R β have a step size of 0.5, and R α is greater than R β . The entire change is presented in the form of a lower left triangular matrix. It can be seen that the lowest average cost occurs at (2.5, 3.5). Both of the values are nearly the pair of thresholds (2.8, 3.8) calculated from the three-way decision. This proves that our model is more efficient than the VPRS model to minimize the average cost.

Discussions
Now we can answer the question at the beginning of this section.
(1) Our HTCR algorithm is efficient. It is linear wrt the number of users or items, hence scalable to large datasets.
In the parallel environment, each user can be recommended independently; therefore, the time complexity is O(n(M + N k)), which is not influence by the number of users. In each round of recommendation, the time complexity is O(nk), which can be finished in a few milliseconds. In other words, our algorithm is appropriate for real-time recommendation.
(2) Our HTCR algorithm performs better than the non-hybrid algorithms in term of average cost. The cost decrease were 3.4906-25.6494%. (3) Three-way model performs better than Pawlak and VPRS models. Compared with Pawlak model, our cost decrease were 13.6765-31.4850%. Compared with VPRS model, our model can find the thresholds more efficiently.
We also found that our HTCR algorithm performed stably, and the cost matrix setting did not influence its superiority.

Conclusions and future work
In this paper, we proposed a hybrid approach to three-way conversational recommendation. The results showed that our algorithm was superior to other existing models. The following research topics deserve further investigation:  We repeated experiments ten times on each dataset and expressed the average costs in the form of "mean value ± standard deviation." Bold fonts indicate the lowest cost of each row Bold fonts indicate the lowest cost of each row. We repeated experiments ten times on each dataset and expressed the average costs in the form of "mean value ± standard deviation" (1) Better cold-start techniques. This work only considers the popularity-based cold-start recommendation. More sophisticated techniques could be applied or designed for this issue.
(2) Richer data sources. This work is limited to the basic rating matrix. We will design new algorithms to process demographic information, item features, reviews and social network. In this way, the recommendation performance can be enhanced. (3) Dynamic updating of the information. This work only considers the conversational recommendation of each user. In reality, many users update their rating information simultaneously. We should design online data updating and off-line subspace updating strategies.