Study on sentiment classification strategies based on the fuzzy logic with crow search algorithm

In recent times, sentiment analysis research has gained wide popularity. That situation causes the importance of online applications that allow users to express their opinions on events, services, or products through social media applications such as Twitter, Facebook, and Amazon. This paper proposes a novel sentiment classification method according to the fuzzy rule-based system (FRBS) with the crow search algorithm (CSA). FRBS is used to classify the polarity of sentences or documents, and the CSA is employed to optimize the best output from the fuzzy logic algorithm. The FRBS is applied to extract the sentiment and classify its polarity into negative, neutral, and positive. Sometimes, the outputs of the FRBS must be enhanced, especially since many variables are present and the rules between them overlap. For such cases, the CSA is used to solve this limitation faced by FRBS to optimize the outputs of FRBS and achieve the best result. This study compares the performance of the proposed model with different machine learning algorithms, such as SVM, maximum entropy, boosting, and SWESA. It tests the model on three famous data sets collected from Amazon, Yelp, and IMDB. Experimental results demonstrate the effectiveness of the proposed model and achieve competitive performance in terms of accuracy, recall, precision, and the F–score.


Introduction
Given the rapid development of online applications, whether social media or those used in e-commerce (Amazon, Facebook, and Twitter), customer reviews are now readily available. Analyzing these reviews and revealing their opinions are valuable for improving the business process. Reviews can help the customer evaluate a specific service or product quickly. Moreover, reviews offer essential evidence for decision-making regarding service providers. Sentiment analysis is also known as ''opinion mining'' or ''emotion Artificial Intelligence'' and alludes to the utilization of natural language processing (NLP), text mining, computational linguistics, and bio-measurements to methodically recognize, extricate, evaluate, and examine emotional states and personal information. Moreover, sentiment analysis is generally concerned with the voice in client materials, such as the surveys and reviews on the Web and in Web-based social networks (Serrano-Guerrero et al. 2015).
As a generic rule, sentiment analysis attempts to determine the author's behavior on articles, speakers, or other topics through intense emotional or emotional responses to archive, appropriately and communicatively. The study shows the response of people on social media. This response is an evaluation of emotion. In other words, the sentimental state of the creator and speaker is an expectation of enthusiastic responses (that is, the effect is intended by the originator or buyer). Numerous customers surveys or recommendations on all topics are available on the Web. The periods and audits may contain surveys on items such as customers or fault-finding of films. Surveys expand rapidly, given that individuals like to provide their views on the Web. A lot of surveys are accessible for solitary items, which makes it problematic for customers as they must peruse each to make a choice. Next, mining this information, special customer assessments organizing them is a vital pledge. Sentiment mining is a task that takes advantage of NLP and information extraction (IE) approaches to analyze an extensive number of archives for ascertaining the sentiments of comments posed by different authors (Serrano-Guerrero et al. 2015;Al Shboul et al. 2015). This process incorporates various strategies, including computational etymology and information retrieval (IR) (Al Shboul et al. 2015). Most of the proposed methods, including supervised and unsupervised methods, tackle the task as binary classification. Supervised learning relies on the learning classifier for each label (Saif et al. 2016;Medhat et al. 2014). Whereas unsupervised learning depends on the glossary of sentiments (Tang et al. 2014;Ravi and Ravi 2015;Poria et al. 2014;Nasim et al. 2021). A fuzzy rule-based system (FRBS) can extract sentimentally and more accurately through the belonging degree of the fuzzy sets. The FRBS (Jefferson et al. 2017) employs the fuzzy degree to analyze the feasibility of a fuzzy rulebased classifier and to obtain the emotion scores. The output is a binary polarity classification (e.g., positive or negative).
Conversely, an unsupervised sentiment analysis method  is in accordance with the fuzzy rules for the Twitter domain. The critical factor involves employing some fuzzy rules to obtain the sentiment of each tweet. That sentiment is determined by the level cross of two input variables (i.e., positive [TweetPos] and negative [TweetNeg] tweets). Unfortunately, this model is unsuitable for a three-class categorization. Meanwhile, a fuzzy logic classifier is used as a sentiment classifier based on emotions (López et al. 2015) and combines deep convolutional neural networks and fuzzy logic.
This paper presents a newly proposed method to analyze the sentiment. The works mentioned above fall under the concept of fuzzy system rules to some extent. That circumstance shows that they help analyze feelings. However, fuzzy logic still has limitations in adapting to exact rules, especially when many variables are connected. Therefore, we added the concept of CSA into FRBS to solve the limitation of this case. Our work consists of two main parts: an application of the FRBS and the crow search algorithm. A novel sentiment classification method that focuses on document-level sentiment analysis tasks is presented in this work. We address these tasks as a three-class classification problem according to fuzzy logic rules. We incorporate the CSA into the concept of a fuzzy logic rules system to optimize the output generated from the fuzzy logic rules. CSA collects all the outcomes of the rules of the fuzzy system and then searches for the best outcomes archived by the system according to specific metrics, namely precision and recall. Regarding the results, we discuss and verify the accuracy of the performance and outcomes by investigating the validity of the performances of three reference data sentiment analysis data sets, including those from Amazon, Yelp, and IMDB. The experimental results reveal the effectiveness of the proposed model and the possibility of improving performance regarding sentiment analysis in the case of three-class predictions. The main contribution of our work can be summarized as follows: analysis in the case of three-class predictions. The main contribution of our work can be summarized as follows: 1. The proposed method can combine positive and negative fuzziness and deal with the mystery and blur. 2. This work integrates CSA with a fuzzy logical rules system to optimize the output results of the FRBS. 3. Our proposed model obtains high classification accuracy relative to existing approaches for all cases of experimentation and with efficiency as a critical factor. 4. The crow search algorithm achieves the best result given any ambiguity or overlap between the membership function degrees when applying the fuzzy rules.

Related work
This section provides a literary survey of works in this domain using various approaches, such as machine learning techniques and artificially intelligent and soft computing algorithms. One of these techniques is a fuzzy logic system. That system can process doubt or ambiguity in a remarkably productive way because of the presence of interference. The writers Vashishtha and Susan (2020a) apply the concept of fuzzy rules to reduce their four fuzzy rules to assess the sentiment of reviews. The highlights of their work are as follows. (i) They develop four fuzzy rules based on a text and speech tick to assess the sentiment of each review. (ii) The suggested decision-level fusion approach performs the best, outperforming unimodal and primary feature-level text-speech fusion using a supervised machine learning classifier, namely support vector machines. (iii) Their work compared the idea of a rulebased system for sentiment analysis with eight modern supervised machine learning techniques. Zohreh Madhoushi et al. investigated sentiment analysis techniques and offered machine-based learning approaches, such as supervised and semi-supervised learning. They used more widely techniques in this field, such as lexical and hybrid techniques. They concluded that there were still some issues, such as the current methods, that could not manage complicated phrases. Thus, the research did not include an investigation of feelings in languages other than English (Madhoushi et al. 2015). Furthermore, other technologies in this subject were investigated, including computational intelligence which includes the evolutionary algorithm, fuzzy sets, rough sets, neural networks, and deep learning (Katarya and Yadav 2018). Formal concept analysis (FCA) was discussed by Medhat et al., and fuzzy formal concept analysis was confused (FFCA). FCA is a mathematical technique that employs relationships that are slightly arranged. FFCA, on the other hand, works on unverified information (Medhat et al. 2014). To assess and summarize the complaints of customers, Kumar Ravi et al. developed a hybrid model that combines fuzzy formal concept analysis (FFCA) and concept-level sentiment analysis (CLSA). They also compared it to FCA and sentiment-level emotional analysis (Ravi et al. 2017). Using decision-level integration, Zadeh et al. attempted to construct an ambiguous, emotive grammar analysis system for multi-speech text. An examination of multimedia sentiment was conducted on a CMU MOSI dataset (Zadeh et al. 2016). In general, three types of fuzzy reasoning models are available: Sugeno, Mamdani, and Tsukamoto. The rule-based fuzzy Tsukamoto system (Medhat et al. 2014;Ravi et al. 2017) was used to analyze feelings. These works employed a trapezoidal blurring organic function to convert numerical values into obscure linguistic terms. The fuzzy logic approach can conduct real-time analysis of tweets to map tweet effects over time (Jefferson et al. 2017). The feelings in these tweets describe the dynamic mood swings of a cricket fan while controlling a cricket match. Mamdani's vague, rule-based system categorizes uncensored feelings for Twitter datasets and is suitable for any glossary and two and three-grade sentiment analysis tasks (Vashishtha and Susan 2020b). This work introduces a genius set of mysterious rules for developing a multimedia emotion classification system using the mysterious Sugeno model. A fuzzy logic system (FLS) has been widely used for sentiment polarity identification in the last few years.
A study Fares et al. (2019) introduced a LISA model, an unsupervised word-level knowledge graph-based LSA framework. The LISA model utilized various variants of shortest-path graph navigation techniques to calculate and propagate effective scores in a lexical-affective graph (LAG), constructed by connecting a typical lexical KB like WordNet, with a reliable effect KB like WordNet-Affect Hierarchy. A new algorithmic framework for autonomous Music Sentiment-based Expression and Composition, called MUSEC, was proposed by Abboud and Tekli (2020). This model perceives an extensible corpus of six primary human emotions (e.g., fear, anger, love, joy, surprise, and sadness). It is expressed by a MIDI musical file, which creates novel polyphonic, (pseudo) thematic, and diversified musical pieces that express these emotions.

Proposed model
This section describes the proposed approach of applying the fuzzy logic system and crow search algorithm to analyze the sentiment. Wherein the first subsection explains the application of the fuzzy logic system (FLS) and the pros and cons of this approach, the second subsection describes CSA for annealing the gap found in the FLS. This section presented a sentiment analysis framework by discussing the different tasks engaged in sentiment analysis. The main idea of this proposed method is to apply the FLS and crow search algorithm to process the posts to overcome the above challenges. In this paper, the proposed approach builds on the FLS to process the computational complexity and its interpretation. At the same time, the suggested technique can preserve the classification performance in line with the performance of the best algorithms for classifying sentiments. The fuzzy rules can be represented in the form of 1 or 0. This form is very close to natural languages and can be interpreted easily. The task of FLS in this work is to classify the polarity of the sentiment into three classes: positive, negative, and natural. The CSA will solve the overlapping of fuzzy sets, especially when several variables that belong to the same category are involved. Moreover, the CSA will achieve the sentiment polarity to enhance the FLS, especially when the result produced by FLS is unclear.
The fuzzy rules-based system with crow search algorithm provides an effective method of sentiment analysis based on an FRBS and a CSA to analyze the personality sentiment during the typing of text. The sentiment analysis process is shown in Fig. 1. The text preparation step performs required text pre-processing and characteristic selection and extraction of the dataset, including the removal of stop words. The sentiment identification step determines the sentiment of people expressed in the text and analyzes it based on selecting the features of writing and building the fuzzy rules. The sentiment classification step is conducted to identify the polarity of sentiment to get the results. It is finally implementing the validation and evaluation of produced results to display the effectiveness and performance of the proposed work.

Fuzzy logic system
The FLSs are the most useful approaches in modeling some complex systems that humans can observe because they use linguistic variables as their antecedents and consequent. These linguistic variables can be represented naturally by these sets' fuzzy sets and logical connectives. I employed three standard methods of fuzzy systems based on linguistic rules in practice: the Mamdani systems, Surgeon models, and Tsukamoto models (Ross 2010). In this study, we adopt the Mamdani systems, which entail four steps: fuzzification of input variables, rule evaluation (inference), aggregation of the rule outputs, and defuzzification. Three different dictionaries have been proposed and investigated for training data: SentiWordNet, AFINN, and LabMT (Baccianella et al. 2010;Nielsen 2011;Dodds et al. 2015). SentiWordNet is divided into two parts: Senti Positive and Senti Negative. Thus, four training data sentiments are generated, and we will classify text polarity according to these lexicons. The FLS deals with these lexicons as input data, and each one is classified into the classes listed below.

Fuzzification
This step is included in the sentiment classification shown in Fig. 1. Using the triangular membership function, we fizzed all data obtained from the second phase's positive, negative, and average scores. All linguistic term T involves three key points, d, e, f, when the triangular the fuzzy membership is used and associated with the change of style of the fuzzy membership. lS: X ? [0,1] defines the membership function (MF) for a fuzzy set S on the universe of discourse X, and each element of X is mapped between the 0 and 1 value. The triangular function equation defined by an upper limit f, an intermediate value e, and a lower limit d, where d \ e \ f, as Eq. 1 (Vashishtha and Susan 2020a): Figure 2 illustrates the triangular fuzzy membership, where the fuzzy sets are created using the triangular fuzzy membership for universe variables. This work uses three datasets (AFINN, SentiWordNet, and labMT) as inputs of the fuzzy system. Where SentiWordNet dataset is classified into two classes Senti Positive and Senti Negative, Fig. 2 shows the fuzzy system of these datasets.
Each one of these datasets has special fuzzy sets; for example, the dataset AFINN in this method has the following five fuzzy membership functions in the range À5; 5 ½ : HighNegative H À N ð Þ¼À5; LowNegative L À N ð Þ¼À2:5; Natural N À N ð Þ¼0; Low Positive L P ð Þ¼2:5; and High Positive H P ð Þ¼5: The output of this stage is the polarity of sentiment analysis: Positive, Negative or Natural.
The dataset LabMT also has five membership functions in range [0 10] (As showing in Fig. 3) and that can be explained as follows: : The output of this stage is the polarity of sentiment analysis: Positive, Negative or Natural.
According to both (labMT and AFINN) datasets (Figs. 3, 4), each Senti Negative and Senti Positive have an individual three membership functions in range [0 1] as Fig. 1 The structure of the proposed approach Low ¼ 0; Medium ¼ 0:5 and High ¼ 1, as shown Figs. 5 and 6. Therefore, the output was classified into three-membership functions, as mentioned in Fig. 7. We calculate the global minimum (min), global medium (nan), global   (2) as follows Vashishtha and Susan (2020a): The required variables for building the triangular fuzzy membership of the fuzzy sets are: Low, Medium, and High. It is shown as follows.

Formulating the rule-base
This step is involved in the sentiment classification shown in Fig. 1. We have four inputs as proposed above, and the first input is AFINN lexicons, which have five membership functions in the domain [-5, 5]. The second input is Senti Positive lexicons, which have three membership functions in the domain [1, 0]. The third input is Senti Negative lexicons, which also have three membership functions in the domain [1, 0]. The fourth input is LabMT Lexicons, which has five membership functions in the range (Serrano-Guerrero et al. 2015; Vashishtha and Susan 2019). These inputs are implemented through two stages using the logical operation called min operation and FRBS. The first stage is implementing the AFINN, Senti Positive, and Senti Negative to fuzzy rules. The second stage implements the Senti Positive, Senti Negative, and LabMT to fuzzy rules. Therefore, the total number of rules becomes ninety rules. These rules are covering all the probabilities and calculated as 5 Ã 3 Ã 3 þ 3 Ã 3 Ã 5 ¼ 90. Figure 8 shows the visualization of a part of ninety rules.

Defuzzification
This step is included in the polarity detection shown in Fig. 1. Defuzzification is the process of identifying the aggregate polarity of the sentiment of a complete dataset. This process returns the overall sentiment for the selected dataset. The Mamdani inference system is employed to compute the center of gravity using the following statements:

Crow search algorithm
Rules to some extent, show that they are effective in feelings analysis. Yet, this technique still has limitations in adapting to the exact rules, especially when many variables are connected. Therefore, we focus on document-level sentiment analysis tasks. We addressed this task as a threeclass classification problem based on fuzzy logic rules. We integrate CSA into the concept of the FLS to optimize the output generated by FLS. CSA collects all outcomes of the rules of the fuzzy system and then looks for the best result archived by the system based on metrics, namely precision and recall.
The CSA is a relatively novel meta-heuristic improvement algorithm based on the intelligent behavior of crows (Askarzadeh 2016). CSA is a population-based optimization algorithm with fundamentally adjustable flight length and potential awareness parameters. These properties make the CSA a viable option for complex engineering optimization problems and mathematically complex optimization problems such as solving the feature selection problem, improving text documents classification, estimating the threshold values for magnetic resonance brain images, and solving the DNA fragment assembly problem. The general mechanism of CSA implementation is mentioned as follows.
Firstly, the position of the hiding place of each crow is created randomly, and the memory of each crow is initialized with this position as the best experience. Secondly, the crow evaluates the quality of its position according to the objective function. Finally, the crow randomly selects one of the flock crows and follows it to discover the position of the foods hidden by this crow. If the found position of the food is tasty, the crow updates its position. Otherwise, the crow stays in the current position and does not move to the generated position.
According to the previous hypothesis, in this study, CSA starts by initialing the variables. In this step, the flock population (N), maximum number of iteration (intermix), awareness probability (AP), flight length (fl), and number of decision variable (d) are defined. Second: represents the memory of crows and locations; in this step, the N number of crows are randomly placed in d-dimensional search space. Each position of crows defines a possible solution to the algorithm. Initially, as the crows have no memory, the starting position is the best memory in the problem. That is at the first iteration. The crow has the initial position as a memory location. Third: calculation of objective function. The quality of each crow's position is determined by dropping the variables into the objective function. Fourth: generation of fresh location Crows produces fresh location in the environment as follows: assume crow ''I need to produce a fresh location.'' To do this propose, this crow arbitrarily chooses one of the flock crows (for example, crow j) and chases it to determine the location of the environment best position by this crow (m j ). The fresh location of crow i found by the cases below.
where r i is any random number uniformly distributed between 0 and 1. m j;iter is a best position of crow j at iteration iter. Fifth: Calculate the objective function of new position. The objective function value for the new position of each crow is figured out. Sixth: Update memory up gradation of memories of crows are computed as follows.
where f Á ð Þ depicts the fitness function value, if the fitness function value of memory is poorer to the fitness function value of the new position, the crows update their memory with present position.
In the last: verification of execution of criterion. These steps are reiterated until intermix is achieved. When the implementation task is completed, the best value of the target function is reported as the general solution and the memory information is reported as possible global points of the optimization problem.
Algorithm 1 shows the pseudo-code of the proposed model Begin Procedure: Algorithm 1: Integrating CSA with fuzzy rules-based system

Evaluation datasets
This subsection describes the data sets and the algorithms used to evaluate the proposed approach. We conduct our experiments on three extensive and well-known datasets. As explained below, these data sets were collected from different fields with varied contexts and vocabularies.
• Amazon: consists of more than1000 product reviews labeled as 'positive' or 'negative.' This dataset has 1865 distinct word tokens. • IMDB: IMDB dataset has been used for sentiment classification of film reviews. IMDB includes 50,000 reviews and consists of binary reviews with positive and negative sentiments. It was evenly split into 25,000 reviews for training and 25,000 for testing. • Yelp: contains more than1000 restaurant reviews labeled with 'positive' or 'negative'. In this dataset, a total of 2049 distinct word tokens are found.

Data preprocessing
Data preprocessing: Data preprocessing is considered as an essential step in machine learning and data mining (Gupta and Malhotra 2015;Verma et al. 2018;Ma et al. 2020;Al-Deen et al. 2021).The reviews usually contain incomplete sentences, much noise, and weak wording such as words without application with high repetition, imperfect words, and incorrect grammar. Unstructured data also have an impact on sentiment classification results. A series of preprocessing on the reviews are needed to maintain a stable structure and reduce such problems. Cleaning data with filters, splitting the data into a part for training and testing, and building data sets with favorite words are just a few of the steps employed in our research. Without going into too much depth, we used the following method to prepare the data:  (or, also, able, etc.). The Natural Language Toolkit (NLTK) library provides a stop words dictionary, including a list of words that have meaning neutral and are not suitable for sentiment analysis. To remove the stop words from the comment's text, we check each word in the list against the dictionary and exclude them.

-Capitalization
Due to documents and texts contain many sentences and include a diversity of capitalization for a sentence. So, diverse capitalization can be a big problem when classifying extensive documents. The best approach to dealing with inconsistent capitalization is to convert each letter to its lower case. This technique shows all words in the same feature distance to the text and document. Still, it reasons a significant issue in the interpretation of some words (e.g., ''US'' (United States of America) to ''us'' (pronoun).

Evaluation metrics
The following metrics were used to evaluate the performance of the proposed sentiment analysis framework.
• Accuracy: defines the ratio of the true results (both true positives and true negatives) to the total number of documents analyzed by the classifier.
where TN, FN, FP, and TP represent the true negatives, false negatives, false, and positives true positives, respectively. • Precision: defines the recommended items that are relevant to correct predictions, divided by the total of predicted recommended items during the testing process • Recall: indicates the ratio of a number of documents whose sentiments are correctly classified as positive to the actual number of documents having positive sentiments in the given text corpus.
• F1 Score: This metric represents the harmonic mean of both recall and precision.

Experimental settings and baselines
Experiments are performed on a PC with Intel(R) Core (TM) i7-6400 CPU at 2.71 GHZ with 8 GB of RAM and running on Windows 10. This method was implemented using three data sets collected from Amazon, Yelp, and IMDB and implemented by the applied fuzzy logic R as a classifier for sentiment analysis and with the CSA as optimizer to achieve the best result in finding out by fuzzy rules-based systems. The implementation was an environment of in MATLAB Programming Version 2019a.

Experimental results and discussion
After collecting the datasets, the FLS applying to determine the polarity of the text. After that, the CSA was used to achieve the best polarity. Demonstrating the performing evaluation and experiment results that are based on different machine-learning classifiers. The classifiers were applied on each dataset, and the performance is compared based on cross-validation, accuracy, precision, recall, and f-score.

Accuracy analysis
The experiment results were obtained using fourfold cross-validation, computing the accuracy for each fold, and then ascertaining the mean outcome. We compared the performance of our proposed method against four different methods. Table 4 presents the accuracies of various algorithms with the review data sets collected from Amazon, Yelp, and IMDB. The outcomes were evaluated with multiple data sets by comparing the performance of different machine learning models and various methods. Table 1 indicates that the difference between our proposed method and the latest techniques is approximately 4.25% in terms of accuracy for the Amazon dataset. Table2 shows that the difference between our proposed method and the latest techniques is approximately 0.46% in terms of accuracy for the Yelp dataset. Finally, Table3 shows the difference between our proposed method and the latest techniques is approximately 3.96% in terms of accuracy for the IMDB dataset. Therefore, according to the Yelp dataset, no significant difference occurred between our method and the maximum entropy method, as shown in Fig. 9. 2. Precision, Recall, and F-score Analysis The precision, recall, and f -score depicted in Table 4 show the impact of the proposed model compared with baseline models on different review datasets. For each proposed algorithm, the performance has been evaluated in each step. Figure 10 shows this superiority.

Conclusions and future work
Sentiment analysis is essential for anyone who makes a decision. It is useful in various areas for measuring feelings, expressing, and identifying emotions. Though the study has achieved interesting results, some changes in our future work will be made to achieve better results and improve the performance. This study provides an empirical analysis of the classification of feelings based on vague rules and crow search algorithms. The results obtained from the classification process showed that our sentiment classification model, which is based on fuzzy vague rules and group search algorithms and achieved great results compared to models based on other various classification techniques such as SVM, Maximum Entropy and Boosting applied to three famous data sets collected from Amazon, Yelp, and IMDB. Consequently, these reviewed data sets are applied to evaluate FLS with CSA models and rank the polarity of sentiments. For each different dataset, the performance, accuracy, and factors (recall, accuracy, and f score) were used to evaluate the results. Finally, the results are analyzed, compared, and presented with a statistical approach.
Author contributions All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by MazenSharaf AL-Deen, Yu lasheng, Gamil R. S. Qaid and Ali Aldhubri. The first draft of the manuscript was written by MazenSharaf AL-Deen, and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.