Query-Based Extractive Text Summarization Using Sense-Oriented Semantic Relatedness Measure

Senseval and SemEval datasets


Introduction
With the growing textual information from the Internet or other sources, extracting useful information in minimum time is a tough job. Text summarization solves the problem of overloading of information. According to Hovy and Lin (1998), text summarization is a process of finding short, accurate, and fluent form of a single or multiple text documents where the length of the summary is not more than half of the original texts. Summarization techniques can be either extractive or abstractive. Extractive summarization techniques extract major snippets, sentences or passages from the input text documents, while the abstractive summarization technique rephrases the information content present in text documents. Query-based text summarization helps in extracting the required information needed by the text summarization method outperforms many existing methods. As an unsupervised learning algorithm, we obtained highest ROUGE (Recall-Oriented Understudy for Gisting Evaluation) score for all three DUC 2005DUC , 2006 and 2007 datasets. Our proposed method is also quite comparable with other supervised learning based algorithms. We also observe that our querybased extractive text summarization method can recognize query relevance sentences which meet the query need.
Keywords Query-based extractive text summarization · Sense-oriented semantic relatedness measure · Word Sense Disambiguation (WSD) technique · Clusters of query-relevant sentences · Redundancy removal technique · Senseval and SemEval datasets · Document Understanding Conference (DUC) · ROUGE (Recall-Oriented Understudy for Gisting Evaluation) progress and problems associated with hydroelectric projects. Hence, TF-IDF measure is not sufficient to measure the query relevance which in turn gives informa-based text summarization. This redundancy removal technique increases the information diversity and reduces the summary size.
Before doing the summarization, we need to find query relevant sentences. To find the relatedness score between query and input text sentence, sense-oriented semantic relatedness measure is used. We know that English language have many ambiguous words. A word carries more than one sense and can be used in many ways. For example: key. Sometimes we use key as a enough to make an informative-rich summary (Han et al. given texts (Wang et al. (2010)). Most of the queries are (2000), Zhao et al. (2009), Carpineto and Romano (2012), complex real-world questions related to the input text progress and problems associated with hydroelectric pro-word order similarity score. Therefore, a word sense text data.
-User gets the information in concise fashion.
-It reduces the reading time.
-User gets the specific data with minimal loss of information without having to read large volume of user. Sources of the text summary either can be single text document or multiple text documents. This query-specific text summary is constructed based on the calculation of the relevancy between sentences in the text documents and the given query. Hence, this type of summarization should be concise and should satisfy the user's need. Three major advantages of automatic query-based text summary are: Afuan et al. (2019), Strzalkowski et al. (1998)). But, indeed, there are some underlying relationships with the query words which are semantically related with the query. Our aim is to find semantically related sentences which will help in getting the summary in a gist way with relevant information.
documents (Canhasi and Kononenko (2014)). In spite of its relevance and significance of query-based text summary, the extracted sentences for query-based text summary has not been fully explored (Bayatmakou et al. (2021)). Our main objective is to create a redundancy free query-based text summary. Existing query-based text summarization methods use many different features. It is also seen that most of the existing methods use to give more focus on query-independent features and usually less consideration is given to the querydependent feature (Damova and Koychev (2010), Afsharizadeh et al. (2018)). Those works do not give much importance to select the features according to the re-Query-based text summarization can produce concise information according to the users queries from the tion less query-oriented summary. It means that only the presence of query words in input sentences are not While doing the query-based text summarization, most of the summarization systems depend on surface features like TF-IDF cosine similarity between input text sentences and a query (Wu et al. (2008), Park et al. (2006), Rastogi (2020), Ramos et al. (2003)). TF-IDF cosine similarity measures the query relevance. However, relevance and similarity both are not same. For example: we consider the document cluster 'd307b' present in DUC 2005. The query is: "What hydroelectric projects are planned or in progress and what problems are associated with them?" Here, hydroelectric projects are the keywords having higher TF-IDF scores. But main purpose of the query is to find out planned, disambiguation method is proposed with less time complexity. We consider accurate sense of the words while finding relatedness score between two words. It helps in increasing the accuracy of relatedness score. After extracting the query relevant sentences, we use cluster based redundancy removal method to create query-jects. These general words are not present in the source text document or may be their frequency of occurrence is quite low. It is obvious that if TF-IDF cosine similarity measure is applied, top scoring sentences will contain mainly the words "hydroelectric" and "projects". Unfortunately, none of them gives the exact planned, small piece of shaped metal or sometimes it is used to describe something which is crucial important. The actual sense of the word could also be calculated by generating contextualized word embeddings from language models such as ELMO, BERT, GPT, etc (Kutuzov and Kuzmenko (2019), Devlin et al. (2018), Radford et al. (2019), Wiedemann et al. (2019), Hadiwinoto et al. (2019)). But sizes of these models are humongous. In contrast, real world applications need small model size, low response times and low computational power wattage (Gupta and Agrawal (2020)). BERT and other neural network models are very compute-intensive at inference time.
Ambiguous word always degrades the quality of querybased text summarization. Ambiguity in words minimizes the query relevance for query-oriented text summarization. Ambiguity removal or detecting senses is an emerging problem in natural language processing and ontology (Yadav et al. (2021), Zamanifar et al. (2008), Abu Nada et al. (2020), Maulud et al. (2021)). Sense makes an important role while finding semantic relatedness score between two content words. To find query relevant sentences, we find sense-oriented sentence semantic relatedness score between query and input text sentence. This relatedness score includes sense wise semantic relatedness score, sense relatedness score and In this paper, we introduce an extractive type redundancy free multi-text document query-based text summarization method. This proposed method investigates mainly following issues : (1) develop a content word sense disambiguation method (2) find query relevance sentences (3) create a cluster-based redundancy free summary so that final summary comes with more information diversity.
The specific contributions are as follows: (1) This query-based text summarization technique extracts sentences based on its sense. To the best of our knowledge, we are the first to use exact sense which will be suitable for the context of sentence. We have done the sense-tagging of our input text sentences. It helps in getting more query relevant sentences. (2) We use sensetences. This total relatedness score combines semantic relatedness score, word sense score and word order score. From sense-oriented semantic relatedness score, we find query relevance sentences. (3) We provide a redundancy removal technique which is suitable for finding redundant sentences in multiple text document querybased text summarization. The proposed method uses the correct sense while finding the redundancy between relatedness measure to find query relevant sentences on the basis of word's sense. This relatedness measure finds the score between input text sentences and query. also help in eliminating irrelevant features. the complete sentence.
ing query relevant sentences (Afsharizadeh et al. (2018), Widyassari et al. (2020)). Extraction of semantically related sentence is a difficult task as machine does not possess a vast knowledge like a human being. We introduce a word-based sense-oriented sentence semantic sentences. Existing query-based text summarization tech-oriented semantic relatedness score between query and niques encounter performance bottlenecks when extract-input text sentence to extract query-based text sen-Query-based text summarization computes semantic relatedness measure between query and input text sentences. It helps in extracting more query relevant vide a word sense disambiguation method to find the correct sense. We match each sense of a word in word level which further helps in finding accurate sense of We need to calculate semantic relatedness measure to find the query relevant sentences for query-based text summary. One important key factor of ontology based semantic relatedness measure between two sentences is that it calculates the score on the basis of words. Existing semantic relatedness measures take the first sense of both words by default and give score or sometimes it uses that sense for which the score is highest (Meng et al. (2013)). It does not always give the correct score based on its actual meaning. Before calculating the measure, it is important to find the correct sense of each word present in the sentence. Therefore, we pro-In real-world applications, there is a high probability that all the query words may not be available in the input text documents. For example: let us take following query from DUC 2005 query-based text summarization dataset: What hydroelectric projects are planned or in progress and what problems are associated with them? Many related text documents are there, but in some text documents hydroelectric word is not present. Considering that issue, our method can find the semantically related sentences from them. In those text sentences a word dam is present which is related with hydroelectric word. Existing methods have not considered to find sense wise query related sentences (Chali and Joty (2008)). Our method has additional advantage of extracting query relevant sentences in case of absence of query words in the text documents. This makes our proposed sentence semantic relatedness measure a different one from the existing measures. The method uses word sense to identify how much these two words are semantically related. Our proposed method can extract the sense of a content word and find its meaning. The method expands each content word sense-wise and finds its relatedness with other words. Finally, word level relatedness score helps in getting sentence level relatedness score. It helps in extracting more semantically related sentences although same words are not present in the sentences. Extracting only query relevant sentences quirement. Many features are irrelevant and redundant according to the requirement (Allahyari et al. (2017)). Keeping in view, to eliminate irrelevant features, proposed method extracts the query relevant sentences. Use of relevant features restrict doing unnecessary computation and enhances the performance (Balachandran et al. (2018)). We use only the relevant and redundancy free sentence-based features. Further we show that use of relevant features help in making clusters. Each cluster contains similar sentences. The probability of redundant sentences present in each cluster is very high. Based on its sense, existing redundancy removal methods fails to find the redundant sentences (Oufaida et al. (2014)). Therefore, we need to develop a redundancy removal method particularly for multiple text query-based text summarization. In this summary, more than one sentences may be present in the prepared summary and they convey the same meaning. Through redundancy removal method, only one sentence will be retained and others will be deleted. It helps in minimizing any redundant or repetitive content. Finally, we will get an informative query-based summary with minimum redundancy. two sentences. It helps in minimizing any redundant or repetitive content. In final stage of the summary, best sentences are selected having most important overlap with query and lowest redundancy among selected sentences. At the end, we get a query-based text summary with relevant information diversity. Experiments are performed on publicly available DUC datasets. Evaluation results show better than baseline systems and many state-of-the-art systems.
The remainder of the paper is as follows: Section 2 presents the literature review on different techniques used in query-based text summarization. Section 3 provides the brief introduction of the proposed technique. Section 4 explains the importance of sense for finding semantic relatedness measure. Section 5 describes the proposed unsupervised method for WSD. Section 6 gives the detailed description of the sense-oriented semantic relatedness calculation between two sentences. Section 7 provides how to generate the redundancy free query-based text summary. Section 8 presents the evaluation of WSD technique. Section 9 and 10 give the dataset description and the evaluation metric for querybased text summarization. Section 11 and 12 include the parameter setting and and the performance comparison for query-based text summarization method. Section 13 discuses the query relevancy. Finally, the paper concludes in section 14 with some feasible future scopes.
Different classical, machine learning and neural network based techniques are used extensively in querybased text summarization from many years. Three different techniques will be reviewed separately in respec-to give scores to each sentence. This method got 44% accuracy while finding similarity between system generated summary and human generated summary. In case of query based text summarization, relevance score (Berger and Mittal (2000)) is used to find the similarity between the query and input text sentences. Term frequency (tf) and inverse document frequency(idf) was used here to calculate the relevance score. The weighted tf and idf is also used to find query based text summary (Ulrich et al. (2008)). Different graph based approaches were used by many researchers. Mohamed and Rajasekaran (2006) have used a document graph where two types of relations "is a" and "related Luhn (1958) gives the first attempt for generating generic summary of a text by finding the frequency of the word present in a text. Before finding the frequency of words in a text document, a few steps like stop word removal, stemming were performed in his work. This work is further known as universal pre-processing step in text analytics. Baxendale (Baxendale (1958)) uses position features while creating the summary of a text. Edmundson (Edmundson (1969)) uses both frequency of words and positional features along with cue words and skeleton of structure of text to create summary. Different weights are assigned with different features tive subsections.

Classsical Techniques Applied in Query-Based Text Summarization
2 Literature Review on Query-Based Text Summarization Techniques to" are used. This document graph is used to represent the text document. They made three attempts in where, they created the centric graph with a little modification for generic summary. After that, they extended their work to query-based summary and finally they used extra information of query by introducing query modification technique. However the performance of their method is better than many baseline systems, but it fails to work when different sub topics are hidden in the query. However, Semantic relatedness is not also considered to find out related words with the query words. Ye and Wei (2008) combine two types of score: first score is based on word overlap between query and the sentence and second score is based on relation between sentences. First measure is calculated by finding the semantic similarity between the sentence and query and second measure is carried out by using the semantic graph. These two measures give high-quality relevant and high density based sentences for creating summary. Their method performs well only on single document query-based text summarization.
TextRank, LexRank are graph based ranking algorithms. They are widely used in text processing specially in text summarization (Mihalcea and Tarau (2004), Erkan and Radev (2004), Akhtar et al. (2019)). Tex-tRank algorithm uses number of common words present in two sentences wher-eas LexRank algorithm is based on cosine similarity of TF-IDF vectors. TexRank algorithm relies on the Google's PageRank algorithm (Brin and Page (1998)). One of the major drawback of Tex-tRank algorithm is that it can only work for single document summarization. TextRank algorithm also neglects about the information about text structure and context (Yu et al. (2016)). In case of LexRank algorithm, the performance is not always constant (Patil and Brazdil (2007)). Semantic relation depends on cosine similarity of TF-IDF vectors. Hence, similarity computation Many supervised and unsupervised algorithms have been proposed by many researchers. Different machine learning approaches like Support Vector Machines (SVM), Support Vector Regression (SVR), Bayesian models, Hidden Markov Models (HMM) and Decision Trees are used extensively in query-based text summarization purquery-based text summarization problem as a weighted archetypal analysis problem. This work also presented how to add query information in Archetypal Analysis and how to add weighted Archetypal Analysis in sentence ranking and clustering. However, the performance of wAA can be improved in following ways (1) they can add semantic similarity by using lexical based ontol-Neural networks process information like a human brain does. The building block of neural network is the neuron. Artificial neural networks are computing systems inspired by the biological neural networks (Anderson (1995)). It is a composition of perceptrons connected in different ways and operating on different activation functions. A neural network with only one hidden layer is known as shallow neural network and more than one hidden layer is known as deep neural network. Deep neural network based techniques are more efficient than traditional machine learning techniques as deep learning skips the manual steps of extracting features. Svore et al. Svore et al. (2007) have proposed a neural network based single document text summarization system called NetSum. This algorithm is capable of doing text summarization using neural network and finding novel features using news search query logs and wikipedia. This technique is influenced by the method of Burges et al. (2005) as similar ranking algorithm is used to rank the sentences. In future, feature selection technique can be added for better performance of NetSum and also should consider extracting content across sentence boundaries. On the basis of the semantic networks, fuzzy logic and evolutionary programming, Shardan and Kulkarni (2010) put forward three approaches and compared with many baseline summarization systems. Semantic network is a knowledge base that represents the semantic relations between different concepts (Bird et al. (2009)). First approach caries part of speech tagging, semantic and pragmatic analysis and cohesion. Second approach performs summarization using a decision module based on fuzzy concepts. Third system takes the consideration of all three evolutionary, fuzzy and connectionist techniques. The limitation of their methods is that they consider only the small details related to small summarization process but not in entire summarization system. Prasad et al. (2009) have applied Recurrent Neural Networks (RNNs) in text summarization .
The deep learning techniques have been successfully applied in many fields like image, vision, speech and natural language processing (NLP) tasks. It is mainly used in optimum feature selection process. In fact, currently use of deep learning in NLP is one of the most promising field of research. Collobert et al. put forward a semi-supervised method for part of speech tagging, pose. HMM model is used as a statistical model for text summarization. The machine learning techniques are mainly based on information retrieval methods. Ouyang et al. (2011) have used a regression style learning model to rank the sentences for query-based text summarization. Here, SVR is used as learning model. The technique is based on feature-based extractive framework. Extraction of sentences are done based on pre-defined features and a composite scoring function. Schilder and Kondadadi (2008) have used SVM model to rank the sentences based on two defined features. One feature is word-based and another is sentence-based. They present a fast query-based text summarization technique known as FastSum. FastSum is based on frequency of words of clusters, documents and topics. Word based feature is calculated based on cluster, document, topic title and description. Sentence based feature is computed depending on length and sentence position. Though both mentioned methods performs better than many baseline systems, but in future they can add more features based on structure of documents and relations between sentences. Peng et al. (2016) have used query-based text summarization technique while doing query reply in social networking sites. Summarization technique based on features like content weight of the sentences, location of the sentence, cosine-similarity between query and input sentence. Their technique also gives more attention in redundancy and noise removal in text messages as social network messages are highly unstructured. They can further enhance their method by incorporating text semantic analysis as features. Canhasi et al. (Canhasi and Kononenko (2014)) have applied a different matrix factorization method named as weighted Archetypal Analysis (wAA) for query-based text summarization purpose. In fact they have formalized the ogy, (2) expansion of query, (3) not only restricted to sentence to sentence relations and sentence to terms relations but also should consider n-grams, phrases and semantic role arguments levels. 2.3 Neural Network based Techniques Applied in Query-Based Text Summarization can be improved by incorporating more advanced techniques (Erkan and Radev (2004)).

Machine Learning Techniques Applied in
Query-Based Text Summarization chunking, named entity recognition, and semantic role labeling purpose. Application of deep learning techniques in text summarization have shown quite satisfactory results. In the works of Duraiswamy (2014), Verma and Nidhi (2017), their techniques use Restricted Boltzmann Machine (RBM) as an unsupervised approach where first two approaches used sentence level features and last approach used tf representations. In future, we can extract more features according to the user's requirement and also more hidden layers can be added in RBM. Yousefi-Azar and Hamey (2017) have used deep Auto-Encoder (AE) to calculate the optimum features based on tf representation of input text document. AE is a feed-forward neural network. AE is used as unsupervised feature learning technique. They propose an Ensemble Noisy Auto-Encoder (ENAE) where this ENAE adds noise to the input text documents and extracts top sentences from an ensemble of noisy runs. This method however is not implemented on newswire summarization. Cao et al. (2015) have used recurrent neural network to rank the sentences for multiple text document summarization. They have taken sentence ranking task as a hierarchical regression process. Ranking of words as well as sentences give more informative and redundancy free summary. This supervised technique can deal with variable length input for text summarization. It calculates the importance of text document sentences as well as phrases. However, their method is only restricted to generic text summarization. Zhong et al. (2015) use unsupervised deep learning technique in creating summary from multiple text documents. A query-oriented extracted sentence technique is proposed to extract the meaningful sentences as per the query requirements. In reconstruction validation phase, the whole deep architecture is adjusted by minimizing the loss of information. Finally, to maximize the importance of a sentence for including in final summary, dynamic programming is used. On the basis of the performance comparison of DUC 2007 datasets, the performance of this method is better than many baseline systems. However, the recall measure is much low than ranking SVM classifier. Afsharizadeh et al. show that use of suitable features can enhance the summary quality. They have used 11 important and best features to extract the sentences for summarization (Afsharizadeh et al. (2018)). The main drawback of their approach is that the lack of semantic similarity computation. For summarization of scientific documents, an interactive query-based approach is proposed by Bayatmakou et al. (Bayatmakou et al. (2021)). Initially, query is refined by user selected keywords or keyphrases. In the second step, system extracts the candidate sentences based on the keywords or keyphrases selected by the user in first step. Sys- (Raffel et al. (2019)), T-NLG (Rosset (2020))and GShard (Lepikhin et al. (2020)) are widely used in text summarization. Risne and Sitova have proposed two summarization approaches: Extractive and Abstractive. In their method, extractive summarization uses submodular functions and the language representation model BERT and abstractive summary uses the language model GPT-2 (RISNE and SIITOVA (2019)). Lamsiyah et al. use order and semantic relation based BERT sentence embedding model (Lamsiyah et al. (2021)). This unsupervised extractive text summarization method uses transfer learning from BERT. However, the high energy consumption and large inference latency are the most serious bottlenecks of transformer based language model (Strubell et al. (2019)).
Recently neural network based models are used widely in abstractive text summarization. Bahdanau et al. (2014) have used neural network for machine translation. Rush et al. (2015) have proposed a fully data-driven abstractive approach by using an attention based feed-forward neural network language model (NNLM). Developing on their work, In their follow-up work, Nallapati et al. (2016a) put forward a set of attentional Encoder Decoder Recurrent Neural Networks. Later on, they extend this work by giving focus on different critical text summarization issues such as modeling key-words, capturing the hierarchy of sentence-to-word structure, and emitting words that are rare or unseen at training time Now-a-days, transformer (Vaswani et al. (2017)) based language models like Bidirectional Encoder Representations from Transformers (BERT) (Devlin et al. (2018) tem also expands the sentences by using Genetic Algorithm (GA) (Zhao and Tang (2010)). Finally, Maximal Marginal Relevance (MMR) algorithm is used for ranking the sentences (Carbonell and Goldstein (1998)). Although proposed method gets o higher degrees of user satisfaction but also it has high computational time and cost complexity. Murarka and Singhal (2020) use a hybrid approach for query relevant single document text summarization. They have used Latent Semantic Analysis (LSA) (Yeh et al. (2005)) technique to to make an intuitive semantic structure. They have also used PageRank algorithm to remove the redundancy (Thakkar et al. (2010)). Their method works well in comparision to many graph-based and semantic-based methods. In future, a semi supervised model can be developed to increase the semantic efficiency of the summary. (Nallapati et al. (2016b)). To do extensive experiments and comparison with other summarization systems, they also propose a novel dataset. Lopyrev (2015) uses an encoder-decoder recurrent neural network with LSTM units and attention to generate abstractive based news article headline. Filippova et al. (2015) use word embeddings and Long Short Term Memory (LSTM) models to obtain a readable and informative rich compression.
In spite of increasing rate of abstractive summarization techniques, extractive summarization techniques are still attractive as they are less complex and inexpensive. In fact, abstractive summarization needs good command of the subject and natural language which both can be difficult task for a machine. Extractive technique provides grammatically and semantically correct summaries in most cases. Cheng and Lapata (2016) have proposed a data-driven approach based on neural networks and continuous sentence features. They proposed a framework for single-document summarizer based on general hierarchical document encoder and an attention-based extractor. Nallapati et al. (2017) have used a recurrent neural network based sequence model for extractive summarization of documents. Their proposed interpretable neural sequence model allows intuitive visualization. Both these deep learning based supervised methods (Cheng et al. Cheng and Lapata (2016) and Nallapati et al. Nallapati et al. (2017)) suffer from domain adaptation issues when they tested on different corpuses. A query-focused text summarization system is developed by Cao et al. (2016) in which both query relevant ranking and sentence saliency ranking tasks are tackled jointly. System is composed of CNN (Convolutional Neural Networks), Pooling and Ranking layers. It is clear from the above mentioned works that though they try to find query-based text summarization but less importance is given to query dependent features. Without these query-dependent features, their performance variation is quite negligible. Hence, these methods fail to find query relevant sentences.
This paper also gives focus on redundancy removal methods. We use it for query-based multi-document text summarization system as probability of redundant information is much more in there. The most important part of removal of redundancy is to find similarity measure. In case of redundancy removal we have to find out how dissimilar two sentences are. There are numerous ways to find similarity between two sentences. Here, we only consider redundancy removal techniques particu-larly for query-based multi-document text summarization. Abdi et al. (2017) have used a redundancy removal technique at summary generation step. According to their method, sentences are ranked using a graph-based ranking model and sentences are selected having highest score values until it get the required summary length. Summary contents multiple extracted sentences from several contents. Since, various sentences may have similar content, it is required to reduce redundancy. To tackle this problem, two main levels are employed: first one is scoring level and second one is comparison level. Before adding a sentence to the final summary, it is compared with significant sentences which are already in final summary. If the sentence is not too similar with other significant sentences, then only it is allowed to be in final summary. They use a greedy algorithm (Wan et al. (2007), Zhang et al. (2005)) to impose a diversity penalty on the sentences. Lloret and Palomar (2013) have used three different levels of language analysis to tackle redundancy in text summarization. Different levels of language analysis are: lexical, syntactic and semantic. Cosine similarity is used in lexical-based redundancy detection method. Textual entailment is used in syntactic-based redundancy detection method and sentence alignment is used in semantic-based redundancy detection method. Result shows that better summary when employing syntactic or semantic based redundancy removal approaches. Mei and Chen (2012) have proposed a soft clustering method for query-based text summarization. A fuzzy medoid-based clustering method is used to create subsets of sentences where each of the subset corresponds to a sub topic of the related topic. Sub topic based feature captures the relevance of each sentence within different sub topics. Finally, it increases the chance of creating the summary with wide coverage and less redundancy. Ye and Wei (2008) have introduced a statistical model for query-oriented summary generation where Maximum Marginal Relevance (MMR) is used to reduce the redundancy. MMR strategy is used by many researchers in finding redundancy in query based text summarization. Ouyang et al. (2011) and Li and Li (2013) also use MMR technique to minimize redundancy. Peng et al. (2016) use simpson distance to find how similar two sentences are. From this survey, it is found that though many different techniques are applied in query-based text summarization, still sense based semantic relatedness measure is not used yet to find the query-based summary.

Proposed Redundancy Free Query-Based Extractive Text Summarizarion Method
The steps to create query-based text summary are described briefly here.

Literature Review on Redundancy Removal in Query-Based Text Summarization
-Pre-processing: To reduce computational time, preprocessing is done on text sentences by applying various techniques proposed by linguists. Pre-processing removes unwanted words from text document and makes it a lighter one. Following techniques are applied to do the pre-processing of text document: Parts of tagging: To classify the words on the basis of parts of speech category, parts of speech tagging (Bird et al. (2009)) is done. Parts of speech tagging classifies the content words. Content words include noun, adjective, verb and adverb. Named Entity Tagging: To distinguish different Names person, location or organization wise, we do the named entity tagging Bird et al. (2009). We will not consider person's name to find semantic relatedness as it is not present in Lexical resources. Stop word removal: It is better to filter out a, an, the, in type of words which do not give any semantic meaning to the sentence. This is known as stop word removal in text mining applications. Here, we use stop word list stores in Natural Language Tool Kit (NLTK) in python.
Stemming: Finally, stemming is done to the content words. Stemming brings the word to its root or base form. For example to convert a word from plural to singular root form (girls to girl) or removing ing from a verb (singing to sing). Number of algorithms are available to do the stemming in natural language processing. -Content word disambiguation: To find out the sense of a word present in the query and input text documents, we take the collocation-based score. Word sense disambiguation is done to improve the accuracy of the sense-oriented sentence semantic relatedness score between query and input text sentence. In section 5 the application of collocation-based feature for finding sense of a word is described.
ness score calculation is presented. -Redundancy free query-based text summary generation: On the basis of distinct features, clusters are created. Each cluster contains the query relevant sentences. There is a high probability of presence of redundant sentences in each cluster. Therefore, a redundancy free sentence extraction method is proposed to create query-based text summary. The method to find redundancy free sentences is already defined in section 7.
The overall process for creating redundancy free querybased text summary is shown in the Algorithm 1.
Data: Query (Q) and Input Text Documents (T ) Result: Query-Based Text Summary 1 Do the pre-processing of input text documents and query described in section 3 2 Do the word sense disambiguation of each content word by using the Algorithm 2 3 Find out the query relevance sentences by using the Algorithm 5 4 Create the redundancy free query-based extractive text summary by using Algorithm 6 and 7 Algorithm 1: Steps to Create a Redundancy Free Query-Based Extractive Text Summary

Importance of Sense for Finding Semantic Relatedness Measure
Query based text summarizer extracts semantically query related sentences from input text sentences. In most cases, to find the semantic relatedness score between two words using WordNet, existing measures find the score with all the senses and give maximum score. WordNet is the most commonly used English lexical database used as a resource for English sense relations created by Miller (Charles (1988)). WordNet is a lexical dictionary which is used to find semantic similarity/relatedness score between two content words. A distinct difference is drawn by Resnik (1995) and Budanitsky and Hirst (2001) between semantic similarity and semantic relatedness. Similarity states near synonyms or can say as roughly substitutable in context. Relatedness implies larger set of potential relationship between words. Antonyms are highly related words but they less similar. Human and women are not similar but highly related than human and car while girl and women have similarity. Thus Similarity is a sub case of Relatedness. We prefer to use semantic relatedness score to find query relevance sentences. Semantic similarity is based on taxonomic relations (based on their meaning, it includes "is a" relations) whereas semantic relatedness is a more general concept. Semantic relatedness score between two content words can be found using WordNet. Miller started the use of WordNet in 1985. This English data-base is designed by Cognitive Science Laboratory of Princeton University. WordNet consists of three separate databases: one is for each Nouns and -Calculating sense-oriented sentence semantic relatedness score between query and input text sentence: To get the query relevant sentences, sense-oriented sentence semantic relatedness score is calculated between query and input text sentence. Here, we are taking 60% as threshold value. We assume that sentences having higher than or equal to the threshold value are all equally important for creating querybased text summary. In section 6, description of the proposed sense-oriented sentence semantic related-Verbs and another is for adverbs and adjectives. Number of content words listed in WordNet 3.0 is shown in following Table 1: In English language, most of the words have more than one senses. In WordNet, concepts are represented by the word, its part of speech and its sense number. For example: for the word bank, the concept is bank#n#1. It means the word bank is a noun here and it has the first sense. Concepts in WordNet are linked together in a hierarchical structure. In Fig. 1 an example of words present in WordNet is shown.
In WordNet, for different types of part of speech of a content word, we get different senses. Senses are the gloss or the definition. For a content word, if it has more than one sense then different number senses have different glosses. A content word may contain different senses for a same part of speech. Table 2 says about different gloss of word interest present in WordNet. Each sysnset of interest contains its parts-of-speech and sense number. Gloss implies a dictionary-style definition.
Ambiguous word always degrades the quality of information retrieval. Ambiguity in words minimizes the query relevance for query-oriented text summarization. According to Jurafsky and Martin (2014), the task of selection of correct sense for a word is known as word sense disambiguation. Ambiguity removal is an emerging problem in natural language processing and ontology. Sense makes an important role while finding semantic relatedness score between two content words. Two words can be said as semantically related/similar if there exist any relation between the words that are described in WordNet. Following Table 3 relations are present in WordNet to find semantic relatedness as well as similarity between content words. WordNet takes only the content words (noun, verb, adverb and adjective) as they carry vital information in a sentence.
For example: we take two sentences: (1) Ram went to the bank to deposit money and (2) Ram went to the bank of river Brahmaputra. Here, we find the semantic relatedness score between the content words of two sentences. In this example: we take two words that are bank and river where: bank word comes from the first sentence and river word comes from the second sentence. Both bank and river words are noun here. While finding the semantic similarity score, we have to give the word, then it's part of speech and sense number. When we do not give any particular sense as an input, WordNet takes automatically that sense for which it gets the highest semantic similarity score. Using Word-Net lexical dictionary, we get the following semantic relatedness scores for different measures listed in Table  4. Table 4 shows that by default almost all semantic relatedness measures take the first sense of bank as it gives maximums score with river. Tables 5 provides the different senses present for the word bank. For the word river only one noun sense is present in the WordNet described in Table 6. Table 7 shows the trace definition present for bank#n #1, bank#n#2, bank#n#3 and river#n#1. Trace definition shows how the word is present in WordNet taxonomy. From these tables, it is quite clear that though the word bank is actually related with the financial institution, here, by default all semantic relatedness measures take an incorrect sense of bank.
From this survey it is found that, till not no semantic relatedness or similarity measure is there in which score can be found on sentence level. In fact, a word should take that sense which is suitable with context of the sentence. Existing semantic measures depend on individual words rather than complete sentence. Therefore, it is important to find exact sense of both words for which the senses are appropriate for the sentences.
It is quite clear that semantic measures needs actual sense while finding its score between two words. Scores obtained from different measures are not appropriate. Default sense does not give the assurance of accuracy of score for specific sentences. A word's sense depends on other words present in the same sentence. Sense detection is important to increase the accuracy of measures. Therefore, finding sense of a word is much essential to get accurate relatedness score between two words as well as between two sentences.

Proposed Unsupervised Method for Word Sense Disambiguation (WSD)
The overall process for finding the sense of a target word is shown in the following Fig. 2: To get the correct sense of a word, we take collocation feature. Following section shows how collocation feature can be applied for finding word sense.    (2006)) is used. The cooccurrence between two terms is calculated by finding its bi-gram frequency. Collocation gives the associativity between two words. For example: car and bike are two words that are semantically similar. They have some common features like wheels or have common function like transport. In contrast, car and petrol both are associated as they occur frequently in language and The relatedness of bank#n#1 and river#n#1 using vector pairs Li et al. (2009) 0.0353 The relatedness of bank#n#3 and river#n#1 using vector Li et al. (2009Li et al. ( ) 0.1958 The relatedness of bank#n#1 and river#n#1 using hso Hirst et al. (1998) 0 The relatedness of bank#n#1 and river#n#1 using Adapted lesk Banerjee and Pedersen (2002)  16 The relatedness of bank#n#1 and river#n#1 using res Resnik (1995) 0.6144 The relatedness of bank#n#1 and river#n#1 using lch Leacock and Chodorow (1998) 1.4917 The relatedness of bank#n#1 and river#n#1 using lin Li et al. (2003) 0.0782 The relatedness of bank#n#1 and river#n#1 using jcn Jiang and Conrath (1997) 0.0691 The relatedness of bank#n#1 and river#n#1 using wup Wu and Palmer (1994) 0.4286 The relatedness of bank#n#1 and river#n#1 using path Rada et al. (1989) 0.1111    (1995)). To find the bi-gram collocation score for each sense of word w 1 (McKeown and Radev (2000)), we find frequency of occurrence of words present in the sense definition for w 1 with other words present in the sentences and take the maximum one. For example, we take one sentence Mary treated John for his injuries. To find the exact sense of the word treat, we first find out the all the senses present for treat in WordNet. Senses are the glosses or the definitions. The method finds the collocation score of each word present in the gloss with the word present in the sentence. Here, one gloss for treat is interact in a certain way. The content words present in the sentence is injuries and in the definitions are interact, certain, way. After finding collocation score of interact, certain, way with injuries, the method takes the highest score. In this way method will calculate for each sense and finally we take that sense for which the collocation score is maximum. The proposed WSD method will work for all the content words present in WordNet except the person's name. We will not consider the person's name, but of course, we will include an organization or location's name using standard Named Entity Tagger (Perkins (2014)). The proposed method considers mainly the content words as they carry the salient information. Content word includes noun, main verb, adjective, and adverb. First, the proposed WSD method finds all the senses of the target words present in the WordNet. Senses are the glosses. For each sense of a target word, we have removed the stop words. We also remove the stop words from the sentence in where target word is present. To find the collocation score between two words (one word is from the gloss of w and other word is from the sentence w'), we use the following equation 1. Here, Wikipedia corpus is used (Denoyer and Gallinari (2006)). While finding the collocation extraction score, we are giving the flexibility that if the two words are not together in Wikipedia, we increase the window size upto 3. We consider the span size as 3 because it works best for our proposed WSD method. We will search for bi-gram frequency where the words may be separated by three other words in the text of Wikipedia. To find the sense of a word present in a sentence, initially, we get a set of probable senses of the word. Now for each sense, we calculate the collocation score between each content word of every gloss of a sense with all other content words present in the same sentence. Same process will be followed for every sense and finally we take that sense for which the collocation score is maximum. The collocation score (CS) of a sense (gloss) for a target word (TW) present in the sentence is: After finding the collocation score of the set of all senses of TW, we consider that sense of TW for which the collocation score is maximum.

Finding Exact Sense of a Word Present in a Sentence
The proposed WSD method is implemented to find the sense of a word which will further helps in calculating semantic relatedness score between query and input text sentences. Following Algorithm 2 gives the systematic steps to find the sense of a word:

Detailed Description of the Sense-Oriented Semantic Relatedness Calculation between Two Sentences
Relatedness implies larger set of potential relationship between words. Antonyms are highly related words but they are less similar. Human and women are not similar but highly related than human and car while girl and women have similarity. Thus Similarity is a sub case of Relatedness. We prefer to use semantic relatedness score to find query relevant sentences. It is quite clear that semantic measures needs actual sense while finding its score between two words. A word's sense depends on other words present in the same sentence. In case of semantic relatedness calculation, default sense does not give the assurance of accuracy of score for specific sentences. Hence, sense detection is important to for each sense (s) of T do Do the pre-processing of s Do the pre-processing of OT Find out the collocation score between s and OT by using equation 2 6 7 8 9 10 11 15 end Extract the sense number and the gloss of T for which we get the maximum collocation score if T has more than one sense having same maximum collocation score then Extract the sense number and gloss which has lower sense number To find the relatedness between two sentences, semantic relatedness score is calculated between two words present in both sentences. Semantic relatedness mea- Get the sense number and the gloss end words, we find the exact sense of that word.
We consider only the content words while calculating the semantic relatedness score between two sentences. Initially, pre-processing step removes the stop words. Stemming is also done to get the root form of a word. Before finding the relatedness score between two sure gives the relatedness score between two sentences on the basis of the meaning of the sentences. This relatedness measure is based on word to word relatedness.
increase accuracy of measures. To find sense-oriented semantic relatedness score between query and input text sentences we combine sentence semantic relatedness score, sense relatedness score and word order relatedness score. Semantic relatedness measure gives how much two sentences are related to each other. Sense relatedness measure defines how much two sentences are related on the basis of its sense definition and word order similarity gives information relations between sentences. All these semantic and same sequence of words information play an important role while finding senseoriented sentence semantic relatedness score between two sentences. In the following section, sense-oriented semantic relatedness score calculation is shown sequentially.
w2 is: This relatedness score uses Path Weight measure (Pedersen et al. (2004)) (Hirst et al. (1998)). It is a semantic relatedness measure between two words based on path which is described in WordNet lexical dictionary. It is found from the literature survey that this measure includes more relations and can find relatedness score between different part of speech (Patwardhan et al. (2003)). It classifies the relations in WordNet as upward, downward or horizontal directions. Higher score implies the shorter path length and less changes of directions. While finding the semantic relatedness score between two words, the proposed method will give the accurate sense number and its part of speech along with the respective words. The equation to find Semantic Relatedness Score (SEM R S) between two words w1 and Here, c and k are the constants and values are c = 8 and k = 1. The maximum semantic relatedness score between two word is 16 which signifies that two content words are identical. The minimum score is 0 which signifies there is no semantic relatedness between them. Following equation 4 is used to find the Semantic Relatedness Score (SEM R S) between two sentences s1 R S (s1, s2) = SEM R S((w1, sense no 1, p o s 1) w1∈ s1,w2∈ s2 , (w2, sense no 2, p o s 2)) The overall process to find semantic relatedness score between two sentences is shown in the Algorithm 3.
(4) M aximum relatedness score the act of going to see some person or place or thing for a short time Synset('visit.n.02') a meeting arranged by the visitor to see someone (such as a doctor or lawyer) for treatment or advice Synset('visit.n.03') the act of visiting in an official capacity (as for an inspection) Synset('visit.n.04') the act of going to see some person in a professional capacity Synset('sojourn.n.01') a temporary stay (e.g., as a guest) Synset ('visit.v.01') go to see a place, as for entertainment Synset ('travel to.v.01') go to certain places as for sightseeing Synset('visit.v.03') pay a brief visit Synset ('visit.v.04') come to see in an official or professional capacity Synset('inflict.v.01') impose something unpleasant Synset('chew the fat.v.01') talk socially without exchanging too much information Synset ('visit.v.07') stay with as a guest Synset('visit.v.08') assail Data: Two Sentences (s1, s2) Result: Semantic Relatedness Score between s1 and s2 (SEM R S(s1, s2)) 1 Do the pre-processing of s1, s2 by using the steps mentioned in section 3 2 for each content word w1 in s1 and w2 in s2 do 3 4 end 5 for each content word w1 and w2 from s1 and s2 do the Algorithm 2 Find out the sense number of w1 and w2 by using 6 Find out the SEM R S ((w1,sense no 1,p o s 1), (w2,sense no 2,p o s 2)) by using the equation 3 7 end 8 Find the SEM R S(s1, s2) between s1 and s2 by using the equation 4 Algorithm 3: Steps to Find Semantic Relatedness Score between Two Sentences

Finding Sense Relatedness Score between Two Sentences
We have already discussed that finding relatedness measure between two sentences on the basis of its sense is quite important. Proposed WSD method finds the sense of each content word present in both text sentences by using the mentioned method described in chapter 3.5.2. After doing the pre-processing of two sentences as mentioned in section 3.5, we have disambiguated the sense of each content word if it has more than one sense. We get the gloss for each content word. We will do the stop word removal and stemming on the content words to get the root form of the content words present in the gloss.
Here, we find the sense relatedness score between two sentences by finding the common content words present in the gloss of a sense of a content word in the first sentence with the words present in the second sentence. The method uses the equations 5, 6 and 7 to find the sense relatedness score (S R S) between two sentences s 1 and s 2 .
{Sense def inition of w ′ ∩W ords present in s 1 } S R S (s 1 , s 2 ) = S R S 1 (s 1 , s 2 ) + S R S 2 (s 1 , s 2 ) T otal number of content words present in s 1 and s 2 The overall process to find the sense relatedness score between two sentences is shown in the Algorithm 4.
Data: Two Sentences (s1, s2) Result: Sense Relatedness Score between s1 and s2 (S R S(s1, s2)) 1 Do the pre-processing of s1 and s2 by using the steps mentioned in section 3 2 Find out the gloss of the sense of the content words by using the Algorithm 2 3 Do the pre-processing of the gloss of the sense of the content words by using the steps mentioned in section 3 4 Find the S R S(s1, s2) between s1 and s2 by using the equation 7 Algorithm 4: Steps to Find Sense Relatedness Score between Two Sentences

Finding Word Order Similarity Score between Two Sentences
On the basis of same sequence of words present in the two sentences, word order similarity provides how much two sentences are similar. Finding longest common substring between two sentences adds more impact on similarity measure. Word order similarity method depends on the order of words present in both sentences. It helps in signifying the relatedness between two sentences though they share same words. Example: (a) Ram killed Shyam and (b) Shyam killed Ram. These two sentences use same content words but we can see that sentence a and sentence b have opposite meaning. Longest common substring can easily distinguish that meaning of sentence a is not similar to the meaning of sentence b. We can take another example: sentence 1 is Narendra Modi's visit to China and sentence 2 is Ram Nath Kovind's visit to China. Both sentences are different though they carry maximum common words. Hence finding longest common substring can identify the differences present in both sentences. It can also find similarity between two sentences if numerical data present in both sentences in a same order. Ontology based semantic relatedness measure can not find this type of similarity as lexical dictionary does not contain numerical data or some proper nouns. Example: sentence 1: In 2006, Ram came to Assam to meet his friend Rahim and sentence 2: In 2006, Ram came to Guwahati to meet Rahim. Here longest common substring is In 2006, Ram came to. Hence, it also helps in finding similarity for numerical and proper noun words. Here, we will not do any pre-processing task. The method uses the following equation 8 to find word order similarity between two sentences s 1 and s 2 : Here, NCW=Number of Common Words between s 1 and s 2 TNWLS=Total Number of Words present in the Longest Sentence between s 1 and s 2

Finding Sense-Oriented Sentence Semantic Relatedness Score between Two Sentences
The sense-oriented sentence semantic relatedness measure between the two sentences (query and input text sentence) is the combination of all these three measures: semantic relatedness, sense relatedness and word order similarity. The sense-oriented sentence semantic relatedness score between sentences s 1 and s 2 is given in equation 9: Here α + β + γ = 1 and α, β and γ are weighting parameters. They specify relative contributions to the sense-oriented sentence semantic relatedness measure of semantic, sense and word order measures. As semantic information caries more relevant information, therefore more weightage is given to semantic relatedness measure Wiemer-Hastings (2000). Considering the view that the word order information plays a subordinate role in finding relatedness between sentences, hence the weightage given to word order information is minimum.
The overall process to find sense-oriented sentence semantic relatedness score between two sentences is shown in the Algorithm 5.
Data: Two Sentences (s1, s2) Result: Sense-Oriented Sentence Semantic Relatedness Score (Sense Sem Rel) of s1, s2 1 Find semantic relatedness score between s1 and s2 by using the Algorithm 3 2 Find sense relatedness score between s1 and s2 by using the Algorithm 4 3 Find word order similarity score between s1 and s2 by using the equation 8 4 Find sense-oriented sentence semantic relatedness score between s1 and s2 by using equation 9 Algorithm 5: Steps to Find Sense-Oriented Sentence Semantic Relatedness Score

Generation of Redundancy Free Text Sentences
To find the redundant sentences from the highly semantically related input text sentences with query, we select certain features thorough which different clusters are created. We have taken five relevant features which are important for generating query-based text summary. Five features exploit a set of characteristics from the input text sentences. From the literature survey, it is seen that if a sentence caries five features, then the sentence must be an informative one Widyassari et al. (2020). Our main motive is to extract those semantically related non-redundant sentences which contain features. As our dataset is a newswire dataset, therefore, these mentioned features are highly feasible which will gives us an informative summary. The five features listed as: 1. Proper noun: Presence of proper noun or entity name in a sentence indicates an important sentence. We use a part of speech tagger (Bird et al. (2009)) to tag the proper nouns. 2. Numerical data: Numerical data gives more significant information in a textual document and consider it as an important one. 3. Thematic word: Frequently occurring words are considered as a thematic words. Thematic word makes the sentence an informative one. Here, we take top ten most frequently used words present in the text document. 4. Cue phrase: Cue phrase like this letter, this report, summary, in conclusion, argue, purpose, development are added as an important sentence. We use the English cue phrase list. 5. Sentence Length: We consider the length of the sentence.

The Process of Creating K Clusters from Sentences
Here, pf s = frequency of occurrence of proper noun in the sentence nf s = frequency of occurrence of numerical data in the sentence tf s = frequency of occurrence of thematic word in the sentence cf s = frequency of occurrence of cue phrase in the sentence sl s = sentence length of the sentence pf c = frequency of occurrence of proper noun in the centroid nf c = frequency of occurrence of numerical data in the centroid tf c = frequency of occurrence of thematic word in the centroid cf c = frequency of occurrence of cue phrase in the centroid sl c = sentence length of the centroid We have to update the centroid value. The equation for updating the centroid value after each iteration is: We have created the clusters of sentences based on the frequency of occurrence of those features present in the sentence. To create clusters, we use K-Means clustering algorithm on text data Andrews and Fox (2007). We remove the stop words and have done the stemming on the sentences. We represent all the sentences by a corpus vector based on the frequency of occurrence of five mentioned features. Initially, five centroids are chosen randomly by the K-Means algorithm {C 1 , C 2 , C 3 , C 4 , C 5 }. The next step is to calculate the Euclidean distance between the other sentences from the each five centroids. The equation to find the Euclidean distance between one sentence vector [pf s , nf s , tf s , cf s , sl s ] and one centroid vector [pf c , nf c , tf c , cf c , sl c ] is: Here, The K-Means algorithm will repeat the whole process until it converges. (12)

Proposed Method to Extract Redundancy Free Sentences
There is a high probability of presence of redundant sentences in different clusters. We need to remove the redundant sentences. If two sentences are highly semantically related, then one of the sentences is redundant. We have already proposed one sense-oriented semantic relatedness measure between two sentences in section 6. We have taken threshold value as above 60%. If the sense-oriented semantic relatedness measure between two sentences is above 60%, then we will remove the redundant sentence. The overall process to remove redundant sentences from text documents is shown in the Algorithm 7.
The quality of cluster c i can be measured by the within cluster variation, which is the sum of squared error between all objects in c i and the centroid where E is the sum of the squared error for all sentence vector in the data set; V S ′ is the point in space representing a given sentence vector; C i is the centroid of cluster c i (both V S ′ and C i are multidimensional).
The overall process to create the clusters with redundancy free sentences is shown in the Algorithm 6. In the Algorithm 7, similarity of a sentence is not computed with all the sentences in all other clusters. Similarity is computed among the sentences present in a cluster. Highly query relevant sentences are extracted to form the initial summary that may contain redundant sentences. Clusters are created for the purpose of removing the redundancy. Each cluster contains similar sentences. Therefore, only a few sentences of a cluster will be selected for inclusion in the summary. Here comes the need for computing similarity of one sentence with other sentences in the cluster. If the similarity of the sentence with any other sentence is found The proposed algorithm first selects the sentence at the top of the text in a cluster and it calculates the semantic relatedness score with each subsequent sentence present in that cluster. If the score is above threshold value, then the other sentence is removed from the cluster. Thus, some sentences may be removed from the cluster in the first pass. Then the control moves to the next sentence below the top sentence which is not removed. The second pass to remove redundant sentence begins. Semantic relatedness score of this sentence with all remaining subsequent sentences are calculated and those sentences having semantic relatedness score above threshold value are removed. The process continues until the last sentence of the cluster is examined for redundancy removal. In the same way redundancy removal is done for all the clusters. Set of sentences remaining in the clusters form the redundancy free summary in the original order. The result depends on the order of the sentences. The algorithm proceeds from top to bottom. If the order of the sentences in the text is changed, then to be greater than a threshold then that sentence is rejected. Clusters are created based on five different features (proper noun, numerical data, thematic word, cue phrase and sentence length). We know that if sentence carries above mentioned five features, then the sentence must be an informative one (Widyassari et al. (2020)). Each of the sentences in the initial summary is represented by frequency of occurrence of the five features, then k-means algorithm is used for clustering. Here k is considered as an user input. Value of k should not be so large, a value in the range [5-10] should be sufficient. We have used 5 clusters in the experimental evaluations.
Say, a cluster contains 3 sentences S1, S2 and S3 and Sentence Semantic relatedness Score(S1, S2) is 0.6 and Sentence Semantic relatedness Score(S1, S3) is 0.3 and Sentence Semantic relatedness Score(S2, S3) is 0.7. If threshold value is 0.5, S1 and S3 sentences will be finally selected from the cluster. Let us assume that the algorithm starts with the S1 sentence. The value of Sentence Semantic relatedness Score(S1, S2) is 0.6. It is greater than the threshold value; therefore, S2 will be removed. The value of Sentence Semantic fore, the proposed method keeps S1 and S3 sentences.
relatedness Score(S1, S3) is 0.3 which is smaller than the threshold value. Sentences S3 will be kept; there-

Evaluation of Proposed Word Sense Disambiguation Technique
Different widely used supervised and knowledge-based word sense disambiguation systems are are included for comparison purpose. All the supervised systems use the same corpus SemCor and Semcor+OMSTI for training purpose. This gives a fair comparison. Supervised WSD systems includes the following techniques: -Support Vector Machine (SVM) (Zhong and Ng (2010)) classifier is used in It Makes Sense (IMS) Word Sense Disambiguation (WSD) system. Different features like: surrounding words, PoS tag of surrounding words and local collocations are taken.
P recision = N umber of correctly disambiguated answers

Evaluation Metric
-A baseline method is included in Table 9 by taking the Most Frequent Sense (MFS) heuristically. Senses are selected based on the highest number of occurrences in the training corpus (Raganato et al. (2017b)).

Knowledge-based system includes following three Word Sense Disambiguation models:
-Lesk is simple knowledge based WSD algorihm (Lesk (1986)) which finds similar words between the definition of each sense with context of the target word. For comparison purpose, extended version of Lesk algorithm is used where definition of related senses are also included. Here, for word weighting, conventional term frequency-inverse document frequency (2017b)). -Now a days, Neural language based model is used widely for WSD task (Melamud et al. (2016), Kågebäck and Salomonsson (2016), Yuan et al. (2016)). Our experiment uses bidirectional LSTM (Melamud et al. (2016)) model. This context2vec neural model learns a generic embedding function for variable length contexts of target words.
its semantic similarity or relatedness score.
We use F-Score criteria for evaluation metric by using the equation 15 (Dongsuk et al. (2018)). It is a harmonic mean of precision and recall value shown in equations 13 and 14.
-A baseline method is chosen to select first sense as a correct sense present in WordNet 3.0 (Bird et al. (2009)). All the existing semantic similarity or relatedness measure uses the first sense while finding -In the works of (Taghipour and Ng (2015), Rothe and Schutzë (2015), Iacobacci et al. (2016)), word embeddings is used. Different methods are proposed by Iacobacci et al. (2016) where word embedding is used in current supervised WSD systems. They have made a deep analysis how different parameters are affecting the performance of WSD system. Here, two best configurations are considered having one with surrounding words (IMS+emb) and other one without surrounding words (IMS -s+emb). In both methods, they integrates word embeddings by using exponential decay. To train the word embeddings, Iacobacci et al.'s suggested learning strategy and hyper parameter's are used (Raganato et al. Rank algorithm (Haveliwala (2002)) is used.
-Babelfy is a graph-based WSD approach where random walk is used to find connections between synsets (Moro et al. (2014)). Babelfy uses random walks with restart (Tong et al. (2006) over BabelNet Navigli and Ponzetto (2012)). Babelfy includes the whole document while finding its sense.
-A graph based WSD system is proposed by Agirre et al. (Agirre and Soroa (2009)) where random walk is used over a WordNet semantic network (Agirre et al. (2014)). In their method, a personalized Page is used (Sparck Jones (1972), Banerjee and Pedersen (2003)). For better analysis, word embedding is added in enhanced version of Lesk which helps in computing similarity between definition and context of the target word (Basile et al. (2014)).

Recall = N umber of correctly disambiguated answers
K=Number Of Words in the test set for which the algorithm made a prediction (13) N umber of all test instances Results in the Table 9 clearly shows that though many supervised methods outperforms existing knowledge-based methods, but our proposed WSD method ganato et al. (2017a)).
We first evaluate our proposed WSD (Word Sense Disambiguation) on publicly available English WSD corpora Senseval-2, Sensval-3 task1, SemEval-2007 task17, Sem Eval-2013 task 12 and SemEval-2015 task 13 (Ra- DUC produces query-oriented text summarization with 250 words. Pre-processing part is done on the sentences. It includes stop word removal and stemming. The popu-  (1998)(1999)(2000) and Xinhua News Agency (1996)(1997)(1998)(1999)(2000). Our proposed method is compared with with existing methods participated in DUC 2005DUC , 2006DUC and 2007 suk et al. (Dongsuk et al. (2018)) and supervised systems are: Zhon et al. (Zhong and Ng (2010)), Weissenborn et al. (Weissenborn et al. (2015)), Raganato et al. (Raganato et al. (2017b)), Pasini et al. (Pasini and Navigli (2017)). From experimental results, it is seen that our proposed WSD method outperforms all listed WSD systems for SemEval-2013 dataset. For the SemEval-2015 dataset, our proposed WSD method has similar performance to the supervised Weissenborn et al. method. However, in terms of macro average F-score of both datasets SemEval-13 and SemEval-15, the proposed WSD method shows higher performance for all WSD systems present in the Table 10. A macro-average calculates the metric independently for each class and then takes the average value. We compare our proposed WSD approach to other BabelNet-based unsupervised and supervised WSD systems (Dongsuk et al. (2018)). BabelNet is a multilingual lexical semantic network. It is automatically created by linking Wikipedia to the WordNet (Navigli and Ponzetto (2012) (2017)), Dongin Table 11. Documents present in each cluster of DUC Here, we have experimented our proposed algorithm with DUC ( Document Understanding Conference) 2005, 2006 and 2007 datasets (http://duc.nist.gov). We try to evaluate the effectiveness of proposed method with existing systems that perform experimental evaluation using DUC and also with the current related systems.
9 Dataset Description for Query-Based Text Summarization Method has performed much better for different datasets. It is also noticed from the Table that the performance of WSD methods for all datasets are not uniform. A large performance gap is seen between the best and worst performing dataset. For dataset SemEval-07, performance is quite low for all WSD systems as this dataset is the most ambiguous one.   (2004)) intrinsic based metric has been adapted by NIST (National Institute for Standards and Technology) and widely used by many researchers for text evaluation purpose. Candidate summary and reference summary are compared in ROUGE. Different systems generate candidate summary and expert human made reference summary. A set of metrics are available in ROUGE for comparing different summaries. It measures the quality of summary in terms of overlapping units such as N-grams, word sequences and word pairs. N-gram Co-Occurrence Statistics, Longest Common Subsequence, Weighted Longest Common Subsequence, Skip-Bigram Co-Occurrence Statistics and Extension of Skip-Bigram Co-Occurrence Statistics are the five standard evaluation metrics available in ROUGE. ROUGE-N compares system generated summary and human summary on the basis of total number of common content words. N-gram recall measure is calculated using following equation: where N= length of N-gram Count match (N − gram) = Number of common N − grams co-occuring in both system summary and human summary Count(N − gram)= Number of N − grams present in reference summary Here, official metrics of ROUGE-1 (unigram-based), ROUGE-2 (bi-gram based) and ROUGE-SU4 are used along with 95 % confidence intervals within the square brackets. We use the average recall scores of ROUGE-1, ROUGE-2 and ROUGE-SU4 on DUC 2005, 2006 and 2007 datasets for our comparison purpose. of unstructured textual information, huge number of on the basis of ROUGE scores. In particular, to evaluate they have performed best on DUC 2006 datasets.
To further prove the performance of our method, we add other recognized and current state-of-the-art existing methods particularly on DUC 2006 for comparing with proposed Method. The following methods are selected for comparison (Abdi et al. (2017)): (1) QSLK by Abadi et al. in 2017(Abdi et al. (2017), (2) CTMSUM by Yang et al. in 2014(Yang (2014), (3) WAASum by Canhasi et al. in 2014 (Canhasi andKononenko (2014)), (4) Topical-N by Yang et al. in 2013(Yang et al. (2013), (4) Qs-MR by Wei et al in 2011 (Wei et al. (2011)), (5) SVR by Ouyang et al. in 2010(Ouyang et al. (2010), (6 ) LEX by Huang et al. in 2010(Huang et al. (2010). We select these methods as From the Table 14 and 15 it is seen that proposed ommend these weightage parameter values for further Method shows considerably better results while apply-analysis and comparison.
is quite comparable to supervised learning based algorithms. In real-world applications, labeled informations   (2000), semantic information caries more relevant information, therefore more weightage is given to semantic relatedness measure. As a result, using current DUC 2005 dataset, we get all ROUGE peak values. Therefore, we can rec-of the results are shown in Table 12: We evaluate our method for each α between 0.1 to 0.9 with β between .1 to .9 and γ between 0 to 1 with a step of 0.1 (e.g. α=0.5, β=0.3 and γ=0.2). We use a nested loop to estimates the values of α, β and γ where γ is outer most loop, β is a inner loop and α is innermost loop. In the first pass of outer loop when value of γ=0, then control enters into next inner loop where β varies from 0 to 1 and finally enters into inner most loop α, which is varied from 0.1 to 0.9. In second pass, again the outer most loop triggers the two inner loops. This repeats until outer most loop finishes. A sample Before tested the proposed method, we first focus on optimizing the all three ( α, β and γ) weighting parameters. α parameter signifies the importance of semantic relatedness information, where as β extracts the query relevant sentences sense wise and γ caries word order information. To set these weighting parameters, we select randomly set of documents from DUC 2005 dataset and run our proposed method to find optimal values of three parameters. All documents are considered as sentences. Pre-processing is done on the sentences to filter out unwanted words.
unlabeled data are available. It is much more practically feasible to construct text summarization under unsupervised learning framework. Therefore, conclusion can be derived from this analysis that our method is able to attain significant performances by providing the comparison with baseline and state-of-the-art systems by applying in different standard datasets.  (2019a) makes the sentences more informative. Other two querybased methods have less query relevant ranking. In fact TF-IDF caries short and simple sentences just having same keywords with query. Table 17 shows when multiple sentences are present in a query, our proposed method responds to all query sentences. Although all the needs present in the query are almost satisfied, the order of respond of each query sentence is not correct. We do not give much importance to the coherency issue as only a small part of DUC dataset contains such complex queries. In fact, it is also hard for human to read attentively when the With manual inspection, the extracted sentences in our proposed method have more detailed information rather than just the exact answer. For instance, when the query is asking about benefits of drug legalization, proposed method catches long sentences having rich information regarding how drug legalize can reduce the drug-related violence.  It is not always seen that our system has significant performance for each of the query. For example, we take this query from DUC 2005 Dataset, "Identify and describe types of organized crime that crosses borders or involves more than one country. Name the countries involved. Also identify the perpetrators involved with each type of crime, including both individuals and organizations if possible." Here, we have to find the name of involved countries. Our system is an extractive text summarization system and there is no such direct information present in the input text documents which can give the names of involved countries. Sometimes, we need to make our own sentence based on the information. Abstractive summary will help us in such situation. If the query is an indirect question, we have to rephrase the sentences. In that case, our system fails to achieve good performance. Another drawback of our system is that there is a lack of coherency among the selected sentences. We can find out the mentioned disadvantages in the following summary generated from our system for the above query: In the 1980s the international spread of drugs (cocaine from Latin America, heroin from Turkey, central Asia and the south-east Asia), advances in telecommunications, liberalisation of banking, freer trade and the opening of eastern Europe provided an enormous stimulus to the growth of Italian organised crime. A report released earlier this year by Censis, the social research institute, said the annual turnover of organised crimes' illicit and 'licit' activities was L19, 389bn. Other guesstimates have put the figure at over L35, 000bn. As Liliano Ferraro, a senior official with Italy's ministry of justice, commented last week: 'The organised crime syndicates have already held this kind of meeting, they just meet in a hotel in eastern Europe and divide the profits. Mr Freeh made it clear that he saw the burgeoning organised crime wave in Russia as being as much a threat to the US and the west as to Russia itself. 'We have to go where the crime is and not do what we did with respect to Italian organised crime and that was wait for 50 years before we got involved,' Mr Freeh said. Though his speech was a model of diplomatic balance, he stressed the many mistakes made by US law enforcement agencies in the original response to the growth of organised crime which 'permitted the expansion of a powerful, well-organised crime syndicate'. Much of the trade, which is centred on the St Lawrence River and Indian reserves which straddle the US-Canada border, has fallen into the hands of

Method Name
Extracted sentences for summary It acknowledges that illegal drugs cannot be kept out of the country by tougher border control and interdiction measures. Much greater resources, derived from taxation of the drugs that are now illegal and untaxed and from the billions saved by not wasting money on more criminal-justice measures, must be devoted to drug treatment and drug prevention. Attsum (Cao et al. (2016)) As is the case with tobacco, legalizing marijuana, cocaine and heroin would not signify an endorsement of their use. The consumption and production of marijuana in the United States is on the decrease, and that criminalization costs society more in terms of increased law-enforcement-related costs and deprived revenues from taxes on pot than legalization would. Drug prices have soared. Drug addicts are not welcome. TF-IDF (Cao et al. (2016)) How refreshing to have so much discourse on drugs and legalization. The only solution now is a controlled policy of drug legalization. The panel, appointed by state agencies and the governor, found that anti-drug laws have been "manifestly unsuccessful in that we are now using more and a greater variety of drugs, legal and illegal." Created by the Legislature in 1969 to study drug abuse and possible remedies, the panel recommended that the Legislature legalize cultivation of marijuana for personal use, consider decriminalizing other drugs on an individual basis and legalize possession of hypodermic needles.

Proposed Method
Ralph Lee White, a former Stockton councilman who was thrown out of office for bribing a voter, announced his candidacy for state controller on a platform of legalizing drugs."We should legalize drugs because it would stop the killing between gangs," White said at a Stockton news conference announcing his candidacy for the Democratic nomination, if incumbent Gray Davis runs for governor as expected. One notable aberration from the campaign's homogeneous tone involves Democrat Abeles' proposal that the use of drugs such as cocaine and methamphetamines be legalized, with the state regulating their production and distribution, as well as that of "legal drugs" such as liquor.
Originally, drugs were legal, and there was no drug-related violence.
To statistically compare the performance of our proposed redundancy free query-based extractive text summary method with other summarization methods, we use a non-parametric Wilcoxon's matched-pairs signed rank based statistical significance test to determine the significance of our results. The statistical significance test for independent samples has been conducted at the 5% significance level of the summarization results. Nine groups, corresponding to the nine widely used and cur-15 Statistical Significance Test characteristics -Cosa Nostra, the Sicilian Mafia; the Camorra, based round Naples and the 'ndrangheta of Calabria. organised crime. THE ITALIAN authorities yesterday achieved a breakthrough in the fight against organised crime with the capture of Mr Salvatore 'Toto' Riina, acknowledged to be the boss of Cosa Nostra, the umbrella organisation of the Sicilian Mafia. Mr Riina's arrest exemplifies the determination of the government of Mr Giuliano Amato to combat organised crime and break the long-standing protection given to the Mafia by elements of the Italian state and among the political parties. The son of a poor smallholder, Mr Riina has been involved in organised crime since his early teens and married into one of the traditional Mafia families of Corleone. Organised crime has traditionally been a 'southern' phenomenon, associated with three groups, each distinguished by clear geographical and historical Boparai also said that wetlands in many developing countries were vital to the sustenance of human beings, not just flora and fauna. EPA says that all water conservation projects,and agriculture and forestry development along China's major rivers must be assessed in accordance with environmental protection standards, and that no projects will be allowed if they pose a threat to the environment. Finland has agreed to help central Chinas Hunan Province improve biodiversity protection, environmental education, subtropical forestry and wetlands protection, according to provincial officials. The EPA had sought as early 1993 to subject all development on wetlands to strict environmental review, but that approach was rejected by the courts, which ruled in favor of arguments made by developers and by the National Mining Association.

TF-IDF
Statistics on wetlands loss vary widely. Mitigation of any impact on wetlands by creating or enhancing other wetlands.
The new regulations would cover about one-fourth of all wetlands. Now more and more people have recognized wetlands great ecological and economic potential and the conservation and utilization of wetlands has become an urgent task.

Proposed Method
Xu said his Ministry will launch a drive to enhance the awareness of wetland protection among the public, establish a dynamic monitoring network and information system for China's wetlands, conduct scientific research in wetland protection as well as proper utilization and management of wetlands resources.
The new organization links the Asian Wetland Bureau, Wetlands for the Americas and the International Waterfowl and Wetlands Research Bureau. Zhen said, for quite a long time, many people believed that the wetlands were useless mud. Now more and more people have recognized wetlands' great ecological and economic potential and the conservation and utilization of wetlands has become an urgent task. The four-day International Workshop on Wetland, sponsored by China's Ministry of Forestry, Japan's Environment Agency and Wetlands International will also arrange investigation of wetlands and water birds in Beidaihe.
each method for the data set are presented in Table  18. Table 18 shows that the median values of ROUGE-1 and ROUGE-2 for our proposed method on data set are better than that for the other methods. To establish that this goodness is statistically significant, Table 19 reports the P values produced by Wilcoxons matchedpairs signed rank test for comparison of two groups (one group corresponding to proposed method and another group corresponding to some other method) at rent methods: (1) LEX, (2) TMR, (3) SVR, (4) Topical-N, (5) QEMD, (6)Qs-MR, (7) CTMSUM, (8) WAA-SUM, (9) QSLK, have been created for data set. Two groups are compared at a time one corresponding to our proposed method and the other corresponding to some other method considered here. Each group consists of the ROUGE-1 and ROUGE-2 scores for the data set produced by each corresponding method. The median values of ROUGE-1 and ROUGE-2 scores of step stores the correct sense of each word.
The redundancy removal step will be of the order of O(N 2 A 2 ) assuming that word sense disambiguation of iterations I which are small.
The k-means clustering step will require O(N ) time, neglecting value of k, number of dimensions D, number will be O(N A 2 S 2 ).
To find query relevant sentences each sentence of the text is to be examined. Each sentence has on an average A word, and each word has S senses. Time complexity The over all multiple text document query-based text summarization method is affected by the evaluation process. DUC datasets only generate the summary with 250 words. That is one major drawback of text summarization evaluation system as it is much difficult to generate the summary for only 250 words which should match with human generated summary. For the redundancy and coherency issues, we have not found proper evaluation method through which importance of redundancy and coherency can be justified. Evaluation process is also affected by reference summary. Although a word, then time requirement will be O(N SA). rect summary.
Word sense disambiguation is done based on the words present in the sentence. If there are S senses for tence is A, and each word has S number of senses.
The processing steps consumes O(N A) time as each word of the text need to be visited.
Let, Total number of sentences in the text to be summarized is N , average number of words in a sen-Although the proposed method works better than many existing query-oriented text summarization methods, we can still improve its performance. In fact, a lot of potential for future work is seen based upon this research. As future work, we plan to identify between active and passive sentences which will help in minimizing redundant information and it will help in finding cor-ity analysis is shown below.
words which are not present in WordNet ontology.
The time complexity of the proposed query-based text summarization method depends upon total number of sentences in the input text to be summarized and the number of words present in an individual sentence. Both of the counts will not be so high. The complete method for the proposed query-based text summarization method is presented in Algorithm 1. The complex-

Complexity Analysis of the Query-based Extractive Text Summarization Using Sense-oriented Semantic Relatedness Measure technique
We have conducted extensive experiments on DUC query-based text summarization datasets. Proposed algorithm achieves competitive performance for all the best performing systems on DUC datasets and as well as with current state-of-the-art query-based text summarization systems. Method also finds those sentences which are relevance in terms of query sense. Literature survey states that existing query-based text summarization systems rarely provide query relevant sentences but our proposed method can find query-based sentences on the basis of word's sense. The proposed WSD technique is also compared with other existing methods and finds its better performance. Our method can also find semantic relatedness score between those In this paper, we proposed a redundancy free querybased extractive text summarization method. Under unsupervised learning methods, the proposed method provides excellent extraction ability and better querybased summary quality even compared with some supervised methods. We also discuss query relevance performance with other query-based text summarization systems. We have also introduced a word sense disambiguation method for query-based text summarization. This method helps in finding the sense-oriented query relevance sentences on the basis of its meaning. a time. As a null hypothesis, it is assumed that there are no significant differences between the median values of two groups. Whereas the alternative hypothesis is that there is significant difference in the median values of the two groups. It is clear from Table 8 that P values are much less than 0.05 (5% significance level). For example, the Wilcoxon's matched-pairs signed rank test between our proposed method and WAASum for DUC 2006 provides a P value of 0.037 (ROUGE-1), which is very small. This is strong evidence against the null hypothesis, indicating that the better median values of the performance metrics produced by our proposed method is statistically significant and has not occurred by chance. Similar results are obtained for all other methods compared to our proposed method, establishing the significant superiority of the proposed method. From the statistical results, we observe that our proposed query-based extractive text summarization method significantly outperforms the other baseline summarization methods.
Thus, overall time requirement is O(N A)+O(N SA)+ O(N A 2 S 2 )+ O(N 2 A 2 ). Values of A, Q and S can be neglected as they will be small constants. Thus, overall complexity becomes O(N 2 ), where N is the number of sentences in the text.  This article does not contain any studies with animals performed by any of the authors. In addition, we can implement our method to generic mary can also be applied. Finally, to improve the linguistic quality of summaries, we need to do more research on how to process natural language on machine proval: This article does not contain any studies with human participants performed by any of the authors. most effectively.

Conclusion and Future Work
multi-document summarization in where abstractive sum-4. (And/or in case humans were involved) Ethical ap-reference summary is produced by expert human, still accuracy can not expect fully from it.