Real-time credible online health information inquiring: a novel search engine misinformation notifier extension (SEMiNExt) during COVID-19-like disease

Public health-related misinformation spread rapidly in online networks, particularly, in social media during any disease outbreak. Misinformation of coronavirus disease 2019 (COVID-19) drug protocol or presentation of its treatment from untrusted sources have shown dramatic consequences on public health. Authorities are utilizing several surveillance tools to detect, and slow down the rapid misinformation spread online, still millions of misinformation are found online. However, there is no currently available tool for receiving real-time misinformation notification during online health or COVID-19 related inquiries. Our proposed novel combinational approach, where we have integrated machine learning techniques with novel search engine misinformation notifier extension (SEMiNExt), helps to understand which news or information is from unreliable sources in real-time. The extension filters the search results and shows notification beforehand; it is a new and unexplored approach to prevent the spread of misinformation. To validate the user query, SEMiNExt transfers the data to a machine learning algorithm or classifier which predicts the authenticity of the search inquiry and sends a binary decision as either true or false. The results show that the supervised learning algorithm works best when 80% of the data set have been used for training purpose. Also, 10-fold cross-validation demonstrate a maximum accuracy and F1-score of 84.3% and 84.1% respectively for the Decision Tree classifier while the K-nearest-neighbor (KNN) algorithm shows the least performance. The SEMiNExt approach has introduced the possibility to improve online health communication system by showing misinformation notifications in real-time which enables safer web-based searching while inquiring on health-related issues.


Introduction
The current internet-based smart technological advancements, such as wearable technology and mobile phone applications (apps) have enabled constant real-time health monitoring of people in various diseases, and users have become more conscious regarding their health, also about their beloved ones (García-Magariño, Sarkar, & Lacuesta, 2019). Even long before this pandemic, in 2012, researchers reported that almost 59% of US adults searched for health information online. However, the number increased to 75% more recently, with more than a billion health-related searches occurring on Google search engines daily.
There is no doubt that individuals are relying more on internet search engines regarding their health-related queries (Rodgers & Massac, 2020). Nevertheless, given a large amount of inaccurate information online, users can quickly become misinformed. For example, there is a misconception found online, where it mentions that eating apricot seeds will cure cancer. Although there is no scientific research report to support the claim; however, it is well established that eating apricot seeds may even cause cyanide poisoning (Swire- Thompson & Lazer, 2020).
Misinformation is fake, unreliable, or not scientifically validated written material regardless of intentional authorship. Most misinformation-oriented discussions have been focused on venomous acts to taint the social media platforms with harmful and inaccurate information. False or fake news, misinterpretation of a drug protocol, or presentation of unrealistic claims have dramatic effects on public health (Southwell et al., 2019). The widespread misinformation, fake news, and rumors draw attention away from the exact information, real public health challenges, and healthcare professionals who are battling against the disease outbreak while protecting, as well as improving treatment modalities, and population health (Southwell et al., 2019). This misinformation and rumors act as a symbolic challenge to administrations combatting any public health crisis or disease outbreak. The flow of biased and inaccurate information can quickly dilute the seriousness of the actual issue and hamper the functioning of healthcare policy, and disaster management (Na, Garrett, & Slater, 2018).
Health care professionals and administrations faced challenges tackling misinformation, such as fake news and rumor during previous disease outbreaks. Misinformation was also widespread during the primary stages of the human immunodeficiency viruses (HIV) epidemic. It was also plagued by conspiracy theories, rumors, and misinformation for many years, with the effects still visible in regions to this day (Mian & Khan, 2020). At the time of the avian influenza H5N1 outbreak in 2004, the World Health Organization's (WHO) Western Pacific Regional Office identified around forty rumors. Among them, nine were verified to be accurate (Samaan et al., 2005). During the West African Ebola virus epidemic in 2014, there was widespread fear and attention among the United States-based users, followers of Western media, and social media platforms such as Twitter (Sell, Hosangadi, & Trotochaud, 2020).
The current ongoing pandemic situation of the novel coronavirus disease 2019 (COVID-19) has potentially affected the capacity of health facilities, even in developed countries where there are proven and robust healthcare systems (Pillai, Siddika, Hoque Apu, & Kabir, 2020). The pandemic has brought unexpected, sudden and unparalleled damages, changes to global health and socio-economic frameworks.
For minimizing COVID-19 spread, most countries have enforced a societal-level lockdown, and citizens have resumed offices remotely and online activities by staying in residences as much as possible (Ding, Del Pozo Cruz, Green, & Bauman, 2020). During the lockdown, people are using internet search engines and social media platforms to gain information about COVID-19. The nature of social media panic's impact varies depending on an individual's gender, age, and level of education. Social media has played a vital role in spreading anxiety about COVID-19 in many territories. The COVID-19 pandemic has been termed as the first social media infodemic (Ahmad & Murad, 2020). At the Italian lockdown period, even near bedtime, people increased the usage of digital media and internet usage (Cinelli et al., 2020). In an Italian CoMuNe laboratory, Gallotti et al. have set up a COVID-19 "infodemic observatory," where they used artificial intelligence (AI) integrated automated software to follow 4.7 million tweets on COVID-19 streaming past every day (Gallotti, Valle, Castaldo, Sacco, & De Domenico, 2020). On the other hand, Cinelli et al. reported about 1.3 million posts and 7.5 million comments on COVID-19 from several social media platforms (Cinelli et al., 2020).
Pandemic fear among the population can promote online searches for unproven and unprescribed therapies. If fake news and misinformation spread in an uncontrolled manner, it may become fatal. False news generally travels faster than reliable and authentic reports on social media platforms such as Twitter (Liu, Caputi, Dredze, Kesselheim, & Ayers, 2020;Vosoughi, Roy, & Aral, 2018). Many misinformation and rumors spread around online search engines and social platforms regarding COVID-19. Technology entrepreneurs and investors shared a document on Twitter, promoting malaria drug, chloroquine for treating COVID-19. Many mentioned successful therapeutic outcomes in China and South Korea. Despite that, when high profile individuals, such as entrepreneur Elon Musk promoted chloroquine, it attracted the attention of the mass people, which could lead to personal decision-making (Ball & Maxmen, 2020;Liu et al., 2020). That misinformation circulated rapidly before the results of a small, non-randomized French trial of the related drug hydroxychloroquine; at that time the article was in press (Gautret et al., 2020). Hospitals have reported poisoning cases, where individuals suffered from toxicity from chloroquine containing pills, which they intended to have for COVID-19 (Ball & Maxmen, 2020). During early July 2020, the WHO released a press note about discontinuing hydroxychloroquine. In the most recently published original article, it has been shown that hydroxychloroquine did not improve clinical status at two weeks, compared with standard care (Cavalcanti et al., 2020).
Moreover, the poisoning of over two thousand Iranians by swallowing methanol took place, since a misleading social media message urged the mass people in Iran to prevent the SARS-CoV-2 infection by drinking alcohol. Around nine hundred illicit alcohol poisoned patients had to get admitted to intensive care unit (ICU) with a fatality of almost three hundred unfortunate deaths (Soltaninejad, 2020). Global coverage about panic-buying in the online media and social networks only served to promote the same behavior, which caused stockpiling of drugs and vaccines as a method of preparation in the COVID-19 pandemic (Arafat et al., 2020).
To control and track the COVID-19 trajectory, different governments have implemented various digital health surveillance tools, such as smartphone-based apps for COVID-19 contact tracing (Jalabneh et al., 2020). Some administrations have utilized AI techniques integrated with 5G technology and aerial drone devices for real-time COVID-19 tracking (Hussein, Apu, Shahabuddin, Shams, & Kabir, 2020). Although there have been some issues with technical limitations, users' data privacy and security concerns, these robust surveillance systems are guiding the health care professionals properly (Hussein, Shams, Apu, Mamun, & Rahman, 2020).
Researchers have suggested that the US Food and Drug Administration (FDA) must warn the general population against collecting unapproved, unprescribed therapies, and medicines. Google integrated an educational website into search results related to the COVID-19 outbreak, and they mentioned that it could be extended for unapproved COVID-19 medication-related searches (Liu et al., 2020). To identify fake news, scientists have developed machine learning-based false news and misinformation credibility inference models, which form a deep diffusive network model to memorize news articles, writers, and topics (Jain, Shakya, Khatter, & Gupta, 2019;Zhang, Dong, & Yu, 2018).
Machine Learning (ML) algorithms are the major applications of artificial intelligence (AI). It uses statistics to look for patterns in a massive data set. The data can be numbers, words, images, etc. ML has been implemented earlier to detect spam emails. This is done by analyzing the texts in a message to determine the possibility if it is from an actual person or a mass-distributed word for advertisement, solicitation, etc. Recently, ML is used by Google, YouTube, and others to suggest recommendations based on the history of visited content. The rise in the use of social media spreads misinformation and unproven treatments about infectious diseases such as COVID-19 at lightning speed. An infinite amount of misinformation is available over the internet and it keeps rising day by day. Therefore, ML is one of the potential candidates as a counterstrategy to authenticate a piece of given information automatically.
To combat the propagation of misinformation, some of the biggest internet and social media companies have introduced technical gatekeepers to control what information can be sent over their networks. Twitter displays warning messages on tweets containing misleading information about COVID-19 and provides an option to direct the user towards reliable and authoritative sources such as the WHO or national health agencies. It also implemented machine learning algorithms to detect accounts that are used to spread misinformation e.g., promoting treatments that are not proven to be effective.
WhatsApp launched a chatbot to connect its millions of users with various fact-checking organizations across the globe (WhatsApp, 2020b). This allows a user to double-check the information. The platform also introduced the WHO's alert notification. This service responds to public queries about Coronavirus and provides official information 24 hours a day, worldwide (WhatsApp, 2020a). Facebook deployed machine learning algorithms to detect the advertisement of false claims such as homeopathic remedies that can prevent, cure, or protect against the COVID-19 (Mosseri, 2020). It also bans the sale of commercial safety products during the outbreak, such as face masks and hand sanitizers, to ease the panic buying. Instagram, to disrupt the sharing of unverified information on its platform, has used several algorithms to identify and track hashtags that are frequently associated with false or misleading information (Instagram, 2020). Google has introduced "Fact Check Explorer," which verifies the information available online through authenticated third-party fact-checkers (Help, 2020). The results of a search query indicate whether the claim from each source is valid, false, partly right or partly wrong.
The fact check service is available in multiple languages and is also applicable to validate an image.
As we have mentioned above, based on the scientific literature, to date, scientists have been utilizing surveillance systems for infodemia tracking and analyzing disease outbreak related fake news, misinformation, and rumor spread in online media and social platforms (Ball & Maxmen, 2020). However, there is an immediate need for a tool for notifying any user about the scandal, fake news, and misinformation, while browsing in the web search engines.
To tackle this public health-related misinformation, rumors and raise awareness among the people panicking in the society, a proper screening approach could play a vital role. Using Natural Language Processing (NLP) technique, several sensitive keywords can be set to trigger a warning methodology.
Internet Society (ISOC) prefers the word "filtering" over "censorship," as the latter provides a sharp negative feeling contrast to former means a moderate approach (Seidler & Rabachevsky, 2017). Therefore, we plan to filter or screen the internet contents before showing to the unsuspecting web users. For this reason, a web browser extension can be built to carry out the screening operation.
A web browser extension for a web search engine can be illustrated as a computer programming code package, which could be installed into a web browser that the user primarily uses for searching on the world wide web (WWW). The extension has the capability of adding a new feature to a web browser's search engine, or augment a current functionality, update a visual theme (Selig, 2004), or in this case, screen a search result content and show warning as per severity. There are several existing browser extensions to block unwanted content (AdBlock, 2017) or provide trust ratings (WOT, 2020) for websites.
But an extension to create awareness specifically for the public health domain is quite a unique area to be explored.
Here, we are proposing a novel approach to prevent the spread of misinformation through a web search engine extension, which, with the help of ML, will provide valuable information and validity notifications for the user before clicking on any of the search results.

Misinformation notifier extension workflow
The comprehensive flowchart of our proposed methodology is shown in Figure 1. Here we have outlined the steps in three parts. At first, we take the textual contents as input and the web browser extension will do this. Google Chrome is chosen as the benchmark web browser, as 65.47% of the netizen, or directly speaking, a user of the internet, uses the mentioned web browser (StatCounter, 2020) as their primary one.
Secondly, using the NLP techniques, sensitive keywords will be detected and sent to ML classifiers if necessary. After applying several ML approaches, the severity of the content will be assessed. In the end, depending on the seriousness of the contents, proper warnings will be displayed to the end-users. For highly sensitive materials, further recommendations will be given to the users for creating awareness.

Data flow-model diagram (DFD)
The data flow-model diagram (DFD) of our system is presented in Figure 2. The web user is the entity that browses web, i.e. world wide web (WWW) via internet through the chosen web browser. The web user then does the online inquiry using a preferred search engine. Next, the inquiry is scanned for the sensitive contents using NLP, i.e. keywords which trigger the extension. After that, the contents are examined, and the seriousness of the analyzed contents are determined using ML classifiers. Finally, the notification is sent back to the web user for creating and raising awareness.

Search engine misinformation notifier extension (SEMiNExt)
The filter extension (

Machine learning module
To train various ML algorithms, we first created a data set. Each sentence of a search query is considered as a single sample and our data set holds about in total 150 samples. The samples were inspired by basic questions found in popular social media platforms such as Twitter, Facebook, and various news articles.

Natural Language Processing
Text pre-processing: At first, we clean all the text samples by removing any punctuation marks e.g., full stop, comma, question mark, etc. to get samples composed of only words. In the second step, all the uppercase letters are converted to their respective smaller cases. The third step involves the removal of words that are not relevant for prediction, such as "the", "that, "in", preposition, grammatical articles, stop words (e.g. "is" and "are"), etc. Now only the words remain that are relevant for prediction, but the inflection remains which may exist in a variety of forms either as prefixes or suffixes. To eliminate the inflections, we first split each sample sentence into its constituent words to remove the unnecessary characters, a process known as stemming. Finally, the words are stitched back to form a sentence.

Bag-of-Words (BoW) model
Machine learning (ML) algorithms only work with numbers and cannot recognize raw text. To circumvent this issue, the text must be translated into a vector of numbers that reflect various linguistic properties of the text. This is where the BoW model steps in. The idea is that texts with similar content are alike and from the content alone, we can learn about the meaning of the text. The BoW model is a text classification method where the occurrence of a word in a sample and its multiplicity are used as a feature to train a classifier. Other information such as grammar, the structure of the words, and position in a text are disregarded (Goldberg, 2017).
For an example, if we have a sample: "John likes statistical tools. He likes neural network too. " it will be converted into a set of distinct words: [ "John", "likes", "statistical", "tools", "He", "neural", "network", "too" ] and based on the frequency of the words the corresponding vector is generated: [ 1 2 1 1 1 1 1 1 ].
Once all the samples in the data set are transformed, we have a matrix of numbers that is now suitable to use in any ML algorithm.

Machine learning classifiers
In our novel study, we have implemented the following ML algorithms: • Logistic Regression

• Random Forest
The working principles of the algorithms mentioned above are beyond the scope of the current study and the details can be found in various literature.

K-fold cross-validation
To use a machine learning algorithm, we must split the input data into training and testing sets. The size of the training data set must be maximized to achieve the best learning outcome and test data size must be optimized for an accurate prediction. More test samples mean fewer data available for the learning process and vice versa. To overcome this tradeoff, a cross-validation approach is generally employed. Cross-Validation is a statistical method of evaluating learning algorithms to estimate the error in the prediction of a model by dividing the input data into two sections: one part is used to train a model and the other is used to validate the model. In this study, we have used the K-fold cross-validation approach (Berrar, 2018). Here, the input data set is randomly partitioned into K disjoint subsets (referred to as folds) of approximately equal size with no overlap between two subsets. Kohavi recommended 10-fold crossvalidation for real-world data sets (Kohavi, 1995). So, if the data set has a total of 150 samples, then for 10-fold cross-validation, the data set is divided into 10 equal subsets each with 15 samples. Then 10 separate (K, in general) learning experiments are carried out. In each run, one subset is selected as the testing set, and the remaining K-1 subset is used to train the classifier. The testing set is then used to validate the trained ML algorithm. This procedure is repeated 10 times (K times in general) with 10 different testing and training sets and the results of all the experiments are averaged. In this process, the assessment of the classifier is more accurate, and all the samples are in turn, used for both training and testing. Figure 5 pictorially illustrates the whole procedure.

Classification parameters
To measure and compare the performance of various algorithms, we need to compute several parameters, namely precision, recall, accuracy, and F1 score. All these classification parameters can be calculated from the confusion matrix (Benfenati, Clook, Fryday, & Hart, 2007 Accuracy can be defined as the ratio of the correctly predicted outcomes over the total number of predictions. A drawback of this parameter is that it does not consider the subtleties of class imbalances.

= + + + +
Recall / Sensitivity is the portion of the actual positives the classifier picks up correctly among the total test data set.

= +
Precision is the portion of correct positive prediction over the total positive predictions made by the classifier.

= +
F1-score measures the accuracy of a model on a given dataset and is calculated as the harmonic mean of the precision and recall. It is a better measure parameter, especially in the presence of an uneven class distribution e.g., many actual negatives. An F1-score of 1 indicated a perfect precision and recall.

SEMiNExt notifies real-time misinformation and false health-related news
The proposed novel SEMiNExt works perfectly with high accuracy in several search engines on the Google Chrome browser. We will be continuously developing the extension with further modifications for increasing accuracy and fast processing and connecting the extension with the machine learning model for real-time notifications. As displayed in Figure 6, the screenshot below, a popup with the trigger keywords present on the Google search result page is displayed along with a hyperlink to the WHO website.

Implemented machine learning algorithms
We have trained six popular ML methods to classify the authenticity of the given information and compared their performance parameters.
• Logistic Regression • KNN: K = 10 nearest neighbors are used for prediction. Minkowski distance metric is used with Euclidian distance as the power parameter.
• SVM: Linear kernel is used.
• Decision Tree: Entropy criterion is used in the decision tree classifier.
• Random Forest: A total number of 300 trees is used with the entropy criterion.

The optimum size of the training set
The total input data set is divided into two segments. One of the segments is used for supervised training of an ML algorithm, while the other segment is used to validate the trained classifier. The more the samples used to train and prepare the classifier, the better is the predicted outcome but at the cost of a reduced test data set. On the contrary, a more extensive test data set will ensure correct evaluation of the trained classifier. Therefore, to circumvent this tradeoff, we first trained the ML algorithms over a range of data size. All the samples used for training and validating are imported as it is sequenced in the input file and no random shuffle is implemented. As can be seen from Figures, 7 and 8, both the accuracy and F1-score rise with the increase in the size of the training data set. This is because the models are better trained from more available samples. But for large size of the training data set, the accuracy and F1-score decrease because the amount of test data is not enough to accurately evaluate the trained classifiers. Besides, SVM, Naive Bayes, Logistic Regression, and Random Forest classifiers operate with maximum accuracy and F1-score when 80% of the input data is used for the training. As Decision Tree and KNN classifiers operate close to their maximum accuracy and F1-score for the same size of trained data set, we use 80% of the input data set to train the ML algorithms in the remainder of the study.

K-fold cross-validation results
In the previous subsection, we used a fixed data set arrangement as is present in the input file. The prediction from this arrangement is biased for this specific order of data set and will vary for an entirely different ordered set. Therefore, for an unbiased supervised training of the classifiers, we used the K-fold cross-validation. This ensures an accurate assessment of the ML algorithms. The results of the 10-fold cross-validation are shown in Figure 9. As can be seen in Figure 9, KNN achieves the least performance among all the categories, and the remaining five classifiers predict with similar accuracy and F1-score. Decision Tree demonstrates the best accuracy and F1-score of 84.3% and 84.1%, respectively. Although Decision Tree may not have the highest precision, it successfully predicts the highest number of test samples correctly, hence producing the best recall. This more than compensates for the comparatively lower precision, resulting in the best accuracy and F1-score. On the other hand, Naive Bayes successfully predicts all the true positives among all the actual positive samples in the test data, and with FP = 0, obtains a 100% precision. But it fails quite significantly to predict the negative samples in the test data set correctly and so it comparatively shows a poor recall, which reduces its overall accuracy and F1-score.

Discussion and Conclusion
The real-time SEMiNExt successfully works on multiple web search engine platforms, such as Google, Bing, Duckduckgo, Yahoo. The keywords used for triggering the extension were -'covid', 'COVID','covid19','COVID19','hydroxychloroquine','alcohol','corona',and 'coronavirus (Samaan et al., 2005). Calisher and colleagues reported a statement of solidarity to fight against COVID-19 and promote scientific evidence and unity over misinformation and conjecture (Calisher et al., 2020). However, during this COVID-19 pandemic, all available current technological tools have focused mostly on misinformation monitoring, digital health care surveillance, and contract tracing Jalabneh et al., 2020). Our proposed online tool, SEMiNExt, enables safe and verified health-related information inquiring in web browsers, and multiple search engines.
Health communicators require to screen more detailed information on misinformation and the search engine, and social media environment during a fear-inducing disease outbreak to improve public health communication practices (Sell et al., 2020). For example, Mesquita et al. proposed several strategies to tackle misinformation and fake medical news concerning public healthcare in their work. The approaches include government actions, creation of trustable content, detection and correction of misinformation, increased collaboration, etc (Mesquita, Oliveira, Seixas, & Paes, 2020). Nevertheless, these approaches were recommended for industrial and large-scale administrative surveillance purposes, where most of them are just general recommendations. However, our combinational approach to detect misinformation is novel and unique. We propose that web search extension would analyze and inform users regarding the misinformation and its severity. It is then the users' responsibility to ascertain the actual information and enhance their knowledge base. Moreover, we solicit for administrative action and draw attention to the trustable content creators to confront fallacies with facts to raise awareness.
The SEMiNExt has successfully identified the sensitive keywords and triggered the warning message to be shown to the users. In the future, more web browsers and search engines will be incorporated for better reach to the end-users. Moreover, only English has been chosen as the natural language here; additional natural languages, such as (by the descending order, according to the speaking population) Spanish, Bangla/Bengali, Russian, German, French, and Italian, will be integrated to our system.
On the other hand, additional ML techniques, along with Data Mining and Deep Learning approaches will be incorporated for enhanced outcomes. The existing dataset would be augmented further and then finetuned soon to improve the current results for endeavor hereafter. This is a novel research area to be explored and enthusiastic researchers may find it suitable as a groundwork study for the groundbreaking discoveries.

Availability of Data
The novel search engine misinformation notifier extension (SEMiNExt) introduced and presented in this original paper have been stored in the GitHub repository, http://doi.org/10.5281/zenodo.3986161.

Contributions of authors
EHA initiated and conceived the study. ABS, EHA, AR and MRH planned, and designed the study. ABS and MSR conducted the machine learning experiments and acquired the data. AR developed the extension and exchanged data with the machine learning classifier. ABS and EHA interpreted the results. ABS, EHA, AR, MRH, NS and EHA drafted the first version of the manuscript. All authors critically reviewed and edited the manuscript. All authors approved final version of the manuscript. ABS and EHA share equal authorship as first author. AR and MSR share equal authorship as second author. EHA takes full responsibility for the contents of the publication as the corresponding author.

Funding
There was no research funding received for this study.