Email databases are continually being updated with the inclusion of active email addresses collected from different sources by hackers and spammers for their illicit purpose, including spamming and sharing. The presence of multiple valid email addresses in the headers of chain and multi-recipient email messages increases the chances of successful harvesting. This paper investigates and exposes a bot-based technique for email address harvesting from email messages, including chain email messages and emails sent to multiple recipients. Experimentation results demonstrate the designed Bot's effectiveness in misusing technologies to collect email addresses from the header and body of email messages. Also, the experimented method and user studies demonstrated the XOAuth authentication mechanism's inefficiency in blocking mailbox access and email address harvesting. The comprehensive illustration of the design shall be beneficial to design techniques for detecting and mitigating such bots. The paper also suggests a few mechanisms that can be put in place to prevent this type of email address harvesting significantly and also designs a mitigation method to detect and mitigate the designed Bots of this nature.
Research Article
Mitigating BOT-based Methods for Email Address Harvesting and Spamming
https://doi.org/10.21203/rs.3.rs-1954011/v1
This work is licensed under a CC BY 4.0 License
Version 1
posted
You are reading this latest preprint version
Email Messages
Email Address Harvesting
E-mail Service Providers
Bot
XOAuth Authentication.
Without a database of active email addresses, it is not possible for spammers to send spam effectively. Without a database of active email addresses, it is not possible for spammers to effectively send spam messages and profit from the process. Spammers are always on a hunt for new and active email addresses for spamming [1], phishing [2], [3], whaling [4], spoofing [5], cyberbullying [4], hoaxes [6], etc. Spamming and phishing attacks are also on the rise in online social networks. The use of new techniques for spamming requires the development of novel procedures to check their spread [7]. A consequence of spamming, i.e., phishing which poses a serious risk for consumers and businesses, has been or rise and requires contemporary measures for its control, such as filtering [8]. They harvest email addresses using different methods, which include mailing lists on sale, social networking sites, group discussion forums, web crawlers, directory harvest attacks, hacking into sites, intercepted user requests to unsubscribe, guessing and cleaning, and free offerings. (http://www.private.org.il/harvest.html). Hiding email addresses is no longer a viable solution to the problem as sharing email addresses with friends, family, customers, business partners, and online service providers are inevitable as it has evolved as a standard mode of communication on the Internet. Moreover, providing an active email to highly in-use social and e-commerce websites has become compulsory. These email addresses and user information are stored in diverse formats, mostly underlying databases of service providers. If hacked by spammers besides others, this database puts users at the risk of spamming. Also, businesses and individuals publish their email addresses on their websites and blogs for their customers to communicate with them. Bots [11], [12] originated for useful purposes, such as web crawlers to perform some predefined functions automatically [13], but their use for numerous malicious purposes has been forged for a long. The illicit use includes spreading spam email messages which are considered one of the major threats to the security and privacy of the Internet [14], [15], [16]. A bot is propagated into the vulnerable systems to gain backdoor access to these systems facilitating the installation of bot malware by uploading or commanding the victim machine to download a copy of the bot malware. The spread of the bot malware includes attacks through software vulnerabilities [17], vulnerabilities caused by other infections, and social engineering, including using email, instant messaging, and malicious web page content [18]. Other means of propagating bot malware are peer-to-peer networks, open file sharing, and direct client-to-client file exchange. Bot malware uses FTP, TFTP, and HTTP to infect computers and spread it until a desired strength of the botnet is assembled [19]. Spammers create bots which crawl the Internet on a large scale and access the contact information page of the website wherefrom they obtain the email addresses [20]. Also, discussion forums usually publish the participants' email addresses on their websites. Users can participate in the discussions directly from their inboxes by replying to emails arriving from the discussion forums. Many discussion forums are designed to convert incoming email messages from registered users into posts on the discussion forum. Spammers design bots which register and create accounts on these discussion forums. Once registered, they can log in automatically, crawl the discussion topics and extract participant email addresses from the web pages. Furthermore, each email message contains one or more valid addresses in its header and body. Local computers or email hosts in diverse formats store these messages. Bots can be designed to collect email addresses from these email sources and launch several types of attacks, including spamming. Web developers take various preventive measures against email harvesting bots when designing a website to prevent email addresses from being automatically stolen on a large scale by these bots. With the constantly evolving techniques, these preventive measures are useful only to some extent in hiding the email addresses from the bots. This paper reports the design of a bot that can harvest email addresses from email messages stored on computers and in email accounts. The bot infects the users’ computers, scans for email messages in local discs and email clients, steals the login information of insecure email accounts, and scans for email addresses to harvest email addresses therein. The experimentation with these basic functionalities incorporated into the designed Bot has shown the high efficiency of the designed bot. A more sophisticated and robust keylogger, logs parsing algorithm, and support for more free email service providers and web-based email clients can highly increase the performance and efficiency of the designed bot. The outstanding efficiency of the bot in harvesting email addresses and its efficiency in using compromised email accounts to send spam messages suggests devising some mechanism to prevent this form of harvesting that can lead to excessive spamming.
A. Contributions:
To the authors' best knowledge, this is the first study that exposed and comprehensively illustrated the design and architecture of an email addressing and harvesting Bot that demonstrates the misuse of technologies intending to make mitigation and detection possible. The contributions of this work are twofold: First, the paper discusses email address harvesting and investigates a bot-based technique for email address harvesting by designing a bot and undertaking a comprehensive experimental analysis of the designed bot that got untraced by popular antivirus applications. The Bot's design illustrates the underlying technologies that spammers and hackers misuse to develop such bots, harvest email addresses, and send spam messages. Any antivirus software shall benefit from the comprehensive and illustrative design of the bot exposed through this study. Secondly, it proposes various mitigation techniques that can be used to strengthen anti-virus software to detect and remove email address harvesting and spamming bot, including the one designed in this work. Further, a technique has been developed and demonstrated to mitigate the email addressing and harvesting bot.
B. Paper Organization:
The paper is organised as follows: after this introduction, section 2 discusses email address harvesting, including harvesting from chain email messages. Then, in section 3 design of a Bot that can be used for email and address harvesting is exposed. It gives a comprehensive process logic and architecture of the designed bot to understand the misuse of technologies for harvesting and developing mitigation processes. Next, section 4 presents experimentation and results obtained and highlights the effectiveness of the designed bot in harvesting email addresses and sending spam on different machine platforms. Section 5 discusses various detection and mitigation methods that can be put in place to detect bots of this kind. It also discusses the design and experimental results of a method to detect and block bots of this kind which is followed by the conclusion.
Email harvesting is collecting email addresses in bulk to spam through various known and unknown methods. These include web crawling, crawling archive sites, malware, malicious sites, dictionary attack, and blind harvesting from social networking sites and search engines [21], [22]. Even personal information available publicly on social networking sites is used to gather email addresses. Many websites replace an email address's character ‘@’ with a small inline image displaying the character ‘@’ to dodge email address harvesting crawlers. Such a character replacement is an effective technique to keep away bots that do not use OCR techniques. Others replace the character ‘@’ and dots with patterns like [at], [dot]. These are somewhat lesser effective preventive measures against email harvesting spiders. Some of the more intelligent bots may still be able to harvest email addresses by converting the patterns back to the characters ‘@’ and dot or recognising the character ‘@’ in the images with OCR techniques. Email address harvesting systems often gather names of people from various social networking websites and predict email addresses based on those they verify by performing email address lookups on different email servers. Stephen suggests a method to detect and respond to a harvesting attack at Internet Service Provider (ISP) level by responding with a false positive response for lookups after a failed email address has crossed a given threshold during an email transaction from a given originating IP address [23]. This method is applicable only when a bot exceeds a certain given threshold. Since the bot receives a false positive for non-existing email addresses, it will consider them valid addresses, send spam, and share them with other spammers. Therefore, email servers may have to deal with spam sent to non-existing email addresses, which may respond with a “Failed Delivery” reply for each such address letting the spammer know that the address is not valid and rendering this preventive method useless. A technique was proposed in [24] to prevent legitimate email addresses from being exposed to spammers. They suggest replacing @ with [at]; on web pages to keep out most of the simple crawlers. To keep away some of the sophisticated crawlers that want to avoid “spamtraps,” they suggest adding simple words like “spamtrap,” “spam harvester”, or “honeypot” somewhere on the webpage in invisible HTML tags. These words or phrases on the web pages have been found to scare many bots and keep them at bay effectively. Other methods to prevent email address harvesting are JavaScript, PHP, HTML obfuscation, images or flash, mail server monitor, and spider traps. Web browsers support JavaScript and can easily decode the obfuscated text and display it on the webpage for users to read. Most web crawler bots do not support JavaScript and depend on reading the HTML response from the web server. These limited techniques attempt to tackle different email harvesting techniques at various sources. These methods are helpful to some extent against selected methods of harvesting. A small fraction of internet users, service providers, and publishers implement these preventive methods; therefore, email harvesting continues to benefit its exploiters. Although the growth of spamming due to email address harvesting has increased exponentially, the underlying harvesting techniques are primarily unknown.
This section presents the design of an email address harvesting bot that combines techniques for harvesting email addresses, especially from email messages. The bot begins by scanning the local storage of the victim machine for email messages stored in files and extracts email addresses from the body and headers of the messages. It then activates a keylogger in the background, which logs the user’s keyboard activities into the log files. A parser parses log files to find and extract email accounts and their credentials. An inbuilt email client then uses the login details to login into email accounts and fetch email messages and address books. Email addresses are extracted from the headers and body of email messages. If the bot cannot log in to the email account, it attempts to trick the user into providing access to the mailbox by performing XOAuth authentication (https://www.eugdpr.org). If authenticated, it again tries to fetch address books and email messages for harvesting email addresses from their headers and body. Finally, the harvested email addresses are passed to a web-based control and command module which stores them online. The Bot also stored a copy of the email addresses and email account login details on the victim’s machine. The bot can also send spam messages to the harvested email addresses using the compromised accounts if instructed by the command-and-control unit.
A. The Process Logic of the Bot
Figure 1 shows the detailed overall process logic of the designed Bot. The Bot works in six significant steps shown in the data flow diagram as A through F. After being installed on the user’s machine like any other bot disguised as some useful program and after being executed begins harvesting email addresses from the user’s computer and email accounts. Once the Bot program gets executed, it starts searching for email files in the local computer. A local file is used to store the search results generated by the Bot. In the next step, parsing of email files is carried out, and email addresses are extracted and stored in a local database. The bot uses different libraries to read the email file contents as various email programs use different file extensions to store emails. The email addresses are next sent to a command-and-control unit of the bot. The bot also attempts to steal email login information and log in to the email account to extract email addresses. In case of login information is not found or a login attempt fails for any reason, it tries to trick the user into obtaining access using XOAuth authentication and downloading email addresses and email messages from the inbox. Both body and headers of email messages downloaded by the bot are also searched for an email address. These email addresses are also stored in the local database. The bot also keeps looking for instructions to send spam email messages. If it receives such instruction from the command-and-control unit, it will download the email message and send it to the harvested email addresses using its built-in email client.
For harvesting by the bot, all steps may not be carried out successfully on all systems. For example, steps A or B followed by steps C, D, and E must successfully run for successful harvesting. Similarly, step F, used for spamming, can work independently and does not depend on the successful execution of other operations.
B. The Application Architecture of the Bot
The architecture of the complete system comprises a Command-and-Control Unit and Bots. While the Command-and-Control unit is installed on the attacker's system or a system under full control, the bots, like any other bot disguised as valuable programs, are installed on victim machines. Figure 2 shows the application architecture of the developed bot.
The Command-and-Control Unit comprises the necessary logic to control all six significant operations described in the above sub-section. The primary constituent modules of the Bot include a user interface, an inbuilt email client, an email parser module, an email address extraction module, and an offline email address search module. The bot harvests email addresses from emails stored locally in the inbox and webmail accounts and sends them to the command and control. In addition, the bot executes instructions from the command-and-control unit and may be commanded to send spam emails to the email addresses it has harvested.
The following section describes the functioning of various constituent modules of the Bot:
1) User Interface
The bot installed on the user’s machine disguised as some useful program has a simple interface that makes a user think of it as some useful utility application or add-in to some application. However, it may be run without a user interface as well. The naïve user has no idea about the background processes and operations performed by the Bot. The Bot starts execution on system startup or by clicking on its disguised application icon and keeps running even when the user closes its disguised user interface.
2) Offline File Search
This module runs at scheduled intervals in the background and scans the directory structure of the infected computer. Next, it looks for email files with predefined file extensions. Finally, this module submits the search results to the email parser module for further processing.
3) Email Parser Module
This module opens email files for reading. Various file formats like .eml., mbox, and .msg are recognised by this module. Different -email file types have different text encoding, and this module identifies the encoding and uses appropriate libraries to extract the header and body of the email messages. The text extracted by this module is submitted to the email address extraction module for further processing.
4) Email Address Extraction Module
This module parses the input text received from the email parser module and extracts email addresses. It uses regular expressions to match the email address pattern and extract all email addresses from the text. After extracting the email addresses, it discards all duplicate email addresses and stores the extracted email addresses in a local database if not already present there.
5) Webmail scanner
The Offline file search module is useful only if the user stores email messages on a local disc. However, some users use email clients and thus don’t download or store email on local machines or use web-based email clients to access their email remotely. Therefore, a webmail scanning module has also been developed to harvest email addresses from remotely stored mailboxes. This Webmail scanner module comprises a Keylogger, Log parser, Email client and XOAuth modules. The working of the Webmail scanner module is shown in Fig. 3. Once the keylogger and email modules are installed and running, the keylogger works in the background, storing the user’s keyboard activity and window titles. The logs are stored in files on the local disc. The log parser module then searches for login details from the logs. If it finds login information, it asks the email client module to attempt login, and in case the login is successful, the email harvesting module will start harvesting. In case of failed login attempts, the XOAuth module will attempt to trick the user into logging in and grant email read access to the module. In case of a successful attempt, the access token is obtained and stored for future use, and the email address extraction module will start harvesting email addresses.
The Keylogger module runs as a background process with the execution of the bot without the knowledge of the victim and monitors the user's activities, and records them in log files. For example, the keylogger captures the text typed in using the keyboard, mouse clicks and active window titles in a sequence. All this captured data is flushed into log files in a hidden location of the victim’s computer.
The Log parser module parses the stored log for email addresses typed in the browser windows and attempts to extract the password, which usually follows the email address strings in the log files. The parser extracts all email addresses, and corresponding passwords from the log files, lists the login details and passes these credentials to the email client module.
The Email client module accepts login details from the log parser module, guesses the email server details of the email addresses and attempts to log in to the email server remotely, just like any regular email client. After a successful login to the victim's email account, it downloads the email messages, reads the contents, and forwards it to the email address extraction module for further processing. In case of login failure, this module sends the login details to the XOAuth module.
The XOAuth module attempts to make the victims provide access to their email accounts using the XOAuth authentication mechanism. The module allures the victim to unlock special features of the application which require a login and sends the user to the login page of the email service provider. After login, the user will be asked to permit the application to manage their email. The user grants access to the bot, and the bot gains full access to the email messages of the victim. If the user does not grant the permissions, this module sends the account's login details to the web service module for manual processing by the spammers.
Web client module: This module is responsible for communication with the command and control unit via the Internet. The harvested email addresses and stolen login details are submitted to the web service module, which stores them in a database for the attacker. It POSTs the email addresses and the login details as JSON data. The web client also frequently keeps looking for email messages from the web service module. It downloads the attacker's email message and instructs the email client module to send spam messages.
6) Command and Control Module
Receive harvested email addresses: This module is an online application that accepts post requests from web clients. The web service takes regular text post fields as well as posted files. The text in the post field and submitted files are parsed by this module, validated and stored in a database. During the validation, it checks if the submitted data contains adequately formatted email addresses and checks for duplicates in the database.
Receive login details: The Web Service has another method which accepts login details of email accounts hacked by the bot. The login details are stored in the database for use by the spammers manually or any other email access method.
Spamming instructions: This module allows the attacker to compose and store the spam messages on the server for the bots. If the web client module of the bot requests the server for messages for spamming, this module sends back the spam message in response.
Some significant free email service providers, including Gmail and Yahoo, have now considered using the XOAuth authentication mechanism in SMTP and IMAP authentication. This mechanism prevents any application's direct use of email username and password for login. Instead, it requires users to login in using the service provider’s web page to create an access token. The SMTP and IMAP applications use the access token to log in and access the email account. This security mechanism can hinder the bot's proper functioning of the Webmail scanner module. Therefore, the bot has been provided with a XOAuth module which asks the victims to log in to their email accounts and obtain an access token. This functionality relies on the ignorance or inattentiveness of the victim while providing access to third-party applications in the XOAuth authentication process.
To test the working and effectiveness of the presented harvesting technique, a bot capable of performing operations, namely user interface, inbuilt email client, email parser, email address extraction module and offline email address search module discussed in section 3.2 above and a command-and-control unit capable of receiving harvested email addresses and stolen login information from the bots and send spamming instructions to the bots were coded. The Bot was coded as a desktop application in multiple programming languages, including Microsoft C# and Java having a disguised interface of being an application for receiving and displaying famous quotes, as shown in Fig. 4. Besides using functions provided by the programming languages, the code also used system calls of the underlying operating system.
The command-and-control unit capable of communicating with the bots to receive harvested email addresses and login information and to send them spamming instructions was coded as a web application in PHP 7.0.31 with the underlying database in MySQL and Apache web server. Command and Control unit uses a centralised approach wherein all communications between bots pass through the centralised C&C unit [25], which is easier to design but still can control many bots and has small message latency and for which inadequate countermeasures exist. Figure 5 shows a sample run of the command-and-control unit showing emails received from a bot and the number of addresses harvested from each. The C&C unit can issue commands that include start/stop harvesting, start/stop disk file scan, interval control command for reading emails/files, send spam, interval control command for sending spam, and send quotes.
The bot was initially installed and tested for its envisaged working within the research laboratory on a few computers connected to the Internet. The designed web application was hosted on a Linux server, serving as the command-and-control unit. The system worked successfully, and harvested email addresses stored locally on the disk and extracted email addresses from the test email accounts created for the purpose. However, due to possible legal issues, an open space large-scale evaluation of the technique wherein the bot could have been distributed through various communication channels such as email, removable media, websites, etc., was impossible. Therefore, to test and evaluate the designed bot's effectiveness on a more massive scale and yet out of the open space, a group of 40 volunteers from among the institution whose computing skills varied from novice to experts were motivated to evaluate the bot. The volunteers willingly installed the bot programme on their systems which they used for their personal use, including checking and downloading their email messages.
In all, 40 volunteers had 56 email accounts accessed from web and desktop email clients. The bot was unable to gain access to all email accounts. For such accounts where login details were either not captured or could not be used to access the email account, the bot asked the volunteers to grant permissions to their email accounts using the XOAuth authentication mechanism. The bot directed the users to web pages to log in with their email service provider’s login page to visit a restricted area. The login page was followed by the consent screen asking users to provide full access to their email accounts.
Table I Harvesting results for desktop email clients
Table 1 shows the results of experiments carried out on desktop email clients. Out of the 56 email accounts, login credentials for 34 accounts were obtained using the underlying keylogger of the Bot. However, it failed to get the credentials for other accounts. The Bot attempted XOAuth authentication on the remaining 33 email accounts, out of which it successfully obtained credentials of 19 accounts. Thus, the bot gained access to 42 out of 56 email accounts tested. In addition, the Bot gained access to 1,42,000 email messages, parsed one at a time for email addresses, and passed each address to the command-and-control unit. In all, as many as 26,766 unique email addresses were harvested.
Table II Harvesting results for web email clients
Table 2 shows the results of experiments carried out using web email clients. The keylogger successfully obtained login details of 44 email accounts out of 56 tested email accounts. However, the bot gained access to only 17 email accounts. The servers of the mailboxes denied access to the remaining email accounts, possibly because of wrong credentials obtained by the keylogger. As such, only 17 out of 56 email accounts were accessed directly. The Bot initiated the XOAuth authentication mechanism to gain access to the remaining 39 email accounts and successfully obtained access to 19 email accounts. In total, the Bot was thus able to gain access to 36 email accounts from 34,200 unique email addresses from 1,59,000 email messages. Regardless of the type of email client, the Bot has been able to crack into the users' email accounts through keylogger obtained credentials or by misuse of the XOAuth mechanism with varying success rates.
Table III Harvesting results from local files
A total of 60,966 unique email addresses were harvested by the bot from both desktop and web-based email clients using just 40 volunteers. The results also show that, on average, unique email addresses were obtained from every second email message. The number of unique email addresses is over half the number of email addresses harvested.
Although all volunteers did not proceed ahead with the XOAuth mechanism, over 50% of them did not pay attention to all the permissions and just allowed login through XOAuth. The inattentiveness or ignorance of users opens an avenue for the bot to manipulate the XOAuth authentication mechanism to gain access to email accounts. In practicality, unintentional consent can be used to misuse the email account for illicit purposes. Furthermore, deploying the designed bot on a more prominent space could yield billions of email addresses and thus can pose severe threats of spamming.
To test the spamming capabilities of the bot, the command and control unit composed email messages and passed them to the bots installed on the systems of the volunteers, along with the spamming instruction and list of email addresses. Each bot submitted email messages using its inbuilt interface less email client to all other email accounts of the volunteers from the email account of the volunteer whose credentials had been obtained by the bot. These spam messages were received by each email account in its inbox, thus proving the effectiveness of the bot in sending spam emails from the volunteer’s computers using volunteers' email accounts.
The designed Bot can harvest email addresses from files stored in the local disc and email accounts. In the former case, the bot's effectiveness depends on the security policies of the underlying operating system, which may or may not grant access to files stored in the local discs. However, such security policies are not frequently set as they can lead to unnecessary blockage of legitimate programs. As such, harvesting email addresses from the files stored in the local discs in most cases gets undetected unless some other bot detection mechanism detects the bot and removes it from the system. In the latter case, the bot's effectiveness depends upon the underlying success of the keylogger to capture keystrokes and deduce email authentication data and yet does not get detected by anti-spyware software packages or even does not show up in system programs such as task manager. It also depends upon the user's consent, who may or may not permit XOAuth authentication to the bot.
Prefixes Myriad contemporary malware detection techniques are available [12], [26], among which signature-based and behaviour-based detection techniques are the two standard keylogger detection techniques. While the signature-based technique detects only known keyloggers, behaviour-based techniques can also detect unknown keyloggers. Keyloggers use some hooking technique (system-wide or thread-specific) to intercept events such as key presses before they reach their intended application. The hooking uses some API of the underlying operating system that prevents function calls or messages pass between software components. To devise an anti-keylogger, an anti-hook technique [27] detects suspicious processes or files which use hooks by scanning all the processes, static executables, Dynamic Link Libraries, etc. However, besides being prone to false positives, such an anti-keylogger may also not detect all types of keylogging activity. Another technique to detect keylogging is by using HoneyID [28], wherein bogus events are generated by the anti-keylogging programme to trap the processes that respond to these fake events. This technique offers an advantage in detecting unknown keyloggers at a higher cost. Also, Dendritic Cell Algorithm [29], which uses the correlation between different behaviours such as keylogging, file access, and network communication, is an efficient technique to detect keyloggers as it has a meagre false alarm rate. These features are incorporated in many anti-spyware utilities that can detect keylogging. Spearman's Rank Correlation algorithm monitors and correlates system activities such as keylogging activity, file access, and outgoing traffic by executing different API functions to detect the presence of bot software [30]. However, this algorithm may fail if the bot performs its activities in random time slots. Use of antivirus, anti-malware, and anti-spyware software that implements one or more of the techniques mentioned above may help to mitigate threats posed by the designed email address harvesting bot, which can warn the user about background keylogging. They may also not allow it to be installed or run and may remove it from the system if installed. Also, a virtual keyboard can provide sensitive data such as email addresses and passwords, which could be made default methods for accepting user input while filling passwords.
Access granted to the bot (a third-party application) to read email and private data stored in emails through the XOAuth authentication mechanism can be controlled through user awareness that can make them more attentive to the risks of granting such permissions. When logging in to apps using OpenID provided by email service providers like Gmail, Yahoo etc., users should pay attention to the permissions being granted. Users can also disable XOAuth authentication mechanisms of their email accounts when not required. Although XOAuth authentication mechanisms are helpful in many situations, their careless use can be hazardous, especially for novice users. Therefore, more strict instructions and passing CAPTCHA tests can be included in the XOAuth authentication process.
Users should always purchase or download software from widespread and trusted sources and avoid downloading software from unknown and un-trusted sources they do not need. Cracks, Keygens, and activators of trusted software from un-trusted sources should be avoided at all costs. There are high chances that this software contains viruses and spyware.
Being careful and monitoring their computers can sometimes help detect and remove such malware. In addition, users should often check monitoring tools provided by the operating system to look for suspicious-looking processes.
Exploitation, installation, and communication with the command-and-control unit are the three primary stages involved in the life cycle of a bot. The first stage is the exploitation stage, wherein vulnerable systems are identified and targeted through various communication channels such as email, removable media, websites, viruses, or any other socially engineered technique. At this stage, mitigation is possible by keeping systems up-to-date with recent releases of patches and a robust defensive mechanism against socially engineered techniques, which is impossible for all systems and users. In the second stage, a Bot is installed on the system. Bot masters code bot programs for different operating systems and environments, ensuring appropriate versions for each release of the current operating system and periodically changing the disguised user interface and application of the bot. An antimalware software running on a system may prevent the bot from being installed and executed on a system. However, its efficiency shall depend on the underlying techniques and database to detect malware signature or behaviour or reputation, etc. As a general rule, antivirus and other malware detection and removal tools must be frequently updated with new signatures and databases to keep it current. If the first two stages fail and the bot finds its way onto a system, it establishes communication with the command-and-control unit. In this stage, some anomaly detection mechanisms may be used to detect a bot; however, depending on its implementation, it may be challenging to differentiate bot traffic from the traffic of a legitimate application.
Bot and botnet detection is an active area of research, with the lead taken through a Honeynet project [32] and later through traffic analysis [33]. Various techniques for their detection can be classified as signature-based [34], anomaly-based [35], host-based [36] and network-based techniques [37]. Signature-based techniques are useful for known bots, and anomaly-based techniques analyse network traffic to detect irregularities but cannot scan encrypted channels. Hosts are scanned for identification of Bot programs based on their behaviour etc., in host-based techniques, while network traffic is monitored in the network-based techniques. Other classifications of technical and non-technical bot mitigation techniques at the application layer have been presented and compared [38].
Multiple network security tools are currently available for capturing, visualising, analysing and detecting network traffic [39]. A traffic analysis of the designed bot was conducted to check for its possible irregular behaviour and subsequent detection. Figure 6 shows a window of the Wireshark displaying the posting of email addresses to the command-and-control unit alongside other communication over the Internet, which did not report any irregular behaviour and as such, antivirus and the anti-malware software installed on the test machine did not detect the bot's communication with the command-and-control unit. Implementing SSL/TLS completely encrypts the communication between the bot and the command-and-control unit, which will make the tools based on network analysis useless because they won’t be able to perform decryption and analysis of the communication data.
During the testing of the bot on systems installed with some anti-spyware/anti-virus software, the underlying keylogger of the bot remained undetected on some of the tested systems despite up-to-date definitions of the installed anti-virus/anti-spyware software. Figure 7 shows the detection of the designed bot's keylogging activity on a system with anti-spyware software installed.
Despite having an active anti-keylogger, there are certain situations, such as cybercafés, Government Offices, academic institutions, etc., where such tools will deliberately not be installed, and keylogging is mischievously facilitated. In such situations, users may conceal their password from being logged by changing focus from password fields in forms to another area while keying in their passwords [31] or by using virtual keyboards whenever provided. In addition, some operating systems also offer built-in virtual keyboards which can be used to type sensitive information instead of typing directly from the keyboard.
To make detecting the keylogger difficult, the bot was reprogrammed to use different hooking methods to intercept key presses before reaching their intended application. Using different system-wide and thread-specific methods for keylogging made keylogging challenging to detect by anti-spyware software. Platform-dependent native code through Java's native interface was leveraged to create low-level system-wide hooks and deliver those events to the bot application. Anti-spyware software, including those which detected the keylogging activity of the designed bot, could not detect the reprogrammed bot that used a different hooking technique for keylogging.
The bot was further tested with many popular anti-virus and anti-spyware software. Because the Java implementation of the bot works on top of JVM, a trusted software, most of the security tools didn’t report its malicious activities. A few software, however, reported Java.exe trying to log keyboard activity.
A. A Mitigation Tool for the Designed Bot
A Java programming language security tool has been developed and tested to mitigate the designed Bot. The tool checks for the presence of Java Security Policies on the user’s machine by looking for the “.java.policy” file in the user's home directory. The tool also tries to create and delete a test file and listen to Mouse and Keyboard events with proper exception handling to check whether or not such Java Security Policies have been implemented. If the Java Security Policies do not deny the tool, permissions to perform the test activities or if the “.java.policy” file is not found, it lets the user create the “.java.policy” file automatically in the user’s home directory, thereby withdrawing the File and AWT Events permissions. The following algorithm is used by the security tool to check for the policies
BEGIN
SET dir = getHomeDirectory()
IF fileExists(dir + “/.java.policy”)
Set policyFileExists = true
policyFileLabel.setColor(color.green)
ELSE
Set policyFileExists = false
policyFileLabel.setColor(color.red)
END IF
TRY
createFile(dir + “/test.file”)
writeToFile(dir + “/test.file”, “Sample Text”)
SET fileSystemAccess = true
filePermissionsLabel.setColor(color.red)
END TRY
CATCH Exception
SET fileSystemAccess = false
filePermissionsLabel.setColor(color.green)
END CATCH
TRY
SET str = readKeyboardInput()
SET keyLoggingEnabled = true
keyloggerLabel.setColor(color.red)
END TRY
CATCH Exception
SET keyLoggingEnabled = false
keyloggerLabel.setColor(color.green)
END CATCH
END
Figures 9 and 10 respectively show the screenshots of the designed tool before and after installing the security policy.
The following algorithm creates the java policy file to prevent keylogging and other activities by JAVA-based applications. The algorithm runs after the previous algorithm, which has already set some required variables:
BEGIN
SET policyFile = dir + “.java.policy”
IF policyFileExists = False THEN
createFile(policyFile)
END IF
openFileForWriting(policyFile);
writeLine(“grant {”)
writeLine(“};”)
closeFile();
END
In the above algorithm, all the permissions are revoked in case a Java policy file already exists. However, a more advanced algorithm can be designed to revoke permissions to File Access and Keyboard Events only, allowing any Java applications to function correctly.
Spammers can gain from spamming when the spam messages a wider audience; therefore, new methods and approaches are explored continuously by spammers to harvest active and current email addresses. The paper presents and provides a comprehensive illustration of such a possible approach that spammers may be using or can use in the future. The illustrative design of the Bot shall be beneficial to understanding the underlying misuse of technologies and the design of remedial technologies for detection and its control. Experimenting with email harvesting through malicious activities such as one presented in this paper that uses keylogging, disguised user application and interface and creation of a hidden bot network is essential as it provides a vital understanding of the underlying process logic and working of such malicious programmes. The detection and prevention of such malicious activities are generally performed through techniques based on signature and behaviour analysis of the malware. Understanding their underlying process logic and working can significantly improve their efficiency and success. The paper also proposes an effective detection and some security measures that can be taken to mitigate bots of this nature.
Compliance with Ethical Standards
Informed consent: Informed consent was obtained from all individual participants who were involved in the user study.
Ethical approval: The research is part of PhD work of the first author. All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional research committee (Department Research Committee, Board of Research Studies of the University of Kashmir) and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Competing Interests
The authors did not receive support from any organisation for the submitted work. All authors certify that they have no affiliations with or involvement in any organisation or entity with any financial interest or non-financial interest in the subject matter or materials discussed in this manuscript.
Data Availability Statement
The datasets generated and programme/script written during and/or analysed during the current study are not publicly available due to potential misuse but are available from the corresponding author on reasonable request.
- Idris I., Selamat A., and Omatu S., (2016), “Hybrid Email Spam Detection Model with Negative Selection Algorithm and Differential Evolution,” Engineering Applications of Artificial Intelligence, Vol. 28, 97–110.
- Tewari A., Jain A. K., and Gupta B. B., (2016), “Recent Survey of Various Defense Mechanisms against Phishing Attacks,” Journal of Information Privacy and Security, Vol. 12, 03-13.
- Dinh S., Azeb T., Fortin F., Mouheb D., and Debbabi M., (2015), “Spam Campaign Detection, Analysis, and Investigation,” Digital Investigation, Vol. 12, S12–S21.
- Ong R., (2015), “Cyber-bullying and Young People: How Hong Kong Keeps the New Playground Safe”, Computer Law & Security Review, Vol. 31, 68–678.
- Dinev T., (2006), "Why spoofing is serious internet fraud", Communications of the ACM, Vol. 49, Issue 10, 76-82.
- Laorden C., Pedrero X. U., Santos I.., Sanz B., J. Nieves and P. G. Bringas, (2014), “Study on the Effectiveness of Anomaly Detection for Spam Filtering”, Information Sciences, Vol. 277, 421–444.
- Kaur R., Singh S., Kumar H., (2018), “Rise of Spam and Compromised Accounts in Online Social Networks: A State-of-the-Art Review of Different Combating Approaches”, Journal of Network and Computer Applications, Vol 112, 53-88.
- Islam R., Abawajy J., (2013), “A multi-tier phishing detection and filtering approach", Journal of Network and Computer Applications, Vol 36, 324–335.
- Mujtaba G., Shuib L., Raj R. G., Majeed N., and Al-Garadi M. A., (2017), Email Classification Research Trends: Review and Open Issues, IEEE Access, Vol. 5, 9044-9064
- Hatton L. and John A., (2017), “Delivering Genuine Emails in an Ocean of Spam,” IEEE Software, Vol. 34, Issue 4, 11-15.
- Castiglione A., Prisco R., Santis A. , Fiore U., Palmieri F., (2014), “A botnet-based command and control approach relying on swarm intelligence”, Journal of Network and Computer Applications, Vol 38, 22–33.
- Razak M.F.A., Anuar N. B., Salleh R., Firdaus A., (2016), “The rise of “malware”: Bibliometric analysis of malware study”, Journal of Network and Computer Application, Vol 75, 58-76.
- Kim W., Jeong O. R., Kim C., and So J., (2010), “On botnets”, Proceedings of the 12th International Conference on Information Integration and Web-based Applications & Services (iiWAS '10), 5-10, Paris, November 08-10.
- Bazydło P., Lasota K., and Kozakiewicz A., (2017), “Botnet Fingerprinting: Anomaly Detection in SMTP Conversations,” IEEE Security & Privacy, Vol. 15, Issue 6, 25-32.
- Stringhini G., Hohlfeld O., Kruegel C., and Vigna G., (2014), “The Harvester the Botmaster and the Spammer: On the Relations between the Different Actors in the Spam Landscape,” Proceedings of the 9th ACM Symposium Information Computer and Communications Security, 353-363, Kyoto, June 04-06.
- Khan W.Z., Khan M.K., Muhaya F.T.B.., Aalsalem M.Y. and Chao H.C., (2015), “A Comprehensive Study of Email Spam Botnet Detection,” IEEE Communications Surveys & Tutorials, Vol. 17, Issue 4, 2271-2295.
- Shannon C. and Moore D., (2004), “The Spread of the Witty Worm,” Security & Privacy Magazine, Vol 2, Issue 4, 46-50
- McCarty B., (2003), “Automated Identity Theft,” IEEE Security & Privacy, Vol. 99, Issue 5, 89-92
- Rajab M.A., Zarfoss J., Monrose F., and Terzis A., (2006), “A Multifaceted Approach to Understanding the Botnet Phenomenon,” Internet Measurements Conference (IMC), 41-52, Rio de Janeiro, October 25–27.
- Ouyang T., Ray S., Allman M., and Rabinovich M., (2014), “A Large-Scale Empirical Analysis of Email Spam Detection through Network Characteristics in a Stand-Alone Enterprise”, Computer Networks, Vol. 59, 101-121
- Saini H.S., Sayal, Rishi, Rawat, and Singh S., (2015), “Innovations in Computer Science and Engineering,” Proceedings of the Third ICICSE, Chapter Effectiveness of Email Address Obfuscation on Internet.
- Polakis I., Kontaxis G., Antonatos S., Gessiou E., Petsas T., Markatos E. P., “Using Social networks to Harvest Email Addresses,” (2010), WPES '10 Proceedings of the 9th annual ACM workshop on privacy in the electronic society, 11-20, Chicago, October 04-04.
- Wood S. K., (2011), “Methods, Systems, and Computer Program Products for Mitigating Email Address Harvest Attacks by Positively Acknowledging Email to Invalid Email Addresses”, United States Patent Application Publication Wood, Pub. No. US 2011/0145922, Pub. Date. June 16, 2011.
- Prince M. B, Holloway L., Langheinrich E., Dahl B. M., and Keller A. M, (2005). “Understanding How Spammers Steal Your Email Address: An Analysis of the First Six Months of Data from Project Honey Pot,” Conference on Email and Anti-Spam (CEAS)
- Cooke E., Jahanian F., and McPherson D., (2005), “The Zombie Roundup: Understanding, Detecting, and Disrupting Botnets,” Proceedings of SRUTI ’05 Workshop, USENIX Association, Berkeley CA, 35–44.
- Khattak S., Ahmed Z., Syed A. A., Khayam S. A., (2015), “BotFlex: A community-driven tool for botnet detection”, Journal of Network and Computer Applications, Vol 58, 144–154.
- Aslam M., Idrees R., Baig M., and Arshad M., (2004), “Anti-Hook Shield against the Software Keyloggers,” Proceeding of the National Conference on Emerging Technologies, 189-191.
- Han J., Kwon J., and Lee H., (2008), “HoneyID: Unveiling Hidden Spywares by Generating Bogus Events,” Proceedings of The Ifip Tc 11 23rd International Information Security Conference. SEC 2008. IFIP-The International Federation for Information Processing, Vol. 278, 669-673, Boston, MA.
- Fu J., Liang Y., Tan C., and Xiong X., (2010), “Detecting Software Keyloggers with Dendritic Cell Algorithm,” Proceeding of the 2010 International Conference on Communications and Mobile Computing, 111-115.
- Al-Hammadi Y. and Aickelin U., (2008), “Detecting Bots Based on Keylogging Activities”, ARES '08 Proceedings of the 2008 Third International Conference on Availability, Reliability and Security, 896-902, March 04-07.
- Herley C. and Florencio D., (2006), “How to Login from an Internet Café Without Worrying About Keyloggers,” Proceeding of the Association for Computing Machinery, 2.
- Spitzner L., (2003), “The Honeynet Project: Trapping the Hackers,” IEEE Security & Privacy, Vol. 1, Issue 2, 15-23.
- Kugisaki Y., Kasahara Y., Hori Y., and Sakurai K., (2007), “Bot Detection Based on Traffic Analysis,” The 2007 International Conference on Intelligent Pervasive Computing IPC, 303-306.
- Goebel J. and Holz T., (2007), “Rishi: identify bot contaminated hosts by IRC nickname evaluation,” HotBots'07 Proceedings of the first conference on First Workshop on Hot Topics in Understanding Botnets, USENIX Association, 8.
- Estévez-Tapiador J.M, García-Teodoro P., Díaz-Verdejo J.E., (2004), “Anomaly detection methods in wired networks: a survey and taxonomy,” Computer Networks, Vol. 27, Issue 16, 1569-84.
- Schiller C., Binkley J., Evron C., Willems C., (2007), “Botnets - The killer web app. Syngress”, Digital Forensics and Cyber Crime, 179-208.
- Zhao D., Traore I., Sayed B., Lu W., Saad S., Ghorbani A., Garant D., (2013), “Botnet detection based on traffic behavior analysis and flow intervals,” Computers & Security, Vol. 39, 2-16.
- Garg S. and Sharma R. M., (2017), "Anatomy of botnet on application layer: Mechanism and mitigation," 2017 2nd International Conference for Convergence in Technology (I2CT), 7-9 April, 024-1029.
- Hoquea N., Bhuyana M.H., Baishyaa R. C., Bhattacharyyaa D. K., Kalitab J. K., (2014), “Network Attacks: Taxonomy, Tools and Systems”, Journal of Network and Computer Applications, Vol 40, 307-324.
No competing interests reported.