A Survey on Honeypot Based Signature Generation Techniques in Computer Network Security

A Survey on Honeypot Based Signature Generation Techniques in Computer Network Security Geetika yadav 1, Ms.Prabhjot Kaur 2 1 M.Tech Student, Department of CSE, B.S.Anangpuria Institute of Technology and Management, Faridabad Haryana, India 2 Assistant Professor, Department of CSE, B.S.Anangpuria Institute of Technology and Management, Faridabad Haryana, India Abstract- Honeypot is a resource that is used in the area of network security, which is intended to be compromised. Honeypots reduces the number of false alerts as each traffic is considered as suspicious.internet worms are of major concern for information and network security. Worms are malicious codes which propagate themselves, after affecting a host will try to infect other hosts. This paper describes Anomaly based detection technique and signature based detection technique to detect the presence of worm and generate signature for the detected worm. Keywords:Cyber attack,honeypots, polymorphic worm,security. I. Introduction A honeypot is a resource whose value is being attacked or compromised. It traps attacks, records intrusion information about tools and activities of the hacking process and prevents attacks. Every traffic to and from a honeypot is considered as unauthorized activity. It utilizes network s unused IP s and analyze attackers behavior and decreases false positives. There are various types of honeypots available based on their aims and the level of interaction. If we look at the aims of the honeypots we can see that there are two types of honeypots which are research honeypots and production honeypot. A.Research honeypot Research honeypots are used by military,research and government organizations. Their aim is to discover new threats and learn more about the blackhat motives and techniques. The objective is how to protect the system better. They capture huge amount of information about the attack. Research honeypot is an excellent tool for capturing automated attacks such as auto-rooters or worms. Research honeypots contribute little to the direct security of an organization. B. Production honeypots Production honeypot is implemented inside the production network to help mitigate risk. They protect the target system by deceiving and detecting attacks, giving alert to administrator. They are capturing limited amount of information. We can categorize honeypots according to the level of interaction. Level of interaction means how much the hacker will be able to interact with the system. More level of interaction brings more risk into the network security. There are three categories of levels of interaction in honeypots these are low interaction honeypot, medium interaction honeypot and high interaction honeypot. A.Low Interaction honeypot Low interaction honeypots are used to detect the hackers and deceive them by emulating the operating system services and port services on the host operating system. The interaction with other hosts is limited which reduces the propogation of attacks. These can be used to identify new worms or viruses and analyzes the traffic that is going on through the network. It captures limited information which is mainly transactional data and very limited interaction ISSN: 2231-2803 http://www.ijcttjournal.org Page 276

therefore it is very easy to fingerprint. Examples of low interaction interaction honeypot are Honeyd,Spector, KFsensor and Dionaea. Honeyd Honeyd is developed by Niels Provos from university of Michiga. Honeyd is an open source solution and designed for UNIX systems. It is configurable so anyone can create their own services and decide which port to open and listen. Honeyd captures TCP traffic that hacker is generating. When the hacker establishes the connection with Honeyd, Honeyd generates fake messages and return them to the hacker to fool the hacker. It can capture the connection on any port and it is being able to change services. Nepenthes Nepenthes is developed with Mwcollect. According to Maggie F. and Zanero S. Nepenthes is working on five modules which are vulnerability, shellcode parsing, fetching, logging and submission modules. Vulnerability function allows us to create vulnerable services. Shellcode parsing takes the payload and examine on it and get information about the extracted data. If any important data is found to examine then fatch functionality gets the malware and submit to the center part. You can log the information that you have by logging function of Nepenthes.Nepenthes is used for mostly malicious software that are spreading over internet automatically. One of the strength of the Nepenthes is that it emulates FTP and TFTP servers so the attacker can upload the malicious software to the honeypot which allows the forensic party to analyze the threat. Fig.Honeyd structure from virtual honeypots:from Botnet tracking to Intrusion Detection B. Medium interaction honeypots Medium Interaction honeypots are most advanced than low interaction honeypots. Still operating system does not exists. More information and more complicated attacks from the hackers can be obtained. MWcollect, Honeytrap and Nepenthes are some of the medium interaction honeypot that are used today. Fig.Nepenthesis architecture from Maggi F. and Zanero S. C. High Interaction Honeypot High interaction honeypots are the most advanced honeypots.unlike Low interaction and Mediun interaction honeypots there is an operating system.more data can be captured from hackers activities. These are also known as GEN-II honeypots and stated development in 2002.They provide better ISSN: 2231-2803 http://www.ijcttjournal.org Page 277

data capture and control mechanisms. These kind of honeypots are very time consuming and difficult to maintain. The number of honeypots in the network is limited. The risk associated with these honeypots is higher because they can be used easily as launch pads for attacks. Example of High interaction honeypot is Honeywall. Honeywall The Honeywall has three virtual network interfaces et0 is bridged to vmnet6,it is the attacker side.eth1 is bridged to vmnet5,it is the honeypot side. Finally eth2 is bridged to vmnet3,it is the management administration and it allows remote administration of Honeywall.Eth0 and eth1 are making a bridge thus none of these interfaces have a network address making these two interfaces invisible. Once managed to install and run all the virtual machines properly, we use the attacker machine in order to hack the honeypot. The first step is to detect any security flow that we could exploit. In order to do that we used two tools : Nmap and Nessus[5]. hosts can communicate. A worm program is selfreplicating: it remotely exploits a software vulnerability on a victim host, such that the victim becomes infected, and itself begins remotely infecting other victims. Researchers attention has turned to methods for containing the spread of a worm. Three chief strategies exist for containing worms by blocking their connections to potential victims: discovering ports on which worms appear to be spreading, and filtering all traffic destined for those ports and discovering source addresses of infected hosts and filtering all traffic from those source addresses; and discovering the payload content string that a worm uses in its infection attempts, and filtering all flows whose payloads contain that content string. Every worm has some invariant byte pattern which is used as signature for detecting a worm. Worm detection algorithms are categorized into two categories Anomaly based detection and Signature based detection. Anomaly based system observe the traffic statistics and host behavior to detect previously known worms to detect malicious traffic it requires to understand normal traffic behavior. this method is found to be effective in detecting unknown worms, it generates high false alarm. Signature based detection looks for specific byte sequence in each packet. If any match found it will be identified as malicious[12]. II. Signature Generation Techniques Attacker Vmnet6 IP:192.168.1.6 Mask:255. 255.255.0 External interface Vmnet6 Eth0 Internal No IP interface Eth1 No IP Management Interface Vmnet3 Eth2 IP:192.168..232.1 Manage ment Vmnet3 IP:192.16 8.232.2 Mask:255.255.255. 0 To generate signature for the detected worm so that they can be detected early and can not propogate our system. For this several techniques are available which are given below: A. Content based Signature generation techniques Honeypot IP:192.168. 1.110 Mask:255.2 55.255.0 Internal Interface Eth1 No IP Fig.Honeywall Implementation Several algorithm have been proposed for anomaly based worm detection and signature based detection.but none can cover entire range of worms.one or the early work in this category is Honeycomb, proposed by Keibach and Crowcroft.Honeycomb combines honeypot technology with automated signature generation scheme to detect malicious network traffic Honeycomb generates signature consisting of a single contiguous substring of a worms payload to match all worms instances. Honeycomb has implemented Longest Common Substring(LCS) algorithm to spot the similarities in packet payloads. Problem with Honeycomb is that it generates single contiguous substrings of worm s payload to match all instances of polymorphic worms. Honeycomb often generate multiple alarms for same attack and unable to detect multiple instances of a polymorphic worms[12]. In recent years, a series of Internet worms has exploited the confluence of the relative lack of diversity in system and server software run by Internet-attached hosts, and the ease with which these Hyang-Ah Kim and Karp describes Autograph a distributed, automated worm signature generation scheme to detect polymorphic worms. Autograph takes input from across DMZ traffic that includes benign traffic and selects suspicious traffic ISSN: 2231-2803 http://www.ijcttjournal.org Page 278

using certain heuristic. Payloads partition is done into different content block using COPP algorithm. The content blocks are analyzed and Autograph selects most frequently occurring byte sequence across the flows in suspicious flow pool. Prevalence histogram is generated for each content block which acts as worm signature. Polymorphic worms may change their payloads in each injection. Autograph fails to address this problem[12]. James Newsome,Brad Karp and Dawn Song address these problems in Polygraph by generating multiple disjoint content substrings to match all instances of a polymorphic worm. They observed that multiple invariant substrings is often present in all variant payloads of a polymorphic worm. Such invariant substrings include protocol framing byte,return addresses and in some cases obfuscated code. Polygraph divides signatures into tokens-a contiguous byte sequence. The system extracts tokens automatically and represents each suspicious flow as a sequence of tokens.the system is noise tolerant the quality of signature depends on the performance of the flow classifier[13]. Zhichun et al., have proposed Hamsa-a network based signature generator scan be connected to routers via a span port or an optical splitter for monitoring the traffic. Hamsa follows the Polygraph token based approach, but replaced suffix tree method of token extraction with light weight suffix array method which increases the speedup of token extraction process 100 folds.hamsa signature quality is also dependent on the performance of the flow classifier chosen. Presence of too much noise will increase the complexity of signature generation algorithm and reduce the quality of signature generated[12]. LISABETH is an improved version of hamsa. All these techniques generate automated signatures for polymorphic worms based on multiple invariant substrings. But these signatures are based on single instances of multiple worms. Hence they can detect only the known worms. Yong Tang et.al has adopted double-honeynet technique which includes two honeypots, one honeypot for inbound traffic with high interaction and other for outbound traffic with low interaction. Since the outbound honeypot is low interactive- it is not able to collect all the worm instances hence it is not able to generate an efficient signature[16]. Mohssen et al., have proposed double honeynet with high interactive honeypot for outbound connections, hence can collect sufficient amount of worm instances. For signature generation different methods like protocol classifier, clustering based on destination port, substring extraction algorithm, an efficient algorithm that converts worm substrings into binary representations and using these binary representation for pattern matching [15], using principal component analysis technique have been used to reduce the dimension of worm payloads[12]. B. Anomaly Based signature generation technique Here a virtual system is set to analyze the behavior of worm and this analysis is used for the detection of the similar worms. A specific worm after attacking a system goes in search of a system with similar vulnerability so this behavior is detected by the virtual machine and is used as signature for worm detection. Pan Xiaohui et al., have designed a hybrid method based on worms propagation model. Authors proposed a hybrid method for detecting polymorphic worm accurately in the early stage. It combines port scan detection and emulation, port scan detects the suspicious packet and emulator first executes every instruction byte and detects is it a worm or not. Song Qing et al., proposes a Worm Terminator which detects and contains the fast spreading worm based on its characteristic a fast spreading worm will start to infect others as soon as it successfully infects one host. Worm Terminator also exploits by observation that a fast spreading worm keeps exploiting the same set of vulnerabilities when infecting new machines. III. Conclusion This paper summarizes some of the techniques to generate signature for detected worms. Among the techniques available Content based signature generation technique is easy to implement because it considers the payload of worm and treat them as strings of bytes which are used to generate signature and these signatures are stored in signature pool but anomaly based signature generation technique analyzes the behavior of the worm which requires efficient training which in real time is difficult to achieve. References [1]Mathew L.Bringer, Christopher A. Chelmecki, Hiroshi Fujinoki A Survey:Recent Advances and Future Trends in Honeypot Research I.J. Computer Network and Information Security,2012. [2]A. Chandra,K.Lalitha Honeypots:A New Mechanism for Network Security IJPaper Vol.04 special Issue 01 2013. [3]Srivastha S Rao,Vinay Hedge,Boruthalupula Maneesh,Jyoti Prasad N M,Suhas Suresh Web based Honeypots Network International Journal of Scientific and Research Publications,Volume3,Issue8,2013. [4]Gary Kelly,Diane Gan Analysis of Attacks Using a Honeypot Springer-Verlag Berlin Heidelberg,2011. ISSN: 2231-2803 http://www.ijcttjournal.org Page 279

[5]Deniz Akkaya-Fabien Thalgott Honeypots in network security-a Thesis Linnaeus University. [6] John P. John, Fang Yu, Yinglian Xie, Arvind Krishnamurthy. Martín Abadi Heat-seeking Honeypots: Design and Experience International World Wide Web Conference Committee,2011. [7]Feng zha g,shijie Zhou,Zinguang Qin,Jinde Liu Honeypot:A supplemented active defense system for network security IEEE 2003. [8] Spitzner, Lance. Honeypots: Definitions and Value of Honeypots, May 2003, accessed: November 2012, URL: http://www.trackinghackers.com/papers/honeypots.html. [9] Robert Lemos, 5 Reasons Every Company Should Have A Honeypot, 1 st October 2013, Accessed 23 March 2014,http://www.darkreading.com/advancedthreats/5-reasons-every-company-should-have-aho/240162106. [10] Almutairi, Abdulrazzaq Survey of High Interaction Honeypot Tools: Merits and Shortcomings, June 2012, Date Accessed: October 2012 http://www.cms.livjm.ac.uk/pgnet2012/proceedings/ Papers/1569604821.pdf. [11]Karthik S. Samudrala,B. And Yang, A.T. Design of network security Projects using honeypots Journal of computing sciences in colleges. [12]Sounak Paul,Bimal Kumar Mishra Honeypot Based Signature for Defense Against Polymorphic Worm Attack in Networks IEEE International Advance Computing Conference(IACC),2013. [13] Newsome J, Karp B, Song D. "Polygraph : Automatically GeneratingSignatures for Polymorphic Worms." IEEE Symposium on Securityand Privacy. 2005. pp 226-241. [14] Zhichun Li, Manan Sanghi, Yan Chen, Ming Yang Kao, Chavez B."Hamsa : Fast Signature Generation for Zero Day PolymorphicWorms with Provable Attach Resilience." IEEE Symposium onsecurity and Privacy.2006. pp15-47. [15]Bimal Kumar Mishra and Dinesh Kumar Saini, SEIRS epidemics model with delay for transmission of malicious objects in computer network,applied Mathematics and Computation,Elsevier,188(2007). [16]R.T. Goswami,Avijit Mondal,Bimal Kumar Mishra and N.C. Mahanti Defending Polymorphic Worms in Computer Network using Honeypot International Journal of Advanced Computer Science and Applications,Vol.3,No.10,2012. ISSN: 2231-2803 http://www.ijcttjournal.org Page 280