Review Study on Techniques for Network worm Signatures Automation

Transcription

1 Review Study on Techniques for Network worm Signatures Automation 1 Mohammed Anbar, 2 Sureswaran Ramadass, 3 Selvakumar Manickam, 4 Syazwina Binti Alias, 5 Alhamza Alalousi, and 6 Mohammed Elhalabi 1, 3,3,4,6 National Advanced IPv6 Centre (NAv6), Universiti Sains Malaysia, {anbar,sures,selva,syazwina}@nav6.org 5 School of Computer and Communication Engineering, Universiti Malaysia Perlis, Perlis, Malaysia, g @studentmail.unimap.edu.my Abstract The network worm signature is a specific string that exists in the packet payload. This string will be used by signature based IDSs such as Snort (Snort) to compare it with the existing signatures in the database, if there is a match found in signatures database, worm can be detected, and otherwise the network worm cannot be detected. This paper presents a review study on network worms signature automation techniques. This study will first define the network worm and worm signatures. Furthermore, it will discuss the severity of presence the network worm in the network. The network propagation and activation schemas will be discussed. In addition, this article explores the current techniques to automate signatures for network worms. 1. Introduction Keywords: Network worm, Worm signature, Intrusion Detection System (IDS) A network worm is a self-propagating, self-duplicating malicious code that spread without human intervention in computer networks and attacks vulnerable hosts and services. Network worms are typically classified based on two attributes: methods used to spread and the techniques used to exploit vulnerabilities. Meanwhile, network worms have destructive effects in the network topology, resources and service [1-7]. Therefore, many researchers attention have been grabbed to propose techniques to automate signatures for network worms. According to Li, Salour & Su [8] the life cycle of a network worm after its release typically includes four phases: target finding or scanning, network worm transferring, network worm activation, and infection. The network worm is active on the network during target finding; network worm transferring and can be detected by network-based intrusion detection systems (NIDSs). The activities in the last two phases (activation and infection) are limited to local machines and are harder to detect by NIDSs because the network worm activities are more focused on individual computers rather than on the entire network. In contrast, the activities in the first two phases (scanning and transferring) are easier to detect because network worm activities are centered on the network, such as the existence of abnormal traffic generated from scanning. Figure 1shows the typical location of NIDS in the network. Figure 1. The typical location of NIDS in the network Advances in information Sciences and Service Sciences(AISS) Volume5, Number17, December

2 Similar to a computer virus or other malicious codes, a network worm has a signature that can be used by IDS in the detection phase. Automated signature generation for new attacks is extremely difficult due to three reasons. First, in order to create an attack signature, we must identify and isolate attack traffic from legitimate traffic. Automatic identification of new worms is critical, which is the foundation of other defense measures. Second, the signature generation must be general enough to capture all attack traffic of certain type while at the mean time specific enough to avoid overlapping with the content of normal traffic in order to reduce false-positives. This problem has so far been handled in an ad-hoc way based on human judgment. Third, the defense system must be flexible enough to deal with the polymorphism in the attack traffic. Otherwise, worms may be programmed to deliberately modify themselves each time they replicate and thus fool the defense system [9]. The packets that were used to transfer malicious code from the sender to the destination have specific patterns and noticeable behaviors. After the malicious code infects the destination host, this new host will act in the same manner as the host that infected it. 2. The severtiy of worms The severity of network worms depends on the propagation process, wherein network scanning is initiated to determine the vulnerability of the host and services. Network scanning will degrade network performance and consume bandwidth and resource (CPU and memory) by making the network machines busy due to the requests that are received and responded in the scanner machine. Once a network worm infects a network, it will automatically begin to propagate, which will cause great destruction throughout the network due to network congestion. This will create unnecessary traffic, which serves only network worm propagation. Figure 2 shows the worm propagation process. 3. Network Worm Propagation Schemes Figure 2. Worm propagation As reported by Weaver et al. [5], there are three network worm propagation schemes which are as follows (1) self-carried (2) embedded (3) second channel. Self-carried network worms are actively transmit itself to the target host (the network worms are fully transmitted to the target during the initial connection), second channel network worms need second communication channel, in this scheme, the network worm communicates with the victim machine using original channel then the victim machine connects back to the infecting machine using another channel to download the network worm payload. The embedded propagation scheme is very stealthy and it s done by append the payload after, or replace, legitimate traffic to hide itself. No anomalous events will be triggered, and it is hard for anomaly-based detection systems to detect. In addition to the three propagation schemes discussed, 9

3 Botnets have been utilized to propagate network worms, spams, spyware, and launching distributed denial-of-service (DDoS) attacks [8]. A Botnet is a group of compromised hosts under the control of a Botmaster. The communication channel for the Botmaster to issue commands that can be implemented using different protocols such as http or point-to-point (P2P) protocols. However, the majority of Botnets use the Internet Relay chat (IRC) protocol for this purpose[14]. 4. Network Worm Activation Schemes Network worm activation means running network worms under certain condition or schedule. Weaver et al. [15] classified the network worm activation as following: 1. Human activated, this type of network worm activation require human intervention to execute the network worm 2. The network worm activate based on specific activity which is performed by user (such as open CD or bin drive) 3. Activated by a scheduled processes. In scheduled process, network worms are activated by a legitimate automated process which has not been properly secured, such as a legitimate program which automatically updates itself from an infected web server. Self-activated, this kind of network worm can activate without human intervention, and it considers the most dangerous one, this thesis focus on detecting this kind of network worms. 5. Techniques for automating network worm signatures Snort open source network-based intrusion detection system (NIDS) has the ability to perform realtime traffic analysis and packet logging on Internet Protocol (IP) networks. Snort performs protocol analysis, content searching, and content matching. Snort can be configured in three main modes: sniffer, packet logger, and network intrusion detection. In sniffer mode, the program will read network packets and display them on the console. In packet logger mode, the program will log packets to the disk [16]. In intrusion detection mode, Snort has the ability to detect different type of malicious code such as network worm. In case of network worm, Snort is a Signature Based Network Worm Detection (SBWD). In other words, Snort will detect the presence of network worm in the network based on the networm worm signature, Snort checks the incoming packet payload and system log files against the network worm signatures that are already stored in the IDS database. An alert will be triggered when a match is found [16]. Kreibich & Crowcroft [17] developed Honeycomb which aims to generate signature for malicious network traffic automatically and it uses pattern-detection techniques and packet header similarities tests on traffic captured from Honeypots. The traffic that bypass the Honeypot is logged into log file, the log file consist of IP, TCP and UDP header as well as payload data. After protocol analysis, Honeycomb proceeds to the analysis of the reassembled flow content. Honeycomb applies the longest common substring (LCS) algorithm on binary strings that built out the exchanged messages. It does this in two different ways which are horizontal detection and vertical detection. The contents of the signature pool are periodically reported to an output module which implements the actual logging of the signature records. At the moment, there are modules that convert the signature records into Bro [18] or pseudo-snort format, and a module that dumps the signature strings to a file. Figure 3 shows Honeycomb architecture. 10

4 Figure 3. Honeycomb architecture (Kreibich & Crowcroft, 2004). Autograph proposed by [19] is a distributed system for automatically generating network worm signatures for Bro [18] and Snort [16]. Autograph aims to automatically generate signatures for unknown network worms that propagate using TCP transport. Autograph generates signatures by analyzing the prevalence of portions of flow payloads, and thus uses no knowledge of protocol semantics above the TCP level. It is designed to produce signatures that exhibit high sensitivity (high true positives) and high specificity (low false positives). Unlike Honeycomb, Autograph s inputs are packet traces from a demilitarized zone (DMZ) that includes benign traffic. Content blocks that match enough suspicious flows are used as input to COPP, an algorithm based on Rabin fingerprints that searches for repeated byte sequences by partitioning the payload into content blocks. Similar to Honeycomb, Autograph generates signatures consisting of a single, contiguous substring of a network worm s payload to match all network worm instances. These signatures, unfortunately, fail to match all polymorphic network worm instances with low false positives and low false negatives. Earlybird is a system proposed by Singh et al [20] for generating signatures to detect network worms based on the assumption that the network worms must generate significant traffic to propagate. This traffic will contain common substring which will transfer from source (attacker) to destinations. Based on this assumption the authors believe that identifying this traffic pattern is sufficient for detecting network worms, for identifying traffic pattern, a content sifting is proposed which work as the follows: (A) for each network packet, the content is extracted and all substrings processed, and (B) each substring is indexed into a log table that increments a count field for a given substring each time it is found. In effect, this table implements a histogram of all observed substrings. To maintain a count of unique source and destination addresses, each table entry also maintains two lists, containing IP addresses that are searched and potentially updated each time a substring count is incremented. Sorting this table on the substring count and the size of the address lists will produce the set of likely network worm traffic. Earlybird, also like Honeycomb and Autograph, generates signatures consisting of a single, contiguous substring of a network worm s payload to match all network worm instances. These signatures, however, fail to match all polymorphic network worm instances with low false positives and low false negatives. A double-honeynet is another system aims to detect new network worms automatically, the proposed system consists of two honeynets namely honetpot1 and honeynet 2. Gate translator at the edge router between the local network and the internet is deployed to detect the unwanted inbound connections and forwards them to Honeynet 1, once Honeynet 1 is compromised, the network worm will attempt to make outbound connections. Each honeynet is associated with an Internal Translator implemented in router that separates the Honeynet from the rest of the network. The internal translator 1 intercepts all outbound connections from honeynet 1 and redirects them to honeynet 2. Honeynet 2 will capture the packets that make outbound connections, and hence the Double-honeynet forwards only packets that make outbound connections. If the Honeynet 2 captures enough instances of network worm payloads, the internal 11

5 translator 2 will automatically forwarded to the signature generator which generates signatures, it receives the packet payloads captured by double-honeynet. These packets are checked by the protocol classifier which classifies packets in terms of different protocols (TCP/UDP) and port numbers. Then the Known-network worms filter component filters out known-network worm samples and pass the remaining samples (unknown network worms) to the Signature Generation Algorithms component which extracts all the distinct tokens in the samples. Then it clusters the distinct tokens according to their similarity. The set of tokens in each cluster is used as a signature for that cluster. The total number of the signatures is equals the total number of clusters [21]. Figure 4 shows double Honeynet system architecture. Table1 shows a summary of techniques for automating network worm signatures. Figure 4. Double Honeynet system architecture Table1: A summary of techniques for automating network worm signatures. Approach Description Disadvantage Honeycomb approach aims to generate signature for malicious network traffic automatically and it Kreibich & Fail to match all polymorphic uses pattern-detection techniques Crowcroft, 2004 network worm instances and packet header similarities tests on traffic captured from Honeypots Kim & Karp, 2004 Singh, Estan, Varghese, & Savage, 2004 Proposed an Autograph which aims to automatically generate signatures for unknown network worms that propagate using TCP transport. Autograph generates signatures by analyzing the prevalence of portions of flow payloads, and thus uses no knowledge of protocol semantics above the TCP level Proposed an Earlybird system for generating signatures to detect network worms based on the assumption that the network worms must generate significant traffic to propagate. This traffic will contain common substring which will transfer from source (attacker) to destinations Fail to match all polymorphic network worm instances Fail to match all polymorphic network worm instances 12

6 6. Conclusion This paper has highlighted the severity of presence the network worm in the network. Then the existing approach for worm signature automations are explored and advantages and drawbacks for each approach are highlighted, the existing approaches which are used to generate network worm signature are unfortunately fail to match all polymorphic network worm instances (because the signature of polymorphic network worm keep on changing every time it send from source to destination and proposed approaches based on generating static signature for detected worm ) [22]. 7. Acknowledgment This research is supported by National Advanced IPv6 Center of Excellence (NAv6), Universiti Sains Malaysia (USM). 8. References [1] S. Staniford, V. Paxson, and N. Weaver, "How to Own the Internet in Your Spare Time," in USENIX Security Symposium, 2002, pp [2] D. Moore, C. Shannon, G. M. Voelker, and S. Savage, "Internet quarantine: Requirements for containing self-propagating code," in INFOCOM Twenty-Second Annual Joint Conference of the IEEE Computer and Communications. IEEE Societies, 2003, pp [3] S. Chen and Y. Tang, "Slowing down internet worms," in Distributed Computing Systems, Proceedings. 24th International Conference on, 2004, pp [4] C. Kruegel and G. Vigna, "Anomaly detection of web-based attacks," in Proceedings of the 10th ACM conference on Computer and communications security, 2003, pp [5] D. Moore, V. Paxson, S. Savage, C. Shannon, S. Staniford, and N. Weaver, "Inside the slammer worm," Security & Privacy, IEEE, vol. 1, pp , [6] C. Cowan, C. Pu, D. Maier, H. Hinton, J. Walpole, P. Bakke, S. Beattie, A. Grier, P. Wagle, and Q. Zhang, "StackGuard: Automatic adaptive detection and prevention of buffer-overflow attacks," in Proceedings of the 7th USENIX Security Symposium, 1998, pp [7] M. W. Eichin and J. A. Rochlis, "With microscope and tweezers: An analysis of the internet virus of november 1988," in Security and Privacy, Proceedings., 1989 IEEE Symposium on, 1989, pp [8] P. Li, M. Salour, and X. Su, "A survey of internet worm detection and containment," Communications Surveys & Tutorials, IEEE, vol. 10, pp , [9] Y. Tang and S. Chen, "Defending against internet worms: A signature-based approach," in INFOCOM th Annual Joint Conference of the IEEE Computer and Communications Societies. Proceedings IEEE, 2005, pp [10] C. C. Zou, W. Gong, and D. Towsley, "Code red worm propagation modeling and analysis," in Proceedings of the 9th ACM conference on Computer and communications security, 2002, pp [11] Z. Chen, L. Gao, and K. Kwiat, "Modeling the spread of active worms," in INFOCOM Twenty-Second Annual Joint Conference of the IEEE Computer and Communications. IEEE Societies, 2003, pp [12] D. Moore and C. Shannon, "Code-Red: a case study on the spread and victims of an Internet worm," in Proceedings of the 2nd ACM SIGCOMM Workshop on Internet measurment, 2002, pp [13] J. O. Kephart and S. R. White, "Directed-graph epidemiological models of computer viruses," in Research in Security and Privacy, Proceedings., 1991 IEEE Computer Society Symposium on, 1991, pp [14] G. Gu, R. Perdisci, J. Zhang, and W. Lee, "BotMiner: Clustering analysis of network traffic for protocol-and structure-independent botnet detection," 2008, pp [15] N. Weaver, V. Paxson, S. Staniford, and R. Cunningham, "A taxonomy of computer worms," in Proceedings of the 2003 ACM workshop on Rapid malcode, 2003, pp [16] Snort, "A free lightweight network intrusion detection system for UNIX and Windows,"

7 [17] C. Kreibich and J. Crowcroft, "Honeycomb: creating intrusion detection signatures using honeypots," ACM SIGCOMM Computer Communication Review, vol. 34, pp , [18] V. Paxson, "Bro: a system for detecting network intruders in real-time," Computer networks, vol. 31, pp , [19] H. Kim and B. Karp, "Autograph: Toward automated, distributed worm signature detection," in USENIX security symposium,2004, p. 19. [20] S. Singh, C. Estan, G. Varghese, and S. Savage, "The earlybird system for real-time detection of unknown worms," Citeseer2003. [21] M. Mohammed, H. Chan, and N. Ventura, "Honeycyber: Automated signature generation for zeroday polymorphic worms,," in Military Communications Conference, MILCOM IEEE, 2008, pp [22] S. Stafford and J. Li, "Behavior-based worm detectors compared," in Recent Advances in Intrusion Detection, 2010, pp