Life After Signatures Pattern Analysis Application for Zombie Detection Blocking server-side polymorphic malware and blended threats before system penetration Amir Lev, President and CTO Commtouch Israel AVAR 2007 Seoul, Korea Abstract Since early 2007, zombies have become a central component in the supply chain of email-borne malware. Zombie computers are being used for everything from generating new malware variants, to sending out vast quantities of malware simultaneously, and even in some cases serving as unwitting hosts for malware websites in blended attacks. Signature- and heuristic-based anti-virus technology is designed to analyze the binary of email attachments and malware code downloaded from infected web sites. This analysis takes precious time, anywhere from several hours to several days. New technologies are needed to defend against malware variants in the first moments of an outbreak, providing protection at the zero hour. A system that could identify zombie senders or zombiecreated web sites would be able to reduce this time significantly. Pattern detection analysis can be used to proactively identify zombie computers sending malware outbreaks. This method recognizes recurring instances of existing patterns in spam and malware messages, and maps these to the IP addresses of zombies. By detecting new emerging patterns, the system identifies zombies in real time, allowing the system to constantly learn about new outbreaks the moment they begin. This creates a reputation-based method for blocking server-side polymorphic malware and blended threats at the perimeter of organizations. Since messages can be blocked based on the sender reputation alone, the need for time-consuming analysis of the binary is reduced. Email-Based Threats Today Email Under Siege Email, one of the most important communication tools, is also the leading vector for viruses, accounting for 23% of all enterprise malware infections. i Every enterprise has some form of antivirus protection in place, yet malware penetration in the enterprise has become commonplace. In fact, 84% of enterprise networks have been penetrated by email-borne viruses, worms or Trojans, according to Osterman Research. ii These penetrations cause millions of dollars in damage and lost productivity. Virus writers have identified the weak-point of traditional AV engines the time it takes to develop protection for new malware variants and have exploited it to their advantage by flooding the Internet with thousands of new distinct virus variants simultaneously, and utilizing blended delivery agents. As Eugene Kaspersky remarked, The anti-virus industry is slowly giving up because it is getting more and more difficult to resist the increasing number of the threats. iii Securing email poses a particularly difficult challenge because allowing the free flow of messages is vital to business operations. Network administrators must balance between solid security and the open flow of information, and costly compromises are commonplace. Most businesses cannot AVAR 2007 Seoul, Korea Page 1 Commtouch Software, Ltd.
tolerate the risk that legitimate email will be misclassified as a virus and blocked. And yet, they are forced to take broad steps like blocking all executable files from entering the organization because their AV solution is unable to recognize and block only the malicious attachments. This type of restrictive policy leads to blocking legitimate messages, or false positives, and frustration on the part of users. Penetrations May Go Unnoticed at First The majority of computer viruses today are virtually invisible to standard users, unlike in the past. These stealthy malwares are designed to generate illicit revenue quietly for the malware underworld, without being detected. The potential for huge profits has spurred the development of a more malicious breed of malware, capable of evading detection by common anti-virus solutions. Most modern malware is designed to quietly go about its malicious activities without creating any noticeable symptoms. Keyloggers gather financial information and passwords; spyware can transmit sensitive information outside the organization; backdoors open up network connections for hackers to enter and send malware and spam directly from the enterprise network. All these activities are carried out in stealth mode, without causing any noticeable interruption, allowing them to quietly continue their malicious activity. The fact that malware infections are going unnoticed means that users often do not complain about infections, and IT managers are less aware of the exposure risks. However the risks of email-borne malware are present, and getting worse with today s server-side polymorphic malware and blended threats. The Growing Threat of Server-Side Polymorphic Malware Earliest computer viruses were most often released in a single variant, in massive amounts of email with a virus attachment. Then, early experimentation by virus writers showed that multiple virus variants could be used to evade signature-based anti-virus engines. A variant is a slightly altered version of malware code. While virus variants may perform basically the same malicious actions, the dissimilarity in the code can fool some signature-based AV engines seeking an exact match. As AV solutions developed faster signature publishing mechanisms to protect against viruses, malware writers changed their tactics. Based on the success of creating a handful of variants, viruswriters took the use of variants to the extreme and developed server-side polymorphic malware. iv Server-side polymorphic malware launches rapid-burst attacks comprised of vast numbers of variants to circumvent common AV defenses. Server-side polymorphic malware refers to the technique of creating huge arsenals of slightly altered variants of malicious code and releasing them in quick bursts. The release of massive amounts of virus variants in just a few hours maximizes penetration by concentrating the outbreak into the brief period before signatures can be released. Polymorphic malware changes its attributes to make it undetectable by signature- and behaviorbased antivirus and intrusion detection defenses. This distribution method proved so effective against traditional AV solutions that it has now become widespread and has been one of the most popular types of email-borne malware in 2007. One early surge of malware of this type was Happy New Year, at the tail end of 2006/early 2007, followed closely by the Tibs/Zhelatin variants that appeared as the Storm Worm, Valentine s Day greetings, various e-card scams, and so on. Even old-fashioned viruses like Bagle, more than three years old, which started out as a run-of-the-mill, single variant virus, is now a full-fledged serverside-polymorph, at times averaging more than 600 new variants per day. AVAR 2007 Seoul, Korea Page 2 Commtouch Software, Ltd.
Most server-side polymorphic malware is sent from zombies, since significant computing power is needed to generate the vast numbers of variants simultaneously. In addition, using zombies as the distribution mechanism ensures that the malware writers and distributors cover their tracks. If they were to send using traditional, legitimate email sending methods, they would be blacklisted rather quickly. Recent research by Commtouch Labs demonstrated that there is significant overlap between the zombies that spew spam and those that distribute viruses; in fact during an 11-day period, there was a 57% overlap. v Server-Side Polymorphic Malware: Hundreds of Overlapping Variants Source: Commtouch Labs AVAR 2007 Seoul, Korea Page 3 Commtouch Software, Ltd.
Server-Side Polymorphic Malware Attack Strategies High velocity server-side polymorphic viruses use the following key strategies to bypass traditional AV defenses: Vast Variant Quantity: These malwares distribute a vast number of variants. For example, Commtouch measured and blocked more than 800 distinct Happy New Year variants in a single five-minute period. Storm-worm distributed more than 7,000 distinct variants on several days of that outbreak, and over 40,000 altogether during a 12-day period. Since each variant or group of variants requires a different signature, it is impossible for anti-virus engines to keep up with this rapid-fire pace. Brief Variant Lifetime: The fleeting lifetime of each variant is two to three hours on average, and each variant rarely makes a second appearance during the outbreak. Since it takes several hours to develop a new signature or heuristic, and up to several days to distribute to end-users, these shortlived variants are typically out of distribution by the time traditional anti-virus defenses are available. Social Engineering: Multiple subject lines and attachment names are used to confuse users; they can no longer be protected simply by avoiding email messages with known subjects or attachments. Topical subjects are designed to entice people to open the messages. For example, the Storm- Worm subject lines had a true irresistible tabloid quality to them. Blended Threats In the second half of 2007, a new type of outbreak started becoming more commonplace: spam outbreaks containing links to malicious websites. The links in the email may appear to be legitimate, for example to YouTube, however most often and most recently, they lead to domains that are just IP addresses. In some cases the IP address is in the body of the email message. The brief messages usually claim to have something cool or unusual to share with recipients, in order to induce them to click on the link. Once the users click through, they are brought to professional-looking web sites that appear to be legitimate. Examples of these include NFL-tracker sites, arcade game sites, and popular social networking sites like YouTube. In some cases, the site initiates a drive-by attack, automatically downloading malware to users computers, and in other cases, the sites use appealing messages to convince users to click on one of the links on the site. Sample Arcade Games Malware Site, Sept. 2007 AVAR 2007 Seoul, Korea Page 4 Commtouch Software, Ltd.
Because the messages do not carry a malicious payload, there is no attached code for the anti-virus engine to scan, and therefore the messages are often delivered right into users inboxes. Also, because the malware may occur in nearly infinite variants on the different web sites, it often passes into users computers undetected by desktop AV solutions. These blended threats are characterized by: Large numbers of randomized email messages and sites: Malware distributors randomize the subject lines, the message text, and the URLs (often zombie IP addresses), in order to evade detection by traditional content-based email filters. Commtouch has seen attacks with thousands of different IP addresses hosting versions of the same malware web site, with only slight differences, if any. Multiple malware variants: Because the malware is hosted on zombie sites, masters can continuously and automatically update the malware over time, and place different versions on different sites, or even change versions on a single site. Even minor differences in the malware code can make it impossible for some signature- and heuristic-based AV engines to block. Zombie senders and zombie hosts: These types of attacks could not occur without a well-oiled zombie botnet. In the outbreaks, zombies are the senders of the email messages, as well as the hosts of the malware web sites. The sites typically expire after a short time sometimes in a matter of hours. Social Engineering: This well-known tactic is used in both in the email messages and the malware web sites in order to trigger users to take action that will cause them to get infected. Fighting Back With Zombie Detection The remainder of the paper describes a new and accurate method for identifying zombies for the purpose of blocking malware in real-time, based on pattern detection technology. Recurrent Pattern Detection Recurrent Pattern Detection or RPD technology is based on the most fundamental characteristic of spam and email-born malware - its mass distribution over the Internet. Originally developed for identifying spam and malware outbreaks, the technology has been in existence and continually refined since 2002. RPD starts by analyzing over one billion messages daily in the Commtouch Global Detection Center. These messages are gathered from over 180 countries, and a broad range of consumer, enterprise, ISP and even SMB traffic. Broad coverage is a must in order to identify distributed attacks, as well as local ones. RPD then identifies new spam, virus and phishing outbreaks based on their characteristic mass distribution patterns. The massive outbreaks which distribute spam and malware consist of millions of messages intentionally composed differently in order to evade commonly-used filters. Nonetheless, all messages within the same outbreak share at least one and often more than one unique, identifiable value which can be used to distinguish the outbreak. These values, called attack patterns, are detected by RPD within the first moments of a new outbreak. Because tactics for distributing spam and malware are constantly evolving, RPD proactively identifies new and unique AVAR 2007 Seoul, Korea Page 5 Commtouch Software, Ltd.
patterns in real-time in order to determine new outbreaks as they are released to the Internet and begin targeting recipients. RPD and Zombies Zombies are typically connected to the Internet via home broadband modems, which have rapid download speeds, but much more limited upload bandwidth. Since zombies utilize the broadband s uplink, zombies are limited as to how much email they can send within a given time-frame, around one message per second. However zombies do not send email alone, they work in concert with massive botnets. If a typical botnet contains, for example, 100,000 zombie computers, this means a coordinated attack would be spewing out 100,000 messages each second. Commtouch s broad global traffic coverage ensures that in such a case, RPD would identify any new attack pattern within mere seconds. Once an attack pattern is detected, all of the source IP addresses sending that pattern are identified and logged. This pattern identification enables real-time mapping of the zombie IP addresses actively participating in each outbreak. Using Zombie-Detection to Block New Email-Borne Malware Protection at the Zero-Hour One of the main advantages of using zombie-detection as a component of anti-virus technology is the fact that it is enables protection in real-time. No time-consuming analysis of the code is necessary since email-borne malware is blocked based on identifying if the sender is a zombie. This ensures that users are protected at the Zero-Hour, that is, at the moment the threat first appears. It is now clear that malware writers have identified the typical time-window until signatures or heuristics will be available, and consciously launch multiple waves of short-lived variants in order to maximize their effectiveness. Commtouch lab research has shown that many server-side polymorphic malwares, for example, have variants that that last on average two to three hours before disappearing. As a result, zombie-based Zero-Hour protection is becoming a crucial complement to traditional AV techniques. Fighting Malware at the Perimeter The key identifier of a zombie at any single point in time is its IP address, which opens up a whole new way of blocking malicious email before it ever enters an organization. Long before an email message is delivered to the gateway filtering tool or Mail Transfer Agent (MTA), the IP address of the sender is a known entity. In fact, the IP address of the sender is identified as soon as the sender initiates the SMTP session. This means that a large portion of malware protection can be done at the network perimeter, offloading enormous amounts of unwanted and dangerous email before it crosses the threshold. Before the SMTP session starts, a small query can be performed about the sender, in order to determine the sender s IP reputation. In most cases, if the sender is a zombie, its IP address will already be logged in the reputation database. In this case, the delivery should be rejected even before the sender submits the message (i.e. the perimeter device can immediately end the session with a response of permfail.) But even in the ideal scenario where RPD identifies zombies within just a few minutes, one could assume that there is still a small chance of a zombie sending malware before it has been identified. AVAR 2007 Seoul, Korea Page 6 Commtouch Software, Ltd.
In order to understand how this scenario is resolved, it is important to grasp a fuller picture of the data that Commtouch collects about each IP address: In addition to logging zombies in real-time, Commtouch gathers and maintains several crucial data points about each IP address that is sending email at a given time. The data can include such items as: Spam characteristics Virus characteristics Average volume of mail sent Changes from this average Riskiness This data enables each sender to be ranked along a reputation scale of good and bad email senders. If a sender is a known good sender based on this IP reputation database, then the the mail message would be allowed to enter the organization. If the source is unknown, as in the case of a new as yet unidentified zombie, then a short sub-process can be initiated, to determine how to classify the sender. The perimeter device sends a response of tempfail (telling the sender to try and send the message later), temporarily ending the SMTP session. The tempfail process creates a short time window, usually 15-20 minutes until the sender will try to resend the message. By the time the sender retries, RPD will have identified any new attack patterns, and classified almost all new zombie IP addresses participating in the current outbreaks. Thus the previously unknown IP address can be classified as friend or foe. The classification time-period for new attack patterns is just seconds; then, after the pattern is identified, any new IP address sending email containing that attack pattern is determined to be a zombie, so within around 15 minutes, the vast majority of zombies participating in that outbreak will have been identified and stored in the reputation database. Zombie IPs Identified over Time 100.0% 90.0% 80.0% % of Zombie IPs Identified 70.0% 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 99% of zombies in a given outbreak are identified within 15 minutes 0.0% 0 5 10 15 20 25 30 35 Minutes Source: Commtouch Reputation Service AVAR 2007 Seoul, Korea Page 7 Commtouch Software, Ltd.
The method described here of tempfailing unknown senders should not be confused with a different practice, known as graylisting. Recently, a mere tempfail response would have been enough to block the majority of zombie traffic, since zombies did not behave like legitimate MTAs, and did not retry sending their messages. However, recently spammers have modified the zombies sending process and many of them do retry, so this is not an effective method. The method described here combines tempfailing with active zombie classification during the time window gained by the tempfail. Saving ISPs from Zombie Traffic The majority of zombies are home computers, with broadband Internet connections. These computers have been infected with Trojan malware, and are forced to perform at the botmasters will. These home users spewing spam and malware are a liability to their ISPs. They generate vast amounts of junk traffic that can clog the ISPs network and slow it down. In addition to the IT headache of lost bandwidth and storage, they can create reputation problems for the ISP itself and other ISP customers by getting the ISP blacklisted. Blacklists are notoriously slow to remove IP addresses from their recommended block lists, and often lack the granular capability to distinguish between a single IP address producing spam or malware, and an entire range. It is easier to simply block the entire range, and suddenly the ISP finds that multiple customers have problems sending even legitimate email. The immediate result is wasted helpdesk hours to try to assist those customers who have been mistakenly blocked. Advanced zombie-detection can save ISPs the unnecessary IT and helpdesk resources by identifying those home computers which have been converted into zombies. The ISP can either simply block outgoing mail from those customers, or can approach them with a method to remediate the zombie. The end result is happier customers both those who have been notified, as well as those other customers that have not been needlessly blocked. Eliminating Zombies from Corporate Networks In addition to home zombie-computers, it is also possible for corporate computers to get similarly infected, and effective zombie detection methods can identify sources of malicious mail originating from within corporations. By crossing-referencing zombie classifications with DNS records, it is possible to determine if a corporate-owned IP address is generating mail that contains known attack patterns. Even if the sender is behind Network Address Translation (NAT), the enterprise itself can be identified and notified. While outside the scope of this paper, zombies within enterprises are a liability, and open up serious compliance issues, and a method for identifying them is a welcome addition to any security portfolio. Concluding Remarks Blocking zombie-generated email serves as a first-line of defense against email-borne malware and reduces the time and resources required to protect customers against new outbreaks. This method is effective against both server side polymorphic malware and blended threats. Detecting zombies is possible by identifying attack patterns in spam and malware messages, crossing-referencing this information with additional data such as volume over time, and tracking the IP addresses of the senders. Advantages of zombie-detection include: AVAR 2007 Seoul, Korea Page 8 Commtouch Software, Ltd.
Blocking malware at the zero-hour, eliminating the threat of unknown malware variants Keeping threats out of the organization by blocking malware at the perimeter Reducing liability and IT waste for ISPs with infected zombie home customers Reducing liability and IT waste by identifying compromised computers within corporations While zombie detection does not replace signatures and heuristics, it can be used as an essential additional layer of protection at the perimeter, to defend against server-side polymorphic malware, blended threats, and whatever new malware the future will bring. Recurrent Pattern Detection, RPD and Zero-Hour are trademarks, and Commtouch is a registered trademark, of Commtouch Software Ltd. U.S. Patent No. 6,330,590 is owned by Commtouch. Copyright 2007 i The 2007 Malware Report, Computer Economics, p. 14 iimessaging Security Market Trends, 2006-2009, Osterman Research, p. 11 iiisearchsecurity SecurityWire Weekly, Episode 7, Eugene Kaspersky http://media.techtarget.com/searchsecurity/downloads/security_wire_weekly_rsa_kaspersky_02_08_2007.mp3 iv Polymorphic malware is malware that self-mutates upon replication, thus making it more difficult for anti-virus engines to catch. Server-side polymorphic malware refers to the fact that the multiple variants are developed on the server-side, that is, before it is distributed to the targets. v VB2007, The Marriage of Spam and Malware, paper by Amir Lev, Commtouch AVAR 2007 Seoul, Korea Page 9 Commtouch Software, Ltd.