ANTI-SPAM SOLUTIONS TECHNOLOGY REPORT FEBRUARY 2006 SurfControl Email Filter
2 ANTI-SPAM SOLUTIONS TECHNOLOGY REPORT Contents SurfControl Email Filter Test objectives and scenario...3 Test network...4 Test methodology...5 Product test reporting...6 Certification...7 The product...8 Test report...10 Test results...15 West Coast Labs conclusion...16 Security features buyers guide...17 West Coast Labs, William Knox House, Britannic Way, Llandarcy, Swansea, SA10 6EL, UK. Tel : +44 1792 324000, Fax : +44 1792 324001.
SURFCONTROL EMAIL FILTER 3 Test objective and scenario The war for control of corporate inboxes has been raging for some years now as Anti-Spam solution providers seek to protect us from unsolicited, inappropriate and often offensive intrusions into our time. The originators of these emails are becoming ever more inventive and so more and more companies are coming to rely on automatic solutions with learning engines to protect their users and machines. The emails themselves are getting more sophisticated. Spam is now no longer just advertising material, but is evolving, and often acting as the precursor to identity theft. This Technology Report examines the functionality and performance of participating Anti-Spam products which are aimed at the small, midsize and corporate network environments. It has been open to both software and appliance-based solutions plus hosted services. The objective of our overall testing program, which is open to all Anti- Spam Vendors is, through a real-world test environment, to provide an independent validation of Anti-Spam solution effectiveness with particular reference to: A detailed view of the features and functions of the solutions Spam detection capability and rates of detection of each solution Integration into a network infrastructure and level of administration required to operate effectively.
4 ANTI-SPAM SOLUTIONS TECHNOLOGY REPORT Test network Software solutions are installed on servers that exceed the minimum specifications required by the vendor. Appliance-based solutions are installed on the network according to the vendor s recommended placing. For hosted services, WCL test through identified email accounts and will change the MX records to divert the mail stream through the hosted service. In order to allow the DNS change to propagate, service providers allow a 2-day settling-in period. Details of the tuning and vendor customer support will form part of the additional feature testing and reporting.
SURFCONTROL EMAIL FILTER 5 Test methodology WCL has a number of domains available which act as honeypots for spam, receiving genuine, not canned spam. These domains receive varying levels of spam and are intended to mirror different email environments. Within each domain are designated user accounts with a variety of email practices and needs - some are subscribed to a variety of newsgroups and mailing lists. Some user accounts actively contribute to mailing lists. The domain designated for testing purposes will be that which currently receives spam at a level consistent with the test requirements. For testing in this Technology Report and for the certification of each of the participating solutions, we used live mail feeds coming in to various extra domains wholly owned and controlled by West Coast Labs. Each domain used contains a number of individual user accounts with established email addresses, along with distribution lists. To maintain the flow of genuine mail, test engineers used several internal and external accounts, to send emails that simulated real life email transactions common in business: for example requesting meetings, sending notifications to groups and non-business related social emails. Emails were also sent from web-based accounts to simulate external users sending non business-related emails and home workers. Individual user accounts were subscribed to several mailing lists and daily newsletters for grey mail purposes. For each solution we configured the device or software to fit in with the test network and placed it into a stream of live mail to see how it would cope in an out-of-the-box configuration with real-world traffic. However, we do recognize that a large part of spam detection relies on an initially intensive learning process. (Hence, we will be placing these devices in the mail feed in coming months for longer periods of time, interactively training them, and updating the performance data included in the online White Papers.)
6 ANTI-SPAM SOLUTIONS TECHNOLOGY REPORT Product test reporting For each product that we test, we will issue a report which will address the following aspects of the product: 1. Management/Administration Ease of Setup/Installation Ease of Use Logging and reporting function Rule creation Customization Content Categories Technical Support Available Program Help Menu 2. Functionality Email Processing Steps Allow/Blocking of Email Quarantine Area Additional functionality reporting Block Email Addresses Blacklist/Whitelist Allow Email Addresses 3. Performance Volume or % of spam detected False positive rate Spam incorrectly passed through Legitimate mail blocked Legitimate subscription mail blocked
SURFCONTROL EMAIL FILTER 7 Certification - Checkmark Upon successful completion of the catch rate testing, participating solutions will be accredited to Checkmark Certifications for Anti-Spam subject to achieving the following catch rates:- Checkmark Anti-Spam Certification PREMIUM 97% and over Catch Rate. www.check-mark.com Checkmark Anti-Spam Certification STANDARD 90% and over Catch Rate. www.check-mark.com
8 ANTI-SPAM SOLUTIONS TECHNOLOGY REPORT The product SurfControl E-mail Filter for SMTP is compatible with all SMTP-based e- mail systems and offers a scalable, secure messaging solution for organisations of all sizes and vertical markets. url : http://www.surfcontrol.com/products/email/ SurfControl says about the EmailFilter Business Benefits SurfControl E-mail Filter is the industry s most accurate and flexible messaging security software. Powered by SurfControl s Adaptive Threat Intelligence, E-mail Filter offers continuous protection against spam, phishing attacks, spyware, viruses, and malicious code. It also provides customizable protection from corporate data leakage, compliance management, and overall management of inappropriate and unwanted e- mail content. No other company offers such dynamic threat protection for inbound and outbound e-mail. Easy to install and administer, and preconfigured for most situations, E-mail Filter gives you the pulse on your e- mail traffic, lets you manage your network resources more efficiently and maximizes employee productivity. url : "http://www.surfcontrol.com/products/email/" http://www.surfcontrol.com/products/email/
SURFCONTROL EMAIL FILTER 9 The product SurfControl says about the EmailFilter Technical Benefits SurfControl E-mail Filter for SMTP works with all SMTP mail servers and can be deployed across multiple servers to provide load balancing and failover protection. E-mail Filter offers market-leading hands-free administration. Isolate queues can be managed without an administrator by using automatic queue management. Employees can manage their anti-spam folders within EUSM, or administration can be passed to designated managers via the Web-based Message Administrator. Reports can be customized and scheduled to run automatically. These can then be automatically e-mailed to the appropriate managers or reporting delegation can be provided to managers to run reports on specific groups or individuals. url : http://www.surfcontrol.com/products/email/
10 ANTI-SPAM SOLUTIONS TECHNOLOGY REPORT Test report Introduction SurfControl s EmailFilter is a software solution that sits upon a Windows 2000 Server or 2003 Server installation. The software itself arrived on a single CD but you certainly can t accuse SurfControl of skimping on the accompanying documentation the package included a Getting Started Guide, a rather stocky Administrator s Guide, and Installation and Administration Guides for both the Report Central component and the End User Spam Management (EUSM) component. The version of EmailFilter provided works for any SMTP host, although SurfControl also provide a solution geared specifically towards Exchange. For the purposes of this test, the software was installed on a Dell PowerEdge 2650 and set to relay the mail to an internal collection server configured with Postfix. EmailFilter itself was hosted on Windows 2000 Server.
SURFCONTROL EMAIL FILTER 11 Test report Installation and Configuration The installation itself was very straightforward, and will be instantly familiar to anyone who has performed Windows software installations before. The routine checks for and provides the necessary prerequisites such as the Microsoft Database Engine (MSDE) if they are not already present on the system. Usefully, these are included these as part of this initial set up. Following this, the Configuration Wizard is launched and this allows the user to define details regarding the corporate network. It is split into four major groups Your Organization, System Details, Mail Routing, and Filtering Options. The entire process is split over several screens and each option has accompanying text explaining why the data is needed. Any ports that are needed are also checked by the installation routine and are easy to configure for alternatives if they are already being used. Overall the process was quite quick, and we had the system and prerequisites installed, configured, and running within just a few minutes.
12 ANTI-SPAM SOLUTIONS TECHNOLOGY REPORT Test report The Interface The EmailFilter software itself has several distinct components. Each of these can be loaded separately and they include Dictionary Management, the Message Administrator, a Monitor application, QueueView, the Rules Administrator, various Scheduler tools, and a Web Administrator interface. Also provided after installation are copies of the documentation and various database tools. Each of these is linked from the Windows Start Menu and even though these are all separate applications, each can spawn the others from the tools menu in each interface. The Report Central and End User Spam management software were installed, but are beyond the scope of this review. The Rules Administrator is where users can really get to grips with how mail is processed by EmailFilter, and is of central importance to how mail is interpreted. This interface is split into three panes - the largest at the top shows the list of rules, whilst the lower two allow for construction of any new rules very easily using a drag and drop interface for building. The rule constructions is split into a Who What Operations Notify Actions methodology and this combined with the intuitive nature of the interface allows for rules to be built and applied very quickly. The nature of the way the product functions means that various Agents can be utilised such as Anti-Spam and Anti-Virus as well as allowing for certain specialized triggers such as MIME format, file attachments and message size to be considered.
SURFCONTROL EMAIL FILTER 13 Test report The Interface continued The Message Administrator shows the user the quarantine folders. Each of these that contains mail has a number next to the title indicating the number of messages within. This gives a good overview of the types of traffic coming through the mail stream. Once one of the queues is highlighted, the window is split into four panes rather than the initial two. These four windows then display the list of quarantine folders, some overview data regarding the individual messages such as subject, recipient, and sender, the component parts of the message that is currently highlighted, and finally a copy of the currently highlighted message. There is the option within the interface to either show all HTML in messages, never show HTML or prompt on a per message basis, with prompt being set as the default. There are several options for mail within this highly intuitive interface - the messages can be released, moved to a different queue, deleted, and a search can be done based upon the recipients of mails. A notable feature is that whilst this application is open, new messages received are heralded by a pop-up dialog box to inform the user so that they can amend any searches or further examine the relevant data.
14 ANTI-SPAM SOLUTIONS TECHNOLOGY REPORT Test report The Interface continued The Monitor interface shows statistics for EmailFilter s Receive Service, Rules Service, and Send Service, and has logging panes for each of these. Within this application it is possible to track the progress of an email from entry into the system through to either quarantine or release. The rules which are being triggered can be traced so that an administrator can immediately see where decisions are made within the software. The Dictionary Management tool shows the user what is in the contents of the Spam filtering dictionaries supplied by SurfControl, and allows the user to add, edit or remove words and phrases as necessary as well as create custom dictionaries. This is straightforward and the methods for adding data are clear with the system using an asterisk as a wildcard operator. The dictionaries that have offensive words or phrases in also provide a pop up warning before the user enters them. The Scheduler application allows the frequency to be set for component rules updates. It also has scheduled tasks for mail queue synchronisation and a weekly database shrink. It is also possible from within this interface to run scheduled jobs manually and to view logs of the previous updates. The Web Administrator brings together several of these applications in one interface providing a cohesive point to assess information. It allows viewing of the traffic logs, rules and system logs, and provides an online version of both the Dictionary Management application and the Message Administrator. Overall, even though there are several distinct applications involved in the general management and monitoring of email they are well matched and cohesive. This solution has a familiarity about it that, even upon first use, means that it can fit in well to Windows-based working practises and be quickly integrated into an existing set up.
SURFCONTROL EMAIL FILTER 15 Results Type of Mail Delivered as Genuine (%) Delivered as Spam (%) GENUINE 100 0 SPAM 6 94 SurfControl s EmailFilter software performed well, delivering 100% of the genuine mail correctly and correctly classifying 94% of the spam mail in a straight out of the box configuration. It is also worth noting that SurfControl s EmailFilter solution delivers a good proportion of grey and list mail as genuine This gives an organisation the flexibility and opportunity to define policies during a training period without missing mail that could be potentially business critical.
16 ANTI-SPAM SOLUTIONS TECHNOLOGY REPORT West Coast Labs conclusion This product is simple to install and simple to use - although there are a few different interfaces and this serves it well differentiating between each module s tasks. The instant familiarity that comes from having a Windows interface along with the capability of the tools and spam engine mean that this is a contender that is worthy of serious consideration for any corporation. The SurfControl EmailFilter software performed consistently well in the tests, and therefore West Coast Labs is pleased to award the SurfControl EmailFilter the Standard level Anti-Spam Checkmark. West Coast Labs, William Knox House, Britannic Way, Llandarcy, Swansea, SA10 6EL, UK. Tel : +44 1792 324000, Fax : +44 1792 324001.
SURFCONTROL EMAIL FILTER 17 Security features buyers guide as stated by SurfControl SPAM FEATURES Does the product block spam out of box or does it require addition or tuning of rules? Spam is blocked out of the box with default rules and the latest download of the Anti-Spam Agent. SurfControl E-mail Filter's pre-defined rules fall under the categories of Network Security, Virus Protection, Spam, Inappropriate Material, Network Resources and Confidential Information. You can change these predefined rules and their associated actions to meet your particular business needs simply by dragging and dropping the related Rules objects onto the Rules workspace. The Network Security, Virus Protection and Spam Rules can be enabled when you install E-mail Filter to provide out of the box protection. Is user feedback required over initial stage of deployment? Minimal user input is required during the deployment stage. The Setup and Configuration Wizards guide the user through the installation procedure. FILTERING Does the product utilise keyword lists? YES - as one of several layers of spam protection. SurfControl E-mail Filter comes with 16 pre-built threat dictionaries, each available in 10 languages, many of which are relevant for spam capture such as the Spam, Spam Misspellings, Gambling and Adult dictionaries. All the words in the dictionaries are weighted and these weights add up to trigger a rule. The user can choose to adjust these weights or add words to or delete words from a dictionary. Administrators can also create their own dictionaries. Does the product utilise Bayesian filtering? SurfControl E-mail Filter does not use Bayesian filtering techniques. However, SurfControl s anti-spam and blended threat protection consists of a defence in depth strategy providing our customers with multiple layers of protection to ensure the attack message is successfully filtered. Statistical analysis methods are used within SurfControl s Anti-Spam Agent and can be leveraged using the Virtual Learning Agent feature.
18 ANTI-SPAM SOLUTIONS TECHNOLOGY REPORT Security features buyers guide as stated by SurfControl FILTERING continued Can white-lists/black-lists be set? YES - white and black lists can be set to reject or allow e-mail from designated addresses or domains. This can be done by the administrator at a global level. In addition, SurfControl s End- User Spam Management (EUSM) lets end-users create their own white list and black lists. Does product support RBL? YES - SurfControl E-mail Filter can check an e-mail sender's domain against those held in a selected Real-time Blackhole List and reject it if found in the list. Does the product support the setting of different confidence levels? Can actions be varied at different confidence levels? YES. Different spam capture techniques have different confidence levels, for instance the spam captured by Digital Fingerprint techniques has a higher confidence level than spam captured by Heuristics techniques. Rules can be set which trigger using different Anti-spam techniques and depending on the rule, different action can be taken. In fact, the default Anti-Spam Agent rules are set up in this manner. In addition, you can specify how sensitive the Heuristics tool is in evaluating e-mails. The higher the sensitivity, the fewer spam-like traits an e-mail needs in order to trigger the rule. Can actions be varied at different confidence levels? YES - Different actions can be taken according to the different confidence level of the particular rule. For instance if an e-mail triggers the Anti-Spam Agent (ASA) Digital Fingerprint Rule then a different action can be taken than say if it triggers the ASA Heuristics Rule. Typically up to 90% of spam is captured by the Digital Fingerprint Layer and because this uses actual fingerprints of spam rather than probability-based techniques this results in almost zero levels of false positives. As a result, many of our customers automatically discard all spam caught by this rule, significantly reducing the burden on the administrator to manage quarantined spam, whereas administrators may choose to allow end-users to manage spam caught by the ASA Heuristics Rule via the End-User Spam Management (EUSM).
SURFCONTROL EMAIL FILTER 19 Security features buyers guide as stated by SurfControl FILTERING continued Can subject line of messages be altered? YES - the subject line of messages can be altered. Can email headers be set/amended? The Header Modification Object can change message header field values including the delivery information of the message, the e-mail address on the To: or cc: list, the sender's identity, the address replies to the message will be sent to, the originator of the message, the text in the subject line, the date and time the e-mail was received and the message identifier. ADMINISTRATION Can the product be automatically updated? All subscription elements of the product can be automatically updated (see below). SurfControl notifies all customers of all significant software service pack and version updates and it is the administrator's decision whether to apply the upgrade or not. Can filters be automatically updated? YES - E-mail Filter integrates new intelligence on viruses, spam, phishing, blended threat attacks, spyware, and inappropriate Web sites automatically via the Scheduler. The Scheduler updates the Anti-Virus Agent, Anti-Spam Agent and the Internet Threat Database (database of web sites categorised by their content eg Adult, Gambling etc.) What are the update methods? You can automatically download updates to the Anti-Spam Agent, Anti-Virus Agent and the Internet Threat Database using the SurfControl E-mail Filter Scheduler to ensure up-to-date protection against spam, viruses, phishing, blended threat attacks, spyware, and inappropriate Web sites. The Scheduler allows the administrator to choose the time and frequency of the updates. In addition, updates can be run immediately if chosen. Can suspected spam be quarantined? YES. Suspected spam can be quarantined in isolation folders.
20 ANTI-SPAM SOLUTIONS TECHNOLOGY REPORT Security features buyers guide as stated by SurfControl ADMINISTRATION continued If so, what type of quarantine (forward to Q mailbox / saved on device / etc.)? E-mails that are isolated are held in dedicated queue folders on the SurfControl E-mail Filter server until they are either discarded, saved or released and sent to their recipient. Administrators can analyse and take action on isolated e-mails quickly and easily via the Message Administrator - the client or Web-based format make it simple to access remotely. Also, Automatic Queue Management can be used to automatically take action on the queue - deleting, releasing or saving e-mails after they have aged a designated length of time for hands free administration. Alternatively, SurfControl s End-User Spam Management (EUSM) enables e-mail users to manage their own suspected spam that has been quarantined. END USER INTERACTION Can users see reports individual to them? YES. SurfControl Reports Central allows delegated access to reports and it is possible to set up reports for individuals. Typically this feature would be used to allow a department manager to view reports on his or her staff. Can users process messages themselves? YES - End User Spam Management (EUSM), an E-mail Filter utility, gives individual users responsibility for their own spam e-mail, and helps ensure that they receive legitimate e-mail. Through EUSM, E-mail Filter regularly notifies end users of the existence of isolated spam and directs them to a secure Web interface that shows their digest of isolated spam messages. End users can then decide whether to release, delete, whitelist or blacklist the e-mail.
SURFCONTROL EMAIL FILTER 21 Security features buyers guide as stated by SurfControl END USER INTERACTION continued Can users review mail marked as spam? YES - End User Spam Management (EUSM), an E-mail Filter utility, gives individual users responsibility for their own spam e-mail, and helps ensure that they receive legitimate e-mail. Through EUSM, E-mail Filter regularly notifies end users of the existence of isolated spam and directs them to a secure Web interface that shows their digest of isolated spam messages. End users can then decide whether to release, delete, whitelist or blacklist the e-mail. Can users free messages from quarantine? YES - Using the EUSM described above. Can users set their own white lists/black lists? YES - Using the EUSM described above. ADDITIONAL SECURITY FEATURES CONNECTION MANAGEMENT PROTECTION Denial of Service Protection Directory Harvest Attack Protection Protected Domain Closed Relay Reverse DNS Lookup for Spoofed E-mail Protection Support for Real-time Blackhole Lists Defined Trusted IPs for protection against spammers Remote User Authentication Blacklists & Whitelists RULES LEVEL PROTECTION HTML parsing and stripping Document decomposition Pre-defined and Custom filtering rules Anti-Virus Protection Anti-Spam & Anti-Phishing Protection: Anti-Spam Agent Anti-Spyware and Anti-Phishing Protection: Internet Threat Database
22 ANTI-SPAM SOLUTIONS TECHNOLOGY REPORT Security features buyers guide as stated by SurfControl RULES LEVEL PROTECTION continued Compliance and Confidentiality Management Offensive Content Management: SurfControl Dictionaries Offensive Image Management: The Virtual Image Agent Confidential Information Management: The Virtual Learning Agent E-mail Bandwidth Management ADAPTIVE THREAT INTELLIGENCE SERVICE DYNAMIC THREAT DATABASES: Anti-Spam Agent: Digital Fingerprint, Heuristics and Lexi-Rules Databases Internet Threat Database Anti-Virus Agent Pre-populated and weighted dictionaries: 16 Categories in 10 Languages REAL-TIME THREAT TECHNOLOGIES: Virtual Learning Agent Virtual Image Agent GLOBAL THREAT EXPERTS: Team of 60+ Global Threat Experts Located in 20 Countries url : http://www.surfcontrol.com/products/email/