ANTI SPAM SOLUTIONS TECHNOLOGY REPORT SurfControl EmailFilter for SMTP JANUARY 2007 www.westcoastlabs.org
2 ANTI SPAM SOLUTIONS TECHNOLOGY REPORT CONTENTS SurfControl EmailFilter for SMTP SurfControl, Inc., 5550 Scotts Valley Drive, Scotts Valley CA, 95066, USA Tel:(831) 440-2500 www.surfcontrol.com SurfControl plc., Riverside, Mountbatten Way, Congleton, Cheshire, CW12 1DY, UK Tel: +44 (0) 1260 296 200 www.surfcontrol.com Introduction...3 Test Network...4 Test Methodology...5 Product Testing Reporting...6 Checkmark Certification...7 The Product...8 Test Report...9 Test Results...14 West Coast Labs Conclusion...15 Security Features Buyers Guide...16 West Coast Labs, William Knox House, Britannic Way, Llandarcy, Swansea, SA10 6EL, UK. Tel : +44 1792 324000, Fax : +44 1792 324001. www.westcoastlabs.org
ANTI SPAM SOLUTIONS TECHNOLOGY REPORT 3 INTRODUCTION As the war for corporate inboxes intensifies, and unmonitored emails disrupt effective and secure working practices, Anti-Spam solutions continue to evolve to deal with this menace. In this, the second Anti-Spam Technology Report, we examine the functionality and performance of the leading products in this market, which are aimed specifically at the SME network environments. A key objective of the testing is to replicate the installation, configuration and use of the solutions in a real-world business environment to enable readers of the White Paper prospective buyers to make a meaningful assessment of the product that is right for protecting their corporate email environment. Test Engineers have evaluated how the solutions install to ensure timely and effective spam protection. Consideration has also been given to the level of security administrator expertise and technical support required to facilitate both out-of-the-box operation and thereafter product training to ensure maximum effective spam protection. This reports provides an independent assessment of effectiveness with regard to: The features and functionality of the solution. Integration into a network infrastructure. The level of user administration required to operate the product effectively. Spam detection capability and rates of detection.
4 ANTI SPAM SOLUTIONS TECHNOLOGY REPORT TEST NETWORK WCL has a number of domains that collect genuine spam. These domains receive varying levels of spam and are consistent with different email environments. To reflect the email usage within a corporate environment, within each domain are a number of designated user accounts with a variety of email practices and needs including some that are subscribed to a variety of newsgroups and mailing lists. Some user accounts actively contribute to mailing lists. The multiple domains designated for testing purposes were those that, between them, receive spam at a level consistent with the defined requirements of testing. Software solutions included in the test program were installed on servers that meet the minimum specifications required by the vendor. Appliance-based solutions were installed on the network according to the vendor's recommended placing. For hosted services, WCL testes through identified email domains and changed the MX records to divert the mail stream through the hosted service.
ANTI SPAM SOLUTIONS TECHNOLOGY REPORT 5 TEST METHODOLOGY WCL initially performed the testing with an out-of-the-box configuration, changing only those settings on the solution needed to ensure correct operation in line with the vendor's recommended installation and configuration procedures. Further testing was then performed following the vendor's advice for the tuning or training of the solution under test. WCL fine-tuned the solution each day of the test, spending no more than half an hour per day undertaking such work. Throughout the course of testing, a mixture of email was sent to the test domains from other email addresses and domains controlled by WCL to mirror genuine email activity common in business, for example, requesting meetings, sending notifications to groups and non-business related social emails. Emails were also sent from web-based accounts such as Hotmail and Google's Gmail in order to simulate external users sending non-business related social emails, and home workers. Thus, during the testing period the domains received some spam, some list/newsgroup mailings and "genuine" individual emails.
6 ANTI SPAM SOLUTIONS TECHNOLOGY REPORT PRODUCT TEST REPORTING Product evaluation addresses three specific areas* - Management/Administration, Functionality, Performance plus Additional Feature Testing. 1. MANAGEMENT/ADMINISTRATION Ease of Setup/Installation Ease of Use Logging and reporting function Rule creation Customization Content Categories 2. FUNCTIONALITY Email Processing Steps Allow/Blocking of Email Quarantine Area Additional functionality reporting Steps to Process Email Block Email Addresses Blacklist/Whitelist Allow Email Addresses 3. PERFORMANCE Volume or Percentage of spam detected False positive rate Spam incorrectly passed through Legitimate mail blocked Legitimate subscription mail blocked
ANTI SPAM SOLUTIONS TECHNOLOGY REPORT 7 CHECKMARK CERTIFICATION Upon completion of the testing, individual product results are analyzed, resulting in accreditation to one of the two Checkmark Certifications for Anti- Spam subject to achieving the following catch rates:- Checkmark Anti-Spam Certification - Premium - 97% and over Catch Rate. Checkmark Anti-Spam Certification - Standard - 90% and over Catch Rate.
8 ANTI SPAM SOLUTIONS TECHNOLOGY REPORT THE PRODUCT SURFCONTROL EMAILFILTER FOR SMTP With continually updated databases, flexible policy setting and market-leading reporting, SurfControl E- mail Filter for SMTP guards against viruses, phishing, confidential data leakage and spam, delivering exceptional visibility, control and protection. http://www.surfcontrol.com/products/email/ SURFCONTROL SAYS ABOUT THE PRODUCT'S BUSINESS BENEFITS Every incoming or outgoing e-mail can expose organizations to viruses, spyware, confidential data loss, regulatory violations and more.to keep your business secure in the face of rapidly evolving threats you need visibility, control and comprehensive protection customized to your own environment and policies - without straining your IT resources. SurfControl E-mail Filter for SMTP messaging security software, powered by SurfControl's Global Threat Experts, offers continuous protection against inbound and outbound threats for any mail server. Easy to install and administer, the solution's automatically updated databases, flexible policy setting and market-leading reporting combine best-in-class protection with exceptional visibility and control. http://www.surfcontrol.com/products/email/ SURFCONTROL SAYS ABOUT THE PRODUCT'S TECHNICAL BENEFITS SurfControl E-mail Filter for SMTP works with all SMTP mail servers and can be deployed across multiple servers to provide load balancing and failover protection. E-mail Filter offers market-leading hands-free administration. Isolation queues can be managed using automatic queue management. Employees can manage their anti-spam folders within Personal E-mail Manager, or administration can be passed to designated managers via the Web-based Message Administrator. Reports can be customized and scheduled to run automatically.these can then be automatically e-mailed to the appropriate managers or reporting delegation can be provided to managers to run reports on specific groups or individuals. http://www.surfcontrol.com/products/email/
ANTI SPAM SOLUTIONS TECHNOLOGY REPORT 9 TEST REPORT INTRODUCTION EmailFilter by SurfControl is a software solution designed to be installed on either a Windows 2000 or 2003 Server. The application itself is provided on CD and comes packaged with an impressive amount of literature in the form of installation and administrator guides as well as a getting started guide. These guides are available for both the Report Central and End User Spam Management consoles, providing an administrator with a wealth of information for use during both the setup and operation of the software. For the duration of this test EmailFilter was installed on a Dell Precision 360 running Windows 2000 Server, which was patched using Service Pack 4. Mail was routed through the service from one of the domains wholly owned and controlled by West Coast Labs.
10 ANTI SPAM SOLUTIONS TECHNOLOGY REPORT TEST REPORT INSTALLATION AND CONFIGURATION The installation process for EmailFilter is both swift and uncomplicated, this is due to a combination of descriptions for each installation step and the use of a commonly used installation routine for Windows-based software. Any applications required for the proper use of EmailFilter are checked for during installation, and are installed should any be missing or of the wrong version. Once this initial process is complete, the Configuration Wizard is launched, allowing an administrator to further define settings relating to the network on which EmailFilter is to be installed. There are four categories to the Configuration Wizard, split across multiple screens: Your Organization, System Details, Mail Routing, and Filtering Options. If during the install procedure a required port is detected as being in-use, EmailFilter provides the ability to specify an alternative. Throughout the installation, EmailFilter provides clear descriptions of each step and informs the user of the reasons for requesting certain information. The entire installation is over in minutes, with EmailFilter quickly configured and running.
ANTI SPAM SOLUTIONS TECHNOLOGY REPORT 11 TEST REPORT INTERFACE Once installed, there are several methods of accessing individual components belonging to EmailFilter. Most of the interfaces are available through two folders located in the start menu: SurfControl E-mail Filter and SurfControl Report Central. Within the E-Mail Filter folder are several programs, all of which can be run independently from a standard Microsoft Console, such as QueueView, Rules Administrator, Monitor, and Message Administrator. Also available within this folder is Dictionary Management, which displays the word and phrase categories that are scanned for by EmailFilter. The initial screen provides a count of the number of entries in each category, and when viewing each category the list of included words and phrases is shown, along with their respective score. Both of these fields can be edited; for example, terms that may normally be associated with pornographic spam can be given a lower score than is awarded by default. Message Administrator provides in-depth information of the messages contained within EmailFilter's various queues. The top half of this console lists the queues, which include virus, offensive, anti-spam agent, and a count of the number of contained messages. Next to this is a window showing the attributes of these messages, including the recipients, subject, dictionary score, and sender. Clicking on each individual message allows the administrator to view the body and any associated files or attachments, and this is displayed in the lower half of this screen. The QueueView is a simplified version of Message Administrator. When launched it lists each queued message along with details including date, time, recipient, sender, and subjects. Also displayed here are the number of attempts to deliver the message and a brief description for the reason of failure. This shortened version allows an administrator to constantly monitor the offending messages that are being detected by EmailFilter. The Rules Administrator console is where the Administrator controls the actions that EmailFilter
12 ANTI SPAM SOLUTIONS TECHNOLOGY REPORT TEST REPORT performs on every categorized message. The top half of the screen lists each rule present on EmailFilter and whether they are currently active. Clicking on one of the rules displays how the rule works in an intuitive If-Then diagram, and the creation of rules from within this console is made easy by a simple drag and drop interface. This method of displaying and creating rules is a refreshing approach and provides a very quick and easy way to configure EmailFilter. For remote administration there is a web interface available, which listens on a non-standard port. The left hand side of the interface provides a menu, from where each of the message queues can be viewed. Clicking on one of the queues displays the messages contained, including standard information such as the sender and recipient addresses, along with the subject and date. Below this the queues are links to the Logs and the Dictionary Management, with the latter providing the same level of customization as available in the console described earlier. Finally a.pdf version of the Administrator's Guide is available for local download.
ANTI SPAM SOLUTIONS TECHNOLOGY REPORT 13 TEST REPORT REPORTING Information is available to the user in two ways, either through the SurfControl Email Monitor or via the Report Central Java-based web interface. The latter, Report Central, may be launched from the second of the two start menu folders mentioned earlier. When the interface is launched, the user is presented with a login screen. Once logged in, the Administrator may choose from two individual report types: Standard Reports and Custom/Scheduled Reports. Selecting one of these displays the categories that are available to the administrator. For example, under Standard Reports, the administrator can select to view reports based on either rules or traffic statistics, each with an array of individual reports available. Rules reports may be viewed by date, various levels of sender details, and various categories of most common rules used by EmailFilter. Each of the ten reports available under the Rules Reports tab contain eight fields that may be further customized, these include rules, weekday, recipients, recipient domains, options, date/time, senders, and sender domains. The second report type to be found under Standard Reports is traffic statistics. Like the Rules Reports this category also contains ten individual reports, which break the traffic down into categories including bandwidth by date, bandwidth by hour, messages by size, and messages by weekday. Also included within the traffic statistics category are reports displaying the top 15 recipients, sender by total messages, and total size. Should the administrator wish to customize what information is generated in a report, these particular settings and configurations can be saved under customized/scheduled reports. By allowing the administrator to save customized reports, very specific reports can be generated time and again without having to continuously specify the same options.
14 ANTI SPAM SOLUTIONS TECHNOLOGY REPORT TEST RESULTS Type of mail Detected as genuine Detected as SPAM GENUINE 100% 0% SPAM 3% 97% SurfControl's Email Filter performed well, delivering 100% of the genuine mail correctly and correctly classifying 97% of the spam mail. It is also worth noting that Email Filter delivers a good proportion of grey and list mail as genuine. This gives an organiaation the flexibility and opportunity to define policies during a training period without missing mail that could potentially be business critical. West Coast Labs is pleased to award Email Filter the Premium Anti-Spam Checkmark.
ANTI SPAM SOLUTIONS TECHNOLOGY REPORT 15 WEST COAST LABS CONCLUSION Installation of this product is both very quick and easy, with a good degree of customization available, allowing EmailFilter to be configured to match the specific needs of an organization. The very short learning curve on the product means the potential defensive capabilities of EmailFilter can be operational in a short space of time. The SurfControl EmailFilter software performed consistently well in the tests, and therefore West Coast Labs is pleased to award the SurfControl EmailFilter the Premium level Anti-Spam Checkmark. West Coast Labs Disclaimer While West Coast Labs is dedicated to ensuring the highest standard of security product testing in the industry, it is not always possible within the scope of any given test to completely and exhaustively validate every variation of the security capabilities and/or functionality of any particular product tested and/or guarantee that any particular product tested is fit for any given purpose. Therefore, the test results published within any given report should not be taken and accepted in isolation. Potential customers interested in deploying any particular product tested by West Coast Labs are recommended to seek further confirmation that said product will meet their individual requirements, technical infrastructure and specific security considerations. All test results represent a snapshot of security capability at one point in time and are not a guarantee of future product effectiveness and security capability. When West Coast Labs provide test results for any particular product tested, said results are most relevant at the time of testing and within the context of the specific scope of testing and relative to the specific test hardware, software, equipment, infrastructure, configurations and tools utilized during that specific test process. West Coast Labs is unable to directly endorse or certify the overall worthiness and reliability of any particular product tested for any given situation or deployment.
16 ANTI SPAM SOLUTIONS TECHNOLOGY REPORT SECURITY FEATURES BUYERS GUIDE DEVELOPMENTS IN THE PRODUCT OVER THE LAST 12 MONTHS AS STATED BY SURFCONTROL Over the last year SurfControl E-mail Filter for SMTP has been enhanced to not only help organization's better address their spam threat, but also address the broader risks posed by e-mail including viruses, phishing attacks, spyware, confidential data loss and regulatory violations. These enhancements include new compliance-related dictionaries and pre-configured compliance rules that help customers meet corporate governance and regulatory requirements. Support of Transport Layer Security (TLS) and Secure SMTP (SMTPS) allows E-mail Filter to send and receive encrypted e-mail traffic, protecting the privacy of e-mails while being transmitted across the Internet. The implementation of Sender Policy Framework (SPF) helps to guard against spoofed e-mails, phishing, fraud and spam and helps distinguish authentic messages from forgeries. Other new features have given administrators and end-users improved visibility and control over spam and other e-mail related threats. New market-leading reporting now incorporates additional high level dashboard and forensic-style, drill-down reports. The addition of connection-level reporting gives customers exceptional visibility of the threats targeted at their e-mail infrastructure and allows them to see the effectiveness of connection-level anti-spam measures such as Real-time Black Lists (RBLs), Directory Harvest Attack and Denial of Service protection and take effective measures to protect against such threats. The message search capability has been significantly enhanced to reduce the administrative burden of managing isolated spam and other e-mails. Also, improvements to E-mail Filter's end-user management tool, Personal E-mail Manager, provide comprehensive administration of isolated e-mails at the end-user level, resulting for greater productivity for both administrators and end-users.
ANTI SPAM SOLUTIONS TECHNOLOGY REPORT 17 SECURITY FEATURES BUYERS GUIDE ADDITIONAL SECURITY FEATURES CONTINUALLY UPDATED DATABASES: Anti-Spam Agent including digital fingerprint and heuristic rules Integrated SurfControl URL Database Anti-Virus Agent powered by McAfee 150+ categorized and weighted dictionaries in multiple languages Team of 70+ located in 20 Countries identifying new threats REAL-TIME THREAT TECHNOLOGIES: Virtual Learning Agent - Recognize your organization's own critical documents Virtual Image Agent - Identify e-mails that contain adult images SURFCONTROL REPORT CENTRAL: Real-time dashboard reporting Drill-down forensic reporting capability Pre-defined, customizable reports Automatic scheduling and e-mailing of regular reports Delegated access via web-based interface CONNECTION LEVEL PROTECTION: Denial of service protection Directory harvest attack protection Protected domain closed relay Reverse DNS Lookup and SPF authentication for spoofed e-mail protection Support for Real-time Blackhole Lists Defined trusted IPs for protection against spammers Remote user authentication Blacklists & whitelists Gateway-to-gateway encryption (TLS and SMTPS)
18 ANTI SPAM SOLUTIONS TECHNOLOGY REPORT SECURITY FEATURES BUYERS GUIDE POLICY LEVEL PROTECTION: Inbound and outbound filtering Pre-defined and custom filtering rules Confidential information management Business compliance management Offensive content and image management HTML parsing and stripping Document decomposition Customizable dictionary threshold filtering E-mail bandwidth management http://www.surfcontrol.com/products/email/