Performance Validation Testing Kaspersky Lab Corporate Security Solutions 1 of 24
Contents Changing Malware Threats in Corporate Networks 3 The Test Objectives 6 Malware Test Suites 8 Malware Detection Test Results 9 Kaspersky Lab Corporate Security Solutions 10 Checkmark Certifications for Kaspersky 14 Checkmark Certification Profile for Kaspersky Lab 15 Conclusion 16 Product Feature Comparisons 17 Kaspersky Anti-Virus 8.0 for Microsoft ISA Server and Forefront TMG Standard Edition 18 Kaspersky Security 8.0 for Microsoft Exchange Servers 19 Kaspersky Anti-Virus 8.0 for Linux File Server 20 Kaspersky Anti-Virus 8.0 for Lotus Domino 21 Kaspersky Anti-Virus 8.0 for Windows Servers Enterprise Edition 22 Disclaimer 23 Contact Information 24 2 of 24 www.westcoastlabs.com
Changing Malware Threats in Corporate Networks The Evolution of Malware, Security Technologies and Services By Lysa Myers, director of research, West Coast Labs There are few who are unaware of the malware landscape changing since the release of the first few viruses decades ago. But it seems there are just as few people outside the computer security industry who understand the nature of that change. No longer is malware as ethereal a threat as an urban legend, and no longer is the virus outbreak of the day making the evening news. Threats now come not by ones and twos but by the many tens of thousands each day with the known total hovering in the tens of millions. And threats come quietly, remaining as far below the radar as possible to maximize their stay on an affected machine. Corporations are now victims of targeted attacks, as well as the regular masses of malware and have specific needs for the protection of corporate information assets. While malware activity has increased, security budgets certainly have not. Many corporate security staff find themselves facing a tidal wave of new threats without extra personnel or resources. They need security software to work faster, harder and require less manual interaction while providing detailed reports as to what actions have been taken. Machines which are infected need to be cleaned completely so as to get systems back up and running quickly and painlessly. Anti-Malware software is only as good as its research and support departments. They are vital in order to have excellent response times to new threats and to provide top-notch customer assistance. As focus in corporate networks shifts away from the desktop, into mobile, cloud and virtual computing resources, security software needs to protect these environments too. The way malware spreads has also changed there is less concern for infecting oneself with a floppy disk (how many of us even have a floppy disk drive now?) or via poorly worded and spelled mass-mailer viruses. When malware authors discovered there was profit to be had in spreading their malicious wares, they began to take many of the tactics used by Search Engine Optimizers and improved their social engineering craft, placing files where people were most likely to run across them. Consequently, the Web is now where the majority of people become infected with malware and, given the extent to which the internet is such an integral part of all corporations business activities, the Web is a potent threat vector. Company s websites are regularly targeted for defacement or infected to spread malware to the site s visitors. Given that the Internet is operating system agnostic and because current scripting languages allow for queries of the specific browser version of each visitor, malware can be spread which in a manner which 3 of 24 www.westcoastlabs.com
infects any particular visit. In the last few years, this has been a tactic which has proved increasingly popular with malware authors, increasing their reach as the market share of new technology increases. Obviously, anti-malware products had to change with the times as the onslaught of malware has increased and the tactics of malware authors has shifted. The first anti-malware products were designed strictly as signature scanners, which only ran when a user specifically initiated a scan. In short order, this was changed to allow the scanner to run continuously in the background so that each file was examined as it was accessed, without users having to think about it. This approach has become more widespread, so that products require little interaction users can automatically have the most up-to-date protection running at all times. Another thing which has changed with the times is the complexity of the scanning processes. No longer are anti-malware products simply signature-based scanners. They now include advanced heuristic technologies and generic signatures which can proactively detect new variants of existing families and new malware families. The best products include a variety of security features such as web or spam filtering, behavioural analysis or a firewall technology which can help protect against brand new threats. With these new, intensive scanning technologies, vendors have come up with many ways to decrease the overall processing load, so that scanning will not noticeably decrease access times or interrupt workflow. As both the malware landscape and anti-malware products have changed, so has the security testing industry. When products under test were updated periodically, used on-demand scanning and the total known malware was in the thousands, it made sense to have only a single pass or fail test which was performed a few times a year over a static test-bed of samples. This is no longer the reality of the current user experience. While it can be a meaningful baseline test of anti-malware functionality, it is far from a complete picture of overall product performance. In order to accurately reflect a user s experience with malware, it is important to gather the full spectrum of malware from a variety of sources from throughout the internet, which circulate on various protocols. This means including not just email-based malware, but malicious files on P2P networks, as well as on the Web and other attack vectors. Because malware does not stop when the work day ends nor does it recognize geographic boundaries, threats must be collected all day from around the world. As anti-malware products have begun to include more wide-ranging technologies including ones which are initiated upon execution of a file, testing must incorporate dynamic functionality by running threats on test machines. This naturally takes more time than scanning an immobile directory of files, so one must take care to select the most relevant sample set which a customer is most likely to encounter. This takes into account not just prevalence, but attack vector popularity on which it s spread, potential for damage on an infected system, as well as geography. Malware authors are always abreast of technology trends where do people share their information, how do people share files? At West Coast Labs we ve already begun to see an increase of attacks on things 4 of 24 www.westcoastlabs.com
like digital picture frames, USB thumb drives, mobile phones and on popular Web 2.0 sites. So, suffice to say, if you know a few people who use one or other or all malware authors are looking to exploit them for financial gain. Likewise, anti-malware vendors are developing technologies to protect them and testers like West Coast Labs are developing methodologies to mirror the user s risk and potential infection experience. In order to keep up to date on the evolving malware landscape, one need only see which new widgets are being used in home and business network environments. But in the corporate world, keeping updated on the latest threats and technologies is not enough TCO and ROI need to be considered. How well do advanced technologies proactively detect? How quickly are new threats added? How is customer support response? How easily can the solution be managed remotely? How much CPU time is used for scanning? To find the answers to many of these questions, take a look at product performance data from leading independent test organisations such as West Coast Labs and the performance validation programmes they deliver such as Real Time Testing. You can also take a close look at how individual vendors are responding to the changing threat landscape and the implications for the security of corporate networks. Nowadays, vendors are defining Protection differently. No longer is it just product performance-related but also related to business and customer service issues, delivering a higher value It s also about the extent of a vendor s product research and development strategy that anticipates threats and trends to ensure proactive network protection. overall service to meet not just security, but also business needs. When considering product performance in a corporate network environment, Protection is more than current malware detection capabilities, it s also about the extent of a vendor s product research and development strategy that anticipates threats and trends to ensure proactive network protection. It can be further defined as the extent to which malware protection is delivered for a multi-platform infrastructure through efficient and easily managed solutions with wide inter-operability capabilities. Protection is also about the extent to which business interests are protected through vendor service strategies that now include optimised and costeffective security plans tailored to individual corporations needs for maximising business productivity, lowering the total cost of ownership and maximising the return on investment. Also, given that corporations are operating in a worldwide e-economy all this needs to be supported by trusted and responsive global support plans. Yes, the threat landscape is continuing to evolve with new malware threats spawned at an alarming rate, but no longer is malware protection and information security in general just a technical issue - it s a business issue. That s why vendors product and service solutions are evolving to suit these changing needs and West Coast Labs is developing independent product performance programmes that ensure that these products and services are tested and validated accordingly. 5 of 24 www.westcoastlabs.com
The Test Objectives Kaspersky Lab commissioned West Coast Labs to carry out the following testing: Checkmark Certification for the Baseline, Dynamic and Real Time testing programme on seven corporate security solutions: Kaspersky Security 8.0 for Microsoft Exchange Servers Kaspersky Anti-Virus 8.0 for Windows Servers Enterprise Edition Kaspersky Anti-Virus 8.0 for Linux File Server Kaspersky Anti-Virus 8.0 for Lotus Domino Kaspersky Anti-Virus 8.0 for Microsoft ISA Server and Forefront TMG Standard Edition Kaspersky Endpoint Security 8 for Mac Kaspersky Endpoint Security 8 for Linux Comparative testing of selected Kaspersky products against a range of competitor products in a static test environment (see below). A comparison of product feature sets using publicly available information on vendor websites and marketing collateral. A comprehensive list of all Kaspersky Lab Checkmark Certifications and Checkmark Platinum Product Awards can be found on page 15. The Comprative Product Testing The comparative testing comprised a basic evaluation of each product s malware detection capability in a static test environment. WCL built a test suite of 100,000 live malware samples* from its own independent resources that covered all appropriate attack vectors. Each solution was installed to a server running the appropriate and commonly supported Operating System and software detailed in the next section of this report. During installation, all default values were kept and, where a choice was required, the course of action recommended by the solution and/or the attendant product documentation was adhered to. Each solution was updated to the latest available definition, engine, and signature releases before a forensic image was taken and stored for later use. Updates were allowed during the test period through any normal scheduled and automatically enabled update mechanism present in the product, and a further forensic image was taken on the last day of testing for each combination of products. Each solution was tested against an appropriate test set extracted from the 100,000 samples mentioned above and made up of real-world, solution capability specific samples taken from West Coast Labs collections, including samples received in the West Coast Labs Global Honeypot Network. For example, the Exchange-based solutions were tested against malware known to propagate over email. Test sets and the methodologies were constructed so as to mirror the experience of a real-life installation as far as possible and not to advantage any one vendor over the others. *For a description of the malware used in this test programme, refer to Appendix 1 of this report. Comparative Product Testing Test Network Testing was carried out on distinct networks which comprised various server and client machines needed to run the respective technologies and operating systems. Corporate Security Solutions Used in the Comparative Product Testing Microsoft Exchange Test Kaspersky Security 8.0 Symantec Mail Security Trend Micro ScanMail McAfee GroupShield Sophos E-mail Security ESET Mail Security Lotus Domino Test Kaspersky Anti-Virus 8.0 Symantec Mail Security Trend Micro ScanMail McAfee GroupShield Sophos E-mail Security ESET Mail Security Microsoft ISA Server (replaced by Forefront TMG 2010) Test Kaspersky Anti-Virus 8.0 Forefront TMG 2010 Windows Server Test Kaspersky Anti-Virus 8.0 Symantec Endpoint Protection Trend Micro Officescan Server McAfee VirusScan Enterprise and VirusScan for Storages Sophos Endpoint Security ESET File Security Linux Test Kaspersky Anti-Virus 8.0 Symantec Endpoint Protection Trend Micro ServerProtect McAfee VirusScan Enterprise Sophos Endpoint Security ESET File Security 6 of 24 www.westcoastlabs.com
In order to provide a balanced reporting process, West Coast Labs recommended that all client machines should run Windows XP and Service Pack 3 and that server platforms ran the highest OS version commonly supported across each of the solutions. In some cases this meant that they may not have been running on the latest version of a particular operating system, but this method meant that any testing carried out was more directly comparable. Details of highest levels of common operating systems per component available at the time of testing are as follows: Network 1 Microsoft Exchange This network comprised 12 systems 6 desktops and 6 servers (one of each for each solution). Each of the desktop machines were paired up with a server system in order to allow an Exchange Server and Outlook client configuration. Server OS: Windows 2003 Server 64 bit, Exchange Release: 2007 64 bit. Network 2 Windows Server This network comprised 12 systems 6 desktops and 6 servers (one of each for each solution). Each of the desktop machines were paired up with a server system in order to allow a server/client configuration. Server OS: Windows 2008 64 bit Network 3 Linux This network comprised 6 systems running the Red Hat Enterprise release 5 version of Linux. Network 4 Lotus Domino This network comprised 12 systems 6 desktops and 6 servers (one of each for each solution). Each of the desktop machines were paired up with a server system in order to allow a Lotus Domino server and Lotus Notes client configuration. Server OS: Windows 2003 32 bit, Lotus Domino Release: R8 Network 5 Microsoft ISA Server (Forefront TMG 2010) This network comprised 4 systems 2 desktop and 2 servers (one of each for each solution). Each of the desktop machines were paired up with a server system in order to allow a server/client configuration. Server OS: Windows 2008 64 bit, Forefront TMG 2010 Supporting these five networks there were a number of servers designed to collect data from each of the tests, along with desktop machines to act as remote points of control and for test management. Comparative Product Testing Test Methodology In each test case, the protocol most likely to be used was employed to test the solutions detailed below. Microsoft Exchange testing: Testing was conducted on an On Access basis. All samples were sent via email from accounts on a real-life, resolvable domain owned and controlled by West Coast Labs to the products under test over a live internet connection with appropriate firewall rules in place to allow only communication between the hosts used in the testing. This enabled West Coast Labs to report on those emails that were stopped at the Exchange Server and track those emails that were bounced to allow for resending to ascertain the gateway protection offered. Windows Server testing: Testing was conducted on an On Demand basis. All samples were copied on to the appropriate server in a number of directories. The solution under test was asked to scan the server Operating System to report any infections it found. Linux testing: Testing was conducted on an On Demand basis. All samples were copied on to the appropriate server in a number of directories. The solution under test was asked to scan the server Operating System to report any infections it found. Lotus Domino testing: Testing was conducted on an On Access basis. All samples were sent via email from accounts on a real-life, resolvable domain owned and controlled by West Coast Labs to the products under test over a live internet connection with appropriate firewall rules in place to allow only communication between the hosts used in the testing. This enabled West Coast Labs to report on those emails that were stopped at the Domino Server and track those emails that might get bounced to allow for resending to ascertain the gateway protection offered. TMG 2010 testing: Testing was conducted on an On Access basis. All samples were provided from a real-life resolvable web, FTP and P2P server on a domain wholly owned and controlled by West Coast Labs. Attempts were made to download the samples over a live internet connection with appropriate firewall rules in place to allow only communication between the hosts used in the testing using HTTP, FTP and P2P to ascertain the gateway protection offered. 7 of 24 www.westcoastlabs.com
Malware Test Suites West Coast Labs puts considerable effort into ensuring the relevance of samples used in testing. There are three key components to this process. The company s research facilities continuously monitor malware attacks and intercept attempts to attack the corporate network of a global company with thousands of users spread over four continents. WCL also has the advantage of an international system of honeypots, machines based in many countries on most continents that sit on open networks waiting to be attacked. When attacks occur the malware is intercepted and reported back to a central repository, where it is de-duped, checked for corruption and validity, stored and can then be used as a sample for testing products. Another method of collection and validation is through honeyclients; systems designed to trawl the Internet to discover drive-by downloads (where malware is downloaded in the background unknown to the user who is looking at an otherwise perfectly acceptable web site), and to download files by visiting these websites and capturing the output. Comparative Test Project Malware Samples For this particular custom test, testing takes place in five different operating environments, namely Microsoft Exchange, Lotus Domino, MS ISA (TMG 2010) Server, Windows Server and Linux File Server. The main test suite is divided into separate sub-suites used for each environment (although some sub-suites are used more than once). For both Microsoft Exchange and Lotus Domino, the main component of the test suite is a group of malware that spreads itself via SMTP. Of course, many different files and types of malware can be attached to emails, and therefore the test suite also includes malware gathered internationally that can be sent by email. Types of malware used in this part of the test include viruses, bots, Trojans, and especially those worms designed to spread by email, all of which have been found in the email intercepts delivered to WCL. Windows Server acts as a network server and repository and so the appropriate test sub-suites include not only those sub-suites as used elsewhere but also network worms as being the malware most likely to infect and spread via these environments. MS ISA Server acts as a network edge gateway and so the suites considered when testing this include a wide range of malware concentrating on network traffic including HTTP, FTP, malware as well as network worms malware transported by the sort of traffic flow that would be associated with a corporate network. Linux has a small selection of malware especially designed to run in that environment, but also needs to recognize Windows malware; although this cannot run natively in this environment, many companies include both Windows and Linux machines on the same networks and any failure to recognize Windows malware might lead to infection of central or shared servers and leave the whole network vulnerable. For this reason the test sub-suites used in this environment include Linux malware but also Windows malware as used in some of the other tests. 8 of 24 www.westcoastlabs.com
Malware Detection Test Results TEST 1 Microsoft Exchange Total Malware Samples 8042 Test Date Detection Rate Test Location Kaspersky Security 8.0 16/09/2009-23/09/2010 100%HH WCL UK Lab Product Performance AverageH 100%HH WCL UK Lab Product A 16/09/2009-23/09/2010 100%HH WCL UK Lab Product B 16/09/2009-23/09/2010 100%HH WCL UK Lab Product C 16/09/2009-23/09/2010 100%HH WCL UK Lab Product D 16/09/2009-23/09/2010 100%HH WCL UK Lab Product E 16/09/2009-23/09/2010 100%HH WCL UK Lab TEST 2 Windows Server Enterprise Total Malware Samples 25640 Test Date Detection Rate Test Location Kaspersky Anti-Virus 8.0 20/09/2010-23/09/2010 99.68% WCL USA Lab Product Performance AverageH 99.54% WCL USA Lab Product A 20/09/2010-23/09/2010 99.45% WCL USA Lab Product B 20/09/2010-23/09/2010 99.50% WCL USA Lab Product C 20/09/2010-23/09/2010 99.36% WCL USA Lab Product D 20/09/2010-23/09/2010 99.69% WCL USA Lab Product E 20/09/2010-23/09/2010 99.57% WCL USA Lab TEST 3 Linux Total Malware Samples 25640 Test Date Detection Rate Test Location Kaspersky Anti-Virus 8.0 05/10/2010-08/10/2010 99.95% WCL USA Lab Product Performance AverageH 99.59% WCL USA Lab Product A 05/10/2010-08/10/2010 99.64% WCL USA Lab Product B 05/10/2010-08/10/2010 99.24% WCL USA Lab Product C 05/10/2010-08/10/2010 99.40% WCL USA Lab Product D 05/10/2010-08/10/2010 99.80% WCL USA Lab Product E 05/10/2010-08/10/2010 99.53% WCL USA Lab TEST 4 Lotus Domino Total Malware Samples 8042 Test Date Detection Rate Test Location Kaspersky Anti-Virus 8.0 06/10/2010-10/10/2010 100%HH WCL UK Lab Product Performance AverageH 100%HH WCL UK Lab Product A 06/10/2010-10/10/2010 100%HH WCL UK Lab Product B 06/10/2010-10/10/2010 100%HH WCL UK Lab Product C 06/10/2010-10/10/2010 100%HH WCL UK Lab Product D 06/10/2010-10/10/2010 100%HH WCL UK Lab Product E 06/10/2010-10/10/2010 100%HH WCL UK Lab TEST 5 ISA Server (Forefront TMG) Total Malware Samples 18680 Test Date Detection Rate Test Location Kaspersky Anti-Virus 8.0 14/10/2010-19/10/2010 99%HH WCL UK Lab Product A 14/10/2010-19/10/2010 99%HH WCL UK Lab HDefined as the performance average of the products included in the tests, which are deemed to be leading solutions in their own rights. HHSamples used in these tests are those found to be in circulation on West Coast Labs SMTP malware feeds immediately prior to the commencement of testing. Although appearing unusual, the 100% detection rates are indicative of two key facts. Firstly, the paranoid behaviour of email protection systems and the degree of protection extended to vital communication systems such as these, Secondly, the changing nature of attempts to compromise end users over this vector. Whilst executables and binaries travelling over this vector are still highly prevalent, they are becoming less diverse, ie that there are not as many frequent outbreaks of email based malware as there were and that the targets are more likely to receive phishing emails and links to websites rather than files. 9 of 24 www.westcoastlabs.com
Kaspersky Lab Corporate Security Solutions Kaspersky Lab Statement Kaspersky Lab has developed highly-effective anti-malware solutions for use in medium and large-scale corporate networks with complex topologies and heavy loads. Combining ease of use with high standards of performance across multiple attack vectors, the products are cost effective solutions which meet both business and technical needs worldwide. West Coast Labs Executive Summary Report The launch of the Kaspersky Lab s range of anti-malware products for the corporate network environment provides security managers with an extended choice of effective solutions for dealing with threats in attack vectors across multiple operating systems. West Coast Labs independent testing and performance validation of the products confirm that they combine ease of use and management with high levels of performance, all of which is driven by Kaspersky Lab s own research, development and customer support programmes. Kaspersky Lab has made a significant commitment to the independent validation of its products efficacy and performance through West Coast Labs Checkmark Certification System. This provides a range of static, dynamic and real-time tests which make these Kaspersky solutions possibly the most intensively tested corporate anti-malware solutions available anywhere in the world today. Details of the specific tests to which the products are exposed are published elsewhere in this report, but the overall outcome of the certification testing is the achievement of the Platinum Product Award for these products, which is the highest level of independent validation from West Coast Labs possible for an anti-malware solution. This is complemented by very respectable malware detection test results which position the performance of Kaspersky Lab products very favourably alongside more widely recognised corporate security solutions. The specific malware detection capability testing of both Kaspersky Lab and a number of competitive anti-malware solutions was carried out in September and October 2010 while the Checkmark Certification testing of its products is performed on an ongoing basis. Custom test reports and details of certification testing are available at www.westcoastlabs.com Kaspersky Security 8.0 for Microsoft Exchange Servers (Kaspersky Security 8.0). Kaspersky Security 8.0 provides anti-malware and anti-spam protection for mail traffic on corporate networks. Its integration with Exchange allows for detection and removal of malware and spam at the gateway level. The product is easy to install and its user-friendly interface, flexible administration and straightforward configuration and reporting system does not place excessive demand upon Administrator s time. No extra setup is required on Exchange and malware protection began immediately. Management of the solution is simple as Kaspersky Security 8.0 employs a Microsoft Management Console (MMC) snap-in, providing an intuitive interface with full access to all features. Database and signature updates run automatically, as often as every two hours, but if required may be run on-demand. Although there are fewer options available compared to other corporate products on the market, it can be argued that all the necessary options are available thus leading to a streamlined user experience. In the ongoing Checkmark Certification Static and Real Time tests, like all the Kaspersky products, this solution has achieved high consistently standards of performance. For the comparative performance testing to measure the product s detection capability of malware t t 10 of 24 www.westcoastlabs.com
known to propagate over SMTP, Kaspersky Security 8.0 achieved 100% detection rate of the 8042 malware samples used in the test. This performance is equivalent to and matches that of the competitor products included in the test. We also test HTTPS. Kaspersky Anti-Virus 8.0 for Microsoft ISA Server and Forefront TMG Standard Edition Kaspersky Anti-Virus 8.0 sits on top of Microsoft Forefront TMG 2010. While TMG acts as a standalone security solution in its own right, the addition of Kaspersky Anti-Virus 8.0 provides a multi-layered security solution. Installation of Kaspersky Anti- Virus 8.0 is simple, using a standard Windows Installer and settings imported from TMG during the install process. The default settings provide fast protection, but a more tailored installation can be achieved if required. The solution is managed via MMC with an additional central monitoring screen and network policies which can be be added to complement those of TMG; making the whole process of management, administration and ongoing use very straightforward. Kaspersky Security 8.0 Update Process Kaspersky Anti-Virus 8.0 allows permission or denial of various traffic types HTTP, FTP, SMTP and POP3 plus the ability to define what, if any, of the protocols should be subject to scanning. Data on network status including the protocols which are being blocked, numbers of files scanned, and the number of resulting infections is readily available. In the performance testing over the HTTP and FTP attack vectors, the combination of Kaspersky Anti-Virus 8.0 and TMG provided 99% detection of the range of malware samples which were included in the test. Kaspersky Anti-Virus 8.0 for Linux File Server Kaspersky Anti-Virus 8.0 for Linux installs from the command line, using a shell-script installer. Although some degree of familiarity with Linux is required, even junior network administrators with a basic t t Test Networks and Methodology In a heterogeneous network situation it is important to know that a security solution is both compliant and compatible. Throughout the comparative test programme for ISA/TMG, Linux, Lotus Domino and WSEE, WCL utilised the following network configuration to simulate a corporate network environment: 64-bit Windows 2008 machine running as a gateway/dns server hosting Forefront TMG/ISA Server 32-bit Windows 2003 machine running Lotus Domino mail server 64-bit servers running Linux and Windows 2008, both acting as file servers While each of the solutions were tested independently of one another, results of these tests and the observations made point to the various Kaspersky Lab solutions providing a multi-faceted security framework for a corporate network. Taking a hypothetical network into account, as below, one can see how each of the solutions would interact with and secure the network. Anti-malware protection, at the gateway level, is provided by scanning email coming into the corporate network over SMTP with an initial scan by Kaspersky Anti- Virus 8.0 sitting on the TMG server. In turn, the email is then received by the Exchange or Domino server and a further scan conducted by the appropriate solution. Should any user require the downloading of email from an external POP3 server, the Kaspersky for TMG solution scans the traffic as it passes through the gateway. When dealing with files any that are downloaded over HTTP/FTP are scanned on the TMG/KAV combined server. Should any network user then attempt to upload any files to either a Windows or Linux based file server then here the respective Kaspersky Lab solution will provide further defense-in-depth. 11 of 24 www.westcoastlabs.com
KAV 8.0 for Linux File Server interface. Application interface of KAV for ISA understanding of Linux should be comfortable with the process. Managed via a web-based GUI running on a non-standard port, Kaspersky Anti-Virus 8.0 is configured from the GUI. No secondary interfaces or files need to be changed and updates are either scheduled or run on-demand. For security admin staff who may be familiar with a file-server anti-malware product, the make-up of the interface is very familiar it is both clear and intuitive. On-Access and On-demand protection are available as standard. Administrators can browse the Quarantine folder from within the product interface to review any malware logged and thus decide what actions to take. Given the complexities involved with porting anti-malware solutions to Linux, it is not always possible to ensure consistency of performance. However, Kaspersky Anti- Virus 8.0 sets itself apart in this regard. It is well implemented, as demonstrated in the comparative performance tests where it led with a 99.95% detection rate on the 25640 malware samples tested compared to an average performance rate of 99.52% for 5 other leading corporate solutions. Kaspersky Anti-Virus 8.0 for Lotus Domino Anyone familiar with Lotus Domino will find the installation straightforward. It is performed using a Lotus.nsf database file which is opened through Lotus Notes to run. Administrators can set various actions to be performed when malware is detected, however they will need to be familiar with Lotus in order to get the best out of the solution when rolling Kaspersky Anti-Virus 8.0 out to a Domino server. Delete or quarantine actions are easily defined for detected malware and for deleting infected attachments. Unlike some of the other vendor prod- Licensing process on Kaspersky Anti-Virus for Lotus t t 12 of 24 www.westcoastlabs.com
ucts included in the comparative performance review, Kaspersky Anti-Virus 8.0 does not need the installation of a desktop antimalware product to be able to use the desktop product s scanning engine signature files. In the comparative testing against 5 other leading corporate solutions, the test methodology employed a sender machines running a Linux distribution. Scripts developed by WCL were used to send the emails that contained infected attachments over a live Internet connection. Emails were sent to servers running Lotus Domino 8.5 on Windows 2003 that each picked up emails for a FQDN owned and controlled by WCL. Client machines running Lotus Notes 8.5 were used to pick-up the messages from the Domino servers and analysed the attachments to aid calculation of the overall detection rate which for Kaspersky Anti-Virus 8.0 was of a particularly high standard which mirrored that of the competitor products included in the test programme. All solutions attained a 100% detection rate during the test period. Kaspersky Anti- Virus 8.0 for Windows Servers Enterprise Edition Kaspersky Anti-Virus 8.0 for WSEE uses the standard Windows Installer interface. Two installations are required, one for the Administration tools and one for the solution itself. However, importing an existing configuration file to keep existing settings is possible when upgrading a previous version. Installation is quick and trouble-free. Managed through an MMC snap-in, the product allows product updates to be rolled-back if needed. It provides a quarantine area and a backup facility just in case the Administrator deletes a file that needs to be restored. The interface, as a whole, provides a rapid means of implementing malware security policies on the solution. All of the available features are easy to locate without the need for drilling down through multiple options screens or hunting for a Update Process on Kaspersky Anti-Virus WSEE required setting. On Demand scans can be set to a pre-defined security level or customized to meet the demands of the organisation. Similarly, On Access protection can be set with a preference for either high speed scans or high protection levels. Throughout the comparative test programme, WCL found the scans ran quickly with an overall detection rate for Kaspersky Anti-Virus 8.0 of 99.68% compared to an average performance of 99.51% for the other 5 security solutions included. WEST COAST LABS VERDICT Combining ease of use with high levels of performance, the Kaspersky Lab solutions under test have delivered comparable and at times, better detection rates to equivalent products. With a consistent level of anti-malware protection across the network topology, users of the Kaspersky Lab products featured in this report can be confident that they are all rigorously tested through the Checkmark Certification and the Real Time testing programme to provide ongoing independent validation on performance. 13 of 24 www.westcoastlabs.com
Checkmark Certifications for Kaspersky The Checkmark Certification System is recognised globally as probably the most comprehensive independent functionality and performance validation program of its kind. With three tiers of certification Baseline, Dynamic and Real Time testing vendors have the opportunity to commit to the System at a level that suits the performance of their products and services in the realworld. The Baseline certifications comprise a series of static benchmarking tests that measure detection capability against a finite suite of known malware threats. Whereas the addition of Dynamic and Real Time testing transforms this certification program into a threefold process that results in the most complete evaluation of an Anti-Malware vendor s products available. Static Testing baseline tests that measure detection capabilities against known threats. Dynamic Testing measures product performance in relation to malware executing as end users and corporations experience them in the real world. Real Time Testing measures critical performance characteristics in a network environment 24x7x365. The testing provides results in metrics including; performance in relation to time, attack vectors, heuristic behavior analysis, signature update and vendor research effectiveness. The combination of these three, distinct test programs provide the highest level certification of product performance available. All the Kaspersky Lab products that form part of this test program are registered in the Checkmark System for all three levels of testing Baseline, Dynamic (where appropriate) and Real Time. In Real Time, the products are tested 24x7x365 against live malware in a range of attack vectors are relevant to each product. These include FTP, HTTP, P2P, SMTP and Malicious Web Sites. Given the nature of the Real Time testing program and the fact that it is probably the most rigorous product performance validation of its kind, the products registered for Real Time testing are eligible for the Checkmark Platinum Product Award. Far more than just a measure of product performance it also acts as recognition of the vendor s commitment to the highest level of independent product validation and a measure of the vendor s responsiveness to emerging threats. The Kaspersky Lab products holding the Checkmark Platinum Product Awards are: Kaspersky Anti-Virus 8.0 for Windows Servers Enterprise Edition Kaspersky Anti-Virus 8.0 for Linux File Server Kaspersky Anti-Virus 8.0 for Lotus Domino Kaspersky Anti-Virus 6.0 for Windows Workstations Kaspersky Anti-Virus 8.0 for Microsoft ISA Server and Forefront TMG Standard Edition Kaspersky Security 8.0 for Microsoft Exchange Server Kaspersky Endpoint Security 8 for Linux 14 of 24 www.westcoastlabs.com
Checkmark Certification Profile Checkmark Anti Virus Anti Virus Trojan Spyware Anti Anti Anti Malware Certifications Detection Disinfection Malware Spam Dynamic Kaspersky Lab Applications Kaspersky Anti-Virus 8.0 for Windows Servers Enterprise Editon l l l l Kaspersky Anti-Virus 8.0 for Linux File Server l l l l Kaspersky Anti-Virus 8.0 for Lotus Domino l l l l Kaspersky Anti-Virus 8.0 for Microsoft ISA Server and Forefront TMG Standard Edition l l l l Kaspersky Security 8.0 for Microsoft Exchange Servers l l l l l Kaspersky Anti-Virus 6.0 for Windows Workstations Windows XP l l l l l l Windows Vista l l l l l l Windows 7 l l l l l l Kaspersky Endpoint Security 8 for Mac l Kaspersky Endpoint Security 8 for Linux l l l l l Kaspersky Anti Spam l Checkmark Real Time Real Time Real Time Real Time Real Time Real Time Certifications FTP HTTP SMTP P2P Mal URL Spam Kaspersky Lab Applications Kaspersky Anti-Virus 8.0 for Windows Servers Enterprise Edition l l Kaspersky Anti-Virus 8.0 for Linux File Server l l Kaspersky Anti-Virus 8.0 for Lotus Domino l Kaspersky Anti-Virus 8.0 for Microsoft ISA Server and Forefront TMG Standard Edition l l l l l Kaspersky Security 8.0 for Microsoft Exchange Servers l l Kaspersky Anti-Virus 6.0 for Windows Workstations Windows XP l l l Windows Vista l l l Windows 7 l l l Kaspersky Endpoint Security 8 for Linux l l The above chart denotes those certifications in which the respective Kaspersky solutions are currently enrolled. It is not reflective of each solution s test results or full protection capabilities. 15 of 24 www.westcoastlabs.com
Conclusion In this test programme, Kaspersky Lab products have undergone probably the most extensive testing carried out by West Coast Labs against a single corporate solution. These tests range from West Coast Labs established Checkmark Certification to ongoing performance validation the Real Time system and the custom malware comparative testing. This programme also includes the first ever product to be awarded the Checkmark Anti-Malware Macintosh certification. Upon completion of the tests covered in this report it can clearly be seen that Kaspersky are offering an extremely competitive and thorough security package to businesses and corporate organisations. For mail-based systems, Kaspersky recorded a 100% detection rate on both Exchange and Lotus against samples which propagate over the SMTP protocol. While this is an impressive detection rate, it should be noted that the other vendors also recorded the same detection levels. This should be an indicator to the level of importance of email coverage and the perceived threat to business communications that is held by the security industry as a whole. On file server-type systems, in this case Windows 2008 and Red Hat Enterprise 5, there is a differential in detection levels. On the Linux OS, Kaspersky recorded the highest detection rate amongst the solutions on test, whilst on the Windows OS Kaspersky recorded the second-highest detection rate. It should be noted that the difference between first and second in the Windows OS test was just 1/100th of a percent, thus putting Kaspersky above the Industry Average as defined in the test results. From the results of the test programme it can be concluded that not only do the Kaspersky solutions offer comparative detection rates to offerings from other vendors, it is clear that the level of protection afforded by Kaspersky Lab solutions is consistently high across the range of platforms. Whether corporate organisations require protection for the desktop environment, a file server, Microsoft Exchange email server, an Apple Mac client, or a server running Lotus Domino, the Kaspersky Lab performance is consistent throughout. Prospective users of Kaspersky Lab products and specifically those featured in this report, can take confidence from the fact that the solutions are rigorously tested on an ongoing basis through the Checkmark certification system and the Real Time testing programme to ensure independent validation of a consistently high standard of product performance. The full West Coast Labs Test Report for this project is available online at www.westcoastlabs.com/ producttestreports/ 16 of 24 www.westcoastlabs.com
Product Feature Set Comparisons West Coast Labs was asked to compile a comparative feature list for each of the products included in this test. This information has been gathered from freely available marketing literature of those companies included in this test. As this information is gathered from marketing and other such materials, the information contained within the following tables should be taken as a high level overview and does not constitute a comparison of those features that were examined as part of the extended malware testing. Research was carried out during September and October 2010 using the reference points detailed on the following pages. 17 of 24 www.westcoastlabs.com
Kaspersky Product Comparison Kaspersky Anti-Virus 8.0 for Microsoft ISA Server and Forefront TMG Standard Edition 1. System Requirements 2. Operating Systems Supported 2. 3rd party platforms/software supported 3. Security Technology components 4. Key Product Features Anti-Virus engine Scanning traffic Anti-Virus Settings Administration Performance Feature KAV 8.0 for Microsoft ISA/TMG SE Microsoft Forefront Threat Management Gateway 2010 Minimum Processor Spec: Minimum RAM Spec: 1 GHz processor for ISA Server 2006 Standard Edition and 64-bit dual-core processor for Forefront TMG Standard Edition Not specified 1 GB RAM for ISA Server 2006 Standard Edition and 2 GB RAM for Forefront TMG Standard Edition 2 GB Minimum available Hard Disk Space 2.5 GB 2.5 GB Supports Windows 2008 R2 Yes Yes Windows 2008 SP2 Yes Yes Microsoft Windows Server 2003 SP2 Yes Yes Microsoft Windows Server 2003 R2 Yes Yes Supports Microsoft Forefront TMG Yes Yes Compatibility with VMware (Vmware Ready) Yes Anti-Virus detection Yes Yes Detected objects: viruses, mass-mailer worms, Trojan horses, spam, spyware Yes Yes Real-time antivirus protection Yes Yes Update rate anti-virus every 1-2 hours not specified Creation of backup copies Yes Yes* Scanning of HTTP and FTP traffic Yes Yes Scanning of HTTPS traffic (Forefront TMG only) Yes Yes Scanning of POP3 and SMTP traffic Yes Provides management, but needs separate product for Exchange Scanning of HTTP and FTP traffic from published servers Yes Yes Scanning of VPN connections Yes Yes Exclusions from scanning Yes Yes Flexible policy settings Yes Yes Management via MMC Yes Yes Monitoring of application status through the administration console Yes Yes Flexible policy management Yes Yes Support for non-standard FTP commands Yes Yes Export and import of settings details Yes Yes Notification system Yes Yes Logging system Yes Yes Detailed reports Yes Yes Control over performance through the Windows Performance Monitor Yes Yes Automatic scalability Yes Yes Server load balancing Yes not specified Optimal use of system resources Yes Yes *This solution offers a comparable technology but is not referred to specifically by this name, or this technology is not specifically documented in the publicly available literature. 18 of 24 www.westcoastlabs.com
Kaspersky Product Comparison Kaspersky Security 8.0 for Microsoft Exchange Servers Feature Kaspersky Security Symantec Mail Security Trend Micro ScanMail McAfee GroupShield Sophos PureMessage ESET Mail Security 4 1. System Requirements RAM 256 MB 1 GB 1GB RAM, 2GB RAM 512 MB minimum, 1 GB 256 MB to 2 GB 2 GB recommended (5MB of recommended recommended (services) RAM per mailboyes) Available disk space required 512? MB 352 MB 1GB 740 MB minimum Console: 150 MB 1.9 GB Services: up to 2 GB 2. Operating Systems Supported Microsoft Exchange Server 2010 Yes Yes Yes Yes Yes Yes Microsoft Exchange Server 2007 Yes Yes Yes Yes Yes Yes Microsoft Windows Server 2008 R2 Yes Yes Yes Yes Other Software Information Microsoft Exchange 2003 is supported by Exchange 2010, 64 bit Native 64-bit support for Windows 2000-2003 Windows 2000-2003 Windows 2000-2003 another version Kaspersky Security for Windows, VMware and Hyper-V Exchange 2010 and 2007; Microsoft Exchange 2003 Virtualized environments 32-bit support for Exchange 2003/2000 3. Security Technology components Anti-Virus detection Yes Yes Yes Yes Yes Yes Anti-Spam detection Yes Yes Yes Yes Yes Yes Heuristic analyzer Yes Yes Yes Yes Yes Yes Linguistic analyzer Yes not specified Yes* not specified Yes No Real-time UDS requests Yes not specified not specified Yes* not specified No Graphical signature analyzer Yes not specified Yes No Yes No SPF and SURBL technologies Yes No No No No No 4. Key Product Features Anti-Virus engine Detected objects: viruses, mass-mailer worms, Trojan horses, spam, spyware Yes Yes Yes Yes Yes Yes Real-time antivirus protection Yes Yes Yes Yes Yes Yes Background on-demand scanning Yes Yes Yes Yes Yes Yes Update rate anti-virus every 1-2 hours rapid release definitions immediate protection AutoUpdate Updates automatically No Anti-Spam engine Classification of incoming messages Yes Yes Yes Yes Yes Yes Spam detection for different languages Yes No* No* No Yes No Update rate antispam every 5 min not specified not specified not specified constantly No Anti-Spam settings Intensity level Yes Yes Yes Yes Yes Yes Black and white listing Yes Yes Yes Yes Yes Yes Configurable scanning eyesceptions Yes Yes* Yes Yes Yes* Yes Anti-Virus Settings Configurable scanning eyesceptions Yes Yes Yes Yes Yes Yes Whitelisting Yes No Yes Yes No No Creation of backup copies Yes No In-memory scanning Yes No No Yes No* No* Administration and notifications via MMC Yes No No No No No Notification system Yes Yes Yes Yes Yes Yes Logging system Yes Yes Yes Yes Yes Yes Detailed reports Yes Yes Yes Yes Yes No* Performance Automatic scalability Yes No No No No No Optimal use of system resources Yes Yes Yes Yes Yes Yes Server Architecture Clusters support Yes Yes Yes Yes No No Compatibility with DAG in Microsoft Exchange 2010 Yes Yes Yes* Yes Yes No VMware ready Yes Yes No No No No *This solution offers a comparable technology but is not referred to specifically by this name, or this technology is not specifically documented in the publicly available literature. 19 of 24 www.westcoastlabs.com
Kaspersky Product Comparison Kaspersky Anti-Virus 8.0 for Linux File Server 1. System Requirements 2. Operating Systems Supported Feature KAV 8.0 for Linux FS Symantec Endpoint Protection Trend Micro Server Protect for Linux t McAfee VirusScan Enterprise Sophos Anti-Virus for Linuxt ESET File Security for Linux/BSD/ Solaris Intel Pentium II processor 400 MHz or higher Intel Pentium processor or compatible) Inte Pentium II 266 MHz or higher Intel x86 or x64; AMD x64 no information i386 (Intel 80386), AMD64 (x86_64) architecture (32-bit and 64-bit 512 MB RAM 1 GB RAM 256 MB min 256 MB min 256 MB 32 MB Cache size 1GB or higher 2 GB hard disk space for installation and 4 GB 50 MB for /opt + 50 MB for /tmp 500 MB 100 MB min 32 MB temporary files. Red Hat Enterprise Linux 5.5 Server Red Hat Enterprise Linux 3.x, 4.x, 5.x Red Hat Enterprise Linux (AS, ES, WS) 4.0 Red Hat Enterprise 4.x, 5.x Red Hat Enterprise 3, 4, 5 Linux Kernel version 2.2.x, 2.4.x or 2.6.x; glibc 2.2.5 or higher; Fedora 13 Fedora Core 10, 11, and 12 CentOS-5.5 CentOS 4.x, 5.x SUSE Linux Enterprise Server 10 SP3, 11 SP1 SuSE Linux Enterprise (server/desktop) SuSE Linux Enterprise Server 9 SuSE Linux Enterprise Server/Desktop 9.x, 10x, 11 SuSE Linux Enterprise Server 8, 9, 10, 11; 9.x, 10.x Desktop 10 Sun Solaris 10 Novell OES 2 SP2 Novell Open Enterprise Server (OES/OES2) Novell Linux Desktop 9 opensuse Linux 11.3 opensuse Linux 10/10.1 Mandriva Enterprise Server 5.1 (32 bit only) TurboLinux 10/11 Server Ubuntu 9.10 Server Edition Ubuntu 7.x, 8.x Ubuntu 8.04, 9.04, 9.10 Ubuntu LTS Server Edition 6.06/8.04 Ubuntu 10.04 LTS Server Edition Debian GNU/Linux 5.0.5 Debian 4.x Debian 3.1 FreeBSD 7.3, 8.1 FreeBSD: Version 5.x, 6.x, 7.x Miracle Linux 4.0 Dazuko kernel module 2.0.0 or higher (optional) Asianux 2.0/3.0 NetBSD 4.x 2. Security Technology components Anti-Virus detection Yes Yes Yes Yes Yes Yes Backup/Quarantine Yes Yes Yes Yes Yes Yes 3. Key Product Features Anti-Virus engine Detected objects: viruses, Trojan horses, spyware Yes Yes Yes Yes Yes Yes Real-time antivirus protection Yes Yes Yes Yes Yes Yes Background on-request or on-demand scanning Yes Yes* Yes Yes* Yes Yes Update rate anti-virus every 1-2 hours daily every 1 hour every 1 hour as often as every 10 minutes daily Creation of backup copies Yes No* No* No* No* No* Scanning of critical system areas Yes Yes Yes Yes Yes Yes* Scans and treats archived files Yes Yes Yes Yes Yes Yes Anti-Virus Settings Assigning trusted zones /users Yes Yes No* No* Yes No* Flexible setting of scan times Yes No* Yes No* Yes No* Additional settings for Samba servers Yes No* No* No* Yes No* Administration Centralized administration Yes Yes Yes Yes Yes Yes Administration via Kaspersky Web Management Console Yes n/a n/a n/a n/a n/a Command line administration Yes Yes No* No* Yes Yes Notification system Yes Yes Yes Yes Yes Yes Logging system Yes Yes Yes Yes Yes Yes Detailed reports (PDF, XLS, CSV, etc.) Yes Yes Yes Yes Yes Yes Performance Automatic scalability Yes Yes Yes* Yes* Yes* Yes* Optimal use of system resources Yes Yes* Yes* Yes* Yes* Yes* Server load balancing Yes Yes* Yes* Yes* Yes* Yes* Continuous server operation Yes Yes Yes Yes Yes Yes t The McAfee and Sophos products support other Linux implementations but only for on-demand scanning, not on-access scanning *This solution offers a comparable technology but is not referred to specifically by this name, or this technology is not specifically documented in the publicly available literature. 20 of 24 www.westcoastlabs.com