Commercial in Confidence. Test Report February McAfee Anti-Malware Custom Test

Transcription

1 Test Report February 2012 McAfee Anti-Malware Custom Test

2 WCL Corporate Offices and Test Facilities US Headquarters and Test Facility West Coast Labs, Von Karman Avenue, Suite 125, Irvine, CA 92606, U.S.A. Tel: +1 (949) , Fax: +1 (949) European Headquarters and Test Facility West Coast Labs, Unit 9 Oak Tree Court, Mulberry Drive, Cardiff Gate Business Park, Cardiff, CF23 8RS, UK. Tel: +44 (0) , Fax: +44 (0) Asia Headquarters and Test Facility West Coast Labs, A2/9 Lower Ground Floor, Safdarjung Enclave, Main Africa Avenue Road, New Delhi , India. Tel: +91 (0) , Fax: +91 (0) Date: 9 th March 2012 Version: 1.0 Authors: Scott Markle, Peter Walker, Mark Thomas, Richard Thomas Page 2 of 15 West Coast Labs 2012

3 Contents Executive Summary 4 Introduction 5 Test Environment / Network 6 Test Cases 7 Test Collections 8 Test Methodology 10 Test Results 12 Conclusion 15 Working with West Coast Labs 16 Disclaimer 17 Revision History 18 Page 3 of 15 West Coast Labs 2012

4 Executive Summary This test report by West Coast Labs was commissioned by McAfee Inc. to examine the protection offered by a complete suite of products which provide a recommended layering or stacking of various security technologies within McAfee s stable of products. The move towards multi-faceted protection has been gathering pace over the last few years, with more and more technologies coming into play in terms of protecting users from the vast and increasing number of threats. Each technology has its own particular area to cover and particular applications within a business environment and, when combined, offer a whole protection profile that is greater than the sum of its parts. This report considers several types of technologies, including Host Intrusion Prevention, Application Whitelisting, Anti-Malware functionality when samples are executed on a machine. We tested a broad sample of over 7300 live malware and McAfee achieved fully 100% protection against all threats. Page 4 of 15 West Coast Labs 2012

5 Introduction Representatives of McAfee Inc. (McAfee) approached West Coast Labs (WCL) with a proposal for an investigation into the different layers of defence as provided by four of their (McAfee) products. The products in question were: McAfee Application Control (MAC) McAfee VirusScan Enterprise (VSE) McAfee Host Intrusion Prevention System (HIPS) The associated setup, methodology, and results of testing are included in this report. As agreed between both McAfee and WCL, this report is cleared for EXTERNAL use. All information contained within this report remains the property of WCL. Testing was conducted in WCL s UK lab during February Page 5 of 15 West Coast Labs 2012

6 Test Environment / Network One test network was constructed in order to carry out the testing outlined in this report, the details of which are outlined below: Network 1 This network was used to conduct Anti-Malware tests as detailed in the Test Methodologies section of this report. Five individual hosts were used in the following configuration: 1. Windows Server 2008 R2 Enterprise Edition 1 64-bit running epolicy Orchestrator 2. Windows 7 Professional 64-bit running VSE 3. Windows 7 Professional 64-bit running VSE and HIPS 4. Windows 7 Professional 64-bit running VSE, HIPS, and MAC Each of the protected hosts (numbered 2-4 above) was configured to report back to the epolicy Orchestrator server (machine 1) and had direct access to a live Internet connection. Each host was running a standard installation of the respective Operating System (OS) with only the addition of specific software required for testing. Only those updates required by the McAfee products were allowed, for example DAT file updates. 1 Running Windows Server 2008 Service Pack 2 Page 6 of 15 West Coast Labs 2012

7 Test Case The purpose of this test is to examine any potential increase in protection when adopting a layered security approach using McAfee solutions, otherwise referred to as "stacking". Testing proceeded to look at how well the solutions performed when blocking attempts to execute malware on the protected system and whether enhanced protection would be provided by stacking the solutions. Page 7 of 15 West Coast Labs 2012

8 Test Collections For this test, a malware test suite was constructed that consisted of three separate sub-collections Viruses, Trojans, and Spyware. Malware WCL s malware sources, reflected in those files used in this test, are split into three main groups WildList Organisation, WCL s global honeypot system, and industry sources. Of the files used in this test, the virus files, apart from those in the WildList, are of many families, with the most frequently found being Allaple, Conficker and Virut variants. For the Trojan and Spyware collections, the most frequently found families include Agent, Alureon, Backdoor, Bancos, Banker, Banload, Downloader, Hupigon, Lineage and VB. The first recorded attack associated with this collection was July 2008 and ranging up to November With the former date in mind, it is worth stating that all of the samples used in this test were found to be "in the wild" in the 12 months leading up to December the date construction began on the test environment. In order to give a clear indicator of how prevalent the samples used in this test are, the 7,313 malicious files have been associated with 4,982,215 attacks against the WCL global honeypot system to date. Page 8 of 15 West Coast Labs 2012

9 Test Methodology Anti-malware As the purpose of this test is to examine the ability of the solution(s) to block execution of malware, it was conducted as per the ISO accredited Checkmark On-Access test procedure. Each sample included in the selected malware collection, constructed prior to testing, was placed on the desktop of each of the protected hosts. Engineers then attempted to run the malware samples and recorded the outcomes. For the purposes of this test, a successful detection was recorded if the solution(s) detected the presence of malware either upon browsing to the file location, upon highlighting the file, or on the attempt at execution. The resulting log file entries were also consulted, both on the local host and the central epolicy Orchestrator server. If no block message was displayed in either location, then the host machine was rebooted to a Linux-variant and analysis conducted to determine if infection had taken place. Page 9 of 15 West Coast Labs 2012

10 Test Results Anti-Malware Protection - Comparative Results The following results show the ability of the combined, or "stacked", McAfee solutions to block malware when an execution attempt is made on a protected host machine. VSE On-Access, HIPS, and MAC Collection Missed Detected Total Detected (%) Missed (%) Anti-Malware Desktop Spyware Trojan Overall (Detection capability with all products deployed) During the tests, it became obvious that the VSE solution acted as a first line of defence, often nullifying the malware sample before any execution attempt could be made. This was then ably followed up by the protection mechanisms of both HIPS and MAC for those samples not immediately detected by VSE. Page 10 of 15 West Coast Labs 2012

11 Conclusion Looking at the outcomes of the malware testing, the most obvious take away is the high level of protection provided by VirusScan Enterprise from the outset, before any of the other protection mechanisms are called into action. This is the case whether the user opts to conduct a manual, on-demand scan of the infected file or jumps straight to an execution attempt. When looking at the addition of HIPS, results show a not insignificant 34% decrease in missed samples. Moreover, when considering the possible ramifications had those samples been missed, such as stolen user information or possible compromising of a user s bank security, then this added detection becomes significantly more important. Finally, there is the 100% block rate associated with the use of the MAC solution. Any potential administrator should keep in mind that each of the security solutions used in this test has a specific role, and specific environment, in which they're best suited. Due to the protection mechanisms used by MAC, it is possibly best suited to more machines such as servers and standardized desktops, that have less dynamic configurations, where users are not regularly installing new software and applications. The VSE and HIPS combination, would provide a far greater degree of flexibility and could possibly reduce an otherwise increased burden on administrators, if used to protect more dynamic workspaces such desktop-based systems. Overall, the fact that these solutions are able to start with an already high detection rate, and then build on that detection rate, should be taken as testament to the capability and effectiveness of both the HIPS and MAC solutions. Page 11 of 15 West Coast Labs 2012

12 Working with West Coast Labs The West Coast Labs (WCL) business philosophy is founded on quality and excellence with all testing activities carried out within the framework of the ISO 17025:2005 accreditation standard for Test Laboratories. WCL prides itself on its open and proactive working relationship with all its clients. All testing is carried out within a framework of confidentiality that ensures integrity of information and test data for both parties. Positive and meaningful communication between WCL test teams and client technical representatives, ensures that individual products are tested properly and effectively within an independent and secure test environment. WCL encourages positive engagement with client technical staff at the outset to ensure that products under test are set-up, configured and operating properly before testing commences. Such contact takes the form of teleconferences or online technical briefing sessions followed either by remote or on-site set-up and configuration. Testing itself is conducted by the designated WCL test teams with the provision for ongoing technical feedback to clients during this process. The outcome is a sound and proactive technical working relationship that ensures the client derives maximum benefit from working with an independent test facility. Page 12 of 15 West Coast Labs 2012

13 West Coast Labs Disclaimer While West Coast Labs is dedicated to ensuring the highest standard of security product testing in the industry, it is not always possible within the scope of any given test to completely and exhaustively validate every variation of the security capabilities and/or functionality of any particular product tested and/or guarantee that any particular product tested is fit for any given purpose. Therefore, the test results published within any given report should not be taken and accepted in isolation. Potential customers interested in deploying any particular product tested by West Coast Labs are recommended to seek further confirmation that the said product will meet their individual requirements, technical infrastructure and specific security considerations. All test results represent a snapshot of security capability at one point in time and are not a guarantee of future product effectiveness and security capability. West Coast Labs provide test results for any particular product tested, most relevant at the time of testing and within the specified scope of testing and relative to the specific test hardware, software, equipment, infrastructure, configurations and tools used during the specific test process. West Coast Labs is unable to directly endorse or certify the overall worthiness and reliability of any particular product tested for any given situation or deployment. Page 13 of 15 West Coast Labs 2012

14 Revision History Revision History Issue Description of Changes Date Issued 1.0 First version of McAfee Test Report 9th Mar 2012 Page 14 of 15 West Coast Labs 2012

15 US SALES T +1 (949) EUROPE SALES T +44 (0) CHINA SALES T CORPORATE OFFICES AND TEST FACILITIES US Headquarters and Test Facility West Coast Labs Von Karman Avenue, Suite 125, Irvine, California, CA92606, USA T +1 (949) , F +1 (949) European Headquarters and Test Facility West Coast Labs Unit 9, Oak Tree Court, Mulberry Drive Cardiff Gate Business Park, Cardiff CF23 8RS, UK T +44 (0) , F +44 (0) Asia Headquarters and Test Facility A2/9 Lower Ground floor, Safdarjung Enclave, Main Africa Avenue Road, New Delhi , India. Facilities also in Hong Kong, Singapore and Sydney E info@westcoast.com W Facilities also in Hong Kong, Singapore and Sydney Page 15 of 15 West Coast Labs 2012