ESET Mail Security 4 April 2010

Transcription

1 ESET Mail Security 4 April 2010 Performance Test & Review Language: English April 2010 Last Revision: 6 th August 2010 Commissioned by ESET

2 Tested Product ESET Mail Security 4 Installed on MS Exchange 2003 and MS Exchange

3 Tested Product We tested the ESET Mail Security 4 product from ESET ( with last signature update in mid of April. All tests were done with up-to-date products; the tested products were not frozen. Performance/Stress Test The test was splitted in two categories. 1. Mail-Sending-Test 2. Exchange Server Stress and Performance Test (MS-Toolkit) Hardware Configuration We used for the tests (Exchange 2003 / Exchange 2010) 40 identical workstations and one Server. To simulate a real office environment, we did not use newest hardware in this test. Workstations HP 7600 SFF Intel Pentium IV 2.8 GHz 1 GB Ram 40 GB SATA HDD DVD-RW Server HP ProLiant DL360G4, 2x Xeon HT 3.40 GHz 8GB, 1x36GB 10k Hotplug, DVD/CD-RW SmartArray 6i, 1x PS, 1U Rack 3 -

4 Mail-Sending-Test In this test, we send and receive the mails simultaneously with 40 clients, connected via a 1GB LAN connection in order to avoid latency from the internet. This test was done using real hardware. We sent mixed mails (60% Spam, 30% Ham, 10% Malware) from 40 workstations; about 1500 mails per workstation. The result of the impact of an Security product depends mainly on the used hardware. You also have to keep in mind, that it will very rarely happen that a server (except ISP s) have to handle such a massive amount of s. In the real world, it will not happen that a SMB will receive mails during a period less than two hours through an internet connection. Usually the bottleneck is not the Exchange server with an installed security product, the bottleneck is the slow internet connection. The mails were sent within the same domain and organization. Results MS Server 2003 with Exchange Server 2003 There was no problem to process ~60000 mixed mails in a period of less than two hours. With installed ESET Mail Security 4, there was a delay of about 21%. The CPU workload toddles between 1% and 5%, with or without ESET Mail Security. The RAM usage with ESET Mail Security was about 2,8 GB (2003) and without 1,24 GB. Results MS Server 2008 R2 with Exchange Server 2010 There was no problem to process ~60000 mixed mails in a period of less than two hours. With installed ESET Mail Security 4, there was a delay of about 22%. The CPU workload toddles between 1% and 5%, with or without ESET Mail Security. The RAM usage with ESET Mail Security was about 2,8 GB and without 2GB. Exchange 2003 vs. Exchange 2010 The Exchange Server 2010 was about 19% faster than the Exchange Server 2003, both with installed ESET Mail Security

5 Table of Performance Overview 2003 with installed ESET Mail Security without installed ESET Mail Security 103 minutes + 21 % 83 minutes 2008 R2 with installed ESET Mail Security without installed ESET Mail Security 85 minutes + 22 % 67 minutes 2003 with ESET % 2008 R2 with ESET 85 Please keep in mind that the loss of time depends on the configuration of the hardware and the mails, which were used. It is not only the count of mails: It depends on the size and the type of mails (spam, malware, clean). There was no system crash and no temporarily freeze during the whole test. The solution ran absolutely stable. 5 -

6 Exchange Server Stress and Performance Test Methodology We used the automated Test Procedure provided by Microsoft that simulates a huge count of several Most common Tasks and analyzed how many tasks were able to complete in a defined period of time. This test is not done with real clients, it is only a simulation. Tasks Overview: Task Name BrowseCalendarTask BrowseContactsTask CreateContactTask CreateTaskTask LogoffTask MakeAppointmentTask ModuleInitTask PostFreeBusyTask ReadAndProcessMessagesTask RequestMeetingTask SendMailTask Browsing Calendar Tasks Browsing Contact Tasks Creating Contacts Creating Tasks Log-off and Log-on Tasks Creating Appointments Module initialization Creating and posting Free/Busy Reading and processing Message Tasks Meeting Requests Send Mail With this bunch of thousands of tasks, we were able to simulate an extremely Hardware and Software resource intensive situation. 6 -

7 Windows Server 2003 & Exchange 2003 With ESET Mail Security EXC SRV: Exchange 2003 Clients: Outlook 2003 & 2007 (online) Users: 40 Concurrent Connections: 5000 Name Succeeded Client Type Action Profile TestClients x 40 Succeeded Outlook 2003 Online Active Users Statistics Active User Count Duration 40 00:10:00 Task Execution Statistics Task Name Count BrowseCalendarTask 108 BrowseContactsTask 114 CreateContactTask 9 CreateTaskTask 6 LogoffTask 17 MakeAppointmentTask 23 ModuleInitTask 1 PostFreeBusyTask 33 ReadAndProcessMessagesTask 174 RequestMeetingTask 5 SendMailTask 39 User Count Tasks per User Day Tasks Completed Average

8 Without ESET Mail Security EXC SRV: Exchange 2003 Clients: Outlook 2003 & 2007 (online) Users: 40 Concurrent Connections: 5000 Name Succeeded Client Type TestClients x 40 Succeeded Outlook 2003 Online Active Users Statistics Action Profile User Count Tasks per User Day Tasks Completed Average Active User Count Duration 40 00:10:00 Task Execution Statistics Task Name Count BrowseCalendarTask 119 BrowseContactsTask 125 CreateContactTask 7 CreateTaskTask 9 LogoffTask 24 MakeAppointmentTask 34 ModuleInitTask 1 PostFreeBusyTask 29 ReadAndProcessMessagesTask 245 RequestMeetingTask 8 SendMailTask 94 Conclusion: The impact of ESET Mail Security was about 30%. 8 -

9 Windows Server 2008 R2 & Exchange 2010 With ESET Mail Security EXC SRV: Exchange 2010 Clients: Outlook 2003 & 2007 (online) Users: 40 Concurrent Connections: 5000 Name Succeeded Client Type Action Profile TestClients x 40 Succeeded Active Users Statistics Active User Count Outlook 2003 Online Duration 40 00:10:00 Task Execution Statistics Task Name Count BrowseCalendarTask 155 BrowseContactsTask 138 CreateContactTask 15 CreateTaskTask 13 LogoffTask 39 MakeAppointmentTask 38 ModuleInitTask 1 PostFreeBusyTask 51 ReadAndProcessMessagesTask 267 RequestMeetingTask 5 SendMailTask 46 User Count Tasks per User Day Average Tasks Completed 9 -

10 Without ESET Mail Security EXC SRV: Exchange 2010 Clients: Outlook 2003 & 2007 (online) Users: 40 Concurrent Connections: 5000 Name Succeeded Client Type Action Profile TestClients x 40 Active Users Statistics Active User Count Succeeded Outlook 2003 Online Duration 40 00:10:00 Task Execution Statistics Task Name Count BrowseCalendarTask 162 BrowseContactsTask 130 CreateContactTask 12 CreateTaskTask 16 LogoffTask 43 MakeAppointmentTask 28 ModuleInitTask 1 PostFreeBusyTask 49 ReadAndProcessMessagesTask 525 RequestMeetingTask 17 SendMailTask 108 User Count Tasks per User Day Tasks Completed Average Conclusion: The impact of ESET Mail Security was about 30% in the MS Test. 10 -

11 ESET Mail Security Simple Review Installation: First, we are greeted by the Welcome page of the installation wizard: The next step is to enter the licence details, which are sent to the user by after purchasing the program. The next part of installation is the registration of the licence. After the obligatory acceptance of the End User Licence Agreement, the user can choose the installation method. We chose to use Typical, i.e. standard settings. The next stage is not very different from the installation of other ESET products. The ThreatSense.Net Early Warning System allows ESET to react quickly to new, unknown threats. When a new threat is detected, information about it is sent to ESET. 11 -

12 This function offers a high level of protection against unknown threats. However, for organisations with a high security environment in which unmonitored external communication is restricted, the feature can be disabled. The option for doing this is shown below: The potentially unwanted applications detection is also part of the Mail Security program. This option can be configured with a simple drop down menu: After this setting has been selected, the installer has enough information to complete the setup process. One last click confirms the settings and starts the installation: 12 -

13 When installation has successfully completed, ESET Mail Security starts up. The Control Center can be started from the icon in the system tray: A first glance gives the user an overview of the state of the system. You can see whether all three aspects of system security are working properly. One look at the Control Center shows that the interface is seamlessly integrated with that of other ESET products. Anyone familiar with ESET will immediately find their way around and be able to make changes easily. The interface of ESET Mail Security can be described as optimal, and the clarity of the layout makes it easy to find what you need. Under Protection Status => Watch Activity you can see a graphic representation of File Activity and Mail Server Protection Performance. Under Protection Status => Statistics you can see the history of infections and clean objects. 13 -

14 The Setup page is divided into two categories. The first is Antivirus and antispyware, and the second is Antispam protection. The sub components of each category are listed, and the enabled/disabled status is shown. These components can be configured by clicking on the blue category titles, or the additional links at the bottom of the page. Computer Scan allows the user to check for malicious code on the system. This interface is identical to that of other ESET security products. The configurable components of the Antivirus and antispyware protection category are as follows: Real time protection client protection The Update page is also familiar, and informs the user of the current update status, and versions of the program and signatures. Mail server protection Web access protection The last item on this configuration page allows temporary deactivation of the malware detection with a single click. 14 -

15 The configuration of the Antispam protection does not differ significantly from that of Antivirus protection. client protection can be configured here, and whitelists and blacklists can be defined. There are also more detailed options for configuring Mail server protection. The Help and support page follows ESET s familiar exemplary style. It provides centralised access to the following: ESET Suite Help Files Extensive FAQ Knowledge base search Version numbers for all components of the suite The opportunity to submit Customer Care enquiries The Tools page offers the following familiar functions: Log files: This provides easy access to all log files. Quarantine: Any suspicious objects which have been quarantined are shown here. Scheduler: Allows specific tasks to be run at a pre defined time. SysInspector: This provides both a summarised and a detailed Sys Inspect Log. 15 -

16 Conclusion ESET Mail Security provides optimal protection for mail servers, and combines all necessary features including Anti Virus protection of the server. The installation and configuration follow ESET s familiar simple pattern. The Control Center is well designed, and the clear structure enables users to find their way around quickly and easily, even if they are not already familiar with ESET products. There is thus no absolute need for server administrators to look in a manual. The malware detection and performance reach ESET s normal high standard, and the suite can be wholeheartedly recommended. Our only minor gripe is that there is effectively no spam logging function for the Mail Server Protection, which makes a detailed analysis of mail attacks impossible. There is just a log of detected malware in the mails. However, we hope that there may be future improvements to this. Highlights One of the important things is the very low false positive rate, which is under 0,5 % at personal mails. Even if there is system impact of round about 20%, this can be taken out of account, because it is very unusual to receive mails in a single blow. There was no system crash and also no temporarily freeze in the whole test. The solution runs absolutely stable. 16 -

17 Copyright and Disclaimer This publication is Copyright 2010 by AV-Comparatives e.v.. Any use of the results, etc. in whole or in part, is ONLY permitted after the explicit written agreement of the management board of AV- Comparatives e.v., prior to any publication. AV-Comparatives e.v. and its testers cannot be held liable for any damage or loss, which might occur as result of, or in connection with, the use of the information provided in this paper. We take every possible care to ensure the correctness of the basic data, but a liability for the correctness of the test results cannot be taken by any representative of AV-Comparatives e.v. We do not give any guarantee of the correctness, completeness, or suitability for a specific purpose of any of the information/content provided at any given time. No one else involved in creating, producing or delivering test results shall be liable for any indirect, special or consequential damage, or loss of profits, arising out of, or related to, the use or inability to use, the services provided by the website, test documents or any related data. AV-Comparatives e.v. is a registered Austrian Non-Profit-Organization. For more information about AV-Comparatives and the testing methodologies, please visit our website. AV-Comparatives e.v. (April 2010) 17 -