Performance test November 2014 / 1 INTRODUCTION... 1 TESTED PROGRAM VERSIONS..2 WHAT AND HOW WE TESTED. 3 OTHER PRINCIPLES...

Transcription

1 PE RF ORMANCET E S TOFF RE E ANT I VI RUSS OF T WARE ANDI NT E RNE TS E CURI T YS UI T E S NOVE MBE R14 Per f or medt estdat e: Oct ober-november14

2 Performance test November 14 / 1 INTRODUCTION... 1 TESTED PROGRAM VERSIONS..2 WHAT AND HOW WE TESTED. 3 METHODOLOGY OTHER PRINCIPLES...5 GRAPHS HOW WE RATED 16 CERTIFICATES GRANTED.. As in last year as now we prepared for our readers performance test for the most popular paid and free security software. Unfortunately during test pending from October to November several producers haven t released their new versions of software. We are talking about Czech TrustPort, which has released only TrustPort Internet Security 14 version unlike vast majority of competition. In similar situation is Polish Arcabit, which hasn t published its latest solution yet, as well as American Comodo, but we can expect stable version shortly, so we decided to test BETA version of Comodo Internet Security 8. Regarding test criteria this year we decided not to measure full system scan time. Such a thing is certainly interesting, but we know that scanning should be above all effective. Regarding resources usage during scan, we should take into consideration that some programs may use technologies under different names, which initiate smart scan when user isn t using computer, scanning process receive more system resources. Certainly many user noticed that antivirus programs can slow down operations such as archiving and data extracting or copying and opening files. It s because of on-access file scanning in real time during mentioned operations. Antivirus programs use different solutions to reduce scan time, for example they don t scan files twice (only if they are modified) and files that don t carry viruses. All similar options can be found in program settings. Our advice for all antivirus software is as follows: perform full system scan immediately after installation which will result in measurable benefits for user. Firstly you will see if system is clean, secondly antivirus program will scan all files, so there is no need to use system resources later during work and entertainment, because most files will be already scanned.

3 Performance test November 14 / 2 TESTED PROGRAM VERSIONS Program name Country Version Arcabit Internet Security 14 Poland 14/ avast! Free Antivirus Czech Republic avast! Internet Security Czech Republic AVG Antivirus Free Czech Republic.0.53 AVG Internet Security Czech Republic.0.53 Avira FreeA ntivirus Germany Avira Internet Security Germany Baidu Antivirus 5 China Bitdefender Antivirus Free Edition Romania Bitdefender Internet Security Romania..0. BullGuard Internet Security United Kingdom Check Point Zone Alarm Internet Security Suite USA Comodo Internet Security 8 BETA USA Dr.Web Security Space 10 Russia Emsisoft Internet Security 9 Austria escan Internet Security with Cloud Security 14 India ESET Smart Security 8 Slovakia FileMedic Internet Security 14 Poland..0.9 Fortinet FortiClient 5 USA F-Secure SAFE / Internet Security Finland G Data Internet Security Germany Kaspersky Internet Security Russia Lavasoft Ad-Aware Free Antivirus+ Malta Malwarebytes Anti-Malware Pro USA McAfee Internet Security USA Microsoft Security Essentials USA Panda Free Antivirus Spain.0.4 Panda Internet Security Spain.0.4 Qihoo 360 Total Security 5 China Symantec Norton Security USA Trend Micro Internet Security Japan TrustPort Internet Security 14 Czech Republic Webroot SecureAnywhere Internet Security Plus USA

4 Performance test November 14 / 3 WHAT AND HOW WE TESTED? This year we took into consideration following aspects: system startup time with antivirus installed, CPU and RAM usage during idle and scanning, copying time, archiving time, extracting time and program size after installation including all virus signatures and files. To verify all aspects mentioned above we prepared VMWare Workstation 9 on host //CPU: Athlon II X ,8GHz, //RAM: 2x4GB DDR MHz, //HDD: Seagate 500 GB SATA II 70 RPM //32MB Virtual Cache Windows 7 x32 Professional + SP1 + critical updates dated 1 September (2 cores assigned, 2GB RAM) which we were restoring after each test. In addition we installed following programs: Windows Update has been disabled because of influence on the test results. Antivirus programs may use internet connection and update their signatures. Methodology 1. Restore system to initial state. 2. Download antivirus program from producer website and install with default settings. 3. Restart system and measure startup time first time. 3. Update signatures database and program files. 4. Restart system and measure startup time second time. 5. Check resources usage during idle and scanning. 6. Check copying time. 7. Check archiving and extracting time. 8. Check program size. 9. Restart system and third measurement, back to point 1.

5 Performance test November 14 / 4 System startup time We used performance monitor to accurately measure system startup time, which is build into Windows. Time was measured from selecting user account to full system start. Procedure was repeated three times: after program installation, system was restarted, we measured time after second logging in, then we were updating signatures and program files, system was restarted and after that we measure time again. Last measurement followed after all other tests. RAM and CPU During idle to verify demand for resources by antivirus program, we checked usage of resources by its all processes, not including process related to GUI, unless it automatically worked together with other processes. After signatures and files update and system restart, we waited 2-3 minutes and after that we run performance monitor. During data collection procedure, when program automatically was updating its signature or scanning files, we aimed for smallest resources usage, so action was repeated couple of times. Results were collected every second for 5 minutes, we calculated average value for each process, RAM memory and CPU was summed which gave us accurate CPU and RAM usage by antivirus processes. Procedure of measurement during scanning looked exactly the same with difference that we added process related to graphical user interface, that was used to run full system scan. Data was also collected for 5 minutes. Copying time To see impact on data copying speed during on-access scan, we prepared folder on C partition 2,5GB in size containing 1 files (ISO, EXE, JPG, ZIP, PNG, OTHER). Data from location above were copied to C:\user\Desktop, time was measured with ExtremeCopy software. Archiving and extracting time To see impact on archiving and extracting process, we used Easy-7 ZIP. Prepared folder 3MB in size contained 1 files was packed in zip and then unpacked. Time was measured with mentioned program. Program size Last thing is program size. Although all programs more or less use cloud technology, it doesn t mean that in all cases their size is reduced to minimum. This aspect wasn t taken into consideration in the final result.

6 Performance test November 14 / 5 WE TAKE INTO CONSIDERATION All program were tested on one virtual machine, which had restorable system. This way, we don t ignore caching access to files by file system, so that each program had same testing environment. I most cases, antivirus program scan files only once until they are modified. Special algorithm can judge, if file has been already scanned and antivirus could have better result in our test. Running (without restoring clean system, even after restart) scan on-access or on-demand causes second test can be finished much faster. Test performed that way aren t reliable, because the most important thing in this test is to measure drop in performance during scan on clean files. Scan time largely depends on process priority or relevant technology (local or cloud) which lies in fact that if user is using computer during scanning, process priority is automatically limited not to slow down system. Principle of equality In comparative tests all programs should be tested on the same machine. Operation system and installed software should be the same as well. Creating system or partition images is good way to map identical test conditions. Appropriate methodology All performance measurements should be tested on clean files. Testing infected samples during scan may be interesting, but such cases are extremely rare and most users don t have hundreds or thousands infected files on their hard drives. In addition, program settings should reflect the most common configuration, in most cases default settings are the most appropriate solution. Updates Products should be fully updated before performance test (AMTSO recommendations), because some programs may be installed without virus signatures, which may affect test results. Testers should be aware of Windows Update and its potential impact in performance tests. Automatic system update should be disabled during test. Update process may affect on research and changes made by it can lead to different results.

7 Performance test November 14 / 6 STARTUP TIME[ S ] Startup 1 Startup 2 Startup 3 Average Windows 7 Professional x32 SP Lavasoft Ad-Aware Free Antivirus+ 12 Webroot SecureAnywhere Antivirus Malwarebytes Anti-Malware Pro Symantec Norton Security Qihoo 360 Total Security 5 Panda Free Antivirus 16 McAfee Internet Security Kaspersky Internet Security TrustPort Internet Security 14 F-Secure SAFE / Internet Security G Data Internet Security Fortinet FortiClient 5 ESET Smart Security 8 16 Trend Micro Internet Security avast! Internet Security Baidu Antivirus 5 AVG Internet Security avast! Free Antivirus Arcabit Internet Security Panda Internet Security FileMedic Internet Security escan Internet Security with Cloud Security AVG Antivirus Free Check Point ZoneAlarm Internet Security Suite Bitdefender Internet Security 24 Bitdefender Antivirus Free Edition 24 Emsisoft Internet Security 9 27 BullGuard Internet Security Comodo Internet security 8 BETA Avira Internet Security Dr.Web Security Space Avira Free Antivirus Microsoft Security Essentials

8 Performance test November 14 / 7 A V E R A G E S Y S T E M S T A R T U P T I M E [ S ] AV Windows W i n d o w s 7 P r o f e s s i o n a l x 3 2 S P 1 13 L a v a s o f t A d - A w a r e F r e e A n t i v i r u s + W e b r o o t S e c u r e A n y w h e r e A n t i v i r u s 8 M a l w a r e b y t e s A n t i - M a l w a r e P r o S y m a n t e c N o r t o n S e c u r i t y Q i h o o T o t a l S e c u r i t y 5 P a n d a F r e e A n t i v i r u s M c A f e e I n t e r n e t S e c u r i t y K a s p e r s k y I n t e r n e t S e c u r i t y T r u s t P o r t I n t e r n e t S e c u r i t y F - S e c u r e S A F E / I n t e r n e t S e c u r i t y G D a t a I n t e r n e t S e c u r i t y F o r t i n e t F o r t i C l i e n t 5 E S E T S m a r t S e c u r i t y 8 T r e n d M i c r o I n t e r n e t S e c u r i t y a v a s t! I n t e r n e t S e c u r i t y B a i d u A n t i v i r u s 5 A V G I n t e r n e t S e c u r i t y a v a s t! F r e e A n t i v i r u s A r c a b i t I n t e r n e t S e c u r i t y P a n d a I n t e r n e t S e c u r i t y F i l e M e d i c I n t e r n e t S e c u r i t y e S c a n I n t e r n e t S e c u r i t y w i t h C l o u d S e c u r i t y 1 4 A V G A n t i v i r u s F r e e C h e c k P o i n t Z o n e A l a r m I n t e r n e t S e c u r i t y S u i t e B i t d e f e n d e r I n t e r n e t S e c u r i t y B i t d e f e n d e r A n t i v i r u s F r e e E d i t i o n E m s i s o f t I n t e r n e t S e c u r i t y 9 B u l l G u a r d I n t e r n e t S e c u r i t y C o m o d o I n t e r n e t s e c u r i t y 8 B E T A A v i r a I n t e r n e t S e c u r i t y D r. W e b S e c u r i t y S p a c e 1 0 A v i r a F r e e A n t i v i r u s M i c r o s o f t S e c u r i t y E s s e n t i a l s 32

9 Performance test November 14 / 8 CPU in idle [ % ] Lavasoft Ad-Aware Free Antivirus+ Webroot SecureAnywhere Antivirus 8 Kaspersky Internet Security ESET Smart Security 8 Qihoo 360 Total Security 5 Panda Internet Security AVG Internet Security TrustPort Internet Security 14 G Data Internet Security McAfee Internet Security F-Secure SAFE / Internet Security Fortinet FortiClient 5 Bitdefender Internet Security BullGuard Internet Security Panda Free Antivirus escan Internet Security with Cloud Security 14 Malwarebytes Anti-Malware Pro Bitdefender Antivirus Free Edition AVG Antivirus Free Arcabit Internet Security 14 Baidu Antivirus 5 Emsisoft Internet Security 9 avast! Internet Security Avira Free Antivirus avast! Free Antivirus Comodo Internet security 8 BETA Avira Internet Security Symantec Norton Security Trend Micro Internet Security Check Point ZoneAlarm Internet Security Suite FileMedic Internet Security 14 0,005 0,010 0,031 0,036 0,037 0,047 0,047 0,048 0,058 0,063 0,063 0,063 0,074 0,084 0,092 0,114 0,126 0,4 0,167 0,9 0,2 0,3 0,2 0,5 0,348 0,364 0,464 0,502 0,565 0,830 0,850 Dr.Web Security Space 10 5,5

10 Performance test November 14 / 9 RAM in idle [ MB ] Microsoft Security Essentials Bitdefender Antivirus Free Edition 5 6 Comodo Internet security 8 BETA escan Internet Security with Cloud Security 14 Symantec Norton Security Panda Internet Security Avira Free Antivirus avast! Free Antivirus Avira Internet Security avast! Internet Security G Data Internet Security Panda Free Antivirus Trend Micro Internet Security Baidu Antivirus 5 FileMedic Internet Security TrustPort Internet Security 14 Qihoo 360 Total Security 5 AVG Internet Security AVG Antivirus Free Kaspersky Internet Security Fortinet FortiClient 5 Emsisoft Internet Security 9 Dr.Web Security Space 10 F-Secure SAFE / Internet Security ESET Smart Security 8 Malwarebytes Anti-Malware Pro McAfee Internet Security BullGuard Internet Security Arcabit Internet Security 14 Bitdefender Internet Security Lavasoft Ad-Aware Free Antivirus Check Point ZoneAlarm Internet Security Suite 2

11 Performance test November 14 / 10 CPU scanning [ % ] Microsoft Security Essentials escan Internet Security with Cloud Security 14 Fortinet FortiClient 5 Avira Free Antivirus avast! Free Antivirus Avira Internet Security avast! Internet Security Symantec Norton Security Webroot SecureAnywhere Antivirus 8 Comodo Internet security 8 BETA Panda Internet Security Qihoo 360 Total Security 5 Panda Free Antivirus AVG Internet Security G Data Internet Security Baidu Antivirus 5 AVG Antivirus Free Check Point ZoneAlarm Internet Security Suite Bitdefender Antivirus Free Edition Kaspersky Internet Security F-Secure SAFE / Internet Security Dr.Web Security Space 10 FileMedic Internet Security 14 Arcabit Internet Security 14 TrustPort Internet Security 14 Lavasoft Ad-Aware Free Antivirus+ ESET Smart Security 8 Bitdefender Internet Security Emsisoft Internet Security 9 Malwarebytes Anti-Malware Pro 0,835 1,098 2,599 3,076 3,4 3,814 4,391 6,4 6,451 7,168 7,576 8, 9,847 11,8 11,368 12,38 13,03 14,024 14,702 16,541,351,395,424,086,262 26,551 27,5 30,934 32,447 33,857 BullGuard Internet Security Trend Micro Internet Security 47,609 51,664 McAfee Internet Security 63,9

12 Performance test November 14 / 11 RAM scanning [ MB ] Webroot SecureAnywhere Antivirus 8 Comodo Internet security 8 BETA Microsoft Security Essentials escan Internet Security with Cloud Security 14 avast! Free Antivirus avast! Internet Security Symantec Norton Security FileMedic Internet Security 14 Panda Internet Security Baidu Antivirus 5 AVG Internet Security Bitdefender Antivirus Free Edition Panda Free Antivirus AVG Antivirus Free Fortinet FortiClient 5 92 G Data Internet Security ESET Smart Security 8 Kaspersky Internet Security Trend Micro Internet Security Dr.Web Security Space 10 Qihoo 360 Total Security F-Secure SAFE / Internet Security 144 Avira Free Antivirus TrustPort Internet Security 14 Arcabit Internet Security 14 Avira Internet Security BullGuard Internet Security Bitdefender Internet Security Emsisoft Internet Security 9 McAfee Internet Security Lavasoft Ad-Aware Free Antivirus+ 2 Check Point ZoneAlarm Internet Security Suite 2 Malwarebytes Anti-Malware Pro 268

13 Performance test November 14 / 12 Copying time [ s ] Lavasoft Ad-Aware Free Antivirus+ escan Internet Security with Cloud Security 14 FileMedic Internet Security 14 Windows 7 Professional x32 SP1 Trend Micro Internet Security Malwarebytes Anti-Malware Pro Emsisoft Internet Security 9 AVG Internet Security Panda Internet Security avast! Internet Security Fortinet FortiClient 5 Avira Free Antivirus McAfee Internet Security Qihoo 360 Total Security 5 ESET Smart Security 8 Baidu Antivirus 5 Panda Free Antivirus Bitdefender Internet Security Avira Internet Security Webroot SecureAnywhere Antivirus 8 Comodo Internet security 8 BETA G Data Internet Security AVG Antivirus Free F-Secure SAFE / Internet Security Bitdefender Antivirus Free Edition Check Point ZoneAlarm Internet Security Suite BullGuard Internet Security Symantec Norton Security avast! Free Antivirus Dr.Web Security Space 10 TrustPort Internet Security 14 Microsoft Security Essentials Kaspersky Internet Security Arcabit Internet Security

14 Performance test November 14 / 13 Archiving [ s ] Panda Free Antivirus Malwarebytes Anti-Malware Pro Qihoo 360 Total Security 5 Panda Internet Security escan Internet Security with Cloud Security 14 Emsisoft Internet Security 9 McAfee Internet Security Lavasoft Ad-Aware Free Antivirus+ ESET Smart Security 8 BullGuard Internet Security Bitdefender Internet Security Symantec Norton Security FileMedic Internet Security 14 Windows 7 Professional x32 SP1 G Data Internet Security Webroot SecureAnywhere Antivirus 8 Baidu Antivirus 5 avast! Free Antivirus Avira Internet Security AVG Internet Security F-Secure SAFE / Internet Security Check Point ZoneAlarm Internet Security Suite Avira Free Antivirus TrustPort Internet Security 14 avast! Internet Security Dr.Web Security Space 10 Comodo Internet security 8 BETA Arcabit Internet Security 14 Trend Micro Internet Security Fortinet FortiClient 5 AVG Antivirus Free Kaspersky Internet Security 68 Microsoft Security Essentials 75 Bitdefender Antivirus Free Edition 160

15 Performance test November 14 / 14 Extracting [ s ] escan Internet Security with Cloud Security 14 8 Qihoo 360 Total Security 5 Dr.Web Security Space 10 Bitdefender Internet Security Lavasoft Ad-Aware Free Antivirus+ Fortinet FortiClient 5 Comodo Internet security 8 BETA FileMedic Internet Security 14 Symantec Norton Security Windows 7 Professional x32 SP1 Malwarebytes Anti-Malware Pro Avira Free Antivirus AVG Internet Security avast! Internet Security Kaspersky Internet Security McAfee Internet Security avast! Free Antivirus Panda Free Antivirus Arcabit Internet Security 14 TrustPort Internet Security 14 Avira Internet Security Emsisoft Internet Security 9 G Data Internet Security Trend Micro Internet Security F-Secure SAFE / Internet Security Panda Internet Security Webroot SecureAnywhere Antivirus 8 37 Baidu Antivirus 5 Bitdefender Antivirus Free Edition BullGuard Internet Security 51 AVG Antivirus Free 54 Microsoft Security Essentials 59 ESET Smart Security 8 62 Check Point ZoneAlarm Internet Security Suite 68

16 Performance test November 14 / Size after installation [ MB ] Webroot SecureAnywhere Antivirus 8 1 Microsoft Security Essentials Malwarebytes Anti-Malware Pro Dr.Web Security Space AVG Antivirus Free BullGuard Internet Security ESET Smart Security 8 AVG Internet Security G Data Internet Security Panda Internet Security Qihoo 360 Total Security 5 Panda Free Antivirus Kaspersky Internet Security McAfee Internet Security Bitdefender Antivirus Free Edition Check Point ZoneAlarm Internet Security Suite Baidu Antivirus 5 Arcabit Internet Security 14 Emsisoft Internet Security Symantec Norton Security avast! Free Antivirus Bitdefender Internet Security Comodo Internet security 8 BETA FileMedic Internet Security 14 F-Secure SAFE / Internet Security TrustPort Internet Security 14 Avira Free Antivirus Avira Internet Security Lavasoft Ad-Aware Free Antivirus+ Trend Micro Internet Security Fortinet FortiClient 5 avast! Internet Security escan Internet Security with Cloud Security

17 Performance test November 14 / 16 Average system startup time, Windows 7: 13 seconds 13-16: 4 points -: 3 points -24: 2 points -28: 1 points 29< : 0 points Copying, Windows 7: 7 seconds 000-0: 4 points 1-300: 3 points : 2 points : 1 points 401<: 0 points Archiving, Windows 7: 36 seconds 00-45: 4 points 46-54: 3 points 55-63: 2 points 64-72: 1 points 73<: 0 points Extracting, Windows 7: seconds 00-: 4 points -: 3 points 24-27: 2 points 28-31: 1 points 32<: 0 points CPU [ % ] idle and scan 00-10: 4 points 11-: 3 points -30: 2 points 30-40: 1 points 41<: 0 points RAM [ MB ] idle and scan 00-75: 4 points 76-0: 3 points 1-5: 2 points 6-300: 1 points 301< : 0 points Final score 32-28: certificate Best : certificate Best++ -: certificate Good+ -0: average quality

18 Performance test November 14 / Short program name Avg. CU* Points Copying Points Archiving Points Extracting Points CPU Points RAM Points CPU** Points RAM** Points Total avast! IS , , AVG IS , , escan IS with Cloud , , TS , , avast! Free , , FortiClient , , Panda Free , , Norton Security , , Comodo IS 8 BETA , , FileMedic IS , , G Data IS , , Webroot SA , , Avira Free , , Ad-Aware Free , , Panda IS , , Avira IS , , Baidu , , Kaspersky IS , ,

19 Performance test November 14 / MBAM Pro , , McAfee IS , , TrustPort IS , , Bitdefender IS , , Dr.Web SS , , Emsisoft IS , , ESET SS , , Trend Micro IS , , AVG Free , , F-Secure SAFE / IS , , Arcabit , , Bitdefender Free , , ZoneAlarm IS , , BullGuard IS , , Microsoft SE , *Average system startup time, **CPU and RAM during scanning.

20 Performance test November 14 / Points avast! Internet Security AVG Internet Security escan Internet Security with Cloud Total Security 5 avast! Free Antivirus Fortinet FortiClient 5 Panda Free Antivirus Symantec Norton Security Comodo Internet Security 8 BETA FileMedic Internet Security 14 G Data Internet Security 27- Points Webroot SecureAnywhere Internet Security Plus 8 Avira Free Antivirus Lavasoft Ad-Aware Free Antivirus+ Panda Internet Security Avira Internet Security BaiduAntivirus5 Kaspersky Internet Security Malwarebytes Anti-Malware Pro McAfee Internet Security TrustPort Internet Security 14 Bitdefender Internet Security Dr.Web Security Space 10.0 Emsisoft Internet Security 9 ESET Smart Security 8 Trend Micro Internet Security - Points AVG Antivirus Free F-Secure SAFE / Internet Security Arcabit Internet Security 14 Bitdefender Antivirus Free Edition Check Point Zone Alarm Internet Security Suite BullGuard Internet Security Microsoft Security Essentials --- Contact about tests for producers: kontakt@avlab.pl Certificates granted for download in high resolution:

21 Performance test November 14 / About AVLab AVLab brings together enthusiasts of antivirus software and web safety. Our activities include program testing and sharing results of our analyzes with all internet users. We are not controlled and/or linked with software producer in any way. AVLab tests are independent and take place in conditions close to reality. Don t be guided by our results when making final decision in choosing antivirus program. In order to make final choice we suggest read tests from other independent laboratories that use different methods and techniques for testing antivirus software. In addition, decisions depends on personal preferences, specified features, efficiency, detection rate, impact on system performance, user interface, price, usability, compatibility, language, technical support quality and many other things.