Test of the Antivirus Software For antivirus solutions, the test was divided up into two typical infection scenarios.

Transcription

1 29th July 2014 created by Markus Selinger 17 software packages in a repair performance test after malware attacks Can antivirus software packages or recovery/clean-up tools completely clean and repair a Windows system after a virus infection? Lab experts at AV-TEST resolved this question in an extremely comprehensive ten-month endurance test. The result: Yes, they can! It's a worst-case scenario many users have experienced: Malware defeats the security barriers of their system and becomes embedded there. Is it even possible, in the aftermath of such an attack, to restore the Windows system to its previous condition? It ought to work with the help of antivirus software or clean-up tools. But how effectively do these first responders do their job? The laboratory at AV-TEST has resolved this question in a major test in which 10 antivirus solutions and 7 special cleaning tools had to not only clean the infected systems but also completely repair them. The following security packages were examined in the test: - 3 popular freeware solutions: Avast! Free Antivirus 9.0, AVG AntiVirus Free 2014, Avira Free Antivirus; - Malwarebytes Anti-Malware Free, a solution recommended by many IT magazines; - Microsoft Security Essentials, included in Windows; - five additional well-known paid software solutions from Bitdefender, ESET, F-Secure, Kaspersky and Norton. In addition, the following clean-up and recovery tools were tested: - The tools Avira Cleaner and Hitman Pro recommended by the initiative and anti-botnet advisory center "Botfrei.de"; - Disinfect2013 from IT trade publisher Heise; - the four well-known tools F-Secure Removal Tool, Kaspersky Removal Tool, Panda Cloud Cleaner and Norton Power Eraser. The Test Procedure In this endurance test, the software solutions were tested multiple times over a period of 10 months, from September 2013 to June 2014 in terms of their clean-up and repair performance. Throughout the entire period, various malware families were repeatedly tested, as they also undergo constant evolution. The malware samples used were already known to all the solutions and thus should have been clearly detected by all of them. In the test tables, the clean-up and repair quality was recorded according to the following priority: 1. Was the malware detected or not? 2. Were the active components completely removed? 3. Did any harmless file remnants remain, and were all the changes to the system reversed? 4. Did the security and clean-up software perfectly remove and restore everything? As a result, the table also includes a percentage rating of the overall clean-up and repair performance. Test of the Antivirus Software For antivirus solutions, the test was divided up into two typical infection scenarios. 1. On a system already infected with malware, the protection software was installed and the subsequent detection, clean-up and repair of the damage was logged. 2. The protection packages were briefly deactivated, the malware was loaded, and protection was reactivated. Here the detection, clean-up and repair were logged again.

2 This two-part scenario was intended to simulate what happens if a user installs the software retroactively and if an installed security package only detects malware after a certain period of time and then attempts to remedy the damage. In each step, the system watchdogs had to deal with 30 malware samples. Test of the Recovery and Repair Tools The recovery tools are generally only deployed after a malware infection has occurred. Which was exactly the scenario examined in the test. Each of the 7 tools had to remove 55 intrusions of malware samples and their damage. These threats were known to the tools as well; after all, the object was to examine the repair performance and not the detection ability. The Test Platform and Environment This endurance test as with all others conducted by AV-TEST was performed exclusively on real hardware, this time under Windows 7, however. The rationale: some malware samples can detect whether they are in a virtual environment. In this case, there was a possibility that some of the malware code would not fully execute. With the use of real hardware, the scenario is as realistic as the user's everyday environment. The lab test was extremely comprehensive, as each platform had to be infected individually with a malware sample. Afterwards, the protection packages were installed or reactivated, and clean-up tools were launched respectively. Finally, the cleaned system always had to be compared to the original system. For a total of 60 and 55 malware samples respectively and 10 security packages, plus 7 tools, this added up to 985 individual tests over a period of 10 months. Evaluating Results For both test groups, security packages and recovery tools, the individual test results were entered into the table according to the following descending risk schema: 1. Malware not detected 2. Active malware component not removed 3. Only harmless file remnants left behind 4. Complete removal, clean system Result 1: Malware not detected The result in this category actually should have been 0 for all the test candidates, because it was established prior to the test that each malware sample was known to the solution. Among the security packages, known malware was not detected as such by Microsoft Security Essentials and Avira Free Antivirus. That actually should not have happened. The seven clean-up tools revealed no errors in basic detection. Result 2: Active malware components not removed In this case, the malware was detected and some files were removed, but the core of the malware was not rendered harmless. Thus, the system remains infected. Among the 10 security packages, the freeware solutions from Avira, Avast and AVG, as well as Microsoft's Security Essentials, had several problems. They failed to remove active components in the system between 1 and 7 times. The additional paid software products and the freeware Malwarebytes Anti-Malware Free cleaned error-free in this case. Among the 7 clean-up tools, more than half the participants also failed on this key task: Hitman Pro, Panda Cloud Cleaner, Avira Cleaner and the F-Secure Removal Tool were unable to remove 1 to 11 active components. Only the Kaspersky Removal Tool, the Norton Power Eraser and Disinfect2013 from Heise performed reliably in this category. Result 3: Harmless file remnants left behind This segment documented the harmless code remnants left behind in the aftermath of malware attack clean-up. These remnants mainly included additional ineffective files or orphaned entries in the Windows registry.

3 The security packages from Bitdefender, F-Secure, Kaspersky, Norton and ESET only left small file remnants and insignificant entries in the registry between 1 and 9 times. In terms of quality, this put them just slightly below the perfect score attained by Malwarebytes Anti-Malware Free. Among the clean-up tools, the Kaspersky Removal Tool only missed removal of a harmless file in one instance. It performed all other tasks with flying colors. Norton Power Eraser fell short of a perfect clean-up 11 times, and Disinfect2013 from Heise even missed it 48 times. All it left behind, however, was digital garbage without any dangerous components. Result 4: Complete removal, clean system Of all the security packages, only the Malwarebytes Anti-Malware Free utility left the system completely cleaned and repaired after 60 tests. The package from Bitdefender was able to do this 59 times, whereas the packages from F-Secure and Kaspersky were each successful 56 times. The two security solutions merely overlooked harmless registry entries. Otherwise they achieved an almost perfect performance. Among the clean-up tools, none of the tools was able to render the infected test systems completely clean. Only the Kaspersky Removal Tool achieved top performance. It would have left the system totally error-free except for one overlooked, yet harmless, file remnant. Summary: There is software for the morning after The test results disproved a statement frequently posted in forums, that all you can do is delete your Windows system if it becomes infected with malware. Among the security suites, the solutions from Malwarebytes, Bitdefender and Kaspersky demonstrated the best performance among all the packages tested. All in all, however, the rest of the field still showed quite solid performance, even though a few active components were left behind. Among the freeware clean-up tools, the Kaspersky Removal Tool is worth recommending. While the Norton tool, as well as Disinfect2013 from Heise, left behind quite a lot of data garbage, neither failed to detect a single active malware component. In final analysis: In case an existing security suite ever fails, there are reliable rescue options for having a Windows system cleaned and repaired. The best part of all: in case of emergency, most of the tools can even be used free of Charge.

4 Repair test: In two typical attack scenarios, the 10 antivirus solutions were tested in terms of their clean-up and repair performance (AV-TEST Endurance Test from 9/13 to 6/14). Overall result: Most antivirus solutions showed excellent performance in the repair test. Several of them did leave some digital garbage behind, but nothing dangerous (AV-TEST Endurance Test from 9/13 to 6/14).

5 Recovery and Repair Tools: The traditional first responders were also tested following two typical attack scenarios in terms of their clean-up and repair performance (AV-TEST Endurance Test from 9/13 to 6/14). Recovery and Repair Tools: Most of the first responders after a malware attack achieved a favorable result in test. They were also quite reliable in clean-up and repair (AV-TEST Endurance Test from 9/13 to 6/14).

6 Malwarebytes Anti-Malware Free: Among the antivirus solutions, this application achieved a perfect repair result. Coming in close behind in terms of performance were the packages from Bitdefender, Kaspersky, Norton, ESET and F-Secure.

7 F-Secure Internet Security 2014: This antivirus software also impressed the test engineers with its quality of repair in the major endurance test.

8 Kaspersky Removal Tool: This freeware first-responder tool rendered the system completely repaired in the test after 55 attacks. Only once did it overlook an unimportant file. Copyright 2016 by AV-TEST GmbH, Klewitzstr. 7, Magdeburg, Germany Phone +49 (0) , Fax +49 (0) ,